Example projects (project hub)

Potato aims to improve the replicability of data annotation and reduce the cost for researchers to set up new annotation tasks. Therefore, Potato comes with a list of predefined example projects, and welcome public contribution to the project hub. If you have used potato for your own annotation, you are encouraged to create a pull request and release your annotation setup.

Schema examples

Potato provides a list of example templates for all the supporting annotation schemas and you can easily download them from project hub using:

[fetch] potato get simple_schema_examples
[launch] potato start simple_schema_examples -p 8000

>>> multiple config files found, please select the one you want to use (number 0-13)
[0] simple-active-learning.yaml
[1] simple-best-worst-scaling.yaml
[2] simple-pairwise-comparison.yaml
[3] simple-check-box-dynamic-labels.yaml
[4] simple-video-as-label.yaml
[5] simple-span-labeling.yaml
[6] simple-likert.yaml
[7] simple-html-annotation.yaml
[8] empty-to-copy.yaml
[9] simple-video-annotation.yaml
[10] simple-single-choice-selection.yaml
[11] simple-text-box.yaml
[12] simple-check-box.yaml
[13] kwargs_example.yaml
[14] simple-check-box-with-free-response.yaml

[Annotate] http://localhost:8000

Annotation projects

Offensiveness rating

yaml config | Dataset

[fetch] potato get offensiveness
[launch] potato start offensiveness -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Question Answering / Reading comprehension (SQuAD style)

yaml config | Dataset

[fetch] potato get reading_comprehension
[launch] potato start reading_comprehension -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Text rewriting

yaml config | Dataset

[fetch] potato get text_rewriting
[launch] potato start text_rewriting -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Politeness rating

yaml config | Dataset

[fetch] potato get politeness_rating
[launch] potato start politeness_rating -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Dialogue analysis (span + categorization)

yaml config

[fetch] potato get dialogue_analysis
[launch] potato start dialogue_analysis -p 8000
[Annotate] http://localhost:8000

Sentiment analysis (categorization)

yaml config

[fetch] potato get sentiment_analysis
[launch] potato start sentiment_analysis -p 8000
[Annotate] http://localhost:8000

Summarization evaluation (likert + categorization)

[fetch] potato get summarization_evaluation
[launch] potato start summarization_evaluation -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Match findings in papers and news (likert + prescreening questions + multi-task)

yaml config | Paper | Dataset

[fetch] potato get match_finding
[launch] potato start match_finding -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Match findings in papers and news (prestudy test)

yaml config

[fetch] potato get match_finding_with_prestudy
[launch] potato start match_finding_with_prestudy -p 8000
[Annotate] http://localhost:8000/?PROLIFIC_PID=user

Textual uncertainty (likert + categorization)

yaml config | Paper | Dataset

[fetch] potato get textual_uncertainty
[launch] potato start textual_uncertainty -p 8000
[Annotate] http://localhost:8000/

Immigration framing in tweets (Multi-schema categorization)

yaml config | Paper | Dataset

[fetch] potato get immigration_framing
[launch] potato start immigration_framing -p 8000
[Annotate] http://localhost:8000/

GIF Reply Appropriateness (video as label)

yaml config | Paper | Dataset

[fetch] potato get gif_reply
[launch] potato start gif_reply -p 8000
[Annotate] http://localhost:8000/

Empathy as Appraisal (paired texts + likert)

yaml config | Paper

[fetch] potato get empathy
[launch] potato start empathy -p 8000
[Annotate] http://localhost:8000/