Annotation Job: Chatbot Evaluation

Emory NLP is hiring Emory students for a comparative evaluation of Conversational AI Research. Please fill this form to apply for this job.

• Prerequisite: fluent in English.
• The project begins in the week of Oct 31st, 2022.
• You must be a current Emory student to apply for this job.
• Your name will be acknowledged in research papers describing this annotation work.

This project involves reading two 30-turn conversations and then selecting which of the two better fits a given description. A single task will consist of performing this selection for each of 8 different characteristics. An example is shown below:

To access your assigned tasks, you will be emailed URL links to our web-based application. Each link is tied to one evaluation task that you will need to complete. When you complete all of the links, you will have finished all of your assigned work.

• There are 200 total tasks that will be distributed among those who participate.
• Each task takes about 4-5 minutes and the compensation rate is $1.00 per task.
• You will be assigned with 10-25 tasks and have one week to finish all of your assignments once the project begins.
• You must complete all of your assigned tasks in the given timeframe in order to be compensated.