Emory NLP is hiring Emory students for annotation related to relation extraction research. Accepted applicants are expected to go through a training session before the actual annotation. Please fill this form if you want to apply for this job.
This project aims to extract relations for three types of NLP tasks, coreference resolution (CR), temporal ordering (TO), and discourse/causal inference (DCI), in 1,000 social media posts from Reddit. Here are details about these tasks:
Thus, each annotator is expected to complete 50 - 500 posts per task (CR: 6 - 60 hours, TO: 10 - 100 hours, DCI: 10 - 100 hours). Note that you do not need to work on all three tasks. Once the annotation is done and passes our quality assurance, we will pay $1.2, $2, and $2 per post ($60 - $600, $100 - $1,000, $100 - $1,000) for CR, TO, and DCI respectively, with an Amazon gift card.
This task aims to find a set of mentions that refer to the same entity. In the following example, "The boy" and "himself" refer to the same entity. Moreover, the pronoun "it" refers to the event described by "likes".
[The boy]E0 [likes]V0 [himself]E0too much. [It]V0is embarrassing.
We will also annotate part-whole relations between entities and events. In the following example, the pronoun "my" refers to the speaker who is a part of "my group". Moreover, the pronoun "they" refers to "some", which is a part of my group.
[Some of [[my]E0group]E0]E1 are useless, [they]E1never help.
Finally, generic nouns, typically used when making generalizations, are also annotated.
[Doctors]Ghelp [doctors]G
This task aims to find the temporal order of events within the post. Specifically, we will identify the following relations:
before/aftercontained/containsoverlapTThe events are ordered by the Document Creation Time (DCT). In the following example, DCT is after the time the boy was eating, and the boy ate his dinner before he became sick:
The boy ate his dinner. He later was sick.
after(ate, DCT)
before(ate, sick)In the following example, the bear comes up to the man during the time he is pretending; thus, the contains relation is used:
He threw himself on the ground and pretended to be dead. The bear came up and sniffed all around him.
after(threw, DCT)before(threw, pretended)contains(pretended, came up)before(came up, sniffed)Finally, the following example shows two events thatoverlap:
The insurer’s earnings from commercial property/casualty lines fell 59% in the latest quarter, while it lost $7.2 million in its personal property/casualty business.
after(fell, DCT)overlap(fell, lost)This task aims to infer causal dependencies between events and discourse relations, such as elaboration, contrast, restatement, etc.
| Role | Situation |
|---|---|
cause | If A did not occur, B would not have either. |
enable | If A did not occur, B may have occurred anyway. |
prevent | A reduces the likelihood of B occurring. |
condition | Only when B happens, can A happen |
contrast | A and B share a significantly different predicate or property |
parallel | A and B share a significantly similar predicate or property |
concession | A should result in the occurrence of C, but B indicates the opposite of C occurring. |
exception | B is an exception to A; A is not true because of B |
elaboration | B is a more detailed description of A |
restatement | B restates the semantic meaning of A |
option | A and B are alternative situations of each other |
chosenoption | A and B are alternative situations of each other, but the author prefers A |
The following example shows an instance of a contrast between the size of TCNJ and Rutgers.
TCNJ is a super small college. Rutgers is humongous, though, totally different vibe.
contrast(a super small college, humongous)In the following example, the author is making a concession: even though Fauci has done so much, people dislike him.
People still dislike Fauci. Despite all he has done.
dislike(concession, done)The following is an example of a restatement.
I wasn't entirely sure what to write for my essay. In fact, I was pretty confused about where to start.
restatement(sure, confused)