Call for Participation
FIRE 2016 Microblog Track
To be organized at FIRE 2016
8 — 10 December, Indian Statistical Institute, Kolkata
Information Extraction from Microblogs Posted during Disasters
User-generated content in microblogging sites like Twitter is known to be important sources of realtime information on various events, including disaster events like floods, earthquakes, and terrorist attacks. In this track, our aim is to develop IR methodologies for extracting important information from microblogs posted during disasters.
The data will contain:
- Around 50,000 microblogs (tweets) from Twitter, that were posted during the Nepal earthquake in April 2015. Tweet ids along with a script to download the tweets will be provided to the participants.
- A set of 5 – 8 topics in TREC format, each containing a title, a brief description, and a more detailed narrative on what type of tweets will be considered relevant to the topic.
Since the aim of this track is to extract a set of tweets that are relevant to each topic, set-based evaluation metrics like precision, recall, and F-score will be used. The gold standard, against which the set of tweets identified by the participants will be matched, will be generated by a “manual run” where human volunteers (assessors) will be given the same set of tweets and topics, and asked to identify all possible relevant tweets using a search engine (Indri).
While judging the participants’ runs, we will also arrange for a second round of assessments, if necessary, to judge the relevance of tweets that are identified by the participants but not identified during the first round of human assessment.
- July 1, 2016: Data and topics released.
- August 15, 2016: Run submission deadline.
- September 15, 2016: Results declared.
- October 15, 2016: Working notes due.