РОМИП объявил дорожку по машинному переводу. Результаты будут объявлены в мае на «Диалоге», дедлайн подачи результатов — 22 января.
Не упустите свой шанс!
AIM & SCOPE
The aims of the workshop are (1) to develop a common testbed and
(2) to evaluate different machine translation approaches for English-Russian pair. Translation will be evaluated on an unseen test set using different machine translation evaluation methods.
The workshop is open to all kinds of MT systems and technologies.
Experienced and early-stage researchers, as well as industrial developers, are welcome to participate in the evaluation campaign and the workshop.
The workshop will take place at the Dialog conference on computational linguistics and intelligent technologies ( http://www.dialog-21.ru/
A test dataset of about 150,000 sentences originally written in English will be made available to the participants. The participants will be requested to submit the whole dataset translated into Russian.
The participants are free to use their own systems and any data to complete the task. For example, participants can use the following freely available resources:
— 1M sentences English-Russian parallel corpus released by Yandex;
— 119K English-Russian parallel corpus from the TAUS Data Repository.
Organizers leave to participants’ discretion to use or not to use these data.
The evaluation will be completed on about 1,000 sentences from the test dataset. Human translators will translate these sentences to ensure the gold standard quality. We will employ two types of evaluation measures:
— automated metrics widely adopted by statistical machine translation community;
— blind pairwise evaluation of systems’ output performed by human assessors.
We expect the participants to share organizational costs either by taking part in human assessment or in monetary form.
20 December 2012 — announcement & data samples
15 January 2013 — release of complete dataset
22 January 2013 — deadline for runs submission
22 February 2013 — announcement of evaluation results
20 March 2013 — deadline for report submission
29 May — 2 June 2013 — workshop @ Dialog conference
Pavel Braslavski (Kontur Labs/Ural Federal University) Maxim Khalilov (TAUS Labs) Sergey Sharoff (University of Leeds)