We are very happy to announce the release of SimVerb-3500 : a new
evaluation resource for verb pair similarity. SimVerb-3500 consists of 3500
verb pairs, each is rated by 10 human judges for similarity, with scores on
a 1-10 scale. The annotation guidelines are adopted from SimLex-999 so that
the judges are guided to judge similarity rather than association.
SimVerb-3500 is a high coverage resource: it covers all normed verb types
from the USF free-association database, and consists of at least three verb
type examples from every VerbNet class. It is divided into train,
development and test sets to facilitate principled machine learning
The resource can be downloaded from:
A paper describing the dataset and analysing the performance of various
state-of-the-art vector space models in predicting its scores has recently
been accepted to EMNLP 2016:
*SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity*
Daniela Gerz, Ivan Vuli?, Felix Hill, Roi Reichart and Anna Korhonen.
EMNLP 2016. [pdf