Post-Evaluation Data for SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection

Post-Evaluation Data for SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection

Post-Evaluation Data for SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection

Type

Dataset

Author

Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky und Nina Tahmasebi

Description

This data collection contains the post-evaluation data for SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection:

  • the true binary change scores of the targets for subtask 1, and their true graded change scores for subtask 2 (test_data_truth/),
  • the scoring program used to score submissions against the true test data in the evaluation and post-evaluation phase (scoring_program/),
  • the results of the evaluation phase including
    • the submitted files of each team (results/submissions/),
    • an overview of the results for each submission ordered by team (results/submissions_results.csv),
    • analysis plots (plots/) displaying the results:
      • under per_target/ we provide the gold change scores and the normalized prediction error of target words plotted against their frequency and polysemy statistics,
      • under per_team/ we provide the model predictions from the best submission per team (per subtask) plotted against frequency/polysemy statistics and performance on gold data (gray lines give the correlation with the respective variable in the gold data); we also provide plots of visualizing the teams' prediction similarities.

Some remarks:

  • the paper referenced below remains the only source for the rankings between teams,
  • some teams were disqualified, and are thus removed from the analyses and the rankings present in the paper,
  • some teams have changed names, resulting in a discrepancy between team names under results/ and team names in the paper. The paper contains a key to match old names with new names.

Please find more information on the provided data in the paper referenced below.

Reference

Dominik Schlechtweg, Barbara McGillivray, Simon Hengchen, Haim Dubossarsky and Nina Tahmasebi. 2020. SemEval-2020 Task 1: Unsupervised Lexical Semantic Change Detection. SemEval@COLING2020.

Sabine Schulte im Walde
Apl. Prof. Dr.

Sabine Schulte im Walde

Akademische Rätin (Associate Professor)

To the top of the page