Data and Implementation for State-of-the-Art Sentiment Model Evaluation

Data and Implementation for State-of-the-Art Sentiment Model Evaluation

Data and Implementation for State-of-the-Art Sentiment Model Evaluation

Type
ExperimentData
Author
Jeremy Barnes, Roman Klinger, Sabine Schulte im Walde
Description

In sentiment analysis there is a tendency to compare models only on one or two datasets, either because of time restraints or because the model is tailored to a specific task. Accordingly, it is hard to understand how well a certain model generalizes across different tasks and datasets.

In this work, we contribute to improving this situation by comparing several models on six different benchmarks, which belong to different domains and additionally have different levels of granularity (binary, 3-class, 4-class and 5-class). We show that Bi-LSTMs perform well across datasets and that both LSTMs and Bi-LSTMs are particularly good at fine-grained sentiment tasks (i. e., with more than two classes). Incorporating sentiment information into word embeddings during training gives good results for datasets that are lexically similar to the training data.

Reference

Jeremy Barnes, Roman Klinger, and Sabine Schule im Walde. Assessing state-of-the-art sentiment models on state-of-the-art sentiment datasets. In Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis, Copenhagen, Denmark, 2017. Workshop at the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics

A preprint of the paper is available at http://www.romanklinger.de/publications/BarnesKlingerSchulteImWalde2017.pdf

Download

The embeddings are available at https://www2.ims.uni-stuttgart.de/data/sota_sentiment/embeddings.zip

Sourcecode

Code repository: https://github.com/jbarnesspain/sota_sentiment

Roman Klinger
Dr.

Roman Klinger

Senior Lecturer (Akademischer Oberrat)

Sabine Schulte im Walde
Apl. Prof. Dr.

Sabine Schulte im Walde

Akademische Rätin (Associate Professor)

To the top of the page