SFB 732 (Area D) Workshop:
Linguistically-aware Models of Distributional Semantics in Computational Linguistics

Institute for Natural Language Processing (IMS)
University of Stuttgart
July 9-10, 2015




Background:

Distributional semantic models (DSMs) of various flavors currently play a major role in mainstream computational linguistics, but much of this work is primarily technical, with little relationship to linguistic insights or principles. The aim of this workshop is to promote discussion of how distributional semantic models can be connected to linguistic insight by bringing together researchers who are interested in linguistic modeling with data, employing distributional methods.

We are pleased to announce five invited talks from researchers doing exciting work in various subfields within distributional semantics, including those whose work focuses on lexicography, compositionality, multi-modal modeling, psycholinguistics, and application-oriented uses of DSMs.


Program:
July 8, 2015
19.30 Dinner and Drinks: Brauhaus Schönbuch
July 9, 2015
9.30-10.00 Opening Session
10.00-11.00 Invited Talk: Harald Baayen and Cyrus Shaoul (Tübingen): A discriminative perspective on vector semantics
11.00-11.30 Coffee Break
11.30-12.30 Invited Talk: Diana McCarthy (Cambridge): Word meaning representations: one size might not fit all
12.30-13.00 SFB Project Teaser Talks
13.00-14.30 Lunch Break at Commundo (self-service cafeteria next to IMS; reserved tables for workshop participants)
14.30-16.00 SFB Project Poster Session:
B9: Distributional Characterization of Derivation
D2: Combining Contextual Information Sources for Disambiguation in Parsing and Choice in Generation
D8: Data-driven Dependency Parsing - Context Factors in Dependency Classification
D10: Incrementality in Compositional Distributional Semantics
D11: A Crosslingual Approach to the Analysis of Compound Nouns
D12: Sense Discrimination and Regular Meaning Shifts of German Particle Verbs
INF: Information Infrastructure
16.00-16.30 Coffee Break
16.30-17.30 Invited Talk: Ivan Titov (Amsterdam): Learning shallow semantics with little or no supervision
19.00 Dinner: Zum Spätzleschwob
July 10, 2015
9.30-10.30 Invited Talk: Ed Grefenstette (Google Research): How much linguistics is needed for NLP?
10.30-11.00 Coffee Break
11.00-12.30 Panel Discussion
12.30-14.00 Lunch Break at Commundo (self-service cafeteria next to IMS; reserved tables for workshop participants)
14.00-15.00 Invited Talk: Stephen Clark (Cambridge): Multimodal distributional semantics
15.00-15.30 Closing Remarks
post-workshop Beer Garden or similar (depending on the weather)

Venue:

The workshop takes place at the Institute for Natural Language Processing (IMS), in the seminar rooms V5.01 and V5.02 on the ground floor. See here on how to get to the IMS. Note: Currently, there is a building site next to the IMS, so it's quite difficult to find the entrance. We recommend to print this PDF and bring it with you.


Abstracts:

Stephen Clark
"Multimodal distributional semantics"

One of the criticisms often levelled at distributional semantics is that the meaning representations are still linguistic, with no ``grounding" or denotation in the world. A recent attempt at addressing this problem, by various researchers, has been to ground distributional vectors in images, and also psychological feature norms, in the latter case as a proxy for perceptual experience. In this talk I will describe attempts to induce mappings between textual spaces and spaces defined in terms of other modalities, including feature norms and images. We have also begun obtaining vector representations for concepts in terms of how the concept sounds (its audio representation) and even how it smells (its olfactory representation). Evaluations have been conducted using standard semantic similarity datasets, as well as ``zero-shot" learning scenarios where, for example, it is possible to guess what a concept might look, or sound, or smell like, based on its textual representation, even if the concept has never been encountered before.

Ed Grefenstette
"How much linguistics is needed for NLP?"

 Many problems in Natural Language Processing, from Machine Translation to Parsing, can be viewed as transduction tasks. Recently, sequence-to-sequence mapping approaches using recurrent networks and parallel corpora have shown themselves to be capable of learning fairly complex transductions without the need for heavy (or any) annotation or alignment data. Traditional linguistically-motivated features such as syntactic types and dependencies are entirely latent in such models, reducing the need for expert linguistic knowledge in designing new solutions in NLP. In this talk, I will discuss the strengths and weaknesses of such approaches, before presenting some ameliorations based on attention mechanisms and working memory enhancements to standard recurrent neural networks.

Diana McCarthy
"Word meaning representations: one size might not fit all"

 The word sense disambiguation literature is full of neat examples of ambiguity and the necessity of resolving it but there is a large body of literature external to computational linguistics, and some from within, that highlights the fact that in a great many cases sense distinctions are not clear cut. There are already options for building softer, subtler more nuanced models of word sense but this may bring a great deal of additional complexity, and effort when creating gold-standards, which begs the question -- is this complexity always worth it? In this talk I'll discuss some different perspectives on word meaning from linguistics, lexicography and computational linguistics and how these relate to various representations in distributional semantics models. I'll talk about some preliminary work analysing datasets aimed at determining how readily a lemma partitions into senses and end by arguing that we could and perhaps should use different representations for different lemmas.

Ivan Titov
"Learning shallow semantics with little or no supervision"

 Inducing meaning representations from text is one of the key objectives of natural language processing. Most existing statistical semantic analyzers rely on large human-annotated datasets, which are expensive to create and exist only for a very limited number of languages. Even then, they are not very robust, cover only a small proportion of semantic constructions appearing in the labeled data, and are domain-dependent. We investigate approaches which do not use any labeled data but induce shallow semantic representations (i.e. semantic roles and frames) from unannotated texts. Unlike semantically-annotated data, unannotated texts are plentiful and available for many languages and many domains which makes our approach particularly promising. I will contrast the generative framework (including our non-parametric Bayesian model) and a new approach called reconstruction-error minimization (REM) for semantics. Unlike the more traditional generative framework, REM lets us effectively train expressive feature-rich models in an unsupervised way. Moreover, it allows us to specialize our representations to be useful for basic forms of semantic inference. We show that REM achieves state-of-the-art results on the unsupervised semantic role labeling task (across languages, without any language-specific tuning) and significantly outperforms generative counterparts on the unsupervised relation discovery task. I will also discuss how evidence from annotated data as well as other forms of linguistic knowledge can be incorporated as soft constraints to guide induction of semantic representations.

Joint work with Ehsan Khoddam, Alex Klementiev and Diego Marcheggiani.