Language-Knowledge Interaction

Ein IBM AI Horizons Network Projekt zwischen dem IMS Stuttgart und IBM Research Zürich

Five doctoral researchers and three PIs from IMS Stuttgart involved in collaborative AI research

Project runtime

This collaboration was started in early 2020.

Principal investigators

The PIs at the Stuttgart end are Jonas Kuhn (lead-PI), Sebastian Padó and Thang Vu

Project description

Language has always been tightly interwoven with the formation, aggregation and exchange of knowledge and beliefs. Even today, most of the exchange of knowledge and ideas is expressed through speech and language: Language is the fabric of the internet, of news providers and social media, of political and scientific communication. Crucially, language is both powerful and flexible: we can use it to talk about arbitrary concepts, words can be used to refer to different things depending on context, and language can be extended as the world changes (e.g. by introducing the verb“to text” for sending text messages). Last but not least, language provides conventionalized ways of linking into other modalities, such as the visual presentation of information.

In contrast to this so-called unstructured information, computers excel at working with structured information, that is, formal representations of knowledge, mathematical objects which enable them to check queries against large knowledge bases, derive new facts, or to identify contradictions, and do so reliably and efficiently.

The challenge is how to build an interface between unstructured and structured information: It is exactly the power and flexibility of language that make the translation into formal representationsa great challenge. Manually created mappings have only been able to capture selected aspects. Recent advances in machine learning, using so-called deep learning models, now provide the means to automatically learn rich, multi-dimensional formal representations of language from samples of natural usage. This makes the mapping dynamic: a recurring pattern in the knowledge exchange of the respective language community will be (implicitly) captured in the representation –even when no system designer has anticipated it.

The project “Knowledge-Language Interaction” within the IBM AI Horizons Network brings together the Institute for Natural Language Processing at the University of Stuttgart and IBM Research Zurich to work together on this focal point of current AI research.

The new learning methods present a huge spectrum of technological opportunities for advanced language interfaces. At the same time, they require a re-thinking of the relationship between learned models on the one hand and their basis –a combination of linguistic competence and contextually grounded pieces of knowledge and beliefs –on the other. In many applications, it is crucial to be able to trace back system predictions to an interpretable distinction, such that further decisions can be justified and reflected. Linking predictions to hypotheses that are expressed in some conceptual framework is also necessary for further progress in modeling approaches. Last but not least, the need to learn from ever smaller datasets (which is imperative for many practical problem settings) calls for systematic strategies to combine knowledge representations obtained from different sources and modalities.


The project is part of the IBM AI Horizons network


Kontakt IMS

Pfaffenwaldring 5 b, 70569 Stuttgart


Webmaster des IMS

Zum Seitenanfang