We present a novel dataset for physical and abstract plausibility of events in
English. The ability to discern plausible from implausible events is crucial for
natural language processing and comprehension. Most previous work on
computational models of plausibility however focuses on semantic knowledge
relevant for distinguishing physically plausible events such as “cat-eat-sardine”
from implausible ones such as “rain-break-belly”. Furthermore, while recent
datasets include events with conceptually abstract participants as to our
knowledge no previous work has systematically investigated the interaction of
event plausibility and abstractness of the involved concepts. The current study
extends the traditional focus and proposes to systematically discern abstract
plausible events such as “law-prohibit-discrimination” from implausible ones
such as “humor-require-merger”. Based on naturally occurring sentences in
the English Wikipedia, we infiltrate degrees of abstractness, and automatically
generate perturbed pseudo-implausible events. We annotate plausibility using
crowd-sourcing, and perform extensive cleansing to ensure annotation quality.
As human intuition regarding the assessment of plausibility is highly individual,
we represent and examine annotation disagreement. In-depth quantitative
analyses indicate that annotators favor plausibility over implausibility, and
disagree more on implausible events. We further find that event abstractness
has an impact on plausibility ratings: more concrete event participants trigger
a perception of implausibility.