Uncertain Natural Language Inference


UNLI (Uncertain natural language inference) is a refinement of Natural Language Inference (NLI) that shifts away from categorical labels, targeting instead the direct prediction of subjective probability assessments to model subtle distinctions on the likelihood of a hypothesis conditioned on a premise.

Read our paper here.


A subset of the popular SNLI dataset is labeled under a probabilistic scale to create our proof-of-concept UNLI dataset, u-SNLI, which can be downloaded here. The dataset comprises of 3 files, {train|dev|test}.csv, where each file contains the following columns: ID, Premise, Hypothesis, NLI label, UNLI label.

Example annotations:

The code to reproduce the results reported in the paper can be found here.


  title={Uncertain Natural Language Inference},
  author={Tongfei Chen and
           Zhengping Jiang and
           Adam Poliak and
           Keisuke Sakaguchi and
           Benjamin {Van Durme}},
  booktitle={Proceedings of ACL},