StAtutory Reasoning Assessment (SARA)

Read the paper; listen to the NLP Highlights Podcast

Download SARA.tar.gz

Example statute

Example statute in Prolog

Example entailment case

Example numerical case

Dataset

The statutes are located under statutes/source/ and the cases under cases/. The cases are split into train and test, following splits/{train,test}. This Github repo contains instructions and scripts to format the dataset into .txt files, typically used for machine reading and to run experiments. Complementary statistics to the dataset can be found here.

Prolog program

To run the Prolog program, you need SWI-Prolog version 7.2.3 for amd64. Run prolog statutes/prolog/init.pl. This will load into memory the entirety of the Prolog version of the statutes, and open an interactive Prolog shell. From there, you can enter facts and call predicates.

Each case can be run individually, via e.g. prolog cases/s3306_a_2_B_neg.pl. The final :- halt statement ensures that no interactive shell remains open. If you want the shell to remain open, e.g. to try out other predicates with the facts of the case, comment the halt statement out.

You may run all the cases one after the other with bash code/run_cases_prolog.sh. Running this command is a good initial check to see whether the Prolog program is being interpreted and run correctly.

Citation

A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering [paper] [bibtex] [featured on AI2's NLP Highlights podcast]
Nils Holzenberger, Andrew Blair-Stanek and Benjamin Van Durme
Proceedings of the 2020 Natural Legal Language Processing (NLLP) Workshop, 24 August 2020, San Diego, US

back to main page