TextEE: A Standardized, Fair, and Reproducible Benchmark for Event Extraction

Kuan-Hao Huang1, I-Hung Hsu2, Tanmay Parekh3, Zhiyu Xie4, Zixuan Zhang1, Premkumar Natarajan2, Kai-Wei Chang3, Nanyun Peng3, Heng Ji1
1University of Illinois Urbana-Champaign, 2University of Southern California, 3University of California, Los Angeles, 4Standford University
Paper arXiv Code

Introduction

TextEE is a standardized, fair, and reproducible benchmark for evaluating event extraction approaches.

  • Standardized data preprocessing for 10+ datasets.
  • Standardized data splits for reducing performance variance.
  • 10+ implemented event extraction approaches published in recent years.
  • Comprehensive reevaluation results for future references.

We will keep adding new datasets and new models!

News

  • [2023/11/15] We release TextEE. Feel free to contact us if you want to contribute your models or datasets!

Supported Datasets

Supported Models

Results

End-to-End Event Extraction

Event Detection

Event Argument Extraction

BibTeX

@article{Huang23textee,
  author       = {Kuan{-}Hao Huang and
                  I{-}Hung Hsu and
                  Tanmay Parekh and 
                  Zhiyu Xie and
                  Zixuan Zhang and
                  Premkumar Natarajan and
                  Kai{-}Wei Chang and
                  Nanyun Peng and
                  Heng Ji},
  title        = {A Reevaluation of Event Extraction: Past, Present, and Future Challenges},
  journal      = {arXiv preprint},
  year         = {2023},
}