TextEE: A Standardized, Fair, and Reproducible Benchmark for Event Extraction

1University of Illinois Urbana-Champaign, 2University of Southern California, 3University of California, Los Angeles, 4Standford University

Introduction

TextEE is a standardized, fair, and reproducible benchmark for evaluating event extraction approaches.

  • Standardized data preprocessing for 10+ datasets.
  • Standardized data splits for reducing performance variance.
  • 10+ implemented event extraction approaches published in recent years.
  • Comprehensive reevaluation results for future references.

We will keep adding new datasets and new models!

News

  • [2024/02/23] TextEE supports the CEDAR approach now!
  • [2023/12/26] TextEE supports three more datasets: MLEE, Genia2011, Genia2013!
  • [2023/11/15] We release TextEE. Feel free to contact us if you want to contribute your models or datasets!

Results

End-to-End Event Extraction

Event Detection

Event Argument Extraction

BibTeX

@article{Huang23textee,
  author       = {Kuan{-}Hao Huang and
                  I{-}Hung Hsu and
                  Tanmay Parekh and 
                  Zhiyu Xie and
                  Zixuan Zhang and
                  Premkumar Natarajan and
                  Kai{-}Wei Chang and
                  Nanyun Peng and
                  Heng Ji},
  title        = {TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction},
  journal      = {arXiv preprint arXiv:2311.09562},
  year         = {2023},
}