TextEE: A Standardized, Fair, and Reproducible Benchmark for Event Extraction

Kuan-Hao Huang1, I-Hung Hsu2, Tanmay Parekh3, Zhiyu Xie4, Zixuan Zhang1, Premkumar Natarajan2, Kai-Wei Chang3, Nanyun Peng3, Heng Ji1
1University of Illinois Urbana-Champaign, 2University of Southern California, 3University of California, Los Angeles, 4Stanford University
Paper arXiv Code

Introduction

TextEE is a standardized, fair, and reproducible benchmark for evaluating event extraction approaches.

  • Standardized data preprocessing for 10+ datasets.
  • Standardized data splits for reducing performance variance.
  • 10+ implemented event extraction approaches published in recent years.
  • Comprehensive reevaluation results for future references.

We will keep adding new datasets and new models!

News

  • [04/21/2024] TextEE supports two more datasets: SPEED and MUC-4.
  • [02/23/2024] TextEE supports the CEDAR approach now.
  • [12/26/2023] TextEE supports three more datasets: MLEE, Genia2011, Genia2013.
  • [11/15/2023] We release TextEE. Feel free to contact us if you want to contribute your models or datasets!

Supported Datasets

Supported Models

Results

End-to-End Event Extraction

Event Detection

Event Argument Extraction

BibTeX

@article{Huang23textee,
  author       = {Kuan{-}Hao Huang and
                  I{-}Hung Hsu and
                  Tanmay Parekh and 
                  Zhiyu Xie and
                  Zixuan Zhang and
                  Premkumar Natarajan and
                  Kai{-}Wei Chang and
                  Nanyun Peng and
                  Heng Ji},
  title        = {TextEE: Benchmark, Reevaluation, Reflections, and Future Challenges in Event Extraction},
  journal      = {arXiv preprint arXiv:2311.09562},
  year         = {2023},
}