LinCE Benchmark

Welcome to LinCE!

Recent trends in NLP research have raised an interest in linguistic code-switching (CS); modern approaches have been proposed to solve a wide range of NLP tasks on multiple language pairs. Unfortunately, these proposed methods are hardly generalizable to different code-switched languages. In addition, it is unclear whether a model architecture is applicable for a different task while still being compatible with the code-switching setting.

This is mainly because of the lack of a centralized benchmark and the sparse corpora that researchers employ based on their specific needs and interests. To facilitate research in this direction, we propose a centralized benchmark for Linguistic Code-switching Evaluation (LinCE) that combines ten corpora covering four different code-switched language pairs and four tasks. As part of the benchmark centralization effort, we provide an online platform in this website, where researchers can submit their results while comparing with others in real-time.

LinCE is a continuous effort, and we will expand it with more low-resource languages and tasks.

Language Pairs	LID	POS	NER	SA	MT
Spanish-English
Hindi-English
Nepali-English
Modern Standard Arabic-Egyptian Arabic
English-Hinglish
Spanglish-English
English-Spanglish
(Modern Standard Arabic-Egyptian Arabic)-English
English-(Modern Standard Arabic-Egyptian Arabic)

LinCE was accepted at the 12th Edition of the Language Resources and Evaluation Conference, LREC 2020. The paper is available here, and you can cite our work with the following BibTex:

@inproceedings{aguilar-etal-2020-lince,
                    title = "{L}in{CE}: {A} {C}entralized {B}enchmark for {L}inguistic {C}ode-switching {E}valuation",
                    author = "Aguilar, Gustavo  and
                    Kar, Sudipta  and
                    Solorio, Thamar",
                    booktitle = "Proceedings of The 12th Language Resources and Evaluation Conference",
                    month = may,
                    year = "2020",
                    address = "Marseille, France",
                    publisher = "European Language Resources Association",
                    url = "https://www.aclweb.org/anthology/2020.lrec-1.223",
                    pages = "1803--1813",
                    language = "English",
                    ISBN = "979-10-95546-34-4",
                }

Contact

Shuguang Chen (current contact person) (schen52 AT uh DOT edu)
Gustavo Aguilar (gaguilaralas AT uh DOT edu)
Sudipta Kar (skar3 AT uh DOT edu)
Thamar Solorio (tsolorio AT uh DOT edu)

Linguistic Code-Switching Evaluation Benchmark

Welcome to LinCE!

Contact

Powered By