Report
Are Expert-Level Language Models Expert-Level Annotators?
| العنوان: | Are Expert-Level Language Models Expert-Level Annotators? |
|---|---|
| المؤلفون: | Tseng, Yu-Min, Chen, Wei-Lin, Chen, Chung-Chi, Chen, Hsin-Hsi |
| سنة النشر: | 2024 |
| المجموعة: | Computer Science |
| مصطلحات موضوعية: | Computer Science - Computation and Language |
| الوصف: | Data annotation refers to the labeling or tagging of textual data with relevant information. A large body of works have reported positive results on leveraging LLMs as an alternative to human annotators. However, existing studies focus on classic NLP tasks, and the extent to which LLMs as data annotators perform in domains requiring expert knowledge remains underexplored. In this work, we investigate comprehensive approaches across three highly specialized domains and discuss practical suggestions from a cost-effectiveness perspective. To the best of our knowledge, we present the first systematic evaluation of LLMs as expert-level data annotators. Comment: Accepted to WiML @ NeurIPS 2024 (extended version) |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2410.03254 |
| رقم الانضمام: | edsarx.2410.03254 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |