Camellia: Benchmarking Cultural Biases in LLMs for Asian Languages

Abstract

Camellia is a benchmark for evaluating entity-centric cultural biases in large language models across nine Asian languages and six cultures. It includes 19,530 manually annotated entities and 2,173 masked social-media contexts. Evaluations across cultural context adaptation, sentiment association, and entity extractive question answering show challenges in cultural adaptation, model-specific sentiment associations, and context understanding gaps across Asian languages.

Publication
arXiv preprint arXiv:2510.05291; accepted at EMNLP 2026 Main Conference
Sahajpreet Singh
Sahajpreet Singh
CTIC Doctoral Student (Jan ‘25)
Co-Supervised by Kokil Jaidka

PhD Candidate January 2025 Intake