Brasilidade: Measuring Brazilian Identity for Large Language Models

brasilidade:-measuring-brazilian-identity-for-large-language-models

Brasilidade: Measuring Brazilian Identity for Large Language Models

[Submitted on 15 Oct 2026]

View PDF HTML (experimental)

Abstract:This work presents a new benchmark that aims to address the limitations of current methods for evaluating LLMs in Brazilian Portuguese. We introduced Brasilidade, an assessment methodology comprised of standardized tasks distributed across four different domains, and we are making available the tasks from its first domain, which address nuances of the language, grammar, and variations of Brazilian Portuguese. The results show that model size, by itself, does not determine performance; several medium-sized models outperformed larger models in the protocols evaluated. Specific models for Portuguese, in general, did not perform above average, despite their targeted training. Different task levels revealed significant nuances: while some tasks proved easy for most models, others posed extreme challenges, and some yielded mixed results, with wide variations in performance. This clearly illustrates the need for benchmarks that address the limitations of the models in Brazilian Portuguese, allowing future training cycles to seek to overcome them. Project repository this https URL

Submission history

From: William Alberto Cruz CastaƱeda PhD [view email]
[v1] Thu, 15 Oct 2026 02:14:13 UTC (4,237 KB)