[Submitted on 14 Jul 2026 (v1), last revised 9 Oct 2026 (this version, v4)]
Abstract:When a language model must choose one answer from a large space of equally valid options, which answer does it choose, and how often is it the answer every other model chooses? Asked to “pick a word,” 105 language models from more than twenty labs chose serendipity 46% of the time. We measure this convergence, and each model’s share in it, with 96 single-turn prompts that each name a category with many valid one-word answers (“Name a tree.”), asked eight times per model and scored by exact match, with no embeddings and no judge. A model’s answer-choice surprisal is the average -log2 probability of its answers under the pooled answers of all other models. In 28 of 96 categories a single answer takes at least 80% of all answers. The concentration does not depend on small or persona-tuned models: the 87 major-lab models are at least as concentrated as the full field. Lightly post-trained and persona-tuned models are the most divergent; heavily post-trained assistants from the major labs are the most conformist. Models that avoid the modal answer mostly land on the same runner-up. Within the major providers’ lineages, release order shows no panel-wide trend once model tier is controlled; GPT, Gemini, Grok and Qwen become more conformist across releases, and Claude’s generation-5 releases reverse. On open checkpoints of three post-training pipelines, supervised fine-tuning is the largest step toward the field’s answers. Against human category-production norms, the field is more concentrated than people in 18 of 20 shared categories. All prompts, transcripts, and code are public.
Submission history
From: Tapan Parikh [view email]
[v1] Tue, 14 Jul 2026 14:12:05 UTC (293 KB)
[v2] Sat, 25 Jul 2026 18:18:32 UTC (297 KB)
[v3] Wed, 7 Oct 2026 20:52:36 UTC (123 KB)
[v4] Fri, 9 Oct 2026 14:01:46 UTC (123 KB)