Contextual Pictograms for Repairing Multilingual ASR Errors

contextual-pictograms-for-repairing-multilingual-asr-errors

Contextual Pictograms for Repairing Multilingual ASR Errors

[Submitted on 3 Nov 2026]

View PDF

Abstract:Speech transcription errors can be difficult to recognize when competing word hypotheses are unfamiliar or visually similar. We describe an interface that combines alternative transcripts, replay and contextually checked pictograms for selected uncertain concepts. The system will initially support French and Spanish functional utterances. A counterbalanced study will assess whether pictorial feedback helps listeners recover the speaker’s intended action beyond text alternatives alone. We will also quantify semantic mismatch and additional interaction time, since an inappropriate image can reinforce the wrong interpretation rather than repair it.

Submission history

From: Carlos Bruno [view email]
[v1] Tue, 3 Nov 2026 21:21:18 UTC (274 KB)