[Submitted on 24 Jul 2026 (v1), last revised 18 Oct 2026 (this version, v2)]
Abstract:Objective:To characterize the kinds of internal documentation inconsistencies a general-domain large language model (LLM) can surface from real-world discharge summaries, and to identify recurring failure modes that limit reliability at scale. Materials and Methods: We applied a two-stage LLM pipeline—open-ended candidate identification (Gemini 2.5 Pro) followed by context-grounded verification (Gemini 2.5 Flash)—to 3000 MIMIC-IV-Note discharge summaries drawn as three contiguous 1000-note batches from distinct positions in the corpus. Two clinicians independently reviewed a domain-stratified sample of 268 retained candidates; a third adjudicated disagreements.
Results: Stage~1 proposed 11333 candidates; 3461 were retained after Stage~2 verification—machine-generated candidates—with at least one in 69.7% of admissions. Medication (43.2%) and diagnosis or clinical-finding (25.3%) candidates predominated. In the reviewed sample, reviewers agreed on 91.0% of candidates, and 253 of 268 (94.4%) were judged valid after adjudication. Representative examples from all seven domains carried direct implications for clinical reasoning or patient safety. Review also revealed recurring failure modes involving temporal reasoning, evolving diagnoses, and outpatient-prescribing conventions.
Discussion: Detection is highly context-dependent: judging a flagged pair requires locating each statement and assessing whether the conflict reflects a true contradiction or missing context. We propose a graded ontology spanning strict contradiction, ambiguity, and gap, with a schema characterizing each flagged case by type, section, scope, domain, attribute, and inconsistency axis.
Conclusion: This formative study establishes a methodological foundation and conceptual framework to guide subsequent validated, large-scale EHR-inconsistency analysis.
Submission history
From: Jian Lu [view email]
[v1] Fri, 24 Jul 2026 23:43:35 UTC (99 KB)
[v2] Sun, 18 Oct 2026 14:08:42 UTC (201 KB)
