Learn how to evaluate AI medical coding accuracy, compare vendor claims and design a representative validation process for your health system.
There is no verified industry-wide accuracy figure for AI medical coding. In July 2026 the US Government Accountability Office reported that few independent studies evaluate these tools and that their accuracy can be difficult to verify. Vendor-reported figures cluster above 95 percent; published research on general-purpose models is far lower.
The gap between those two numbers is not mainly a gap in capability. It is a gap in what is being measured, on which charts, judged by whom.
What the GAO said in 2026
The GAO's July 16, 2026 science and technology spotlight on AI for medical notes and coding lists difficulty verifying accuracy as the first challenge facing the category. It notes that few independent studies evaluate these tools, that some tools do not retain the underlying records in a way that supports independent assessment, and that inaccuracies can produce over-reimbursement or under-reimbursement (GAO-26-109116).
The same report relays a vendor-side figure: one coding software developer said its software generated codes with more than 95 percent accuracy across a five-hospital system, alongside a reduction in emergency department coding costs. That is a developer's account, reported as such. It is the shape of nearly every public accuracy claim in this market.
What independent research measures
Peer-reviewed studies test models, not commercial coding platforms, and each defines accuracy differently. That makes them poor proxies for vendor performance and useful evidence of how much the definition matters.
| Study | What was measured | Result | What it does not tell you |
|---|---|---|---|
| Soroush et al., NEJM AI, 2024 | General-purpose models asked to produce a billing code from a code description | GPT-4 exact match: 33.9 percent for ICD-10-CM, 49.8 percent for CPT | Nothing about coding a chart; the task was code lookup, not abstraction |
| npj Health Systems, 2025 | Fine-tuned models coding real clinical notes | 69.2 percent exact match, about 87 percent at the category level | How the model performs on a full inpatient record or a payer-specific rule |
| Klang et al., 2024 (preprint) | Retrieval-augmented models on 500 emergency department visits, compared with provider-assigned codes | Exact match rose to roughly 18 to 26 percent for the models tested | Whether the provider-assigned codes were themselves right; agreement is not adjudication |
Purpose-built coding systems are trained on coded charts, constrained to a codebook and tuned to a specialty, so they should outperform a general model prompted cold. The published literature does not tell you by how much, because those systems are rarely tested independently.
Four things that change the number
Ask these before accepting any accuracy figure, vendor or study.
- The denominator. Accuracy measured only on charts the system chose to code automatically excludes every chart it found hard. That is the single largest source of inflated figures.
- The adjudicator. Agreement with your coders is not accuracy; both can be wrong. An adjudicated figure needs an independent auditor and a documented disagreement process.
- The unit. Code-level accuracy, chart-level accuracy and MS-DRG-level accuracy give different numbers from the same test. One wrong secondary diagnosis can leave code-level accuracy near 99 percent and still change the Medicare Severity Diagnosis Related Group (MS-DRG) and the payment.
- The direction. Over-coding and under-coding carry different consequences. A net accuracy figure hides both.
The human baseline is not 100 percent
AI accuracy only means something against your own baseline, and that baseline has error in it too. A 2025 preprint validating 13,102 ICD-10-CM codes across 64,764 admissions reported a positive predictive value of 93.9 percent for principal diagnoses and 83.8 percent for secondary diagnoses, and noted published validity estimates for ICD-10 codes ranging from 53 to 98 percent across conditions and settings (medRxiv, 2025). Those figures are not peer reviewed and are not a benchmark for your department.
The practical consequence: run the same blind test on your own coders and on the software, on the same charts, with the same adjudicator. Any comparison built on your baseline measured one way and the vendor's accuracy measured another is not a comparison.
MEDIQCODE™ does not publish an accuracy percentage. Evaluations run against the hospital's own baseline on its own closed charts, and its coders adjudicate the disagreements, which we report as a disagreement analysis rather than an accuracy claim. A coder approves or adjusts every code before release.
What to require instead of a number
Put the test in the contract, not the figure. Specify the chart population, the sample size, who adjudicates, the unit of measurement, separate reporting of over-coding and under-coding, and what happens if the result misses. A vendor confident in its system will agree to be measured on your charts; a figure in a slide deck commits nobody to anything.
Related questions
Is AI medical coding more accurate than human coders? There is no general answer. It depends on the chart type, the documentation quality and how both are measured. The only comparison that holds is a blind test of both on the same charts, adjudicated by the same auditors.
What accuracy should we require? Match the level your own audits currently hold coders to, on the same chart types, and require that the figure be adjudicated rather than self-reported. A number without a denominator and an adjudicator is not a requirement.
Does accuracy vary by specialty? Yes. Short, templated documentation with a narrow code set, such as radiology or urgent care, is easier for any system than long inpatient records where principal diagnosis selection drives payment. Ask for results on your specialty mix.
This article is general education about coding technology and measurement. It is not legal advice or a coding determination for any encounter.
