KnowBench: Effort Reduction as a Unified, Deployment-Grounded Benchmark for Clinical AI

Score clinical AI by how much of its draft doctors sign without editing.

97.99%

Aggregate Effort Reduction, clinical notes

Over one million signed encounters across thirteen specialties

How it works

Wrong yardstick
Clinical AI is scored by similarity to reference notes or rubric panels. These scores do not measure work saved.
Use the clinician's edits
Compare the AI draft with the signed version. The share of draft content kept unchanged is Effort Reduction.
High retention reported
Knowtex's notes kept 97.99% of tokens across one million encounters. Some companion statistics were withheld.

Which comes out on top on Aggregate ER?

try it

Slide the review strictness and see which drafts get flagged versus signed.

Review strictness0.5

High acceptance alone can hide errors; ER needs a safety audit and review-time checks.Illustrative model, not the paper's data

6 terms

Effort Reduction (ER) Share of AI-generated content the clinician keeps and signs unchanged.ERROR_FIX

Like grading a junior writer by how much of their draft the editor publishes untouched, rather than how much it resembles a model essay.

Where it breaks

Doctors may sign without reading, so untouched text isn't always approved text.

Acceptance is not correctness; signed notes can still contain errors.
Completeness and signed-note rate were withheld, so templated retention can't be ruled out.
Self-reported by the vendor on its own proprietary system, no other systems compared.

From the paper