HealthBench

Reported in recent model launches

These model reports include this benchmark. Their harnesses, task subsets, scoring policies, and resource budgets can differ; follow each report’s evaluation notes before joining the numbers.

What it measures

Physician-authored rubrics evaluate health conversations. Original HealthBench and the Professional evaluation use distinct measures and settings.

How to read the result

Inspect the task mix and the length adjustment before interpreting an average as clinical quality.

Limits and comparison traps

The directory also references the original HealthBench. The Professional subset and original full benchmark are distinct contracts.

Benchmark reference notes

5,000 realistic multi-turn health conversations graded against 48,562 physician-written rubric criteria developed by 262 doctors from 60 countries.

Epoch data and evaluation setting

Epoch AI publishes this benchmark in its capability collection. Imported rows retain the published score column and model configuration. Original authors and evaluation providers remain credited in the result references.

Epoch attribution and score coverage

Data is attributed to Epoch AI under CC BY 4.0, with original evaluation providers credited in the linked source snapshots. Scores are imported only where a published mapping identifies the field and its scale; missing or incompatible records are omitted rather than assigned zero.

Versions and evaluation settings

Create a HealthBench family page with separate original and Professional measures. The weighted 16-run mean and Professional length-adjusted score are not interchangeable.

Where this fits in an evaluation suite

Read accuracy together with error and abstention rates. Domain averages can hide language gaps, specialist failures, and poorly calibrated confidence.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Use the exact implementation, version, task subset, tool access, attempt budget, and judge described by the source.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.