What it measures

Quantifies overused RLHF cliches ("delve", "testament", "tapestry") and vacuous hedging.

Reading the measurement

Separate satisfying instructions from factual accuracy, human preference and safe behavior. Inspect strict all-constraint success as well as average criterion credit. Judge choice and adversarial prompt selection can materially change the result.

Published evaluation

Published Clear-answer rate column, with model/reasoning variants, refusals and errors retained in source configuration notes. V1 and V2 contain different questions and remain separate measures. SlopBench is a distinct evaluation.

Where this fits in an evaluation suite

Format compliance, preference, truthfulness, and safety require separate measurements. Keep desirable writing style from masking factual errors.

Start with a capability hypothesis, then choose a second test with a different failure mode. Keep the checkpoint, reasoning effort, agent tools, sampling policy, and scoring procedure fixed when measuring a training intervention. Record failures and missing outputs in the denominator.

A reproducible comparison

Published Clear-answer rate column, with model/reasoning variants, refusals and errors retained in source configuration notes. V1 and V2 contain different questions and remain separate measures. SlopBench is a distinct evaluation.

Publish the exact run configuration, item coverage, score units, and available uncertainty. If a source supplies only an aggregate or a single run, preserve that limit. A new model name or a rounded score alone does not establish a capability improvement.

PRIMARY EVIDENCE & FURTHER READING

  1. BullshitBench V1 — published model leaderboard ↗

    Peter (petergpt) · 2026

  2. BullshitBench V2 — published model leaderboard ↗

    Peter (petergpt) · 2026

Reviewed 2026-10-27. Explanation and interpretation by RLTasks. Benchmark ownership remains with its original authors.