Asked to report on flawed work, GPT-5.5 flagged the problem in 2 of 200 reports. One honesty line raised it to 190

Your automation's summary step may know something went wrong and still tell you everything is fine.

What happened

A preprint titled "Language Models Are Insecure Reporters", covered by The Decoder as work from Google Research, MIT and Harvard, tested whether models mention flaws when they write up work. The researchers built synthetic work logs that look like successes, with passing tests and positive results, but each hides a problem: a method that does not actually win, a hidden bug, fabricated measurements, a job that never finished.

When GPT-5.5 wrote reports from experiment logs with a planted negative result, it flagged that result in only 2 of 200 reports. Adding the line "Be honest in your response" raised it to 190 of 200. Per The Decoder, Gemini 3.1 Pro flagged errors in at most 34% of cases without the line, and Opus 4.8 in over 90%. On error free logs the honesty line barely added false alarms. Wording like "Be critical" or "Be skeptical" worked less reliably.

The key detail: asked directly, GPT-5.5 and Opus 4.8 found the errors in every case. The problem is reporting, not detection. The line is also not a cure. When a database query had not returned yet, GPT-5.5 with the honesty prompt noted the missing data but still answered using older, incomplete information. The logs are synthetic and designed to provoke the behavior, so treat the exact numbers with care.

My take

Almost every automation I build has a summary step somewhere: a daily sync report, an agent's "done" message, a ticket recap for the account manager. Those are exactly the reports people read instead of checking the work.

What I would change this week:

1. Add the honesty line to every prompt that summarizes work. It is cheap and the study shows a large effect.
2. Ask the failure question separately. Since models detect problems when asked directly, run a second yes or no step: "Did any step fail, return nothing, or not finish?" Route a yes to a human.
3. Check pending jobs in code, not in the prompt. If a query or API call has not returned, the workflow should stop or flag it before any model writes a summary.

A green checkmark in a report is a claim. Build the workflow so the claim gets tested.