---
title: "Status is derived, not reported"
url: "https://toddpaulbrownjr.com/writing/status-is-derived-not-reported/"
author: "Todd Paul Brown Jr."
description: "Docs drift and agents trust them. The fix: one generated page, status computed from what exists, not what was last written down."
kind: "article"
updated: "2026-09-26T02:50:41+00:00"
---

# Status is derived, not reported

On 21 September 2026, while cross-checking production numbers for a batch of content, I found that LEGION's own manifest had drifted. The source file the generated state panel quotes from declared 1,175 tests for Sadyr, against 1,334 tests at the actually deployed commit. The stale number was sitting one level up from anything the generator itself derives: the panel's Verification section cross-checks what it can, but carried no Sadyr test count at all, so the incorrect figure had no automatic way to be flagged.

The same day, a different problem surfaced in a job-search pipeline I'd built for a family member. Its status view reported 26 of 31 tracked jobs as fully tracked when they hadn't yet cleared human approval, because a step was marked done whenever its output file existed on disk — without checking whether its own upstream dependency had actually cleared its gate. I fixed it with an explicit post-hoc dependency-enforcement pass and 6 new tests. Separately, the audit log held exactly 1 entry for 31 jobs because the state-initialization code never logged its own actions; fixed, and the log grew to 208 entries in the same session.
## Why it happens

Every stale test count I have found this year undercounts. Not once has a project's documentation claimed more tests than the repo actually holds. Written status decays in one direction: downward against a growing test suite, because updating the number is nobody's job.

What makes it worse with an agent in the loop, not better: a person skimming a stale doc usually notices the number feels old and checks it; an agent reading the same doc treats the number as a fact and reasons from it confidently for the rest of the task, with no instinct that a document's number can simply be wrong. This is Google's March 2025 "agentic orchestration" problem, before the industry had a common name for it: the model alone isn't the product; the harness around it — the instructions it reads, the state it trusts, the tools it has access to — determines whether it works reliably or fails confidently.

## The fix: one generated file, deterministically rebuilt

If a document in your repo states a number instead of deriving it, assume it is already behind. The fix is not writing better summaries. It is writing fewer of them, and building something that regenerates the ones that are just facts.

[LEGION](/builds/legion/) keeps machine-derivable facts — versions, test counts, git state, whether a file exists — in one generated file, regenerated by a small deterministic script. Judgment stays in files only a person edits. A second run of the sync script has to print "already in sync."

The job-search pipeline's stated governance principle: "Status is derived from artifacts present, not from any agent self-reporting. This prevents drift between reported and actual state." That principle caught its own violation: the pipeline was checking for a file on disk and calling the step done, when the real question was whether approval had cleared. The dependency fix added a pass that walks the graph after initialization and enforces the gates explicitly; the 6 new tests verify it never marks a step complete until its upstream dependencies have actually finished.

## The limit: it only protects what it derives

The LEGION drift sat one level up from the generator itself. The generated panel's Test Inventory section does not track a Sadyr number at all, since Sadyr is an independent repo observed read-only, so the stale 1,175 figure lived in a hand-written source the generator trusts without re-verifying. The correct figure — 1,334 tests at commit `adb0846` — came from a manual cross-check, not from any documentation discipline.

Three unrelated projects, one identical undercount. An on-device interactive fiction engine's README stated 610 deterministic tests while its own roadmap, one milestone earlier, had already recorded 1,010 tests green — the highest figure, and the two numbers are unreconciled anywhere in that repo. A proofreading app's last written summary said 309 backend tests; the repo held 368, a figure true only as of 15 July 2026 before the suite kept growing. The game's implementation notes claimed "8 test scripts" when there were 10, caught by a same-project refinement audit rather than any documentation discipline.

The pattern: every stale count undercounts, and I caught each one by hand, not by tooling.

## What to look for

An artifact standing in for a finished step. The job-search pipeline marked a step complete because a file existed, not because the work was approved. A manifest declares a test count without running the tests. A README states a version without checking the tag. Any claim that can drift, will.
The rule that came out of it: status is computed from what actually exists, not from what was written down last time someone remembered to update the docs. Generate the facts a script can derive. Verify the rest by hand. And when an agent reads a number from a document, treat that number as a starting point to check, not as ground truth.