The citation is real. Does the restart advice follow?

The summary recommends restarting a worker and quotes the runbook word for word. There is just one problem: the worker runs release 2.4, whose procedure preserves a diagnostic buffer first. The quote came from 2.3. In this synthetic case, the sentence is genuine; its fit for the decision is not.

The local checker below compares a proposed citation with a packet prepared by the reviewer. It checks captured words and the selected release, leaving their interpretation to a person. Python’s standard library is enough, with no model or production calls.

A frequently repeated source need not be the applicable one. One current exception can matter more than several older examples. Keep versions, dates and the decisive passage attached so the review can examine what changed rather than count how many links agree.

Keep the saved pages outside the assistant’s edit access

Capture the 2.3 and 2.4 pages before giving the assistant its assignment. Select 2.4 independently as the release being investigated, and keep the saved copies outside the assistant’s write access. Otherwise the assistant could replace the captured words with its preferred version and make its own quotation appear to pass.

Both pages earn their place in this fictional packet. Release 2.3 explains where the restart advice came from; 2.4 supplies the preservation condition. A newer date alone does not settle applicability: that page might describe another service or an unapproved proposal. The reviewer chooses which release this investigation concerns.

Download the source packet, candidate and checker to an empty local directory. Inspect them before using Python 3.11 or newer to run the example. No packages are required. Begin with these invented files, not a sensitive incident transcript.

Each captured page stores its text, release, effective date and source ID, plus a SHA-256 digest calculated from the text. Recalculate the digest to detect changed words. It cannot establish whether the original capture was authentic, so protecting and choosing the source remain part of the review. The hashlib reference explains the calculation.

Try the convincing quote from the wrong release

The supplied candidate recommends immediate restart and cites 2.3. Set the target release separately as reviewer input, then run these commands from the download directory:

python3 incident-citation-check.py --self-test
python3 incident-citation-check.py incident-citation-sources.json incident-citation-candidate.json 2.4

The self-test exercises expected failures. The second command should return citation-check-failed, the reason source version differs from reviewer target, and exit one. The quoted sentence exists, but not in the selected release. Choosing 2.3 merely to make it pass would change the question; do so only if the investigation actually concerns 2.3.

The checker also rejects unknown source IDs, changed digests and quotations absent from captured text. A candidate must contain exactly three nonempty fields: claim, source_id and quote. Duplicate JSON keys are rejected rather than silently taking the last value. The JSON reference documents the decoding hook and recommends bounding untrusted input; this fixture limits each file to 64 KiB.

That failure points the reader toward 2.4. Open its procedure and consider the diagnostic-preservation exception before revising the recommendation. The question was never whether the old sentence existed. It was whether that sentence governed this worker.

Now make the quotation pass and the claim fail

Change the source to runbook-24 and quote the preservation condition exactly. With the claim “Check the diagnostic-preservation requirement before considering a restart,” the checker should return citation-integrity-passed. The words now match the selected page. A reader can examine whether they support the claim.

Keep that same 2.4 quotation and change only the claim to “Restart immediately.” The check still passes. The sentence is present in the correct release, while the recommendation has discarded its condition. This case is included in the self-test: string matching cannot do the reading that notices the diagnostic step has vanished from the advice.

Put the condition beside the proposed action. Must the buffer be preserved first? Does an exception sit just beyond the quotation? Does the procedure apply to this worker and failure? Another model can help identify disagreement, but its answer also needs comparison with the page.

The twenty-character quotation minimum is only a fixture heuristic to discourage trivial matches. A long quote can omit the next sentence, and a short configuration value can decide the issue. The number is not a universal rule for good evidence.

Tomorrow’s deployment is not in today’s packet

If the worker moves to a later release tomorrow, these saved copies still contain 2.3 and 2.4. They cannot notice the deployment or fetch a new procedure. Before using the result for a real restart, establish the running version, current applicable instructions and authority to act. A repeatable old comparison does not become current on its own.

The fixed packet is useful for testing a changed prompt or retrieval setup. Does the result select 2.4 and retain the diagnostic condition? Keep wrong-release and wrong-claim answers marked as failures, even when the checker finds matching words. Counting matches alone would reward the second mistake.

The agent-evaluation guide follows an assistant’s answer into service behavior, while the source-comparison guide keeps conflicting procedures visible. The saved pages here give those reviews a concrete starting point: which release said what, before anyone changes a worker.

The first recommendation borrowed its instruction from 2.3. The second quoted 2.4 accurately but skipped what had to happen first. The checker catches the version mismatch; the reader catches the missing condition. When the release changes again, that division of work explains why the procedure must be reopened before the buffer is preserved and a restart decision is made.