Skip to main content

21 docs tagged with "governance"

View all tags

The grep that was read backwards

A four-model audit cycle reviewed a Charter that added a new public method — the only method that made a feature work. Three of four auditors reported no critical or high findings. The feature had zero production callers, 3,376 passing tests, and did not work end to end. The defect was mechanically detectable with a single grep; one auditor ran that exact grep and read the result backwards. The post-mortem produced a one-rule addition to the audit prompt — and a more uncomfortable finding about the implementer's own verification, which could not produce a red result.

What a million assertions can't settle

Bun's Rust rewrite ran AI adversarial review at a scale worth studying — one model family, split context windows, and 1,386,826 test assertions as the final judge. StrayMark's audits look different because the target is different, and the difference isn't stylistic. When a passing test suite can be the judge, you isolate reviewers by context. When the judge is a human judgment call, you diversify by model family — and 25 audit cycles of real data show exactly why, including one family that went quietly blind.

What the bash script said was in sync

Follow-ups became StrayMark's second first-class entity — and within a day, two external migrations stress-tested the design. The reference adopter's deprecated bash script had been reporting "in sync" while 29 entries sat invisible to it.

What the feature flag compiled away

A second adopter, in a different language, validated the "surface declaration without wiring" anti-pattern — and the CLI helper we deferred on purpose finally shipped

What the follow-up took for granted

A greenfield adopter cleared a backlog of follow-ups before shipping and found that three of them — the three that were real work — each carried a premise that was false the moment you checked it. Not because the code had changed under them, but because a follow-up is written at the one moment you are least equipped to verify it. The lesson isn't "write better follow-ups." It's that a follow-up is a dated hypothesis, and the cheap place to test it is when you read it, not when you write it — so that is where StrayMark now puts the check.

What the open format left to the producer

Google published the Open Knowledge Format — markdown, YAML frontmatter, a graph of links, an agent that maintains them. StrayMark reached the same primitives independently. The convergence is real; so is the divergence — OKF builds cognition for the AI, StrayMark builds it for the AI and the human engineer together.

Who the audit thought it was

StrayMark's audit is only worth anything if independent model families converge on a finding by themselves — agreement is the signal. A run of releases hardened that guarantee against three ways to fake it. The freshest is the sharpest — a router CLI injects its own product name, so an auditor stamped its report "qwen-code" even after the operator had switched the backend model. The convergence math was being fed a lie about who did the work.