All writing
AIMemoryEvalsResearch

Write the Kill Criteria Before the First Run

In June I tested a memory design for agents against thresholds committed before the first result. It failed on a cheap model, a frontier model's edge precision was no better, and the design was dead in an afternoon.

Max KellyJune 28, 20265 min read
Write the Kill Criteria Before the First Run

On June 11 a memory design I'd been researching for agents failed its kill test in an afternoon. Its kill criteria were committed to the repository before the first result came in.

The research is Maat, a private exploration of agent memory that stays correct when facts change. The design under test was specific. When an agent learns that something it believed is no longer true, the beliefs that depended on it should be retracted too. A model proposes the dependency edges between beliefs, and a symbolic engine walks them. Nothing is deleted: an old belief's valid interval is closed, so you can still ask what the agent believed at any point.

The whole design rests on one number that, as far as the research found, nobody had published for agent memory: how often a model's proposed dependency edges are actually right.

The bars went in first

The first commit of the test harness included the criteria, with a line underneath: "These thresholds do not move after seeing data."

  • Go if the design beat a plain baseline by at least 10 points on the benchmark, with under 5% of beliefs wrongly retracted.
  • Iterate once if the lift was 3 to 10 points.
  • Kill if the lift was under 3 points, or wrongful retraction was over 10%.

The project's decision log gives the reason for writing them down first: so the test would be "falsification, not confirmation."

The baseline was the cheap version anyone would ship: a vector store of extracted memories, plus a model asked on each new fact which existing memories it invalidates. The benchmark was STALE, published in May, which tests whether agents notice that a later observation has quietly invalidated an earlier memory. Before trusting it, the harness audited the answer key and reproduced one of the paper's numbers on a four-scenario check: GPT-5.4 reading the raw context scored 16.7%, against the paper's 15.7%.

Stage one: a loss, but not an answer

On a cheap model, Gemini 3 Flash, the baseline scored 40.0% and the design scored 26.7%. That's a lift of minus 13.3 points, well past the kill line.

The edge numbers showed why. On a small hand-built set of beliefs, edge precision was 0.47: about half of the dependency edges the model proposed were wrong. A retraction that cascades along wrong edges retracts correct beliefs, and the design still missed the stale facts it was supposed to catch.

But the question the harness was built to answer was whether the design works on a current frontier model. The design had assumed edge precision of 0.6 to 0.85, the range published for models extracting causal graphs from text. A cheap model scoring 0.47 couldn't settle that.

Stage two: the same coin flip

So stage two measured the one number the design depended on, on GPT-5.4, before spending anything on full benchmark runs. The commit that records the result also records the bar: proceed at 0.65 or above, stop at 0.55 or below.

GPT-5.4 scored 0.50 at low reasoning effort and 0.49 at high effort. Across three configurations, precision sat at 0.47, 0.50 and 0.49. More effort cut wrongful retractions from 20% to 11.4%, and missed retractions rose from 8.9% to 20%. The model got more conservative, not more correct.

Under that rule the full benchmark runs weren't run. The published precision, measured on causal graphs extracted from text, didn't transfer to small, atomized memory beliefs, and it took one day to find that out.

The second design, held to its own bar

That evening the test moved to a simpler mechanism. On each new fact, the model answers one constrained question about the current belief on that subject: add it, supersede it, or ignore it. Its bar was 50% to go and under 43% to kill.

It scored 20% on the cheap model, and the diagnosis changed what the whole test could say. The update was never reaching the store. STALE's updates are implied rather than stated: a run of sessions about enrolling a child in daycare implies that a strict savings budget has changed, and the extractor recorded only the surface event. That extraction step was shared by every arm, baseline included, so the benchmark scores so far said as much about extraction as about any mechanism. The edge-precision result didn't depend on it, because that set was built by hand.

The final run fixed extraction and replaced a prompt that had picked up patterns from the two diagnostic scenarios with generic examples. On GPT-5.4 it scored 40.0%, under both bars.

What survived

Every mechanism failed on accuracy. The layer underneath passed on every run: the append-only event log, answers to as-of queries, a provenance trace for every belief and a hash-chained log that detects tampering. Propagation along the edges took 0.18 ms at the 99th percentile.

So the result was narrow. The spike was kept as a reference for the substrate it had validated. The next day one more mechanism, resolving conflicts at read time instead of write time, ran on five scenarios against a floor fixed before the run. It scored 6.7% against a floor of 20%, the thresholds didn't move, and the accuracy work stopped there.

Two weeks later, a paper measuring memory updates on a different benchmark reached a similar conclusion about its own gap: it "is not closed by a stronger model."

The samples were small. The benchmark arms ran 10 scenarios each, and the edge set was hand-built, with labels still marked as drafts awaiting human review. What I can say is narrower: on that set, two models and two effort levels landed within 0.03 of each other, so neither a stronger model nor more effort moved the number.