All writing
AIMemorySearchEvals

My Search Miss Wasn't a Missing Graph. It Was Link Markup in the Index.

MxMind's eval found that searching for one of my products by name missed that product's own note. A graph was the planned fix. The cause was the search index reading link markup as text, and the same eval caught a regression the fix introduced.

Max KellyAugust 9, 20264 min read
My Search Miss Wasn't a Missing Graph. It Was Link Markup in the Index.

MxMind is the memory layer my agents share. My notes stay as markdown files, and agents find what they need through one search tool. On August 1 a small eval showed that search had a blind spot: asked for one of my products by name, it didn't return that product's own overview note anywhere in the top ten.

The planned fix was a graph. The cause turned out to be the search index reading link markup as if it were text.

An eval before any ranking change

On July 29 search became hybrid: a lexical arm over Postgres full-text vectors and a semantic arm over embeddings from a small local model, fused with reciprocal rank fusion. No API keys are involved, and at this corpus size an exact scan over the vectors replaces an approximate index.

On August 1, before ranking changed again, a 12-case eval went in. Each case is a query a caller would actually type, plus the notes it should find, grouped by kind: exact names, entity questions, questions answered by meeting summaries, decisions buried late in long notes, paraphrases, a query that should find nothing, and a misleading hub page. The fixtures hold queries and expected paths only, never note bodies. Ten of the twelve cases are scored. The baseline was recall at 5 of 0.9 and mean reciprocal rank of 0.73.

The rule written next to it: retrieval changes are accepted "on measured quality, not intuition," and only when they improve the named cohorts "without material regression on exact-name lexical queries."

The graph stayed out of ranking

The same day, a graph went in: a plain database view of 12,469 typed edges derived from links, entity registrations and other provenance, with two bounded tools to walk it. One lists a note's neighbors. The other explains how two notes connect.

It did not go into search. The commit calls the graph "standalone by design" and leaves it out of ranking until the eval set can gate that step. The baseline commit names the miss on my product's note as the case graph seeding was meant to improve. With the graph out of ranking, any change in that case had to come from something else.

Markup in the index

That evening, a cleanup of the vault's entity notes added many more links between notes, written as full-path wikilinks like [[Folder/Product Overview|Name]]. The eval moved. Recall at 10 rose from 0.9 to 1.0, and the missing note appeared at 9th. But one exact-name query dropped from 2nd to 5th.

The cause was in how the lexical arm built its index. The full-text vectors were generated from raw note bodies, link markup included. So every note that linked to a canonical note carried that note's path words in its own index: the folder name, "Product", "Overview". For a query using those words, the notes that mention an entity outranked the entity's own note, and the more carefully the vault linked, the worse it got.

The fix strips link markup in the index and in search snippets: [[a/b|Display]] becomes "Display", and [[a/b]] becomes "b". Note bodies stay verbatim, and reading a note returns exactly what's on disk. A companion fix taught the entity resolver to accept those full-path links. Its commit message puts the rule this way: "the resolver bends, not the vault." The index strip follows the same rule.

What the eval said

After the strip, compared with the run before it:

  • my product's overview note moved from 9th to 5th
  • the exact-name query that had dropped to 5th went back to 2nd
  • mean reciprocal rank went from 0.781 to 0.787, and from 0.733 at the August 1 baseline
  • another exact-name query fell from 2nd to 6th, and it had been 3rd at the baseline

That last one is a regression on exactly the cohort the acceptance rule protects. The strip stayed in, and the regression is a known cost rather than a hidden one: it's the case to watch before any graph arm goes into search.

The eval is twelve cases, so one case moves recall by a tenth. That makes it a regression catcher, not a benchmark, and it caught both regressions: the exact-name drop that led to the real cause, and the one the fix introduced.