Skip to content

feat(memory): ask the vault a question in plain words - #67

Merged
arthware-dev merged 4 commits into
mainfrom
feat/memory-owns-query-rewrite
Aug 4, 2026
Merged

feat(memory): ask the vault a question in plain words#67
arthware-dev merged 4 commits into
mainfrom
feat/memory-owns-query-rewrite

Conversation

@arthware-dev

@arthware-dev arthware-dev commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

The question-to-keywords rewrite moves out of the archivist into memory.lib, where the vault is, and takes the caller's LLM rather than opening one. stack memory search --nl reaches it through the bot-runner the same way stack memory wiki does, so a sentence finally finds something from a terminal and from Stacky, not just in chat.

The default stays a regex: no model, no container, same speed. A single word never calls a model even with --nl, and every failure falls back to searching the query literally. Chat keeps inferring from a trailing ?, because here ? is a quantifier.

Two defects the rig turned up along the way: the rewrite sampled, so one question gave different answers run to run, and it answered from the ontology's category names instead of words documents actually contain. Both fixed, both shared with chat.

Refs FAM-21

The rewrite that turns "What do we still need to buy for the camping
trip?" into keywords lived in the archivist, so a family question
answered in chat and returned nothing from `stack memory search`.
It now lives with the vault it searches, in memory.lib, and takes
the caller's LLM rather than opening one.

The archivist keeps what is its own: the `?` trigger, the Whoosh
rendering for Paperless, and the "Searched for: ..." line.

Refs FAM-21
@arthware-dev

Copy link
Copy Markdown
Contributor Author

Verification.

Unit

  • tests/stacklets/test_recall.py (8 cases, the pre-existing contract) passes with zero edits, not even an import path: pipeline.py puts stacklets/ on sys.path before recall imports memory.lib.
  • New tests/stacklets/test_memory_query_rewrite.py, 24 cases: 12 moved prompt/parse tests, 6 driving rewrite_query against a stub LLM (recall role plus json_mode, ontology reaches the prompt, unreachable / timeout / off-shape answers all degrade to []), 3 for keywords_to_regex, and one class asserting both callers share the same rewrite. That last one is the gap that hid the original bug: eight passing tests on one caller while the other got nothing.
  • New guard in test_memory_host_stdlib.py: a subprocess imports memory/lib.py with loguru, openai and aiohttp blocked. The rewrite's third-party imports sit inside the function, and hoisting one would kill stack memory search on every clean host without any bot suite noticing.
  • Lanes: tests/stacklets/ 1903 passed, 8 skipped. tests/framework/ (minus test_config_to_container.py) 487 passed. uvx ruff check clean. basedpyright against a pre-change baseline: 0 new.

Rig, against the Simpsons demo instance with a real model:

Check Result
messages send documents "What do we still need to buy for the camping trip?" --as marge Searched for: Travel, Shopping, Trip plus the same cited answer as before the move
German, as homer: "Was kostet die Autoversicherung dieses Jahr?" Searched for: Autoversicherung, Versicherung, Kosten, correct EUR 340.00
"Which shots has Maggie already had?" Searched for: Maggie, shots, Vaccination, all five from the immunization record
"Krusty" (no question mark) No rewrite, literal hit list, fast path intact
"Do we have anything about a boat licence?" Searched, found nothing relevant, said so
memory.lib alone, no archivist Question to `Travel
stack memory search Regex path unchanged, natural-language question still exit 1, no --natural flag
Agent's route to the CLI Unchanged: camping trip 1 hit, camping 8, raw sentence nothing

One judgement call to review. Classifier.rewrite_query was not deleted; it is now a four-line delegate handing memory the LLM the bot already built. test_recall.py's stub classifier implements rewrite_query, so deleting the method would have meant rewriting that fixture into a stub LLM and pulling memory's JSON parsing into recall's tests. The card says an edit beyond the import path means the move changed behaviour, so this is the version where those 8 cases stand untouched. Prompt, ontology priming, parsing and regex assembly all live in memory; the method contributes only the client.

Noticed, not fixed here. Rewrite keywords are matched as a literal regex with no stemming, so Booking missed a note that says "booked" and an older campsite note answered instead. Visible only because of the Searched for: line, which is the argument for keeping it.

`stack memory search --nl "what do we still need for the camping trip"`
turns the question into the words that are actually on disk, searches
for those, and prints what it searched for so a bad guess is visible:

    Searched for: Zelt, Schlafsack

The default is unchanged and still a regex: no model, no container,
same speed. A single word never calls a model even with --nl, and when
no model is reachable the query is searched literally instead of
failing. Stacky now passes the flag, so its own searches stop coming
back empty on every question.

Refs FAM-21
The rewrite left sampling to the model, so asking twice picked
different keywords and returned different results. One run of "what do
we still need to buy for the camping trip" searched Travel, Holiday,
Shopping and surfaced a vaccination record; the next run found the
camping notes. Picking words a document already contains is a lookup,
not a creative act, so it now decodes greedily, like the wiki
generator does for the same reason.

Refs FAM-21
Asked "what do we still need to buy for the camping trip", the rewrite
answered Travel, Shopping, Receipt. All three are real topic names from
the family's ontology, none of them appear in any camping note, and the
search came back with a Kwik-E-Mart receipt. Handed a list of
categories, the model was answering from the list.

The question's own subject now comes first, and the topic list is
context for spelling and language rather than a menu. Same question now
searches camping, trip, shopping.

Refs FAM-21
@arthware-dev arthware-dev changed the title refactor(memory): let any caller turn a question into a vault search feat(memory): ask the vault a question in plain words Aug 4, 2026
@arthware-dev

Copy link
Copy Markdown
Contributor Author

Rig evidence, all three surfaces, on the demo instance at d176660.

CLI, the question that started this

$ ./stack memory search "What do we still need to buy for the camping trip?" --nl
Searched for: camping, trip, shopping

2026-08-04 [Bart]  family/camping/notes/.../mount-splashmore-camping-reservation...
2026-08-03 [Marge] family/camping/notes/.../vorbereitung-camping-wetter-reservierung-zeltheringe...
2026-08-03 [Homer] family/camping/notes/.../packliste-fur-campingausflug...
exit=0

Agent. One memory_search({"query": "Santa's Little Helper vet bill"}), hit on the first try, one read_file, correct answer with the amount. Before the flag, that same phrase cost four searches and 26 seconds of guessing, which is ADR-012 lesson 5 in the wild:

memory_search({"query": "camping trip", ...})
memory_search({"query": "camping trip todo list buy", ...})
memory_search({"query": "camping shopping list", ...})
memory_search({"query": "camping"})            <- first one that could ever match

The same string through the CLI, both ways:

$ ./stack memory search "Santa's Little Helper vet bill"          # exit 1, nothing
$ ./stack memory search "Santa's Little Helper vet bill" --nl     # exit 0, the vet invoice

Archivist. Unchanged path, and better from the shared rewrite: the same question cited one camping note before, four now, including one it had been silently missing.

The two rig-found defects. First --nl run searched Travel, Holiday, Shopping and returned a vaccination record. Sampling made it non-repeatable; the ontology made it answer from the category list, and search_memory strips frontmatter so a tag-only category can never match. The prompt wording was iterated against the live model before being committed: the German insurance case came back byte-identical, so it is not a regression trade.

Cost. Container exec plus imports 0.69s, the rewrite 3.84s on Qwen3.6-35B, vault walk ~0.1s. So --nl is a ~4.5s command and the default stays ~0.2s. A single bare word with --nl is 0.35s and never leaves the host.

Other scenarios. German question returns Autoversicherung, Versicherung, Kosten and the right invoices. A question nobody can answer prints what it searched for instead of going quiet. With an out-of-date container (host updated, bot-runner not restarted) the degradation is one line, not the entry point's usage dump.

Checks: 1922 stacklet tests + 487 framework, uvx ruff check . clean, basedpyright 0 new against a baseline written on this tree. tests/stacklets/test_recall.py never needed an edit, which was the move's own gate.

@arthware-dev
arthware-dev merged commit a17fad8 into main Aug 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant