memshelf — Roadmap

Milestones are deliberately thin. M0 validates the pattern with zero new code; every later milestone must justify itself against what M0 already achieves.

M0 — Pattern validation, no code — complete (2026-07-13 → 2026-07-22)

Epic: #164 (closed).

Exit criteria met: recall test 5/5, ledger + recall-cost numbers written down (demo.md), and the annoyance log became the M1 backlog verbatim. Case B closed 2026-07-22 (33 episodes, zero loss; verdict episode on the shelf).

Prove the loop works with docshelf as-is plus conventions. Protocol, kit, and measurement methodology: docs/M0.md; the prompt-only skill and recall-rule snippet live in adapters/claude-code/.

Exit criteria: 5 known-answer recall questions answered correctly from a fresh session via INDEX navigation; ledger numbers written down; the annoyance log filled. That log is the M1 backlog.

M1 — memshelf-mcp thin server — complete (closed 2026-09-27)

Epic: #165 (closed). Closed by the owner on 2026-09-27: every scope item shipped (0.1.0, 2026-07-25, plugin with it), and the exit clause holds on the record. In the 14 days to 2026-09-27 all 37 derived-layer renders on main-memshelf were made by the shelf bot and none by hand, doctor on its main is clean, and five project shelves ran shelve → doctor → rebuild on 2026-09-26. The one manual fix in CHANGELOG 0.2.0 (#56, 2026-07-27) predates that window.

Only what M0 proved annoying, expected:

Exit criteria: dogfooded on two real projects for two weeks; a full shelve→compact→recall cycle survives without manual repair; doctor clean.

M2 — Policy, hygiene & the context advisor

Epic: #166 — the current period since 2026-09-27.

Exit criteria: a shelf with 100+ episodes keeps INDEX within its budget — INDEX_BASE_TOKENS + INDEX_TOKENS_PER_ENTRY × listed episodes, i.e. the price of a line stays flat as the shelf grows — and recall precision doesn’t degrade (re-run the M0 question set); the advisor’s shelve proposals are accepted (not overridden) most of the time in dogfood use.

Revised 2026-08-21. This used to read “100+ episodes and INDEX under ~10 KB”, and the two clauses contradicted each other. INDEX lists episodes, so its size is O(episodes) by construction; a fixed ceiling is therefore unreachable past some shelf size, and the only mechanism that lowers the number afterwards is rollup — which buys compliance by archiving live memory. Measured on the author’s 113-episode shelf: the structural floor, with every description deleted and the link de-duplicated, is ~3800 tokens (~15 KB), so “100+ episodes under 10 KB” was not merely tight but arithmetically impossible. The derived constant INDEX_BUDGET_TOKENS = 2500 inherited the contradiction and added a unit error — “~10 KB at chars/4” holds only if one character is one byte, and this shelf’s Cyrillic runs ~1.42 bytes per character, making the two clauses two different budgets (10 KB ≈ 1800 tokens; 2500 tokens ≈ 14 KB). The budget is now linear in shelf size, which puts the check on the quantity formatting can actually control. Rollups stay in M2, triggered by INDEX’s share of the context window (INDEX_CONTEXT_SHARE) rather than by a threshold that growth alone would breach.

Measured 2026-09-28 (memshelf-166, part of this epic; two of three exit criteria, method and numbers in docs/demo.md). (a) INDEX budget on the dogfood shelf at 174 live entries (273 rows in ledger.tsv, 101 archived): memshelf doctor reports INDEX at ~13,486 tokens against a budget of 14,120 (200 + 80×174) — under budget, no index-bloat; ~76.4 tokens/entry against the 80 allowance, down from ~81.1 tokens/entry at 113 entries (measurement recorded above, 2026-08-21). Per-line price did not rise as entries grew +54% — flat both by construction and on this measurement. (b) Recall: M0’s original 5-question set targets Case A episodes now two rollup generations deep (absorbed into 2026-07-27-rollup, itself absorbed into 2026-08-09-rollup) and carrying no keyword into the live INDEX’s rollup entries — 0/5 resolve via plain INDEX-then-fetch navigation today. That is a rollup-depth effect (rollup working as designed since #145), not a measured accuracy regression: the source material was deliberately archived, so replaying the identical questions is no longer a like-for-like test. A substitute pair of known-answer questions against currently-live episodes, same INDEX-first-then-targeted-fetch rule, resolved 2/2 — one unambiguous INDEX match each out of 174 entries. No nested test agent was available in the executing session; both checks were self-answered with the M0.md answer key already known — disclosed as contamination in docs/demo.md. (c) unchanged: the advisor-proposals-accepted-in-dogfood measurement is the owner’s to make and stays open.

Decided 2026-09-28, in response to (b) above (#172, owner comment): a rollup now carries the absorbed episodes’ keywords into its own keywords frontmatter, its digest, and its description — the field the INDEX line actually renders — inherited transitively across generations (core/keywords.py; full rationale in docs/DECISIONS.md). This closes the mechanism gap (b) found, not the measurement itself: re-running the M0 question set against the dogfood shelf’s existing rollups needs a backfill this change deliberately does not perform (see the PR).

M3 — Retrieval upgrades, reuse layer & second surface

Epic: #167.

Exit criteria: search-miss rate measurably better than grep baseline on the dogfood shelves (met, #17: 30 hand-written queries on the dogfood shelf — 10 Russian paraphrases, 10 English queries against Russian episodes, 10 short keyword queries — grep misses 29/30, the hybrid 14/30 at k=5 and 9/30 at k=10; MRR 0.03 → 0.43; no query where the hybrid lost a grep hit); one non-author user runs the chat-project flow from docs alone; one real “fork from episode” session succeeds end-to-end (met, #18).

Explicitly deferred