ENSUREOK
← 返回 Blog
E1 · repository testsSource ↗

This article belongs to the repository-tested set. The related public retrieval artifact is Single-threshold versus three-region retrieval; it remains a fixture-level result, not a production benchmark.

Runbook: Stop Paying for Repeated Repository Discovery

Use this runbook when agent traces show repeated repository searches, configuration reads, or test-discovery commands across sessions.

1. Confirm repetition before adding memory

Sample at least ten sessions. Group tool calls by repository and task intent. If repeated work is rare, a semantic cache adds complexity without much return.

2. Build and capture safely

go build -o semantix ./cmd/semantix
go vet ./...
go test ./...

Store sanitized session JSONL locally. Semantix's file store is designed for 0600 files and 0700 directories, atomic replacement, and symlink defenses. That does not remove the operator's obligation to exclude credentials.

3. Extract, retrieve, inject

semantix extract --input session.jsonl --db .semantix/project.db --project demo
semantix search --query "fix the failing Go test" --db .semantix/project.db --retriever hybrid
semantix inject --query "fix the failing Go test" --db .semantix/project.db
semantix verify --session ./sessions --project demo > eval.tsv

Run BM25 first because it is easiest to explain, then compare vector and hybrid. Set scope deliberately. Inspect the injection block before enabling it in a harness.

4. Define rollback

If retrieval fails, continue without the cache. If injected content degrades tasks, disable injection and clear the affected slices. Do not enable direct L3 reuse until dependency verification is covered for the task type.

5. Measure the right outcomes

Track retrieval relevance, repeated tool calls avoided, completion tokens, task success, and user rejection. A lower token count paired with a worse fix is not a win. The project's synthetic cost report is a useful calculation template; replace every assumed value with observations from your own workload.

Baseline evidence before claiming savings

I reproduced the cache and retrieval packages on 2026-08-12 from main e93668e, Go 1.26.5, Windows/amd64:

go test -count=1 ./kernel/cache ./kernel/bm25 ./kernel/inject

Observed result:

ok  semantix/kernel/cache
ok  semantix/kernel/bm25
ok  semantix/kernel/inject

This proves only that the implemented cache, ranker, and injection behavior satisfy repository tests. It contains no production token bill, wall-clock comparison, or independent task-success score. I would publish a savings claim only with paired runs on the same repository tasks, including cache-off baselines, failures, rejected suggestions, token counts, and confidence intervals. Until that dataset exists, the defensible benefit is deterministic reuse infrastructure, not guaranteed lower cost.

Sources and limitations