03 / extraction

Thirteen stages,
two kinds of executor

Code handles everything that can be settled mechanically: ordinals, timestamps, who spoke, tool calls, who was mentioned, dedupe, provenance. The model makes semantic judgements only, one at a time, and its output carries only seq and a verbatim quote — no id, no timestamp, no pointer, and no "how sure I am".

Code P1–P17 LLM L1–L12 Code checks + model review
01 / Import
Bring the words in first

The converter writes sessions into one format; unknown fields are rejected and secrets are scanned first.

02 / Distillation
Cut into small batches

One batch per pass, and it can be re-run; a failed batch stops there and the next run resumes from the break.

03 / Extraction
Code locates, model judges

Code supplies ordinals, timestamps and speakers; the model only judges meaning. Whatever code can check is checked first, and model review comes after.

04 / Use
Take out what is needed

Recall gives a one-sentence brief, search gives the raw words, a session gives context; all of them expand the source text by the handle at the start of the line.

No LLM

Searchable once ingested

Every turn is written into the full-text index at import: session lists, raw-word search and replay all return results without calling a model.

With an LLM

Only then does extraction run

Memory cards, review and the always-on brief wait for the model; without it no raw text is lost, and it is filled in when the model comes back.

01

Click a stage, or let it run on its own

Each diamond is one step: brass goes to the model, steel blue runs in code; wherever the two colours alternate, there is a handoff.

extraction pipeline

This draws the main trunk of 13 stages; the full numbering is P1–P17 in code and L1–L12 in the model, and the extra ones are subdivisions and wrap-ups of the same stages. On phones, swipe sideways to see all 13 stages.

program
ingest

Translates the agent's own session format into one unified format; unknown fields and scanned secrets are rejected outright.

Input
session.json (agent-private format) → converter
Output
events[] · each with seq / speaker_id / ts_ms / text
02

Hard-code who does what

Switch between three views to see where the flow breaks when only one side is left. This is not an animation; it is the real trade-off this system makes.

CodeP1–P17
SanitizeLocateTimeSpeaker Candidate filterDedupeProvenanceValidate WriteLedgerMetrics

It does not judge meaning; it only produces the exact position and provenance.

ModelL1–L12
NamingAliasReferenceDisambiguation Tone and roleFactEventRelation StateTime mappingReviewCausality

It never touches position; within the given turn it only judges what this sentence means.

Both sides: Code produces the exact position and provenance; the model makes one semantic judgement on that track. Every output can be traced back to seq and a verbatim quote.
03

Code checks first, then the model reviews

What code can decide never goes to the model; model review runs as a separate pass that only flags problems and never rewrites the draft. Below is a draft that will be sent back — click "Run review" to see which layers it breaks into.

draft"Xiao Wang changed the deploy script to build-release.sh last week, because the Raspberry Pi would not start."
subjectXiao Wang
time_textlast week
valuebuild-release.sh
evidenceseq 12 · "Swapped the release script for build-release.sh; the Raspberry Pi side will not come up."

After it runs, each issue is listed by layer: code check / model review. The draft itself is never rewritten.

Waiting to run 0 issue
Criteria may not use an ok field; tests assert on row counts,
and writing zero rows is an error.
04

How small can one output be

The model produces only "which turn + which quote + what was extracted". Every other field is assembled by code, so the model has nothing to invent.

llm output contract · v4frozen
// only these fields; no id, timestamp, source position, code, logs or confidence score { "seq": 12, "quote": "Swapped the release script for build-release.sh; the Raspberry Pi side will not come up.", "facts": [ { "subject": "deploy script", "predicate": "single entry point", "value": "build-release.sh" } ] } // no content returns an empty array — no guessing, no padding, no "none" { "seq": 13, "quote": "", "facts": [] }