experiment 6 · W2a · exploratory · complete · grading sensitivity pending
Who Should Drive the Join?
The point estimate favored agent-side joining, but the interval is wide. No accuracy difference was detected, equivalence was not established, and the agent used 4.061 times the cumulative input tokens. The 53 discordant questions make a hybrid worth exploring on burned data.
what we asked
The pre-join test showed the deterministic join wins the visit questions, but every earlier agent held join-less tools; “a real traversal tool is untested” was the published caveat. So: given tools that can express the join, does an agent match the deterministic pre-join? Either answer matters. Agent far below: the pre-computed join is the product. Agent equal: the product is the tool contract, at whatever token cost the agent pays.
what we ran
The agent got two tools carrying the same intended join semantics: list_visits and visit_events, alongside bounded search and read tools. No packet. The preserved server hash is 6672b41e…. Deterministic grading covered 46 answers; a 3-vote same-family panel covered 130. That historical panel was not arm-blind, and the answer model was not pinned in the manifest.
what moved
72.2% vs 68.2%: +4.0 points, McNemar p=0.41, 95% CI [−8.7, +17.6] points. The point estimate leaned agent, but the run does not establish superiority, equivalence, or noninferiority.
no difference detected; equivalence not established (p=0.41)
4.061× the cumulative input tokens. This is a measured efficiency difference; the accuracy comparison remains unresolved.
The first dev attempt ran the tools through an SSH tunnel that died mid-run, and every tool failure surfaced as what looked like an agent abstention: an infrastructure failure masquerading as a model failure. The contaminated run was wiped and the transport switched to a direct LAN connection before the freeze; the registered run then completed 176/176 with the FHIR server healthy throughout and zero stragglers. If we had graded the tunnel run, the agent would have looked far worse for reasons that had nothing to do with the model.
What this supports. Agent-side and deterministic joining fail on different questions, and deterministic prejoin is much cheaper in cumulative input tokens. It does not establish that their accuracy is equal, that the agent is better, or that the join alone caused the difference. The historical arm-label leak and missing answer-model pin require a fresh sensitivity grade before a stronger claim. See the commit-pinned result receipt.
The hybrid is the next hypothesis.
Develop packet-first retrieval with tools for the residue on burned data, then test it once on a new patient-disjoint holdout.