Build an AI scribe in a weekend without hand-rolling the data layer

An AI medical scribe is the perfect first build in healthcare: clinicians feel the pain daily, the demo lands in thirty seconds, and the workflow is genuinely buildable with AI tools. What stops most weekend builds isn't the AI — it's the question that arrives on Monday: "where does the note live, who signed it, and what vendors touched PHI?" This walks the build with the regulated data-layer questions explicit from the start.

Can you build an AI medical scribe in a weekend?

Yes, as a prototype. The workflow — record a visit, transcribe it, draft a SOAP note, let the clinician edit and sign — is a weekend project for one developer. The hard part is never the AI; it is storing the note as durable clinical data with provenance, audit, encryption, and the right vendor agreements. Put that on a clinical backend and the prototype stays focused on the workflow. (The backend calls shown here use bonfireDB's early-access SDK.)

What the build actually is

An AI scribe is a pipeline, not a model. Six steps, each small:

• Capture the visit audio in the browser.
• Transcribe it to text with a speech-to-text model.
• Draft a structured note (SOAP/DAP) with an LLM.
• Sign — the clinician edits the draft and attests, which records who authored what and when (provenance).
• Store the signed note as a FHIR DocumentReference, exportable and audited.

Steps 1–3 are where you build freely with AI tools. Steps 4–5 are the regulated data layer — the part you should not hand-roll. That split is the whole point of bonfireDB: the open-source clinical backend designed to own the data layer so the prototype does not depend on improvised PHI plumbing.

Step 1 — Capture the audio

This is plain web. Record with MediaRecorder in the browser, hand the resulting blob to your scribe pipeline. Be clear about one thing: this audio is PHI from the moment it's recorded — it just hasn't left the browser yet, so no vendor is in the data path.

const recorder = new MediaRecorder(stream);
const chunks: Blob[] = [];
recorder.ondataavailable = (e) => chunks.push(e.data);
recorder.onstop = async () => {
  const audio = new Blob(chunks, { type: "audio/webm" });
  await draftNoteFromAudio(audio, patientId);
};
recorder.start();

AI-generate this entire step. It's UI and browser APIs — exactly what AI tools are great at.

Step 2 — Transcribe

Send the audio to a speech-to-text model. One thing matters here and it's not the code: the transcription vendor is now processing PHI, so it must be under a Business Associate Agreement. A BAA on your coding tool does not cover the model your running app calls — each data-layer vendor needs its own. Route the call through your backend so the model endpoint and key live server-side, never in the browser.

// server-side — the transcript is PHI from here on
const transcript = await clinical.transcribe(audio);

Step 3 — Draft the note with AI

Feed the transcript to an LLM with a SOAP/DAP template and get a structured draft back. This is a draft, not a decision — it never becomes part of the record until a clinician signs it. Keep the prompt and the model call server-side, and use a model vendor that has signed a BAA.

const draft = await clinical.notes.create({
  patientId,
  type: "progress-note",
  status: "draft",
  text: await clinical.agent.draftSOAP(transcript),
});

The note exists now, but as a draft: not signed, not part of the legal record, fully editable. That distinction belongs in the data layer, not in a boolean you remembered to check. When you want the draft grounded in the patient's history, pull that context through bonfire's planned agent layer (MCP + clean evidence packets) rather than dumping the whole chart into the prompt. The benchmark shows why the raw dump fails, and A6a measured a narrow query-aware selection win; the production context compiler remains a product contract in build.

Step 4 — Clinician edits and signs (provenance)

Render the draft, let the clinician fix what the AI got wrong, then capture the signature. Signing is the moment that matters: it freezes the text, stamps the author and timestamp, and writes a FHIR Provenance resource so the record can always answer "who wrote this, who signed it, and was AI involved?"

await clinical.notes.sign(draft.id, {
  signedBy: clinician.id,
  // provenance is recorded automatically:
  // author, timestamp, and that an AI draft was used
});

You did not build an audit table, a signature schema, or an amendment workflow. Provenance is a first-class primitive — because in a clinical record, who said what when is not metadata, it's the point.

Step 5 — Store as a FHIR DocumentReference, then export

Store a SOAP note as a FHIR DocumentReference. That's the R4-correct home for a clinical document: it carries the text, the type, the author, the status, and links to the encounter and patient. Because it's FHIR underneath, the note is portable from the first day — you can export a patient's whole chart as a standards-compliant bundle without a migration project.

// the signed note is already a DocumentReference under the hood
const bundle = await clinical.fhir.export(patientId);
// → a FHIR R4 Bundle: Patient, Encounter,
//   DocumentReference (the note), Provenance, Observation…

For coded data alongside the note — vitals, a screener result — use the matching primitives so they land as proper FHIR resources, not free text:

await clinical.observations.record({
  patientId, code: "8867-4", value: 72, unit: "/min", // heart rate
});
await clinical.assessments.record("PHQ-9", { patientId, score: 11 });

SNOMED CT is free for US use through the NLM UMLS license, so coding your observations and problems doesn't add a licensing bill. On the read side, your UI queries the same store reactively:

const { data: notes } = useClinicalQuery(
  clinical.notes.list, { patientId }
);

Step 6 — Audit is built into the path

Every read and write above is logged automatically — who accessed which patient's data, when, from where. You don't write the audit log; you query it when a compliance review asks. Same for access control: "only this patient's clinician can read this note" is enforced server-side by the data layer, not by CRUD you scaffolded and hoped covered every route.

Where this gets you — and what's honest about it

By Sunday night you can have a working AI scribe prototype whose notes follow the right clinical-data path: FHIR documents, signing provenance, encryption, audit, and export all treated as first-class requirements. The parts you built with AI tools (capture, draft UI, edit screen) are yours to own; the parts you didn't hand-roll (storage, signing, audit, FHIR, vendor agreements) are the parts you'd least want to improvise under a deadline.

Two honest caveats. First, bonfireDB is pre-launch — early access, presented as product vision, not a track record. Second, we are not the only option: AWS HealthLake is a real FHIR store, and Medblocks already pioneered the code-first, AI-buildable FHIR reframe. bonfire's bet is narrower and opinionated — agent-native primitives, a managed tier designed to sign a BAA (early access today), and an OSS core sized for a pre-seed indie team building exactly this scribe. Pick the tool that fits; just don't let the data layer be the thing you improvise.

Want the deeper version? See the Scribe Fire Starter for the full starter template, how bonfire works end to end, and the AI scribe backend pillar for the architecture behind every step above.

Keep reading

• The backend for an AI medical scribe — the architecture and FHIR data model behind every step above.
• Build an outpatient app — the same build pattern applied to a full clinical workflow.
• The agent benchmark — why raw FHIR dumps are not enough context for reliable clinical agents.

TL;DR

• An AI scribe is a 6-step pipeline: capture → transcribe → AI draft → clinician signs → store as FHIR DocumentReference. Steps 1–3 are AI-buildable; 4–5 are the regulated data layer.
• Signing records provenance (author, timestamp, AI-assisted) — first-class, not a table you build.
• Notes stored as FHIR R4 are portable and exportable from day one via clinical.fhir.export(patientId).
• Audit and per-patient access control are enforced by the data layer, not scaffolded CRUD.
• Every PHI vendor (transcription, the LLM, storage) needs its own BAA — your coding tool's BAA doesn't cover them.
• Honest framing: bonfireDB is early access; HealthLake and Medblocks are real alternatives. Compete on fit, not on a vacuum.