Build an AI scribe in a weekend (without a HIPAA lawyer)

An AI medical scribe is the perfect first build in healthcare: clinicians feel the pain daily, the demo lands in thirty seconds, and the workflow is genuinely vibe-codeable. What stops most weekend builds isn't the AI — it's the question that arrives on Monday: "where does the note live, and is it compliant?" This walks the whole build with that question already answered.

Can you build an AI medical scribe in a weekend?

Yes. The workflow — record a visit, transcribe it, draft a SOAP note, let the clinician edit and sign — is a weekend project for one developer. The hard part is never the AI; it is storing the note as durable clinical data with provenance, audit, encryption, and a BAA. Put that on a clinical backend and the weekend is real.

What the build actually is

An AI scribe is a pipeline, not a model. Five steps, each small:

• Capture the visit audio in the browser.
• Transcribe it to text with a speech-to-text model.
• Draft a structured note (SOAP/DAP) with an LLM.
• Sign — the clinician edits the draft and attests, which records who authored what and when (provenance).
• Store the signed note as a FHIR DocumentReference, exportable and audited.

Steps 1–3 are where you vibe-code freely. Steps 4–5 are the regulated data layer — the part you should not hand-roll. That split is the whole point of bonfireDB: the open-source clinical backend that owns the data layer so your weekend stays a weekend.

Step 1 — Capture the audio

This is plain web. Record with MediaRecorder in the browser, hand the resulting blob to your scribe pipeline. No PHI touches anything regulated yet — it's an audio file in memory.

const recorder = new MediaRecorder(stream);
const chunks: Blob[] = [];
recorder.ondataavailable = (e) => chunks.push(e.data);
recorder.onstop = async () => {
  const audio = new Blob(chunks, { type: "audio/webm" });
  await draftNoteFromAudio(audio, patientId);
};
recorder.start();

Vibe-code this entire step. It's UI and browser APIs — exactly what AI tools are great at.

Step 2 — Transcribe

Send the audio to a speech-to-text model. One thing matters here and it's not the code: the transcription vendor is now processing PHI, so it must be under a Business Associate Agreement. A BAA on your coding tool does not cover the model your running app calls — each data-layer vendor needs its own. Route the call through your backend so the model endpoint and key live server-side, never in the browser.

// server-side — the transcript is PHI from here on
const transcript = await clinical.transcribe(audio);

Step 3 — Draft the note with AI

Feed the transcript to an LLM with a SOAP/DAP template and get a structured draft back. This is a draft, not a decision — it never becomes part of the record until a clinician signs it. Keep the prompt and the model call server-side, and use a model vendor that has signed a BAA.

const draft = await clinical.notes.create({
  patientId,
  type: "progress-note",
  status: "draft",
  text: await clinical.agent.draftSOAP(transcript),
});

The note exists now, but as a draft: not signed, not part of the legal record, fully editable. That distinction is enforced by the data layer, not by a boolean you remembered to check. When you want the draft grounded in the patient's history, pull that context through bonfire's agent layer (MCP + clean projections) rather than dumping the whole chart into the prompt — agents working over raw FHIR cap out around 50% on real tasks (FHIR-AgentBench), so a curated projection is the part designed to do better than the raw resource graph.

Step 4 — Clinician edits and signs (provenance)

Render the draft, let the clinician fix what the AI got wrong, then capture the signature. Signing is the moment that matters: it freezes the text, stamps the author and timestamp, and writes a FHIR Provenance resource so the record can always answer "who wrote this, who signed it, and was AI involved?"

await clinical.notes.sign(draft.id, {
  signedBy: clinician.id,
  // provenance is recorded automatically:
  // author, timestamp, and that an AI draft was used
});

You did not build an audit table, a signature schema, or an amendment workflow. Provenance is a first-class primitive — because in a clinical record, who said what when is not metadata, it's the point.

Step 5 — Store as a FHIR DocumentReference, then export

Store a SOAP note as a FHIR DocumentReference. That's the R4-correct home for a clinical document: it carries the text, the type, the author, the status, and links to the encounter and patient. Because it's FHIR underneath, the note is portable from the first day — you can export a patient's whole chart as a standards-compliant bundle without a migration project.

// the signed note is already a DocumentReference under the hood
const bundle = await clinical.fhir.export(patientId);
// → a FHIR R4 Bundle: Patient, Encounter,
//   DocumentReference (the note), Provenance, Observation…

For coded data alongside the note — vitals, a screener result — use the matching primitives so they land as proper FHIR resources, not free text:

await clinical.observations.record({
  patientId, code: "8867-4", value: 72, unit: "/min", // heart rate
});
await clinical.assessments.record("PHQ-9", { patientId, score: 11 });

SNOMED CT is free for US use through the NLM, so coding your observations and problems doesn't add a licensing bill. On the read side, your UI queries the same store reactively:

const { data: notes } = useClinicalQuery(
  clinical.notes.list, { patientId }
);

Step 6 — Audit comes free

Every read and write above is logged automatically — who accessed which patient's data, when, from where. You don't write the audit log; you query it when a compliance review asks. Same for access control: "only this patient's clinician can read this note" is enforced server-side by the data layer, not by CRUD you scaffolded and hoped covered every route.

Where this gets you — and what's honest about it

By Sunday night you have a working AI scribe whose notes are real FHIR documents, signed with provenance, encrypted, audited, and exportable. The parts you vibe-coded (capture, draft UI, edit screen) are yours to own; the parts you didn't (storage, signing, audit, FHIR, BAA) are the parts you'd least want to hand-roll under a deadline.

Two honest caveats. First, bonfireDB is pre-launch — early access, presented as product vision, not a track record. Second, you are not the only option: AWS HealthLake is a real FHIR store, and Medblocks already pioneered the code-first, AI-buildable FHIR reframe. bonfire's bet is narrower and opinionated — agent-native primitives, a BAA from day one, and an OSS core sized for a pre-seed indie team building exactly this scribe. Pick the tool that fits; just don't let the data layer be the thing you improvise.

Want the deeper version? See the Scribe Fire Starter for the full starter template, how bonfire works end to end, and the AI scribe backend pillar for the architecture behind every step above.

Keep reading

• The backend for an AI medical scribe — the architecture and FHIR data model behind every step above.
• Build an outpatient app — the same build pattern applied to a full clinical workflow.

TL;DR

• An AI scribe is a 5-step pipeline: capture → transcribe → AI draft → clinician signs → store as FHIR DocumentReference. Steps 1–3 are vibe-codeable; 4–5 are the regulated data layer.
• Signing records provenance (author, timestamp, AI-assisted) — first-class, not a table you build.
• Notes stored as FHIR R4 are portable and exportable from day one via clinical.fhir.export(patientId).
• Audit and per-patient access control are enforced by the data layer, not scaffolded CRUD.
• Every PHI vendor (transcription, the LLM, storage) needs its own BAA — your coding tool's BAA doesn't cover them.
• Honest framing: bonfireDB is early access; HealthLake and Medblocks are real alternatives. Compete on fit, not on a vacuum.