A US-based regional health system needed clinical decision support software that could clear low-acuity self-care, queue everything else for a clinician with evidence attached, and page on-call instantly when a red-flag symptom set fired. We shipped clinical decision support software built on Claude Sonnet 4.6, pgvector, and FHIR R4: eval-first, BAA-scoped, with a kill point at week 7 that we used.
A US regional health system shipped a HIPAA-safe clinical triage agent on Claude Sonnet 4.6 across 6 hospitals, 40 clinics, and a 24/7 nurse triage line. Pre-triage wait dropped 38-62 percent across n=14,200 shadow encounters (95% CI). Stack: Claude Sonnet 4.6, pgvector 0.7, HL7 FHIR R4, LangGraph 0.2, AWS Bedrock with PrivateLink, Langfuse. Compliance: HIPAA Business Associate Agreement, HITECH Act, PHI redaction pipeline. Nine weeks discovery to shadow mode.
6 hospitals · 40 clinics · 24/7 nurse triage line handling ~9,500 inbound messages/week across portal, SMS, and post-visit. Too small to staff a 60-seat triage centre during evening peaks; too large for the nurse manager to keep eyes on every queue. Wait times during evening surges were routing wrong-acuity patients to the ER.
Pre-triage queue wait averaged 38–62 minutes at peak, fat tail past 2 hours. The clinical lead estimated 50–70% of inbound messages mapped to documented low-acuity self-care pathways (sore throat with no red flags, medication-refill questions, post-op wound checks healing fine), but patients waited anyway because there was no triage layer in front of the nurse. Worse, the nurse line was occasionally routing borderline patients home when an ER visit was indicated. Generic patient-facing chatbots had been evaluated and turned down on operator-grade objections: no autonomous routing on acuity, no PHI leaving the BAA perimeter, no advice generated without grounding in their own clinical-pathway corpus, no metric not measurable on a frozen eval set.
Medical director named borderline-routing-to-home as the binding constraint in discovery week 1.
Compliance isn't a vendor badge — it's a data-flow choice. PHI starts in your EHR's BAA-scoped environment (Epic, Cerner, athenahealth, Veeva), passes through scrubbing or de-identification before any inference, and audit-logs trace every boundary in reverse. Hover any lane for the controls we ship in that zone.
Where this fails: if your team pastes PHI directly into a consumer ChatGPT window, no diagram saves you. We can't help that — we can only architect the auto-path so the temptation goes away. BAA tiers don't cover the consumer surface. Internal training on "what gets pasted where" is the other half of healthcare AI security and we say so in every audit.
FHIR R4 chart pull (scoped to Patient + Encounter + recent Observation), PHI redaction with reversible token map, hybrid pgvector + BM25 retrieval with bge-reranker-large, Claude Sonnet 4.6 forced-JSON decision, policy + 2-eye guardrails. The agent has zero write tools. Diagram below.
Healthcare AI is governed by clinical-safety scope, not throughput. We map every workflow to an autonomy band before we ship it — autonomous · clinician-approved draft · human-only territory. Here's how we decide.
Scheduling, refill timing, post-visit instructions, eligibility checks. No clinical judgment required at the decision level — but PHI is still in scope.
Symptom interpretation, medication question, diagnostic interpretation, treatment-plan input. Clinical judgment required — the only question is who signs off.
Suicide-risk, pediatric red-flag, chest pain, suspected stroke, behavioral-health crisis, acute psychiatric. AI does not adjudicate these — period.
When something regresses, the per-component metric tells us which stage broke. No single end-to-end number that hides which subsystem moved.
Labelled acuity-band correctness + groundedness on the frozen 412-item eval. Forced-JSON schema requires every claim to cite a retrieved chunk_id whose pathway-id matches the routed clinical context. The model cannot decide without grounded evidence.
Top-k recall on the frozen eval. RRF + reranker tuned against this number, not end-to-end accuracy.
Top-1 precision on the held-out slice. Catches over-confident retrievals before Sonnet ever reasons.
Token recall on labelled PHI spans · reversible map.
Expected calibration error on labelled set.
Policy-routed: pediatric <3y, active pregnancy w/o OB, OOD.
Every patient message enters at the top. It either clears to a self-care pathway, lands in a clinician queue with structured evidence attached, or escalates stat. Hover any stage to see its tool inventory and latency budget.
latency budgets above are p50/p95 on the production traffic mix · end-to-end p95 inside 3.1s target
Everything in the build is a thing your security team can write a question about. Nothing in the build is `our proprietary AI`. Vendor swap-out cost is bounded because the eval set, prompts, and policies are all checked into the customer's repo, not ours.
The numbers below are from the current production cut. Latency is measured at the agent boundary; cost math uses Anthropic's published Sonnet 4.6 pricing as of May 2026; eval composition is the frozen 412-item set the CI gates on.
| stage | p50 | p95 | tooling |
|---|---|---|---|
| FHIR resource pull | 92 | 140 | Epic on-FHIR + athenahealth APIs · cached Patient + scoped Encounter |
| PHI redaction | 78 | 120 | Regex pre-pass + i2b2-fine-tuned clinical NER (DistilBERT base) |
| Hybrid retrieval | 112 | 180 | pgvector cosine top-40 ∥ Postgres tsvector BM25 top-40 → RRF k=60 |
| Cross-encoder rerank | 240 | 340 | BAAI/bge-reranker-large · g5.xlarge in customer VPC · top-12 |
| Claude Sonnet 4.6 decision | 1740 | 2180 | Anthropic API · response_format json_schema · ~3,400 in / ~480 out tokens |
| Policy + 2-eye validation | 14 | 22 | TypeScript runtime · Zod schema · audit-log write |
| Total (end-to-end) | 2280 | 3098 | agent boundary — excludes clinician-side queue render |
p50/p95 from 30-day rolling window over n ≈ 41,200 production decisions. SLO is p95 ≤ 3,500 ms; current burn ≈ 88%.
Retrieval lane was where most of the per-stage tuning went. The corpus is ~1,400 pathway pages, chunked to 480 tokens with 80-token overlap, sentence-anchored. We picked voyage-3-large at 1,024 dimensions specifically because Voyage signs a BAA at the same price tier as voyage-3-lite; the lite variant dropped recall@5 by 4 points and the 35% cost saving wasn't worth a measurably worse retriever. Fusion is RRF with k=60 (paper default; held-out slice did not move on alternatives), top-40 from each lane, deduplicated by chunk id, reranked, top-12 to the model.
// triage/schema/decision.ts
// Forced-JSON decision schema. Validated client-side too; if the
// model produces something that doesn't parse, we retry once with
// a stricter system prompt, then fail closed (queue for clinician).
import { z } from "zod";
export const TriageDecision = z.object({
routing: z.enum([
"clear", // safe for documented self-care; no clinician needed
"queue", // route to nurse queue with this agent's reasoning attached
"escalate", // page on-call clinician now (stat criteria)
]),
acuity_band: z.enum(["1-self-care", "2-routine", "3-same-day", "4-urgent", "5-stat"]),
confidence: z.number().min(0).max(1),
rationale: z.array(z.object({
claim: z.string().min(40).max(420),
evidence_id: z.string().regex(/^chunk_[a-f0-9]{12}$/),
pathway_id: z.string(),
})).min(1).max(8),
refused: z.boolean().describe(
"True if the agent decided it cannot decide: pediatric < 3y, " +
"active pregnancy without OB context, or any rationale failed to ground."
),
});
export type TriageDecision = z.infer<typeof TriageDecision>;
| line item | $ / decision | $ / month (≈ 41k decisions) | note |
|---|---|---|---|
| Claude Sonnet 4.6 — input tokens | $0.0102 | $418 | 3,400 tokens × $3.00 / 1M |
| Claude Sonnet 4.6 — output tokens | $0.0072 | $294 | 480 tokens × $15.00 / 1M |
| voyage-3-large embeddings (avg query) | $0.0004 | $16 | ≈ 3,300 tokens × $0.12 / 1M |
| pgvector + RDS db.m6i.large | — | $284 | BAA-scoped Postgres; embeddings + tsvector |
| g5.xlarge reranker (24/7) | — | $378 | BAAI bge-reranker-large self-host |
| Cloudflare Workers (BAA-eligible) | — | $128 | edge + audit log shipping |
| Langfuse self-hosted (t3.medium) | — | $67 | trace store; 90-day hot / 7-yr cold |
| All-in monthly | ≈ $0.0411 | ≈ $1,585 | vs. ≈ $7,900 / mo to add one triage nurse |
Token costs use Anthropic's public Sonnet 4.6 pricing as of May 2026: $3 / 1M input, $15 / 1M output. Infra costs are AWS US-east-2 list price; client paid less under EDP. Payback period from go-live (including the 9-week build at $185k) was ≈ 6.2 months.
| category | items | what it checks | ci-gate threshold |
|---|---|---|---|
| Acuity-decision golds | 80 | labelled routing + correct acuity band on real (de-identified) encounters | ≥ 0.90 precision @ 1% FPR |
| PHI redaction | 60 | spans of PHI correctly redacted; reversible-token map intact | ≥ 0.99 token recall |
| Retrieval recall | 120 | correct pathway chunk in top-5 after RRF + rerank | ≥ 0.90 recall@5 |
| Groundedness | 100 | every rationale claim points to a retrieved chunk id that supports it | ≥ 0.93 groundedness |
| Refusal / adversarial | 52 | pediatric < 3y, active pregnancy w/o OB, jailbreak attempts, OOD cases | 100% refusal on listed must-refuse |
Eval set is frozen: items only added, never edited. Clinical lead signs off any addition. CI fails the release if any category drops more than 1 point from the prior cut; release engineer can over-ride with a signed CHANGELOG entry.
Production ops is part of the build, not an afterthought. Four controls keep the agent calibrated and the BAA scope honest after cutover.
Every queued case where the agent recommendation differed from the nurse's gets opened. Systematic drift (>3 same pattern/wk) becomes a JIRA ticket against the eval set.
Langfuse in customer VPC. Matches the health system's HIPAA documentation retention policy.
Two engineers per week. 99.5% pipeline-availability SLO + p95 ≤ 3.5s end-to-end decision SLO.
Model version, retrieval candidates, redaction map, policy-check verdict, clinician override.
Five stages, milestone-billed. The week-7 shadow run found a calibration bug on borderline-acuity cases that would have hurt patients in production. We halted cutover, re-fit the calibration head, re-ran the eval, and only then promoted to primary. The honest version of `9 weeks` includes the week we sat on our hands.
Two weeks shadowing the nurse triage line. 412 frozen eval items written by the clinical lead from real (de-identified) past encounters. Each item carries a labelled correct routing decision and the clinical reasoning behind it. We wrote the harness; clinicians wrote the answers.
Ingested the existing clinical-pathway document set (≈ 1,400 chunked pages) into pgvector 0.7 inside the customer VPC. Built the BM25 sidecar over the same chunks. Reciprocal-rank fusion tuned on a held-out eval slice; cross-encoder rerank added when top-1 recall plateaued.
LangGraph 0.2.x agent with three read-only tools. Zero write tools by design. Forced-JSON decision via Anthropic's response_format. Policy-as-code in TypeScript shipping next to the agent: every routing decision is gated and audit-logged before it touches a clinician queue.
Two weeks of silent shadow against the live nurse triage line. Day 4 the clinical lead flagged a calibration drift on borderline-acuity cases: the model was confident on cases where the correct answer was 'queue for clinician', not 'clear'. We halted cutover, re-fit the calibration head on a fresh slice, and re-ran the eval. The honest version of `shipped on time` includes this step.
Promoted to primary triage with the nurse line in active-standby. Four clinician training sessions on the override flow and the audit-log viewer. PagerDuty wired to the stat-escalation lane. Old nurse line stays on for 30 days post-cutover by policy. Every diff between agent + human is logged for review.
The eval set is frozen. Every model change, prompt change, retrieval change, and policy change re-runs the full 412. Nothing ships if any metric red-lights against its target. Numbers below are from the current production cut and the frozen eval slice; live shadow-traffic numbers are within ±2% across all rows over the last 30 days.
Sample size for the production wait-time number is n=14,200 patient encounters across the two-week shadow window; the 38–62% reduction range is the 95% confidence interval, not a point estimate. ECE is expected calibration error on the labelled 412-item set. P95 latency is end-to-end from FHIR pull to JSON decision, measured at the agent boundary (excludes clinician-side queue render). Refusal rate is the share of inputs where the agent legally cannot decide and routes straight to a clinician: by design, not by failure.
A triage agent built on these patterns will hurt patients in any of the following situations. We turn down the engagement before a pilot is scoped.
Pediatric emergency, acute behavioural-health crisis, suspected stroke timeline, anaphylaxis: these are not eval-set problems. They are policy-routed straight to a human. If the workflow needs an AI in that lane, the answer is no.
If the program lead is not going to review agent-vs-clinician diffs weekly for the first six months, the calibration head drifts and nobody catches it. The eval set is necessary, not sufficient.
Every additional chart resource the agent reads is a larger blast radius if the BAA breaks. We scope down to Patient + Encounter + recent Observation. Clients who want the full chart read into the agent are scoping a different (and worse) product.
No BAA from the model vendor, no Cloudflare-or-equivalent BAA on the edge, no audit-log review cadence. The legal posture either exists at week 1 or the pilot doesn't get signed.
Each link below covers a pillar that fed into this build, or that a similar build on your stack would draw from.
The healthcare pillar: BAA-scoped delivery, PHI redaction, clinician-in-loop posture across triage, ambient scribe, and prior-auth.
The agent pillar: ReAct, plan-and-execute, hierarchical multi-agent recipes. Same eval-first loop used on this triage build.
The pillar service this case study is one shipped example of.
Sonnet 4.6 + Haiku 4.5 integration patterns. Forced JSON, Constitutional-AI posture, BAA-eligible deployment options.
Six AI case studies: RAG, agents, voice, and chatbots. Same operator detail across every page.
fixed-fee discovery audit. We map the workflow, scope the eval, and tell you whether it's case-study-shaped.
Policy-as-code, audit-log scaffolding, BAA + DPA templates. The plumbing that made this pilot pass a security review.
How a clinical triage agent fits inside a broader AI development services engagement: EHR integration + retrieval + Sonnet reviewer + audit log.
Clinical triage runs on RAG over patient + protocol docs: the same productized AI knowledge base pattern applied inside a HIPAA boundary.
Book a fixed-fee discovery audit. We'll review the workflow, scope the eval set, recommend a model + retrieval recipe, project token + run-cost, and tell you honestly whether it's case-study-shaped. We'll also tell you if it isn't. About one audit in five ends with `buy the platform, here's the SOW for integration.`