Pre-filling a clinical assessment without taking the clinician out of it.

“Pre-fill the assessment with AI” had been floating around the company for two years. When one of our largest health plan clients became the design partner, the obvious move was to fill the same boxes automatically and call it done.

But the Clinical Assessment was never a data-entry form. It was built as a structured walk through how a nurse decides whether a patient still needs skilled care. Fill every box with AI and you remove the form’s actual job, leaving the nurse to rubber-stamp. Get it wrong and nurses will read it, correctly, as a path to their own obsolescence.

So I reframed it: replace the input, keep the reasoning. The AI populates the same structured record that already feeds the recommendation engine, reporting and PDFs, so nothing downstream changes. The nurse’s job shifts from transcription to verification and judgement. That frame survived every reversal that followed.

We ran four discovery sessions with the design partner’s medical director, review nurses, a discharge planner and their compliance lead. Three findings shaped everything after.

Documents are the source of truth. Nobody could recall a phone call or email changing a determination, which made “AI reads the documents” a safe premise. But documents moved unreliably between the plan’s system and ours, which would come back to bite us.

The bar is high. Pre-fill wasn’t a nice-to-have. The medical director’s view was blunt: nurses doing dropdowns was not a thing they wanted to exist.

Some judgement isn’t ours to set. Medical directors asked us to remove quantitative benchmarks from the prototype. A software vendor shouldn’t be defending clinical thresholds it invented.

Diagnosis codes stay human. No model picks ICD-10 codes reliably, and not because of model quality: the people it learns from choose codes inconsistently, driven by payer rules rather than clinical text. A wrong code a tired nurse accepts becomes a reimbursement defect with no audit trail. So the field opens empty, the AI lists the three documents most likely to inform it, and the nurse decides. The AI helps where it’s reliable, finding the pages, and steps back where it isn’t.

The recommendation moved below the evidence. In clinical review, I had placed the system’s recommendation at the top of the page. Nurses pointed out that reading the verdict before the evidence inverts how they reason, and that the recommendation shifts as sections fill in. It moved to sit directly above the determination. Product never would have caught it.

Clinical sign-off without a meeting series. To get approval on a 260-question mapping, we built an audit worksheet clinical leadership could review row by row, sorted so the handful of high-risk changes got close attention and the clean one-to-one rows got spot-checked.

The design partner signed off on a full click-through prototype in April: new pages, confidence-based routing that guided reviewers to the fields needing judgement, narrative summaries. Then, in a three-day architecture session with engineering, data science, clinical and compliance, we made the hardest call of the project.

Beta would ship on the existing form. Each AI-answered question got two additions, a pre-fill indicator and a citation link, and nothing else. No new pages, no confidence signal. It was less than the client had seen and less than I wanted to build. But it tested the only question that mattered first: is the model accurate enough to be useful? The full prototype became a library of validated patterns for later releases rather than a destination.

It also meant what the client approved in April wasn’t what they got in August. We handled that in weekly sessions with their super users rather than pretending otherwise.

The first spec gave each section a green, yellow or red completeness badge. An engineer read it literally, as “answers present versus missing,” which lost the signal: a section carried forward untouched from last week and one Anna had just filled fresh both read green.

The fix wasn’t a better color. It was writing down the two facts the badge is computed from, then giving each combination its own name. The badge says what Anna did this run; the icon on each question says what’s in the field now. Try it:

A few weeks into beta, the feedback converged on one sentence: the AI isn’t answering, and when it does, it’s wrong. It would have been easy to hand that to data science as an accuracy problem. Tracing the reports back, most of them weren’t.

Beta ran once, on a schedule, against a three-day document window, with no way back in. If documents landed after the morning run, the nurse saw empty or outdated answers and no explanation. Four missing capabilities explained almost every complaint:

The bet behind the first revision: most of “broken” is a freshness and visibility failure, and fixing visibility would show how much was really model quality. I designed a right-hand panel, everything you need to know on the right and the work on the left, carrying when Anna ran, documents received since, a re-run with clear warnings, and in-flight job status. Engineering planned it in four slices, information first and notifications last, because the complexity sits in coordinating running jobs.

In October my product director left, and I took over product ownership two weeks before full rollout to roughly 300 nurses. The handoff surfaced three gaps, and closing them became the work:

• No ratified success metrics. I set the starting bar from beta’s draft thresholds, share of answers pre-filled and share left unchanged by the nurse, against the analytics we already had.
• A triage process that wouldn’t scale. Beta feedback came to me by email. I replaced it with a shared tracker where items stay “pending review” until engineering confirms the root cause, so nobody ships a confident wrong diagnosis.
• No way to flag a silent error. The nurse is already looking at the evidence when they notice a wrong answer, so the citation itself is the cheapest place to report it. That’s the next thing I’m designing.

Usage and accuracy numbers from the full rollout will be added here once there’s a month of data. The throughput goal the product was built against: from three reviews per nurse per day toward ten.

Turn the ask into the problem before you design. “Pre-fill the form” would not have survived the reversals. “Replace the input, keep the reasoning” did.

Ship the smallest thing that tests the premise, even after the prototype is signed off. No mockup would have surfaced the freshness problem.

When users say the AI is wrong, check the inputs first. Accuracy work is real, but visibility was the first-order problem.