Which AI scribe writes the note you can sign?

The three encounters were written as transcripts: one admission with a long medication list and a last dose that matters, one clinic visit with a pertinent negative the patient volunteers, and one handoff in which the patient and the clinician disagree. In this round the transcript was pasted as text into each scribe (Scribeberry: the transcript field, SOAP Note Standard template; Heidi: the context field, SOAP Note template), so the transcription step was not tested. The audio round follows. Each note was graded line by line against the transcript on the five criteria above, 0 to 2 per criterion: 2 clean, 1 one error, 0 more than one.

Scribeberry, text, encounters one to three

Behind each point, by encounter. Column order follows the five criteria above.

1. Admission. Time and attribution 1, no encounter time.
2. Clinic. Time and attribution 1, no encounter time.
3. Handoff. Medications and last doses 0, the last ceftriaxone dose time was dropped and a route was added that nobody said; Time and attribution 1, no encounter time.

Heidi, text, encounters one to three

Behind each point, by encounter. Column order follows the five criteria above.

1. Admission. Medications and last doses 1, the next apixaban dose was dated a month late; Invented findings 0, a rhythm description, a route of oxygen, a non tender abdomen and two diagnoses the patient described only in plain words; Time and attribution 0, no encounter time and the daughter's words not attributed.
2. Clinic. Invented findings 1, cranial nerves listed by number when the clinician said intact; Time and attribution 1, no encounter time.
3. Handoff. Medications and last doses 1, an intravenous route was written on the antibiotic and nobody said it; Invented findings 1, a separate line gave the oxygen a nasal cannula that nobody said; Time and attribution 1, no encounter time.

What both got right. The volunteered negative in the clinic visit, that the aneurysm was in a half sister's other family and not the patient's, reached both notes exactly. The handoff disagreement was recorded as it happened: the patient agreed to stay on a condition, not a discharge against advice. The stopped aspirin was not listed as a current drug, the held lisinopril was held, and the blood culture stayed "no growth at 48 hours, final pending" instead of "negative."

What neither did. Neither note carried the time of the encounter, which was the first line of every transcript. Heidi added findings nobody said and dated the next dose a month late; Scribeberry dropped the time of the last antibiotic dose in the handoff.

A study of feedback from physicians using an AI scribe in a large US hospital system (Dai and colleagues, ML4H 2025, arXiv 2512.04118, cited from a reading on 20 September 2026 and not re read since) found the reported errors cluster in a few places: medication instructions, titration and doses first; confusion between the patient and a caregiver; long discussions of surgery, risks and the patient's decision; dictated phone numbers and referrals that never reach the note; and visits through an interpreter. Round two is three encounters built on those classes, pasted as text into the same two scribes with the same templates: a clinic visit with a dose titration schedule, a nine day taper, a referral phone number and a callback deadline; a pediatric visit with a mother and a grandmother in the room; and a surgical consultation through an interpreter in which the patient defers the decision and signs nothing.

Scribeberry, text, encounters four to six

Behind each point, by encounter. Column order follows the five criteria above.

1. Titration, taper, phone number. Medications and last doses 0, a route was added that nobody said, and the naproxen sentence came out garbled enough to read two ways; Invented findings 2, and the objective section says plainly that no vitals or exam were given; Time and attribution 1, no encounter time.
2. Child and family. Medications and last doses 1, "the blue inhaler" became a drug name by inference; Invented findings 2, the mother's asthma stayed the mother's and the cousin's allergy stayed the cousin's; Time and attribution 0, no encounter time, the child's own words not attributed; Reads at handoff 1, a plan line reading "Not Provided".
3. Surgery through an interpreter. Dropped negatives 2, "no consent was signed today," "by Friday" and "well under one in a hundred" all kept; Time and attribution 1, interpreter and husband attributed, no encounter time.

Heidi, text, encounters four to six

Behind each point, by encounter. Column order follows the five criteria above.

1. Titration, taper, phone number. Medications and last doses 2, the naproxen instruction read correctly; Invented findings 0, a kidney disease diagnosis nobody made, a year added to "June," a toe called swollen when nobody said so; Time and attribution 1, no encounter time. The objective section was also left blank without saying why, which no criterion in this method charges.
2. Child and family. Medications and last doses 1, the drug name by inference; Invented findings 0, "no prior similar episodes" and "no significant past medical history" were never said; Dropped negatives 1, the decision not to order a chest X ray was dropped; Time and attribution 0, no encounter time, the child's words not attributed; Reads at handoff 1, three sentences repeated twice.
3. Surgery through an interpreter. Invented findings 0, "chronic cholecystitis," "cholangitis" and "pancreatitis" were never said by the surgeon; Dropped negatives 0, "not signing consent today" was dropped, "by Friday" became "end of week," and the stated frequency of duct injury was dropped; Time and attribution 1, interpreter and husband attributed, no encounter time.

What round two showed. Both scribes kept the titration schedule, the nine day taper and the referral phone number complete, so the first class of physician complaints did not appear here. Both kept the mother's history away from the child and the cousin's allergy away from the patient. The difference is structural: Scribeberry copies and runs long and does not invent; Heidi compresses into a physician's shorthand and invents diagnoses and complications nobody said, which is the error that stays in the record. Across six encounters, neither scribe ever wrote the time of the encounter.

Totals over six encounters, text input: Scribeberry 47 of 60, 7.8 of 10. Heidi 35 of 60, 5.8 of 10.

Freed took each transcript as an uploaded text file and produced its default Returning Patient note, since its template list has no SOAP; Twofold took the transcript in its Text to Note field with its SOAP template. Nabla offers a microphone and a dictation mode and no text or file input, so it waits for the audio round; it has not been run yet. Same encounters, same five criteria, same 0 to 2 per criterion, nothing edited before grading.

Freed, text, all six

Behind each point, by encounter. Column order follows the five criteria above.

1. Admission. Medications and last doses 0, "PO" on every drug and "chloride" nobody said; Invented findings 0, heart failure with reduced ejection fraction and chronic kidney disease as diagnoses when the patient said "weak" and "a little weak"; Dropped negatives 0, the cardiology appointment on the second of the month and letting cardiology know were dropped; Time and attribution 0, no encounter time, the daughter's words given to "she".
2. Clinic. Medications and last doses 1, "PO" nobody said; Dropped negatives 1, "I am not ordering imaging" dropped from the plan; Time and attribution 1, no encounter time.
3. Handoff. Medications and last doses 2, fourteen hundred and twenty one hundred all kept; Time and attribution 0, no encounter time, the night physician absent.
4. Titration, taper, phone number. Medications and last doses 1, "PO" nobody said; Invented findings 1, "June 2026," a year nobody said; Dropped negatives 1, "No aspirin" dropped; Time and attribution 1, no encounter time.
5. Child and family. Medications and last doses 2, "blue inhaler" kept as the mother said it; Invented findings 0, "Lives with mother and grandmother" and "informed consent provided" never said; Time and attribution 0, no encounter time, the grandmother's question not attributed.
6. Surgery through an interpreter. Invented findings 0, "Cholelithiasis with cholecystitis," "cholangitis," "Lives with husband; has a daughter" and "Informed consent discussion completed" never said; Dropped negatives 1, "understanding confirmed through the interpreter" dropped; Time and attribution 0, the husband's words dropped entirely, no encounter time.

Twofold, text, all six

Behind each point, by encounter. Column order follows the five criteria above.

1. Admission. Medications and last doses 2, every time kept, no route added; Invented findings 0, plan lines nobody said, "optimize diabetes and hypertension" and "reinforce adherence"; Dropped negatives 2, the cardiology appointment, the potassium and the lisinopril all kept; Time and attribution 0, no encounter time, the daughter not attributed.
2. Clinic. Invented findings 1, "the patient is aware to limit use" nobody said; Time and attribution 1, no encounter time.
3. Handoff. Medications and last doses 1, "subcutaneous" nobody said; Time and attribution 1, no encounter time.
4. Titration, taper, phone number. Medications and last doses 2, the naproxen instruction read correctly; Invented findings 0, "Chronic kidney disease" nobody said, and the pronoun "He" when no gender was stated; Time and attribution 1, no encounter time.
5. Child and family. Medications and last doses 2, "blue inhaler" kept; Invented findings 1, an assessment line under differential considerations about exercise induced abdominal pain that nobody said; Time and attribution 0, no encounter time, the grandmother not attributed.
6. Surgery through an interpreter. Dropped negatives 0, "well under one in a hundred" became "rare," and "not signing today" left implicit; Time and attribution 0, the husband's words dropped, no encounter time.

Freed. Its best note was the handoff: every dose time kept, the escalation thresholds exact. Its worst were the admission and the surgery consult, where it wrote diagnoses nobody made (heart failure with reduced ejection fraction and chronic kidney disease from "the pumping is weak" and "my kidneys are a little weak"; cholecystitis and cholangitis from an ultrasound report of stones and a thick wall), added social lines nobody said (who the patient lives with, twice), wrote "informed consent" where the surgeon said consent would be signed on the day, and dropped the husband's words entirely. It added a route to every drug. It kept "the blue inhaler" as the mother said it and read the naproxen instruction correctly.

Twofold. Steady across all six, closest to Scribeberry in behaviour: it copies and runs long rather than compressing, so it invents little. What it did invent were plan lines that sound like a plan ("optimize diabetes and hypertension management," "reinforce adherence") and a chronic kidney disease diagnosis in the titration visit. It gave the patient a gender the transcript never stated, softened "well under one in a hundred" to "rare," and lost the husband's sentence in the surgery consult. Every note ends with a patient letter signed by the account name, which a clinician would need to edit before sending.

Across four scribes and twenty four notes: none wrote the time of the encounter. The scribes that compress into clinical shorthand invent diagnoses; the scribes that copy do not. Titration schedules, tapers and phone numbers survived everywhere. The words of the person who is not the patient, a daughter, a husband, a grandmother, are the weakest point for all four: merged into the patient's voice or dropped.

Standings over six text encounters: Scribeberry 7.8, Twofold 7.0, Heidi 5.8, then Freed 5.7. Nabla waits for the audio round.

On 21 September 2026 the six encounters were fed as audio files, one per encounter, to the scribes that accept an uploaded recording. Freed took each file through the upload icon beside Capture and produced its default Returning Patient note, with the patient name prompt skipped. Twofold took each file through Upload Recording under Capture Conversation with its SOAP template, session type in person, patient name left blank. Scribeberry was not tested: its visit screen has no audio file upload, only a live microphone. The one file input that accepts audio is a context attachment, and an uploaded encounter produced no transcript there and left the generate button disabled. That was measured on 21 September 2026 and will be rerun if the vendor adds an upload. The patients are fictional, written for the test. Same five criteria, same 0 to 2 per criterion, nothing edited before grading. The audio files were built from the speaker turns alone and contain no clock time, so the encounter time half of the fourth criterion could not be applied in this round; the attribution half, who said what, was. Each file was generated from the written transcript with two synthetic voices, one for the physicians and one for everyone else, single take, no room noise and no overlapping speech, so what follows is a floor and not what a real room would produce.

Freed, audio, all six

Behind each point, by encounter. Column order follows the five criteria above.

1. Admission. Medications and last doses 0, a route on every drug and a salt name nobody said; Invented findings 0, heart failure with reduced ejection fraction and chronic kidney disease from plain words again, and a regular rhythm written for a patient whose pulse was called irregular; Dropped negatives 2, the cardiology date, the held lisinopril and the potassium decision all kept this time; Time and attribution 1, the daughter's words given to the patient.
2. Clinic. Medications and last doses 1, a route nobody said; Dropped negatives 2, no imaging and no sleep medicine both kept.
3. Handoff. Medications and last doses 2, both dose times exact, no route added.
4. Titration, taper, phone number. Medications and last doses 1, a route nobody said; Invented findings 0, a year added to June, and a toe called painful when nobody said so; Dropped negatives 1, "no aspirin" was dropped.
5. Child and family. Medications and last doses 1, a puff count written into the current inhaler use when the mother gave none; Invented findings 2, the mother's asthma stayed the mother's, and no social lines this time; Dropped negatives 2, no X ray, no peanut testing, no eczema and normal ears all kept; Time and attribution 1, the grandmother's question not attributed.
6. Surgery through an interpreter. Invented findings 1, "informed consent provided" when the surgeon said it would be signed on the day; Dropped negatives 0, the interpreter vanished entirely, the sentence that no consent was signed today was dropped, and so was the line that no medicine dissolves the stones; Time and attribution 1, the husband's words dropped.

Twofold, audio, all six

Behind each point, by encounter. Column order follows the five criteria above.

1. Admission. Medications and last doses 0, the furosemide dose never appears, and the last apixaban dose lost its hour and the fourteen hour interval; Invented findings 0, heart failure with reduced ejection fraction from plain words, a non tender abdomen, a cardiology appointment moved to two weeks away when the daughter said the second of next month, and monitoring lines nobody said; Time and attribution 1, the daughter not attributed.
2. Clinic. Invented findings 1, the patient written as preferring to consider a sleep aid later, when it was the clinician who said not yet; Dropped negatives 1, the half sister and her mother's side reduced to "a sister with a brain aneurysm".
3. Handoff. Medications and last doses 0, fourteen hundred became 1:40 PM, and a route was added that nobody said; Invented findings 0, "stable saturations above 92%" from one reading of 94, and ambulation "without desaturation" when nothing was measured while walking; Dropped negatives 0, the patient's agreement to stay on a condition and the discussion of leaving against advice never reached the note, and the eight in the morning walk test lost its hour.
4. Titration, taper, phone number. Medications and last doses 1, atorvastatin came out as "Adervasted"; Invented findings 1, the labs moved to ten days after the taper ends when the clinician said after the taper, in about ten days; Dropped negatives 0, the nephrology number dropped and replaced with "at provided number," and "no aspirin" dropped.
5. Child and family. Medications and last doses 0, amoxicillin came out as "moxicillin," and "four times this week" became "about four times per week"; Invented findings 0, the mother's childhood asthma written on the child, and a technique review nobody planned; Dropped negatives 0, "no eczema" dropped, normal ears dropped; Time and attribution 0, neither the mother nor the grandmother was attributed.
6. Surgery through an interpreter. Invented findings 1, a line about contraindications nobody said; Dropped negatives 2, less than one percent, no dissolving medicine, consent on the day and the interpreter all kept; Time and attribution 1, the husband's words dropped.

Freed. 44 of 60, 7.3. Its text score of 34 carried six points lost to the missing encounter time, a criterion the audio round could not apply, so the like for like comparison is 40 on text (34 plus the six time points) against 44 on audio: four points up. The same shape too: the handoff is still its best note, a clean 10, both dose times exact and no route added. On audio it kept the cardiology date it had dropped on text, kept "no imaging," and wrote no invented social lines for the child. It lost where it had lost before and once more: heart failure with reduced ejection fraction and chronic kidney disease from "the pumping is weak" and "a little weak," a route on every drug, the daughter and the grandmother folded into the patient's voice, the husband gone. New on audio: "regular rhythm" and "regular heart rate" for a patient whose pulse the clinician called irregular, a toe called painful when nobody said painful, and in the surgery consult the interpreter vanished entirely; on text only the confirmation of understanding had gone missing.

Twofold. 33 of 60, 5.5. Like for like, with the time point removed from its text score as well, that is 48 on text (42 plus the six time points) against 33 on audio: fifteen points down. Three of the losses are transcription and nothing else: "Adervasted" for atorvastatin, "moxicillin" for amoxicillin, 1:40 PM for a dose given at fourteen hundred. The rest are not transcription. From audio Twofold stopped copying and started inferring: heart failure with reduced ejection fraction and a non tender abdomen in the admission, a cardiology appointment "in two weeks" when the daughter said the second of next month, the mother's childhood asthma written as the child's own history, the nephrology number replaced with "at provided number," the furosemide dose missing from the medication list, and in the handoff "stable saturations above 92%" from a single reading of 94 and the patient's agreement to stay on a condition never written at all. What made Twofold second on text, that it copies and runs long rather than compressing, did not survive the audio step.

What audio changed. Freed gained four points like for like; Twofold lost fifteen. A text round shows you what a scribe does with words it was handed. The audio round shows you what it does with words it had to hear first, and on these six encounters that step cost one scribe a drug name, an antibiotic name and a dose time before any summarising began. The twenty four text notes never carried the time of the encounter; the audio files did not contain one, so that criterion was not applied to the audio round. The words of the person who is not the patient remain the weakest point in every note, text or audio.

Standings. Text, six encounters: Scribeberry 7.8, Twofold 7.0, Heidi 5.8, Freed 5.7. Audio, six encounters: Freed 7.3, Twofold 5.5; Scribeberry not tested, no audio file upload on its visit screen; Heidi and Nabla not yet run on audio.