how automated healthcare fails, how you'd know, and what to do at each tier — every claim sourced, reviewed continuously
Layer 5 of 5
The human handoff
The moment automation hands work back to a person who has stopped watching, stopped practising, or stopped hearing the alarms.
Reviewed 27 September 2026Sources checked when written 26 September 2026Involved in 26 of 39 incidents96 sources (92 primary or secondary)
What this layer is
The human handoff layer is everything that has to be true of people for an automated hospital to fail safely: that clinicians notice when a machine is wrong, that someone owns each alarm and alert, that staff can take over when automation stops, and that they can still run the ward on paper. It covers automation bias (following or waiting for the machine), out-of-the-loop performance (taking over without situation awareness), skill atrophy and deskilling, alarm and alert fatigue, and downtime competence.
The problem is old. Bainbridge's 1983 'Ironies of Automation' observed that automating a process leaves the operator with the tasks the designer could not automate, including taking over in abnormal conditions, while the manual skills needed for that takeover deteriorate when they are not used. Parasuraman and Riley (1997) described misuse (over-reliance) and disuse (ignoring automation, commonly after false alarms). Endsley and Kiris (1995) showed that operators of an automated system were slower to decide after it failed, and linked this to lower situation awareness from passive monitoring.
Healthcare now has direct evidence of each mechanism. Experienced radiologists' accuracy collapsed when a purported AI suggested the wrong BI-RADS category (Dratsch 2023). Endoscopists' adenoma detection rate in non-AI colonoscopies fell from 28.4% to 22.4% after AI was introduced (Budzyń 2025). The Joint Commission logged 98 alarm-related sentinel events, 80 of them deaths, between January 2009 and June 2012. During EHR downtime, safety reports show downtime procedures missing or not followed in 46% of cases (Larsen 2018), and clinicians at Ascension in 2024 described medication errors and delayed labs after the switch to paper.
FailSystems viewFailSystems' view: in an automated hospital the human is no longer the primary operator but the fallback, and a fallback that is never exercised decays. Every other layer on this site eventually fails into this one: when power, network, devices or models go, the plan is 'staff take over', and that plan silently assumes skills, attention and alarm ownership that automation has been eroding. The dangerous moment is not the outage itself but the handback, when a person with stale skills and little context is asked to act fast. We judge that rehearsal (unaided practice at tier 0, drills at tiers 2 and 3) is the only defense that addresses the cause rather than the symptom, and that it is the one hospitals most often cut.
How it fails
Automation bias: following the machine, or waiting for it
Clinicians over-rely on decision support, making errors of commission (following incorrect advice) or omission (not acting because the system did not prompt). Reliance increases with workload, time pressure and task complexity, and when verifying the output is hard. Experience reduces but does not remove it: in a mammography experiment, very experienced readers' accuracy dropped from 82.3% to 45.5% when the AI suggestion was wrong.[1,2,3,4,5,6]
Warning signs
Clinician-AI agreement near 100% with no documented overrides
Decisions made faster after AI deployment with no change in case mix
Staff cannot explain the basis of a recommendation they acted on
AI output placed where it is seen before the clinician's own assessment
When automation disconnects or fails, the person taking over has been monitoring passively and lacks current situation awareness. Endsley and Kiris found slower decisions after an expert system failed, attributed mainly to the shift from active to passive processing. Bainbridge noted that manual operators need 15 to 30 minutes to build a feel for a process before taking over, time an automated handback rarely allows.[7,8,9,10]
Warning signs
Automation disconnects or mode changes without a clear, explained display
Takeover procedures exist on paper but are never practised against realistic failure signatures
Supervisors monitor many automated streams at once with low event rates
Skills that automation performs routinely are practised less and deteriorate, so a formerly experienced operator becomes an inexperienced one at the moment of takeover. In four Polish endoscopy centres, the adenoma detection rate of standard (non-AI) colonoscopy fell 6.0 percentage points in the three months after AI was introduced. Trainees who learn with automation from the start may never build the unaided skill at all (judgement; not yet measured).[11,8]
Warning signs
Unaided performance is not measured after AI rollout
No protected unaided cases or simulator time
New staff onboarded only on the automated workflow
Most device alarm signals do not require clinical intervention (the Joint Commission cites estimates of 85 to 99 percent), so clinicians become desensitised and turn alarms down, off, or outside safe limits. Parasuraman and Riley described this as disuse: false alarms teach operators to ignore automation. Alarm fatigue was the most common contributing factor in alarm-related sentinel events reported to the Joint Commission.[12,13,14,5]
Warning signs
Hundreds of alarm signals per patient per day on a unit
Default alarm limits never tailored to the patient
Alarms found silenced or with widened limits on rounds
ECG electrodes and sensors not changed on schedule
An alarm only protects a patient if a specific person hears it and is responsible for acting. Joint Commission data list alarms not audible in all areas (25 events), alarms inappropriately turned off (36) and inadequate staffing to respond among contributors. The Joint Commission's alarm-safety goal (NPG.01.05.01 from January 2026, previously NPSG.06.01.01) requires hospitals to define who may set, change and turn off alarm parameters, but it generally excludes CPOE and other IT alerts, so AI and EHR alerts can fall outside any alarm-ownership policy.[12,15,16,17,18]
Warning signs
No written answer to 'who responds to this alert, and within how long?'
AI or EHR alerts routed to a shared inbox or a role, not a person
Central-station monitoring without a named watcher on every shift
Decision-support alerts with low specificity are overridden routinely: a review found drug safety alerts overridden in 49% to 96% of cases. A widely deployed sepsis model generated alerts for 18% of all hospitalised patients while missing 67% of sepsis cases at one academic centre, a burden the authors described as alert fatigue. Clinicians learn to click through, including on the rare correct alert.[19,20]
Warning signs
Override rates above 90% for an alert class
Alert volume rises with each new model or rule, none retired
No one reviews overridden alerts that preceded harm
When the EHR or other systems go down, staff must run care on paper, but ONC notes that many organisations have employees who do not know how to work in a paper-based environment. Safety reports tied to downtime cluster in lab orders and results and medication, and patient identification and communication fail. Paper workflows are slower (lab results averaged 62% longer in one study) and the supporting infrastructure, such as forms and fax machines, may no longer exist.[21,22,23,24,25]
Warning signs
No downtime drill in the past 12 months
Paper forms missing, outdated, or stored where no one can find them
Read-only backup EHR credentials unknown to front-line staff
Ontario's Auditor General found every AI scribe on the province's approved vendor list had fabricated, wrong or missing content in procurement tests, including 12 of 20 recording the wrong drug.[26,27]
May 2026Kinney Drugs, Vermont-based pharmacy chain, USAFell to tier 1: assisted operationModels & agentsHuman handoff
A pharmacy chain's AI voice and text assistant for refills mispronounced medications, ordered wrong dosages and duplicate refills, and gave callers no keypad alternative; after hundreds of complaints the chain pulled it back.[28,29]
An automated sepsis alert triggered a protocol for large-volume IV fluids in a dialysis patient; the nurse objected, was told to follow the protocol, and a physician intervened.[30,31]
A Class I software correction found that backlogged EHR-to-pump automated programming requests could load stale rate, dose or volume parameters.[32,33]
Researchers and an AP investigation found OpenAI's Whisper speech-to-text model inserting fabricated sentences, and a Whisper-based clinical scribe used by over 30,000 clinicians erased the source audio, removing the way to check.[34,35]
A faulty Rapid Response Content update to CrowdStrike's Falcon sensor crashed about 8.5 million Windows devices worldwide. Outside-in measurement found disrupted services at 759 of 2,232 US hospitals studied.[36,37,38,39,40,41,42]
Ransomware hit Synnovis, the pathology provider for several south-east London NHS trusts and GP practices. Blood testing and matching collapsed, more than 11,000 appointments and procedures were postponed, O-type blood ran short nationally, and one death was later partly attributed to a delayed result.[43,44,45,46,47,48,49,50]
A ransomware attack took Ascension's electronic records offline for about five weeks. Clinicians told KFF Health News of medication errors and delayed lab results, and one said he had no training for the attack; Ascension said its care teams were trained for such disruptions.[24,51]
Attackers used stolen credentials on a Change Healthcare Citrix remote-access portal that had no multi-factor authentication, then deployed ransomware nine days later. Disconnecting the clearinghouse stalled pharmacy claims, medical claims and payments across the US.[52,53,54,55,56]
A class action alleges UnitedHealth used naviHealth's nH Predict model to cut off post-acute care; a Senate investigation found UnitedHealthcare's post-acute denial rate nearly tripled while it automated prior authorization.[57,58,59,60]
An independent validation of Epic's proprietary sepsis score found it far less accurate than the vendor reported: it missed 67% of sepsis cases while alerting on 18% of all hospitalizations.[20,61]
A month-long ransomware attack on a health system with about 25% of regional inpatient discharges drove patients and ambulances to two unaffected academic EDs, raising their census, waits and stroke activations.[62,63]
Freezing weather knocked out generation and forced the largest controlled load shed in US history. Power loss spread to water systems and hospitals, and to patients at home on powered medical equipment.[64,65,66,67,68,69]
PathPower → Devices → Human handoff
October 2020Mann-Grandstaff VA Medical Center, Spokane, Washington, United StatesNo outage: wrong outputConnectivity & dataHuman handoff
After go-live, the new EHR routed more than 11,000 clinical orders to a hidden queue instead of the intended service, without telling the ordering clinician; VHA identified 149 adverse events.[70]
April 2020University of Michigan Hospital, Ann Arbor, MI, USA; alert surge measured across 24 US hospitalsFell to tier 1: assisted operationModels & agentsHuman handoff
Weeks after its first COVID-19 admissions, the University of Michigan paused Epic sepsis alerts because dataset shift produced spurious alerting; its clinical AI committee decommissioned the model.[71,72]
18 March 2018Tempe, Arizona, USANon-clinical analogueHuman handoff
Non-clinical analogue. The human operator meant to back up an automated driving system was looking at her phone; the NTSB cited automation complacency.[10]
A self-spreading ransomware worm infected 34 English trusts and 603 primary-care and other NHS organisations, and at least 46 more trusts were disrupted. Thousands of appointments were cancelled and five hospitals diverted ambulances.[73,74,75]
Storm surge flooded basements holding fuel tanks and pumps at two Manhattan hospitals whose generators sat on upper floors. Both hospitals evacuated.[78,79,80]
A patient on a cardiac monitor died after the monitor's crisis alarm had been left off; lower-level alarms sounded at the nurses' station but went unheeded.[81,82]
2010Massachusetts, USA (hospital named in the Boston Globe report cited by the Joint Commission)Tier 0: automation stayed upSingle sourceHuman handoffDevices
A 60-year-old ICU patient's monitor alarmed for rising heart rate and falling oxygen saturation; staff responded only after about an hour, when he had stopped breathing.[12]
1 June 2009Atlantic Ocean, Rio de Janeiro-Paris flightNon-clinical analogueHuman handoff
Non-clinical analogue. Iced pitot probes caused inconsistent airspeeds, the autopilot disconnected, and the crew did not recognise or recover from a stall.[9,83]
A network loop took down clinical applications at an academic medical centre for about four days, forcing a return to paper it had abandoned years earlier.[87,88,89,21]
How you'd know
Measure unaided performance after any AI rollout, not just AI-assisted performance. In colonoscopy, the non-AI adenoma detection rate is the metric that exposed deskilling.[11]
Track clinician-AI disagreement and override rates. Near-total agreement on a system with known error rates is a sign of automation bias, not accuracy.[1,3]
Count alarm signals per bed per day and the fraction that were actionable; several hundred per patient per day, with most non-actionable, predicts alarm fatigue.[12]
Track override rates by alert type. Rates approaching the 49 to 96 percent reported for drug alerts mean the alert is being ignored as a class.[19]
Search incident reports for 'downtime' and code whether procedures were in place and followed; in one analysis 46% of downtime-related reports said they were not.[22]
Time key tasks during downtime drills (first paper medication order, first critical lab result delivered) and compare with normal operation.[23,21]
What to do, tier by tier
What should already be in place at each degradation tier for this layer. Tier 0 is normal automated running; tier 3 is paper, batteries and judgement.
These are practices reported or recommended in the cited sources, gathered for reference. They are not a prescription for your organisation; judge what fits your setting, and check the current official text of any standard.
0Full automation
Keep a set of unaided cases for every AI-assisted task and report unaided performance to the department quarterly.[11,8]
Buy or build decision support that shows its inputs and reasoning so the clinician can check it, and treat time-critical uses as highest risk for automation bias.[4,1]
Inventory every alarm- and alert-generating system, AI and EHR alerts included, and name the role that responds to each and the response time expected.[15,12]
Retire or retune any alert class with an override rate above your threshold before adding a new one.[19,20]
For each AI tool, write down which function it automates (information gathering, analysis, choosing an action, or carrying it out) and at what level, and keep high-consequence action selection at a level where a clinician must actively decide.[90]
Train users that automation bias exists and emphasise their accountability for the final decision; both are among the few mitigators with evidence.[1]
1Assisted operation
Make every automation mode change visible and explained on screen: say what is off, since when, and what the clinician now owns.[9,7]
Write and rehearse takeover procedures for each known failure signature (model offline, feed stale, sensor disagreement), not a generic 'use clinical judgement'.[9,8]
Where verification is hard, reduce the clinician's cognitive load before asking them to check the AI; automation bias tracks verification complexity.[2]
Put a named human on every automated monitoring stream with a realistic watch load; do not rely on passive supervision of low-event streams.[10,8]
2Manual operation
Tailor alarm limits to the patient and change ECG electrodes and single-use sensors on the manufacturer's schedule to cut nuisance alarms.[12]
Write down who may set, change and turn off alarm parameters, and audit silenced or widened alarms on rounds.[15]
Test that critical alarm signals are audible in every area where the responder may be, including at night staffing levels.[12]
Specify alarm priority and signal conventions to IEC 60601-1-8 in procurement so devices from different vendors are distinguishable by urgency.[91]
3Analog fallback
Run an unannounced EHR downtime drill at least once a year on every clinical unit, and time the first paper order and first critical result.[21,92]
Train every clinician on paper ordering and charting and on activating the read-only backup EHR, and make sure they can find its login.[21]
Stock current paper forms for key EHR functions on each unit and write a patient-identification procedure for before, during and after downtime.[21,22]
Use one of the two exercises CMS already requires each year for a multi-day loss of the EHR and clinical systems, and revise the plan from what you learn.[93,25]
Keep a downtime communication channel that does not depend on the EHR's computing infrastructure.[21]
Standards and rules (US)
Instrument
What it requires
Joint Commission NPG.01.05.01, clinical alarm safety (hospitals and critical access hospitals, from January 2026; previously NPSG.06.01.01, effective 2014)
Leaders make alarm safety a priority, identify the most important alarm signals, and set policies on alarm settings, when alarms may be disabled or changed, who has authority to set, change or turn them off, and monitoring and response; staff must be educated on the alarm systems they are responsible for. It generally does not cover CPOE or other IT alerts.[16]
42 CFR 482.15(d), CMS Conditions of Participation: emergency preparedness training and testing
Hospitals must train all staff in emergency procedures initially and at least every 2 years, demonstrate staff knowledge, and run at least two exercises a year (one full-scale or functional, one additional such as a tabletop), analysing and documenting each.[93]
Voluntary self-assessment. Recommends paper forms for key EHR functions, training and testing staff on downtime and recovery (including unannounced drills at least yearly and read-only backup EHR use), EHR-independent communication, and review of downtimes over 24 hours.[21]
FDA, Clinical Decision Support Software guidance (January 29, 2026)
Non-binding. Defines automation bias and treats the level of automation and the time-critical nature of the decision as factors in whether a clinician can independently review the basis of a recommendation (criterion 4 for non-device CDS).[4]
IEC 60601-1-8:2006+A1:2012 (Ed. 2.1), alarm systems collateral standard
Requirements and tests for alarm systems in medical electrical equipment: alarm categories by urgency, consistent alarm signals and control states, and their marking.[91]
Elsewhere: EU and UK
The EU AI Act (Regulation (EU) 2024/1689) is the first law to name automation bias. Article 14(4)(b) requires high-risk AI systems to be provided so that overseers can remain aware of the tendency to over-rely on outputs, and 14(4)(d)-(e) require that they can disregard or override an output and stop the system safely. Article 26(2) requires deployers, such as hospitals, to assign human oversight to people with the necessary competence, training, authority and support. In England, DCB0160, mandated under section 250 of the Health and Social Care Act 2012, requires care organisations to apply clinical risk management to the deployment and use of health IT, which is where local handoff and downtime hazards are expected to be logged.[94,95,96]
Severity score v0.1 draft
4Likelihood
3Blast radius
4Detectability (5 = hardest)
48of 125
FailSystems judgementJudgement. Likelihood is high because alarm fatigue, alert override and weak downtime readiness are documented as common, not exceptional. Blast radius is usually one patient or one unit per event, but rises to system-wide when a large outage forces a whole network onto paper. Detectability is poor because skill loss and automation bias stay invisible until the handback, and routine metrics measure assisted rather than unaided performance.
Each factor is scored 1–5 and multiplied, as in a classic FMEA risk priority number. This is our first-draft judgement, not a measurement; see how scoring works and how it will be revised.
What we don't know yet
How fast does unaided skill decay after AI adoption, does it plateau, and does it recover when AI is withdrawn? Budzyń compared only three months before and after, in one specialty.
What drill frequency and format actually preserves paper-mode competence? SAFER recommends at least an annual unannounced drill, but outcome evidence for any frequency is thin.
Does showing the basis of a recommendation reduce automation bias under real time pressure, or only in experiments?
Who owns AI and EHR alerts that fall outside the Joint Commission's alarm-safety goal (NPG.01.05.01), and does any accreditor survey their response?
Will clinicians trained from the start with AI assistance ever develop the unaided skill the fallback plan assumes?
These gaps drive what the nightly research pass looks for. If you have evidence, send it.
Cite this pageFailSystems. “The human handoff.” https://failsystems.health201.com/layers/handoff/ (reviewed 2026-09-27). Health 201 / AstroNexus LLC. CC BY 4.0.
Information only, not advice. FailSystems is an aggregation and synthesis of published sources. It is not consulting, engineering, legal, regulatory or medical advice, and using it creates no professional relationship. Health systems are complex and no approach fits every organisation: anything you adopt is your own decision, at your own risk, and should be checked against the current official sources and by qualified people who know your setting. Full disclaimer.
Dealing with an incident right now? This site is a reference, not an incident-response service. Activate your organisation's emergency operations plan and incident command, and:
Power loss, disaster or resource needs: go through your local or county emergency management. They escalate to the state, and the state requests FEMA support; hospitals do not call FEMA directly.
A medical device problem: report it to the manufacturer and to FDA MedWatch.
Outside the US: your national emergency number and national cyber agency (in the UK, NCSC).