A solid plan to assess the impact of a new schedule must be in place prior to instituting any scheduling intervention (see What needs to be in place before you start?).  Patient safety is the primary outcome that must be quantified using validated instruments to ensure that the quality of care is not compromised.  A ‘best practice’ to assess the impact of a new schedule on patient safety is to form an advisory committee to review patient safety outcomes at regular intervals.

There are various methods for measuring patient safety.  At a minimum, every new schedule should review rates of adverse events, medication errors, and mortality before and after implementing a change, using a systematic active surveillance methodology.  Published instruments such as the Institute of Healthcare Improvement (IHI) Global Trigger Tool have been developed to quantify and track the incidence of adverse events.  Systematic tracking of medical errors and medication errors is also possible.  In addition, many hospitals employ patient safety researchers, risk management experts, or quality improvement officers who may be able to help programs develop objective, appropriate metrics for measuring the effects of an intervention on safety.  The Harvard Work Hours, Health, and Safety Group would be glad to discuss the specifics of assessing change with program directors or others planning to implement a scheduling intervention – please contact us.

There are several potential pitfalls to be aware of when seeking to assess scheduling changes.  First, it should be recognized that existing hospital incident report systems are notoriously unreliable as a means of tracking the incidence of errors and adverse events.  In several studies, spontaneous reporting has been found to capture only 2-6% of all incidents that occur.   In addition, while measuring house staff, faculty, and nurse experience / satisfaction is an important element of any assessment, it should not be used in isolation.  Residents and other providers enmeshed in systemic changes often have significant concerns about the changes being made, but may be less acutely attuned the risks present in existing systems to which they have grown accustomed.  Often, changes are perceived as being far more negative than objective data would indicate.  While measuring provider experience is important in its own right, and as a means of identifying issues that need to be addressed, it cannot be used as the sole means of quantifying the effects of an intervention.