Risk Based Monitoring: Going Beyond Re-training Sites

03 May 2024

In the not so distant past, the typical clinical trial site operated with one physician, nurse, and dedicated exam room. Now, most sites employ many clinicians and a group of exam rooms with different devices.

In this more complex environment we need to explore new tools that can help us visualise what is happening at sites so that we can analyse the root cause of problems. Many of these errors do not stem from training issues, but rather factors that can be more difficult to analyse at face value. Increasingly common error sources — such as inefficient processes, cultural factors, or deficient IT support — require a different approach to identify and interpret the causes.

Furthermore, CRAs need to be able to prioritise the errors that matter; not necessarily those that occur once, but recurrent errors that affect patient and trial outcomes.

Because not all sites and protocols carry equal risk, traditional monitoring (100% SDV and regularly scheduled site visits) can utilise resources on low risk sites while preventing the CRA from intensifying effort on high-risk sites. Implementing 100% SDV does not prevent errors from occurring. In a TransCelerate study to determine the effectiveness of 100% SDV and its impact on overall data quality, only 7.8% of total queries and 2.4% of critical data queries were generated as a result of 100% SDV. These data suggest that a primary focus on data verification has a “negligible effect” on improving data quality.

To prevent the recurrence of errors that matter most to patients, CRAs need the tools to approach monitoring data from a new perspective.

Extending modern monitoring practices with patient centricity

Fixing a mistake is more effective if its source is known and understood. We’ve already discussed how other industries, such as aviation, identify and fix errors through classification of the human factors that contributed to serious incidents. Likewise, in clinical trials, visualisation tools and a systematic approach to error classification reveal hard-to-discern trends can vastly improve a CRA’s ability to identify, track, and mitigate risk that impact patients.

Consider a CRA who is responsible for monitoring 16 sites. This CRA’s time could be focused empirically on the most serious threats to trial integrity by visualising error incidence by site (Figure 1, below). We can see that site P has a large number of errors, but do all of those errors have the same cause?

These errors, in traditional monitoring approaches, would be addressed through a CRA retraining sites regardless of the type and severity of errors. The problem is that retraining can only address errors that occurred as a result of improper training. Errors that result from missteps in the execution of process (process errors) or from lack of communication will continue to propagate despite retraining.

However, using Human Factors Classification (HFC) to identify the root cause of site errors, the CRA can visualise a breakdown of the error sources at site P (Figure 1, below). He or she can then put in place pre-bespoke mitigation strategies to address each error in a way that will be more successful than simply retraining the site staff.

In this case, site P was identified as underreporting concomitant medication. The CRA confirmed at the site monitoring visit that adverse event (AE) and concomitant medications were recorded in source or hand-written logs, but the data were not entered into the EDC system. With the traditional approach, a CRA might retrain on proper usage of the EDC system.

Using HFC, however, the CRA dug deeper to find out the root cause and principal human factor for the problem. During root cause investigation the CRA also discovered three additional, previously undetected issues:

Laboratory values meeting AE criteria were not entered in the EDC system and were not assessed in a timely manner by the Primary Investigator
Source documentation did not contain additional information regarding causality, stop dates or dose for AEs, and frequency and dates for concomitant medication
Source notes and logs were discrepant

Through the HFC analysis, the root cause and principal human factor for the finding was identified as the lack of a consistent process for data oversight and record keeping at this site. The Principal Investigator was asked to develop and implement a specific AE and concomitant medication handling and reporting process. Retraining on the use of the EDC would never have resolved this issue.

Impact

There are obvious benefits to classifying and visualising risk in clinical trials, particularly through access to analytics and trends that demonstrate what is and is not working in the trial.

A far greater, but perhaps less quantifiable benefit, is likely to be improved CRA effectiveness. Instead of performing 100% SDV, CRAs strategically analyse risk, make decisions on necessary corrective actions, know why they are performing a corrective action, and ultimately realise how important their role is in ensuring the trial runs smoothly.

This style of leadership, known in the military as mission command, has been shown to be significantly more effective than when subordinates follow orders from their leadership without receiving insight about the importance of their job, which the military calls command and control.

Moving away from 100% SDV to a patient centric monitoring approach based on the root human causes of error is worth considering. Seven of our pharmaceutical and biotechnology partners are now employing HFC in their trials to generate material benefits in cost efficiency and quality.

Risk Based Monitoring: Going Beyond Re-training Sites

Extending modern monitoring practices with patient centricity

Impact

In this section

Connect with us