Defining Key Performance Indicators and Governance

18 April 2024

A hallmark of high-performing partnerships is the establishment and maintenance of a governance model that aligns the interests of the sponsor and the CRO.

At a time when device manufacturers are under increasing pressure to improve pipeline productivity, a critical area of focus involves the assessment of the key performance indicators (KPIs) that help bridge the gap between strategies and resultant operational achievements and efficiencies.

How KPIs can drive success — or failure

KPIs help organisations order priorities and guide the sponsor and CRO(s) to mutually valued outcomes. No matter what metrics are measured, the aim of any KPI is to bring about improvement(s), provide leading indicators of potential problems, and offer opportunities to course-correct.

KPIs also drive two-way dialogue to inform changes to current operating practices to optimise the partnership’s long-term efficiency and effectiveness. A poorly chosen KPI, however, can lead to errant decision-making and undermine a program’s success. As an additional side effect, it also can have an effect on resource bandwidth since a great deal of time and energy could be spent collecting KPIs that aren’t actually moving the needle.

For example, if a CRO’s performance is judged by measuring recruitment success via the traditional metric of “time to first patient in,” a CRO is incentivised only to achieve a one-off performance. A substantial number of problems that may unexpectedly or unnecessarily expand a study’s duration or cost may be left out of the scope of a CRO’s responsibilities.

A KPI better aligned with a manufacturer’s success is “last patient out.” This places the responsibility on a CRO to resolve any issues that may threaten the overall recruitment speed and retention rate. Other KPIs that better align a CRO’s execution of a study with a manufacturer’s own success include time to database lock, monitoring quality, and even capital burn rate (for small or medium-sized manufacturers).

However, if a KPI’s measurement does not trigger relevant corrective actions, its measurement generates little value. A contingency plan should be established in advance to respond to a KPI that crosses cautionary or critical thresholds. Such a plan should include steps that both partners have agreed upon to resolve potential roadblocks and help bring the KPI back on track.

For example, in a trial in which enrolment is expected to be challenging, when the enrolment rate at a few key sites falls behind, a plan should be outlined specifying how the CRO will investigate the root cause of the delay and address the issue with those sites.

Furthermore, the plan should specify a threshold at which the manufacturer may take more significant actions, such as to approve the addition of a new site to the trial. Setting expectations for adverse scenarios in this manner strengthens a partnership and the resulting outcomes.

Win-win for sponsor and CRO

Manufacturers can benefit from provisions that are set up for CROs to share both risk and reward. Missing KPIs could incur progressive penalties, including the inability to bid on subsequent projects, whereas a certain level of over performance could generate progressive rewards, such as bonus payments or the ability to earn back previously incurred service credits. Here, manufacturers and CROs are incentivized to engage in risk-sharing activities, in which CROs are encouraged to draw on the full range of capabilities that may not otherwise be leveraged or visible.

In addition to considering risk based pricing/contracting models, partnerships should put in place provisions for continuous improvements to drive value throughout and beyond the life of the partnership. This approach sets year-on-year targets for incremental efficiency improvements. Such targets promote the integration of the manufacturer and CRO’s staff and processes to achieve the most efficient collaboration.

Maximising efficiencies will likely require the CRO to adapt processes and operational structures over time to best interface with a manufacturer. Likewise, manufacturers may find that they can achieve enhanced productivity by adapting their organisation(s) to best interface with the CRO partner. In other words, if a manufacturer knows that its long-term strategy and growth is dependent on a significant outsourcing strategy, organisational structure and redesign should be considered. Since these adaptations often extend beyond a single trial, continuous improvement functions work best when governed by a dedicated governance committee, including the presence of process improvement leaders at both the CRO and manufacturer who are empowered to drive change.

For all levels of leadership, the governance and measurement reporting structure driving the overall partnership strategy should specify the degree of transparency into ongoing outsourced trial activities. A CRO should be able to provide a live view of a trial’s progress, including enrolment numbers, quality metrics, and adherence to milestones. More sophisticated CROs provide a single web portal that allows for on-demand reporting and analysis of real-time trial operational data, as well as communication with relevant staff.

The metrics that matter

While the metrics that matter to each manufacturer vary, ICON has found that inclusion of metrics across four categories — finance, quality, operations, and relationship health — provides the most robust representation of a partnership’s overall performance. These partnership-level metrics should not be as granular as the sub-layer of potentially hundreds of operational metrics that detail an individual trial’s progression, but should still provide leading indicators of potential problems and thus opportunities to course correct.

When selecting KPIs, it is important to be specific. For example, a relationship health KPI for “staff retention” should specify whether turnover is calculated based on staff retained on a specific project, programme, or at the CRO itself. At the partnership level, sample KPIs include:

Finance

Turnaround times for work order development
Payments to sites
Revenue burn rate
Plan to actuals for study

Quality

Audit findings
Monitoring turnaround time for closing out action items
Compliance rates

Operations

Protocol amendments
Cycle times
Enrolment rates

Relationship health

Quarterly two-way surveys (or one-on-one interviews) to measure the satisfaction of people working on the alliance, including opinions on training, on boarding, and operating processes
Staff retention rate

Conclusion

Well-defined KPIs have the potential to drive additional value to the outsourcing partnership. They can be used to help organisations become more efficient and provide a useful measure regarding the effectiveness of the CRO/sponsor relationship. With appropriate governance structures and metrics in place, KPIs become a major driver of constant improvement and innovation in outsourcing relationships.