Risk-Based Quality Management (RBQM) uses several score types and calculations to support risk monitoring for studies, sites, and countries. This article explains Raw Scores, Risk Levels, site and country risk scores, Trends, and Trend Lines.
Raw Score
A Raw Score is the value returned by the query configured for an RBQM indicator.
- KRI Site: The Score value returned by the site query configured for the KRI.
- KRI Country: The Score value returned by the country query configured for the KRI.
- QTL: The Score value returned by the query configured for the QTL. Each study has one Raw Score for each QTL.
Risk Level
RBQM converts a Raw Score to a Risk Level based on the configured upper and lower risk limits. Risk Level values are 0, 1, or 2. The corresponding labels depend on the indicator type.
| Risk Level | KRI | QTL |
|---|---|---|
| 0 | Low | In Control |
| 1 | Medium | Approaching Limit |
| 2 | High | Tolerance Exceeded |
RBQM dashboards and listings display the Risk Level as a Risk Level gauge. Hover over the gauge to view the risk level value.
Higher Values Indicate Higher Risk
The following example shows a KRI where higher values indicate higher risk, with a Medium limit of 5 and a High limit of 10.
| Raw Score | Risk Level |
|---|---|
| 3 | 0 (Low) |
| 6 | 1 (Medium) |
| 12 | 2 (High) |
Lower Values Indicate Higher Risk
The following example shows a KRI where lower values indicate higher risk, with a Medium limit of 7 and a High limit of 3.
| Raw Score | Risk Level |
|---|---|
| 9 | 0 (Low) |
| 6 | 1 (Medium) |
| 2 | 2 (High) |
Site and Country Risk Score Calculation
RBQM calculates an overall risk score for each site and country by combining KRI Risk Levels with configured KRI weights. This score is called the Normalized Risk Score.
Key considerations include:
- Risk scores are calculated only for sites and countries.
- Each site has one risk score for each RBQM Snapshot.
- Each country has one risk score for each RBQM Snapshot.
- A KRI that does not calculate a score is excluded from the calculation and displays a gray risk badge. A KRI that calculates a score but returns a null Risk Level is treated as Risk Level 0, or Low.
Important: A KRI configured as a Critical KRI overrides the calculated risk score. When a site exceeds the limit configured for a critical KRI, RBQM flags the site as medium or high risk regardless of the calculated score.
RBQM calculates the Normalized Risk Score using either of two formulas. Both formulas produce the same result.
Note: In both formulas, Sum of KRI Weights includes only the weights of the KRIs that calculated a score for the site or country in the most recent RBQM Snapshot.
Formula 1
For each KRI that calculates a score for the site or country in the most recent RBQM Snapshot, RBQM:
- Divides the KRI weight by the sum of the weights for all KRIs that calculated a score to determine the Indicator Weight.
- Multiplies the Indicator Weight by the KRI Risk Level.
- Adds the resulting values for all KRIs to determine the Normalized Risk Score.
Normalized Risk Score = SUM ((KRI Weight / Sum of KRI Weights) × Risk Level)
Formula 2
RBQM can also calculate the Normalized Risk Score by:
- Multiplying each KRI weight by its Risk Level.
- Adding the resulting values.
- Dividing the total by the sum of the KRI weights.
Normalized Risk Score = SUM (KRI Weight × Risk Level) / Sum of KRI Weights
Formula 1 Example
The following example calculates the Normalized Risk Score for a country where four KRIs calculated a score in the most recent RBQM Snapshot.
| KRI ID | Weight | Risk Level | Indicator Weight | Weighted Risk Level |
|---|---|---|---|---|
| 557 | 3 | 1 | 3 / (3 + 2 + 1 + 1) = 0.43 | 0.43 × 1 = 0.43 |
| 559 | 2 | 1 | 2 / (3 + 2 + 1 + 1) = 0.29 | 0.29 × 1 = 0.29 |
| 560 | 1 | 1 | 1 / (3 + 2 + 1 + 1) = 0.14 | 0.14 × 1 = 0.14 |
| 561 | 1 | 2 | 1 / (3 + 2 + 1 + 1) = 0.14 | 0.14 × 2 = 0.28 |
The resulting Normalized Risk Score for the country is 1.14.
Trend
RBQM compares the current score with the previous RBQM Snapshot to determine the trend. A change within 0.5 is treated as no change.
| Trend | Indicator |
|---|---|
| No change, or a change within 0.5 compared with the last RBQM Snapshot | Straight line |
| Score decreased from the last RBQM Snapshot | Blue down arrow |
| Score decreased across the last two RBQM Snapshots | Double blue down arrow |
| Score increased from the last RBQM Snapshot | Red up arrow |
| Score increased across the last two RBQM Snapshots | Double red up arrow |
Note: QTL trends are calculated by comparing the current score with the average of the three previous scores. A 5% threshold determines whether the trend is Increasing, Decreasing, or Steady.
Trend Line
The Trend Line column displays the full score history. The values represented depend on the dashboard or listing.
- Site or Country Profile listings: The values represent Normalized Risk Scores for the site or country. The column is labeled Site Score Trend Line in the Sites Profile and Country Score Trend Line in the Countries Profile.
- Site or Country listings for a specific KRI: The values represent the non-normalized Risk Scores for the selected KRI.
- KRI-specific listings: Each value represents the average of the non-normalized site scores for the KRI for that RBQM Snapshot.