Risk-Based Quality Management (RBQM) uses several score types and calculations to support risk monitoring for studies, sites, and countries. This article explains Raw Scores, Risk Levels, Normalized Risk Scores, Trends, and Trend Line values.
Raw Score
A Raw Score is the value returned by the query configured for an RBQM indicator.
- KRI Site: The Score value returned by the site query configured for the KRI.
- KRI Country: The Score value returned by the country query configured for the KRI.
- QTL: The Score value returned by the query configured for the QTL. Each study has one Raw Score for each QTL.
Risk Level
RBQM converts a Raw Score to a Risk Level based on the configured upper and lower risk limits. Risk Level values are 0, 1, or 2. The corresponding labels depend on the indicator type.
| Risk Level | KRI | QTL |
|---|---|---|
| 0 | Low | In Control |
| 1 | Medium | Approaching Limit |
| 2 | High | Tolerance Exceeded |
RBQM dashboards and listings display the Risk Level as a Risk Level gauge. Hover over the gauge to view the risk level value.
Higher Values Indicate Higher Risk
The following example shows a KRI where higher values indicate higher risk, with a Medium limit of 5 and a High limit of 10.
| Raw Score | Risk Level |
|---|---|
| 3 | 0 (Low) |
| 6 | 1 (Medium) |
| 9 | 2 (High) |
Lower Values Indicate Higher Risk
The following example shows a KRI where lower values indicate higher risk, with a Medium limit of 7 and a High limit of 3.
| Raw Score | Risk Level |
|---|---|
| 9 | 0 (Low) |
| 6 | 1 (Medium) |
| 2 | 2 (High) |
Normalized Risk Score
A Normalized Risk Score, also called an Aggregate Score, combines KRI Risk Levels and configured KRI weights to calculate an overall score for a site or country.
Key considerations include:
- Normalized Risk Scores are calculated only for sites and countries.
- Each site has one Normalized Risk Score for each RBQM Snapshot.
- Each country has one Normalized Risk Score for each RBQM Snapshot.
- A KRI that does not calculate a score is excluded from the Normalized Risk Score calculation and displays a gray risk badge. A KRI that calculates a score but returns a null Risk Level is treated as Risk Level 0, or Low.
Important: A KRI configured as a Critical KRI overrides the Normalized Risk Score. When a site exceeds the limit configured for a critical KRI, RBQM flags the site as medium or high risk regardless of the Normalized Risk Score.
Normalized Risk Score Calculation
RBQM calculates the Normalized Risk Score using either of two equivalent formulas. Both formulas produce the same result.
Note: In both formulas, Sum of KRI Weights includes only the weights of the KRIs that calculated a score for the site or country in the most recent RBQM Snapshot.
Formula 1
For each KRI that calculates a score for the site or country in the most recent RBQM Snapshot, RBQM:
- Divides the KRI weight by the sum of the weights for all KRIs that calculated a score to determine the Indicator Weight.
- Multiplies the Indicator Weight by the KRI Risk Level.
- Adds the resulting values for all KRIs to determine the Normalized Risk Score.
Normalized Risk Score = SUM ((KRI Weight / Sum of KRI Weights) × Risk Level)
Formula 2
The same Normalized Risk Score can be calculated by multiplying each KRI weight by its Risk Level, adding the resulting values, and dividing that total by the sum of the KRI weights.
Normalized Risk Score = SUM (KRI Weight × Risk Level) / Sum of KRI Weights
Formula 1 Example
The following example calculates the Normalized Risk Score for a country where four KRIs calculated a score in the most recent RBQM Snapshot.
| KRI ID | Weight | Risk Level | Indicator Weight | Weighted Risk Level |
|---|---|---|---|---|
| 557 | 3 | 1 | 3 / (3 + 2 + 1 + 1) = 0.43 | 0.43 × 1 = 0.43 |
| 559 | 2 | 1 | 2 / (3 + 2 + 1 + 1) = 0.29 | 0.29 × 1 = 0.29 |
| 560 | 1 | 1 | 1 / (3 + 2 + 1 + 1) = 0.14 | 0.14 × 1 = 0.14 |
| 561 | 1 | 2 | 1 / (3 + 2 + 1 + 1) = 0.14 | 0.14 × 2 = 0.28 |
The resulting Normalized Risk Score is 1.14.
Trend
RBQM determines the trend by comparing the current Normalized Risk Score with the average of the three previous Normalized Risk Scores. A 5% threshold determines whether the trend is Increasing, Decreasing, or Steady.
Example: If the current Normalized Risk Score is 2 and the previous three scores are 2, 2, and 2, the trend is Steady.
If the current Normalized Risk Score is 2 and the previous three scores are 1, 2, and 2, the average of the previous scores is approximately 1.67. Because the current score is more than 5% higher than the previous average, the trend is Increasing.
Trend Line
The Trend Line column displays historical score values. The values represented depend on the dashboard or listing.
- Site or Country Profile listings: The values represent Normalized Risk Scores for the site or country. The column is labeled Site Score Trend Line in the Sites Profile and Country Score Trend Line in the Countries Profile.
- Site or Country listings for a specific KRI: The values represent the non-normalized Risk Scores for the selected KRI.
- KRI-specific listings: Each value represents the average of the non-normalized site scores for the KRI for that RBQM Snapshot.
How Trend Line Values Are Calculated
The Trend Line does not display the five most recent values. RBQM uses all historical values to create five representative values:
- Retrieves all previous records for the applicable score type.
- Divides the records into five groups.
- Selects the maximum score from each group.
- Uses the five maximum values as the Trend Line values.
Example: A KRI calculates a score for a site across 10 RBQM Snapshots and produces the following scores:
3, 5, 10, 2, 6, 8, 2, 5, 8, 10
RBQM divides the values into five groups:
- (3, 5)
- (10, 2)
- (6, 8)
- (2, 5)
- (8, 10)
RBQM selects the maximum score from each group, resulting in the following Trend Line values:
5, 10, 8, 5, 10