Clinical Trials Operational Analytics (CTOA) provides an overview of operational data such as Enrollment, Query and Audit metrics, Adverse Event rates, and Protocol Deviations.
CTOA can be grouped by compound, therapeutic area, or program. If the CTOA app is configured to be grouped at the URL level, the name will reflect the grouped app.
CTOA, like all Analytics apps, has several common features such as filtering, selection tools, bookmarks, etc. For details on all Analytics functionality, please refer to the Analytics section or the Analytics courses in the Learning Portal.
In this article, you will find information on all the available out-of-the-box CTOA sheets. Use the Table of Contents at the right to navigate to the sheets of interest.
Important: The list of sheets does not include custom sheets specific to your team.
CTOA Home Page
The Home Page displays key summary information and metrics across all studies.
- By default, the stacked bar chart located in the top left corner presents the Historical Subject Status tab. This chart displays the progression of subjects from status to status, allowing you to visually compare segments and observe how they evolve as the trial progresses.
- To explore current subject status, click the Current Subject Status tab. This chart displays where subjects are presently in the trial process.
Tip: Any applied filters (shown in the Selections toolbar) will remain when you move to another sheet. Remove a filter by clicking the 'x' next to the filter in the Selections toolbar.
Enrollment Summary
Use the Enrollment Summary sheet to view enrollment metrics and information by Study, Country, Site, Status, and Group Assignment.
- In the Enrollment Summary chart, click the x-axis drop-down arrow to select from Site Number (default), Country, or Study.
- In the Enrollment Details chart, click the y-axis drop-down arrow to select from Site Number (default), Study, or Country.
Recruitment and Retention
Use the Recruitment and Retention sheet to monitor the subject recruitment and retention metrics for sites based on thresholds you set for key metrics to determine if a site is enrolling at an acceptable rate. Based on the entered threshold rates, the report shows values which are On Target, Under Target, or Action Required.
Planned enrollment numbers must be provided during configuration for the sheet to reflect accurate study details.
If multiple studies are available, select one Study.
- Enter threshold rates for:
- Planned Enrollment on Target % (>=) and Action Required % (<=)
- Screen Failed on Target % (<=) and Action Required % (>=)
- Consented on Target % (>=) and Action Required % (<=)
- Discontinued on Target % (<=) and Action Required % (>=)
Enrollment Trend
Use the Enrollment Trend sheet to view actual, planned, and cumulative enrollment trends in various forms. You can compare the actual enrollment with the enrollment forecast at a site broken out by year and month.
Planned enrollment numbers must be provided during configuration for the sheet to reflect accurate study details.
If multiple studies are available, select one Study.
- Use the tabs across the top to see Cumulative Actual Enrollment (default), Cumulative Enrollment, or Cumulative Screen Failure.
- In the Planned vs Actual Totals by Study/Site/Country chart, click the x-axis drop-down arrow to select from Study (default), Site, or Country.
- In the Actual Study Enrollment vs Actual Site Enrollment chart, to see a site compared to the entire study, select only one site.
Subject Early Discontinuation
Use the Subject Early Discontinuation sheet to get detailed information for subjects who withdrew, dropped out, or discontinued from the study (or studies) for various reasons.
- In the Discontinued Rate by Study/Country/Site chart, click the x-axis drop-down arrow to select from Study (default), Site, or Country.
- In the # of Discontinued by Site / Reason chart, # of Discontinued By Site tab is displayed by default. Click the # of Discontinued By Reason tab to change the view.
- Click the y-axis drop-down arrow to select from # of Discontinued Subjects (default) or % of Discontinued of Enrolled.
Query Metrics
Use the Query Metrics sheet to find information regarding query metrics.
If multiple studies are available, select one Study.
- By default, 20% is the default Open Query Threshold rate. Update the value if you choose. Sites that exceed the defined threshold are highlighted in yellow.
- Click the x-axis drop-down to display the chart by Site (default), Study, or Country.
- Use the tabs across the top to see Queries vs Subjects (default), Current Query Statuses, or # of Queries by Sender-Recipient/Query Status.
- Use the listing at the bottom of the sheet to view the data in tabular form with additional details.
Queries Duration
Use the Queries Duration sheet to view the duration of queries in the opened status, the average number of days it took for the opened queries to be answered, and the average number of days it took for the answered queries to be closed, along with other relevant metrics.
- Enter an unacceptable number of Average Days Outstanding (>=) in the threshold field to focus the data on the sites with average days outstanding greater than or equal to the number entered.
- In all charts, click the y-axis drop-down arrow to select from # of Queries (default) or % of Queries.
- In all but the first chart, click the x-axis drop-down arrow to display the % of Queries instead of the default value.
Query Details
Use the Queries Details sheet for detailed information on all queries by site, folder, form, or data field.
- In the Queries by Sender / Site / Recipient chart, click the y-axis drop-down arrows to select from Query Sender (default) or Query Recipient, and Site (default) or Query Recipient.
- In the Queries by Sender-Recipient and Site (default) / Folder / Form / Data Field chart, use the 4-button group above the chart to toggle between the options.
- Click the drop-down at the top left of the chart to view All (default) data, or for the Top 10 sites only.
Queries by Open Date
Use the Queries by Open Date sheet to view the cumulative number of queries opened over time by Site and User / Sender-Recipient. Additionally, view Query Details by Site and Subject.
Click a year, month, week, and then day in one of the top two charts to drill down to when the queries were opened.
Query Response
Use the Query Response sheet to view the details on query response time.
- For the Average Response in Days by Site Over Time chart, enter the graph's desired timeframe in months, 12 (months) is entered by default.
- The chart will go from the current day back to the number of months entered.
- In the Queries by Open to Answered Duration chart, click the y-axis drop-down arrows to select from Open to Answered Duration (default), Country, or Study, and Site (default), Country, or Study.
Protocol Deviations - Overview
Use the Protocol Deviations - Overview sheet to find details related to subjects who had protocol deviations.
- Use the Protocol Deviation Type filter in the top row to enable easier group analysis.
- The Level filter in the top row can be used to toggle the view of all charts and the table on the sheet by severity level.
- The Number of Protocol Deviations chart contains:
- A reference line for # of PDs/Subject that dynamically responds to the selection in the filters.
- Use the x-axis drop-down to view the chart by Site of Deviation (default), Country, PD Type, Current Site, Subject, Class, and Study.
- In the Protocol Deviation Details by Study, Country, or Site chart, use the y-axis drop-down arrows to select from Site of Deviation (default), Current Site, Country, Protocol Deviation, Study, Field, or Form, and then choose by Class (default), Current Site, Country, Protocol Deviation, Study, Field, Form.
- In the Protocol Deviations Over Time chart:
- Click the y-axis drop-down to select from # of PDs (default) or # of PDs/Subject.
- Click the x-axis drop-down to select Date (default) or Visit. If a protocol deviation happened that was unrelated to a visit, the visit will be shown as 'missing.'
- The Protocol Deviation table presents the data in a tabular format, including additional details.
Protocol Deviation Details
Use the Protocol Deviation Details sheet to view details regarding subjects with protocol deviations, as well as the number of eligible vs ineligible subjects who had protocol deviations. Also, identify the site of deviation details for subject transfers.
- Use the Protocol Deviation Type filter in the top row to enable easier group analysis.
- In the Eligible vs Ineligible Subjects (%) with PDs chart, click the y-axis down-down to select from Site of Deviation (default) or Current Site and Eligibility (default) or Current Site.
- In the Subjects with PDs by Study, Country, or Site chart, click the y-axis down-down to select from Site of Deviation (default), Current Site, Study, or Country.
- The Protocol Deviation table presents the data in a tabular format, including additional details.
AE and SAE Rate
Use the AE and SAE Rate sheet to view detailed information and metrics related to adverse events and serious adverse events (AEs and SAEs).
- Select the AEs (default) or SAEs button at the top left to view charts by AEs or SAEs.
- Enter a value in the # of AEs/Subject (>=) threshold field to quickly identify outliers.
- In the Subject / Event / Unresolved Ratio chart, select the Subject (default), Event, or Unresolved button.
Time to SDV and Details
Use the Time to SDV and Details sheet to view metrics related to Source Data Verification.
If multiple studies are available, select one Study.
Enter a threshold in the Average Days to SDV (>=) to quickly identify outliers.
Timeliness
Use the Timeliness sheet to view various metrics related to source data.
- In all charts, click the x-axis drop-down arrow to select from Site (default), Form, Subject, Folder, Study or Field.
- In the Audits by Audit Date chart, there is another x-axis drop-down arrow to select from, Audit.TimeStamp (Year) -> Month (default), Subject, Study, Form, Folder, or Field.
Post SDV and Post Query Changes
Use the Post SDV and Post Query Changes sheet to view various metrics related to post source data verification and post query changes.
- In the Data Changes After SDV, Query Actions After SDV, and Number of Queries Resulting in a Data Change charts, click the y-axis drop-down arrow to select from Site (default), Form, Subject, Folder, Study, or Field.
- In the Number of Queries Resulting in a Data Change chart, click the x-axis drop-down arrow to select from # of Queries Resulting in a Data Change (default) or # of Subjects.
Data Changes
Use the Data Changes sheet to view data changes related to folders, forms, fields, and sites.
- For the Data Changes Over Time chart, enter the graph's desired timeframe in months, 12 (months) is entered by default.
- In the Studies or Sites with Changes chart, click the x-axis drop-down arrow to select from Study (default) or Site.
- In the Forms or Folders with Changes chart, click the x-axis drop-down arrow to select from Form (default) or Folder.
Audit Summary
Use the Audit Summary sheet to view metrics related to data changes by change reason and audits by user and action type.
- In the Data Changes by Change Reason chart, select a change reason (within the treemap), and / or a particular site (either within the treemap or the legend).
- In the Audits by Action Type chart, click the y-axis drop-down arrows on to select from Site (default), Folder, Form, Field, Subject, or Study, and Action Type (default), Folder, Form, Field, Subject, or Study.
Application Information
Use the Application Information sheet to view application Information, such as what's new in CTOA for the version you are using, the last data refresh date (by domain), a glossary of terms used in CTOA, a link to the Standards module, a Data Model Profile chart which displays the Actual Cells (number of base data cells) vs the Target Cells (number of standard application cells), and eCS support contact information.
Download Sheet Descriptions
Click the links below to Download the PDF with full sheet descriptions.