Clinical Trials Operational Analytics (CTOA) provides pre-defined sheets with visualizations of operational data, including enrollment, query and audit metrics, adverse event rates, and protocol deviations.

CTOA can be grouped by compound, therapeutic area, or program. If the CTOA app is configured for URL-level grouping, the application name reflects the grouped app.

Use the filters at the top of each sheet to refine the displayed data.

Tip: Applied filters remain when navigating between sheets. Clear a selection from the Selections toolbar to remove the filter.

Important: The list of sheets does not include custom sheets specific to your organization.

CTOA Home Page

The Home Page displays key summary information and metrics across all studies.

• By default, the stacked bar chart in the top-left corner displays the Historical Subject Status tab, which shows the progression of subjects from one status to another throughout the trial.
  • Click the Current Subject Status tab to view the current status of subjects in the trial.

Enrollment Summary

Use the Enrollment Summary sheet to view enrollment metrics and information by Study, Country, Site, Status, and Group Assignment.

• The Total Planned Enrollment KPI supports planned enrollment values stored at the study level, in addition to more granular levels such as country, cohort, or site.
• In the Enrollment Summary chart, click the x-axis drop-down arrow to select from Site Number (default), Country, or Study.
• In the Enrollment Details chart, click the y-axis drop-down arrow to select from Site Number (default), Study, or Country.

Recruitment and Retention

Use the Recruitment and Retention sheet to monitor subject recruitment and retention metrics with configurable thresholds that categorize sites as On Target, Under Target, or Action Required.

Planned enrollment values defined at the study level are fully supported, ensuring metrics accurately reflect data whether stored at the study, country, site, or cohort level.

• An instructional banner in the upper right corner of the sheet provides in-app guidance on how to use the threshold input boxes and interpret the resulting status indicators.
• Default target values are pre-populated for all threshold inputs and can be modified to match study-specific expectations.
• Use the filters across the top of the sheet to refine the analysis by Study, Country, Site, and Group Assignment, enabling cohort-level review in addition to site-level analysis.
• Enter or adjust threshold values for the following metrics:
  • Planned Enrollment: On Target % (>=) and Action Required % (<=)
  • Screen Failed: On Target % (<=) and Action Required % (>=)
  • Consented: On Target % (>=) and Action Required % (<=)
  • Discontinued: On Target % (<=) and Action Required % (>=)

Enrollment Trend

Use the Enrollment Trend sheet to view actual, planned, and cumulative enrollment trends in various forms. The sheet compares actual enrollment with forecast enrollment at the study or site level by year and month.

Planned enrollment values defined at the study level are fully supported. This ensures accurate reporting whether data is at the study level or more granular levels such as country, site, or cohort. The charts and KPIs on the sheet fully accommodate this flexibility, even if detailed breakdowns are not included.

Planned enrollment numbers must be provided during configuration for the sheet to reflect accurate study details.

• Use the tabs across the top to see Cumulative Actual Enrollment (default), Cumulative Enrollment, or Cumulative Screen Failure.
• In the Planned vs Actual Totals by Study/Site/Country chart, click the x-axis drop-down arrow to select from Study (default), Site, or Country.
• In the Actual Study Enrollment vs Actual Site Enrollment chart, to see a site compared to the entire study, select only one site.

Subject Early Discontinuation

Use the Subject Early Discontinuation sheet to get detailed information for subjects who withdrew, dropped out, or discontinued from the study (or studies) for various reasons.

• In the Discontinued Rate by Study/Country/Site chart, click the x-axis drop-down arrow to select from Study (default), Site, or Country.
• In the # of Discontinued by Site / Reason chart, # of Discontinued By Site tab is displayed by default. Click the # of Discontinued By Reason tab to change the view.
  • Click the y-axis drop-down arrow to select from # of Discontinued Subjects (default) or % of Discontinued of Enrolled.

Query Metrics

Use the Query Metrics sheet to review query metrics and status information.

If multiple studies are available, select one study from the Study filter.

• The default Open Query Threshold rate is 20%. Update the value as needed. Sites that exceed the defined threshold are highlighted in yellow.
• Click the x-axis drop-down to display the chart by Site (default), Study, or Country.
• Use the tabs across the top to see Queries vs Subjects (default), Current Query Statuses, or # of Queries by Sender-Recipient/Query Status.
• Use the listing at the bottom of the sheet to view the data in tabular form with additional details.

Queries Duration

Use the Queries Duration sheet to review how long queries remain open, the average number of days from open to answered, and the average number of days from answered to closed.

• Enter an Average Days Outstanding (>=) threshold to focus the data on sites with average days outstanding greater than or equal to the entered value.
• In all charts, click the y-axis drop-down arrow to select from # of Queries (default) or % of Queries.
• In all but the first chart, click the x-axis drop-down arrow to display the % of Queries instead of the default value.

Query Details

Use the Query Details sheet to explore query activity across studies, countries, sites, and multiple dimensions such as folder, form, and data field. This sheet includes visual summaries, pivot-based breakdowns, and a table to review details.

• The three-tab container on the left includes horizontal stacked bar charts for Queries by Sender and Dimension, Queries by Recipient and Dimension, and Queries by Sender-Recipient and Dimension.
• Use the dimension selector above the charts to view data by Study (default), Country, Site, Folder, Form, or Data Field. The selected dimension updates all visualizations on the sheet except the query table.
• The pivot table groups rows by the selected dimension and the user who opened the query. Columns display counts by Sender-Recipient and Query Status. Expand to see additional details.
• The table in the bottom right corner of the sheet includes key fields such as Study, Site, Query Text, Query Response, Data Field, and Field.

Queries by Open Date

Use the Queries by Open Date sheet to analyze query trends over time, monitor resolution progress, and compare query volume across sites, users, and sender-recipients.

• The Queries By Date & Site line chart displays query activity over time by site. Use the x-axis drop-down to select Open Date (default), Answered Date, or Closed Date. Use the y-axis to toggle between Cumulative # of Queries (default) or # of Queries. Click a value on the x-axis to drill down from Year > Month > Week > Day.
• The Queries By Date & User and Queries By Date & Sender-Recipient line charts are displayed in a tabbed container. The charts show cumulative queries (default) or the number of queries opened over time by user or sender-recipient. Use the y-axis to toggle between Cumulative # of Queries (default) or # of Queries. Click a value on the x-axis to drill down from Year > Month > Week > Day.
• The pivot tables are organized into two tabs: Queries By Site & User and Queries By Subject.
  • Each pivot table displays a count of queries, broken down by Sender-Recipient and Query Status.
• The Current Query Status Over Time stacked bar chart displays the number of queries opened each year as well as their current status, with color-coded counts for query statuses (Open, Answered, Closed, Cancelled). Click a bar to drill down by Year > Month > Week > Day.

Query Response

Use the Query Response sheet to view the details on query response time.

• For the Average Response in Days by Site Over Time chart, enter the timeframe in months. 12 is entered by default.
  • The chart goes from the current day back to the number of months entered.
• In the Queries by Open to Answered Duration chart, click the y-axis drop-down arrows to select from Open to Answered Duration (default), Country, or Study, and Site (default), Country, or Study.

Protocol Deviations - Overview

Use the Protocol Deviations - Overview sheet to find details related to subjects who had protocol deviations.

• Use the Protocol Deviation Type filter in the top row to refine group analysis.
• Use the Level filter in the top row to display all charts and the table by severity level.
• The Number of Protocol Deviations chart contains:
  • A reference line for # of PDs/Subject that dynamically responds to the selection in the filters.
  • Use the x-axis drop-down to view the chart by Site of Deviation (default), Country, PD Type, Current Site, Subject, Class, and Study.
• In the Protocol Deviation Details by Study, Country, or Site chart, use the y-axis drop-down arrows to select from Site of Deviation (default), Current Site, Country, Protocol Deviation, Study, Field, or Form, and then choose by Class (default), Current Site, Country, Protocol Deviation, Study, Field, or Form.
• In the Protocol Deviations Over Time chart:
  • Click the y-axis drop-down to select from # of PDs (default) or # of PDs/Subject.
  • Click the x-axis drop-down to select Date (default) or Visit. If a protocol deviation happened that was unrelated to a visit, the visit shows as 'missing.'
• The Protocol Deviation table presents the data in a tabular format, including additional details.

Protocol Deviation Details

Use the Protocol Deviation Details sheet to review protocol deviation details, compare eligible and ineligible subjects with protocol deviations, and identify the site of deviation for subject transfers.

• Use the Protocol Deviation Type filter in the top row to refine group analysis.
• In the Eligible vs Ineligible Subjects (%) with PDs chart, click the y-axis drop-down to select from Site of Deviation (default) or Current Site and Eligibility (default) or Current Site.
• In the Subjects with PDs by Study, Country, or Site chart, click the y-axis drop-down to select from Site of Deviation (default), Current Site, Study, or Country.
• The Protocol Deviation table presents the data in a tabular format, including additional details.

AE and SAE Rate

Use the AE and SAE Rate sheet to view detailed information and metrics related to adverse events and serious adverse events (AEs and SAEs).

• Select AEs (default) or SAEs at the top left to display the corresponding rates, sorting, and reference lines in the charts.
• Enter a value in the # of AEs/Subject (≥) threshold field to highlight outliers.
• Use the buttons above the Subject Ratio chart to switch between Subject (default), Event, and Unresolved views. The chart title updates to reflect the selected ratio.
• The table at the bottom of the sheet provides AE and SAE details.

Time to SDV and Details

Use the Time to SDV and Details sheet to view metrics related to source data verification.

If multiple studies are available, select one study from the Study filter.

Enter a threshold in the Average Days to SDV (>=) to identify outliers.

Timeliness

Use the Timeliness sheet to analyze how quickly data is reviewed, verified, or queried after initial entry.

• All four charts on the sheet are controlled by a shared dimension selector in the upper right corner. Select from Study (default), Country, Site, Subject, Folder, Form, or Data Field to update all charts simultaneously. Selecting a value updates each chart's x-axis and title.
• The three combo charts display average timelines (blue bars) along with the number of enrolled subjects (red diamonds).
• The Audits by Audit Date and Dimension line chart displays the number of audits over time for the selected dimension, with the year on the x-axis. Click a year to drill down to the months.
• A drill-down pivot table at the bottom of the sheet displays all three average day metrics. Expand rows to view values by Study, Country, Site, and Subject.

Post SDV and Post Query Changes

Use the Post SDV and Post Query Changes sheet to review metrics related to data changes and query activity that happen after source data verification (SDV).

• The three horizontal bar charts are controlled by a shared dimension selector located in the upper right corner of the sheet. Select from Study (default), Country, Site, Subject, Folder, Form, or Data Field to update all three charts at once.
• The Number of Queries Resulting in a Data Change by Role chart shows how many queries caused data changes, grouped by user role. A secondary y-axis shows the percentage of queries that resulted in data changes for each role.

Data Changes

Use the Data Changes sheet to view data changes related to folders, forms, fields, and sites.

• For the Data Changes Over Time chart, enter the timeframe in months. 12 is entered by default.
• In the Studies or Sites with Changes chart, click the x-axis drop-down arrow to select from Study (default) or Site.
• In the Forms or Folders with Changes chart, click the x-axis drop-down arrow to select from Form (default) or Folder.

Audit Summary

Use the Audit Summary sheet to view metrics related to data changes by change reason and audits by user and action type.

• In the Data Changes by Change Reason chart, select a change reason in the treemap, a site in the treemap or legend, or both.
• In the Audits by Action Type chart, click the y-axis drop-down arrows to select from Site (default), Folder, Form, Field, Subject, or Study, and Action Type (default), Folder, Form, Field, Subject, or Study.

Application Information

Use the Application Information sheet to view CTOA reference information, including a glossary of terms, a link to the Standards module, the Data Model Profile chart, eClinical Solutions Support contact information, and other application details. The Data Model Profile chart compares the number of actual data cells with the target number of standard application cells.