Permissions: eIQ Review: Allows licensed users access to eIQ Review in Data Central. No user will be granted default access to eIQ Review. Access to eIQ Review must be manually enabled within User Management.

eIQ Review can be enabled for studies in Data Central. eIQ Review provides users a way to conduct assisted data review. Users can see a list of data points that need human review with pre-set visualizations that support their review. The output can be used to improve data quality and increase efficiency in the data review process. They can drill down to underlying subject data to better understand and visualize the atypical data, ask further questions about the data, and finally take review actions, all without leaving the eIQ Review panel.

The models available in eIQ Review include:

• Central Statistical Monitoring applied to Labs and Vitals - CSM Labs and Vitals

  A subject-level univariate (1-dimensional) anomaly detection model that runs on SDTM mapped data and identifies subjects that have atypical labs and / or vitals. Users get a list of subjects that have an anomalous lab or vital sign. 

• Anomalous AE Durations - AE Durations

  A Machine Learning (ML) model trained on historic clinical data. The model predicts if reported AEs have an anomalous duration. Users get a list of records that have an anomalous AE duration. This model requires using SDTM mapped data. 

• Anomalous Shift in Labs and Vitals - Shift in Labs and Vitals

  A Machine Learning (ML) model trained on historic data. The model predicts data discrepancies that have an atypical shift in lab values and vital signs within both normal and out of range values. Users get a list of records that have an anomalous shift in labs and vital signs. This model requires using SDTM mapped data. 

• General Data Review - Univariate Outlier Detection

  A study-agnostic outlier detection model that runs on raw data and identifies atypical values in several domains (Labs, Safety, CM, MH, etc.). Output is based on selected variables within configuration. Users get a list of values that are atypical according to patterns identified in the current study. 

• General Data Review - Incorrect Item Type
  A study-agnostic item type detection model that runs on raw data and identifies incorrect item types in several domains (Labs, Safety, CM, MH, etc.). Output is based on selected variables within configuration. Users get a list of values that have the incorrect type according to patterns identified in the current study (e.g., non-numeric value in a numeric field). 
• Anomalous CM Durations - CM Durations
  A Machine Learning (ML) model trained on historic data. The model predicts if reported Concomitant Medications (CMs) have an anomalous duration by reviewing records with an end date or that are ongoing. This model requires using SDTM mapped data. 
• Domain Classification - MH and AE - Domain Classification
  A classification predictive model that predicts the correct domain based on coded and free text fields. The model uses natural language processing (NLP) and large language models (LLM), using supervised learning and is trained on historical data with the target ground truth labelled by data review experts. The model identifies records and data that may have been entered in the incorrect form or domain and requires correction from the site to ensure data quality. It automates the following checks:
  • Medical History condition / event entry corresponds to a Condition / Event
  • Adverse Event entry corresponds to a Condition / Event
• Concomitant Medication (CM) Consistency - CM Consistency
  A Machine Learning (ML) model trained on historical data using semi-supervised learning with some records ground truth labelled by data review experts. It uses an anomaly detection algorithm to identify fields within CM records that are inconsistent, incorrectly entered as other or have missing data, and provides suggestions for correct data entry. The model includes predictions for dose unit in the recorded data set.

• Concomitant Medication (CM) Indication - CM Indication
  A Machine Learning (ML) model identifying inappropriate medication-indication pairs in the study data by cross-referencing recorded indications with the open-source dataset. It assigns a matching score between the recorded indications and the indications retrieved from the open-source dataset to curate a list of anomalous pairs that need human review. Detection of such data discrepancies helps clinical data review teams identify data quality issues in a more efficient manner, thereby reducing the need for manual checks.

• Irregular Data Patterns - Audit Trail Review Workspace
  A model that identifies site-level irregular data patterns across multiple indicators such as irregular amounts of data deletions, data changes per domain, per user, after verification, and per month. Each indicator has a pre-defined sheet in the Audit Trail Review Workspace that includes both visualizations supporting the review and data listings supporting issue creation.

Tip: Usual Data Central functionality is available, such as filtering and sorting columns, using panel icons to export (download) a listing, creating issues, marking records as reviewed, and maximizing / restoring a panel. Users can also float additional lists and charts on an eIQ Review sheet.

Review Workflow

Records identified by eIQ Review are configurable for review. A single record or multiple records can be marked as reviewed or unreviewed, as well as all records within a panel. Records marked as reviewed will not appear as requiring review again unless the data record changes before a subsequent import. By default, most listings have pre-programmed filters applied to show only records that are new or updated since reviewed. Review statuses are tracked separately for each reviewer role. Only the reviewer roles that have been configured to review that domain display. Additional reviewer roles can be configured based on the needs of the study. Steps to mark records as reviewed / unreviewed are the same for marking records as reviewed in Data Central.

Model Feedback Option

For all eIQ Review models, except CSM Labs and Vitals and Irregular Data Patterns, users performing review can provide feedback within the Anomalous Records listing indicating if the anomalous prediction is accurate or not.

To provide feedback on the model prediction:

• For a single record: Click the thumbs up icon in a row to accept the prediction, the thumbs down to mark the prediction as incorrect and mark the record as reviewed, or right-click on a record and select the from the Feedback options.
• For multiple records:
  1. Click the checkboxes at the left of the rows.
  2. Click the Thumbs Up icon in the panel toolbar.
  3. Select from the drop-down: Accept prediction, Incorrect prediction, Mark as Reviewed, or Remove Feedback(s).

Issue Creation

Users may also create issues within the eIQ Review models. Issue text is auto populated based on the model, eliminating the need to manually input the issue text. Steps to create issues are the same as creating issues in Data Central.

Tip: When creating a record-level issue, the Issue Text is auto populated based on the model. You can use the auto-populated text or update it before saving.

Access eIQ Review

1. Within Data Central, click eIQ Review in the left navigation. The left navigation expands displaying model listing names within eIQ Review: CSM Labs and Vitals, AE Durations, Shift in Labs and Vitals, Univariate Outlier Detection, Incorrect Item Type, CM Durations, Domain Classification, and CM Consistency.
2. Click on any of the model listing names and a listing with anomalies displays on a sheet labelled EIQReview. 

CSM Labs and Vitals

Lab and Vitals Tests Containing Anomalies

1. Click on CSM Labs and Vitals. The listing opens. This view provides a summary of all tests that have at least one anomalous subject.
   

   Note: CSM requires at least 5 subjects with 5 tests each to analyze a test type.

2. The sensitivity value is selected by default, click the threshold icon, and use the slider to adjust the sensitivity value. Changing the sensitivity updates the number of anomalous subjects shown in the listing and dashboards. Use the Reset all variables to default threshold button to return to the default sensitivity value.
3. Click on a row (Test Type) and the test-level sheet opens with three panels. 

Test-level Sheet

Lab and Vitals Tests Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple tests without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (Test Type) and the other two panels update based on the selected row.

Tip: Hover over a data point in the box and whisker chart to see details in a tooltip.

Anomalous Subjects for TEST TYPE (for example ALP): This listing displays the reviewer role(s) (e.g., DM, MDR, etc.), Subject ID, # of Analyzed Measurements, and Site ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues and mark records as reviewed.

Central Subject vs Anomalous Subjects: A box and whisker chart displaying details for the selected Test Type. 

• Click on a box for a subject and the subject-level sheet displays (same sheet as above).

Tip: Use the box and whisker chart to select the subject you want to start your review with.

Subject-level Sheet

Tip: Use the breadcrumbs at the top left to return to the previous sheet and use the test name drop-down to select a different test. 

Anomalous Subjects for TEST TYPE (for example ALP): This is the same listing but reduced in size to allow users to select other anomalous subjects. The selected row is highlighted blue.

Central Subject vs Anomalous Subjects: A box and whisker chart displaying details for the selected Test Type and subject. 

Graphical Patient Profile (GPP): Usual Data Central functionality is available, such as creating issues, and clicking on a data point to open the record details as a floating panel. 

AE Durations

AEs Containing Anomalies

1. Click on AE Durations. The listing opens in the EIQReview sheet. This view provides a summary of all reported AEs that have at least one anomalous AE duration.
2. Click on a row (AE) and the AE-level sheet with two panels displays.

AE-level Sheet

AEs Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple AEs without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (AEDECOD) and the other panel updates based on the selected row.

Anomalous Records for AE (for example Arthralgia): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), Subject ID, Record ID, Confidence Indicator, Value, and Site ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.
• The Confidence Indicator column displays the confidence level associated with each prediction. Records with the highest confidence levels are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row (record) and the record-level sheet opens displaying five panels.

Record-level Sheet

Anomalous Records for AE (for example Arthralgia): This is the same listing but reduced in size to allow users to select multiple records without having to go to the previous screen. The selected row is highlighted blue.

Medical History Listing: Medical History listing.

AE Durations: Box and whisker chart.

Graphical Patient Profile (GPP): Usual Data Central functionality is available, such as creating issues, and clicking on a data point to open the record details as a floating panel.

Record Details: Use this panel to review record details and create issues directly.

Shift in Labs and Vitals

Lab and Vitals Tests Containing Anomalies

1. Click on Shift in Labs and Vitals. The listing opens. This view provides a summary of all labs that have at least one anomalous shift in labs or vitals.
2. Click on a row (Test Type) and the test-level sheet with two panels displays. 

Test-level Sheet

Lab and Vitals Tests Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple lab tests without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (Test Type) and the other panel updates based on the selected row.

Anomalous Records for Lab Test (for example BASO): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), Subject ID, Record ID, Confidence Indicator, Value, % Shift, and Site ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.
• The Confidence Indicator column displays the confidence level associated with each prediction. Records with the highest confidence levels are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row (record) and the record-level sheet opens displaying five panels.

Record-level Sheet

Anomalous Records for TEST TYPE (for example BASO): This is the same listing but reduced in size to allow users to select other anomalous records. The selected row is highlighted blue.

Medical History Listing: Medical History listing.

Measurements over time graph: A line chart showing the shift in measurements for labs and vitals across study days. 
The model requires the first two measurements to learn the base trend, therefore the first two dots are grey.

Graphical Patient Profile (GPP): Usual Data Central functionality is available, such as creating issues, and clicking on a data point to open the record details as a floating panel.

Record Details: Use this panel to review record details and create issues directly.

Univariate Outlier Detection

Tests Containing Anomalies

1. Click on Univariate Outlier Detection. The listing opens. This view provides a summary of all configured variables and groups of variables that are outliers.
2. The confidence value is selected by default, click the threshold icon, and use the slider to adjust the confidence value. Changing the confidence updates the number of outlier subjects shown in the listing and dashboards. Use the Reset all variables to default threshold button to return to the default confidence value.
3. Click on a row (Variable) and a variable-level sheet with three panels displays. 

Variable-level Sheet

Tests Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple variables without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (Variable) and the other panels update based on the selected row.

Distribution of data: This chart identifies outlier data points.

• Click on another row (Variable) and the other panels update based on the selected row.

Anomalous Records for Variable (for example LBSTRESN-LBCAT-HEMATOLOGY-LBORRESU) - hover over to see the full variable in a tooltip): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), Record ID, Confidence Indicator, Value, and Subject ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.
• The Confidence Indicator column displays the confidence level associated with each prediction. Records with the highest confidence levels are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row (record) and the record-level sheet opens displaying four panels.

Record-level Sheet

Anomalous Records for Variable (for example LBSTRESN-LBCAT-HEMATOLOGY-LBORRESU- hover over to see the full variable in a tooltip): This is the same listing but reduced in size to allow users to select other anomalous records. The selected row is highlighted blue.

Medical History Listing: Medical History listing.

Distribution of data for subject chart: A one-line bubble chart displaying all measurement instances. The more instances a measurement contains, the larger the bubble. Bubbles are colored differently depending on how many models detect the instance as an outlier.

Record Details: View the record details, add an Issue by clicking on the plus sign at the top right, scroll to the bottom of the record to view comments, related queries, and related issues. Usual Data Central functionality is available. 

Incorrect Item Type

Tests Containing Anomalies

1. Click on Incorrect Item Type. The listing opens. This view provides a summary of all configured variables and groups of variables that have incorrect item types.
2. Click on a row (Variable) and a test-level sheet with two panels displays. 

Test-level Sheet

Tests Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple variables without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (Variable) and the other panels update based on the selected row.

Anomalous Records for Variable (for example ANALYTEVALUE - hover over to see the full variable in a tooltip): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), Record ID, Value, and Subject ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.

CM Durations

CMs Containing Anomalies

1. Click on CM Durations. The listing opens in the EIQReview sheet. This view provides a summary of all reported CMs that have at least one anomalous CM duration.
2. Click on a row (CM) and the CM-level sheet with two panels displays.

CM-level Sheet

CMs Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple CMs without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (CMDECOD) and the other panel updates based on the selected row.

Anomalous Records for CM (for example Ibuprofen): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), Subject ID, Record ID, Confidence Indicator, Value, CMINDC, CMDOSE, CMDOSEU, and Site ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.
• The Confidence Indicator column displays the confidence level associated with each prediction. Records with the highest confidence levels are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row (record) and the record-level sheet opens displaying three panels.

CM Durations chart: A chart displaying the dose by duration (days). Color-coded markers indicate Range Area, Not Anomalous, and Anomalous.

Record-level Sheet

Anomalous Records for CM (for example Ibuprofen): This is the same listing but reduced in size to allow users to select multiple records without having to go to the previous screen. The selected row is highlighted blue.

CM Durations: A chart displaying the dose by duration (days). Color-coded markers indicate Range Area, Not Anomalous, and Anomalous.

Graphical Patient Profile (GPP): Usual Data Central functionality is available, such as creating issues, and clicking on a data point to open the record details as a floating panel. 

Domain Classification

Tests Containing Anomalies

1. Click on Domain Classification. The listing opens in the EIQReview sheet. This view provides a summary of records identified as entered into the incorrect form or domain.
2. Click on a row (Data Store / Domain) and the Data Store / Domain-level sheet with four panels displays.

Data Store / Domain-level Sheet

Tests Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple rows without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (Data Store / Domain) and the other panels update based on the selected row.

Anomalous Records for Data Store-Domain (for example Clinical-MH): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), Subject ID, Record ID, Confidence Indicator, Site ID, Predicted Domain, Misclassified Variables, and Reason. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.
• The Confidence Indicator column displays the confidence level associated with each prediction. Records with the highest confidence levels are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row (record) and the two panels on the right update.

Domain for Subject (for example MH for Subject CLINTEK-064-001-008): The AE or MH listing for the selected subject.

Record Details: View the Record Details of the highlighted record.

• Create an issue on the record or on an anomalous field. Anomalous fields are highlighted red for easy identification. 

Tip: When creating a record-level issue, the Issue Text is auto populated based on the model. You can use the auto-populated text or update it before saving.

CM Consistency

CM Consistency between Dose, Dose unit, Route of Administration, and Frequency Containing Anomalies

1. Click on CM Consistency. The listing opens in the EIQReview sheet. This view provides a summary of records with fields identified as inconsistent, incorrectly entered as other or have missing data, and provides suggestions for correct data entry.
2. Click on a row (CM) and the CM / Record-level sheet with three panels displays.

CM / Record-level Sheet

CM Consistency between Dose, Dose Unit, Route of Administration, and Frequency Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple rows without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (CM) and the other panels update based on the selected row.

Anomalous Records for CM (for example Biotin): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.), number of fields containing inconsistencies, Subject ID, Record ID, Confidence Indicator, and Site ID. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can provide feedback on the model prediction, create issues, and mark records as reviewed.
• The Confidence Indicator column displays the confidence level associated with each prediction. Records with the highest confidence levels are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row (subject) and that subject's CM record displays in the Record Details.

Record Details: View the Details view of the highlighted subject.

• Create an issue on the record or on an anomalous field. Anomalous fields are highlighted based on the Confidence Indicator value for easy identification. 
• Hover over the anomalous data variable to view a tooltip that contains the confidence level and description.

CM Indication

CMs Containing Anomalies

1. Click on CM Indication. The listing opens in the EIQReview sheet. This view provides a summary of Concomitant Medications (CMs) that have an indication identified as anomalous in the recorded study data.
2. Click on a row (CM) and the CM / Record-level sheet with three panels displays.

CM / Record-level Sheet

CMs Containing Anomalies: This is the same listing but reduced in size to allow users to select multiple rows without having to go to the previous screen. The selected row is highlighted blue.

• Click on another row (CM) and the other panels update based on the selected row.

Anomalous Records for CM (for example Pyridoxine hydrochloride): This listing displays a checkbox (if you are assigned a reviewer role), Feedback options, assigned reviewer roles (e.g., DM, MDR, etc.),  Subject ID, Record ID, confidence Score, CMINDC, Site ID, and Reason. The reviewer role columns have a pre-set filter displaying records that are new and updated since reviewed based on the user's reviewer role.

• From this listing, users can create issues, provide feedback on the model prediction, and mark records as reviewed.
• The Score column displays the confidence level associated with each prediction. Records with the lowest scores are listed at the top of the list, ensuring the most anomalous records appear first.
• Click on a row and that subject's preset Graphical Patient Profile (GPP) displays at the right.

Subject Graphical Patient Profile (GPP): Usual Data Central functionality is available, such as creating issues, and clicking on a data point to open the record details as a floating panel. 

Access Irregular Data Patterns via the Audit Trail Review Workspace

The Irregular Data Patterns model is accessed by selecting the Audit Trail Review Workspace.

1. Click the Workspace drop-down in the master header.
2. Scroll to locate the Audit Trail Review folder.
3. Select the Audit Trail Review Workspace.
4. By default, the Data Deletions sheet is in view.
5. To access other sheets within this model, click on the sheet name at the bottom of the window. Available sheets include Data Changes, Data Changes per Month, Data Changes per User, Data Changes after Verify, Data Deletions per Month, Data Changes per User & Month, and Changes after Verify per Month.