Use Data Sources to define the location and connection details for accessing the source data to be imported into elluminate, as well as its target location within the elluminate platform.

Data Sources can be added at the system, study, or global data store level. Data sources created at the study or global data store level can later be promoted to system data sources.

Before importing data for the first time, the data store must be defined. This requires specifying the import type and providing the details associated with that data source type.

If no data source has been defined, no data sources display. Multiple data sources can be created as needed to align with the data being imported.

Users with the Importer privilege can select system, study, or global data store data sources when creating an import definition. Changes made to fields within a data source apply only to the associated import definition.

Important: Data Source Templates have been converted to System Data Sources for global use within elluminate. Study or global data store data sources derived from templates inherit information from the system data source. Changes made at the system level update all instances of the data source at the study, global data store, and import levels.

System Data Sources

Users with the System Configure privilege can create, edit, and delete system data sources. When creating a new data source, users can choose from data sources already configured at the system level and can edit all fields. 

Columns within the list of system data sources are Name, Description, Type, Last Modified, Last Modified By, and Usage Count. The Details view for a data source displays at the right. 

Create a New System Data Source

1. From the Platform Menu, select System Data Sources under Platform Administration.
2. Click the New button in the master header. A New Connection window opens.
   
3. Name: Enter the name of the new data source (name must be unique). Any combination of letters, numbers, characters, or spaces can be used.
4. Description: Enter a description of the data source.
5. Use System Data Source: Optionally, click the drop-down to select an existing data source. All details are automatically added from the selected data source except for the Name and Description.
6. Data Source Type: Click the drop-down to select the data source type. This option is grayed out if an existing data source is selected in the above drop-down. For details on each data source type, see below.
7. Click Save.

Edit a System Data Source

Changes made to a system data source automatically update all study data sources derived from that system data source.

1. From the Platform Menu, select System Data Sources under Platform Administration.
2. Click the row of the data source to edit.
3. Click the Edit button. The Edit Connection window opens.
4. Make the changes.
5. Click Save.

Delete a System Data Source

1. From the Platform Menu, select System Data Sources under Platform Administration.
2. Click the row of the data source to delete.

3. Click the Delete button. A confirmation message displays.

   Note: A message may display indicating it cannot be deleted because other connections depend on it.

4. Click OK to delete or Cancel to cancel the action.

Study / Global Data Store Data Sources

Users with the Study Configure privilege, can create, edit, and delete study / global data store level data sources. These users can choose from data sources configured at the study / global data store or system level and can edit all fields at the study / global data store level. Edits made at the study / global data store level only apply to data source usage within the study / global data store. 

Create a New Study Data Source

1. From the Platform Menu, select Data Sources under Study / Global Data Stores Management.
2. Select a study / global data store from the right side of the Platform Menu.
   OR
3. Access a study from the Studies Home Page / global data store from the Global Data Stores Home Page.
4. Select Data Sources from the Module drop-down. The list of existing data sources is displayed.
5. Click the New button in the master header. A New Connection window opens.
6. Name: Enter the name of the new data source (name must be unique). Any combination of letters, numbers, characters, or spaces can be used.
7. Description: Enter a description of the data source.
8. Use System Data Source: Optionally, click the drop-down to select an existing system data source. All details are automatically added from the selected data source except for the Name and Description.
9. Data Source Type: Click the drop-down to select the data source type. This option is grayed out if an existing data source is selected in the above drop-down. For details on each data source type, see below.
10. Click Save.

Edit a Study Data Source

1. From the Data Sources page for a study or global data store, click the row of the data source to edit.
2. Click the Edit button. The Edit Connection window opens.
3. Make the changes.
4. Click Save.

Delete a Study Data Source

1. From the Data Sources page for a study or global data store, click the row of the data source to delete.
2. Click the Delete button. A confirmation message displays.
3. Click OK to delete or Cancel to cancel the action.

Promote a Study Data Source to a System Data Source

Users with the System Configure privilege can promote a study data source to a system data source.

1. From the Data Sources page for a study or global data store, click the row of the data source to promote.
2. Click the Promote button. The Promote Connection window opens with all fields populated from the selected data source except for the Name and Description.
3. Name: Enter the name of the new data source (name must be unique).
4. Description: Enter a description of the data source.
5. Click Promote. The promoted data source is added to the System Data Sources. 

Note: When creating a Data Source for a Global Data Store, only the following data sources are available: Amazon S3, Amazon S3 Outbound, Box, Box Outbound, FTP, FTP Outbound, and Nurocor MDR.

Important: All Rave data sources require a Rave user account created specifically for this purpose. Please complete the steps detailed in the Set up Rave Web Service User Account section prior to creating the Rave data source.

Data Source Types

Click a data source from the list below for specific instructions relevant to that data source type:

• Amazon S3
  For use with Amazon S3 storage bucket. 
  
  Name and Description: Confirm they have been entered appropriately.
  Bucket Name: The name of the S3 bucket that contains the data.
  Access Key: The Access key for the Amazon account can be read from the S3 bucket.
  Secret Key: The Secret key associated with the source system account.
  Amazon S3 Region: The Amazon Region where the S3 bucket containing the data resides.
  Remote Files: Enter the data files to import from the source system folder. Use the * as the wildcard. For example, *.xlsx imports all xlsx files from the selected folder.
  Remote Folder: Enter the folder for the source system folder.
  Role: Roles can be specified in the S3 bucket.
   
• Box
  The Box for Enterprise (Box) data storage software is a data source for imports to elluminate. Used as an alternative to Amazon S3 and SFTP.
  
  Name and Description: Confirm they have been entered appropriately.
  Box Application Configuration: Cut and paste the contents of the Box Application Configuration JSON file.
  Remote Files: Enter the data files to import from the source system folder. Use the * as the wildcard. For example, *.xlsx imports all xlsx files from the selected folder.
  Remote Folder: Enter the folder for the source system folder.
   
• Clario eCOA
  This enables the importation of current patient-reported outcomes, symptom ratings, and quality-of-life data from Clario's eCOA Live platform into elluminate.
  
  Name: Confirm that the name has been entered correctly; maximum length is 50 characters.
  Description: Provide a description of the data to be imported.
  Address/URL: Enter the URL from which the data is imported.
  Client ID: Input the Client ID used to access the Clario eCOA system.
  Secret Key: Enter the Secret Key provided by Clario.
  Remote Study ID: Specify the Remote Study ID to be accessed within Clario's system.
  Environment: Indicate the name of the environment used at Clario.
   
• Formedix MDR
  The Formedix repository can be used to import and upload custom standards into the elluminate Standards module. Only available for system data sources.
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the data is imported.
  API Key: Web Service key provided by the source system to define the study and authenticate access.
  Secret Key: The Secret key associated with the source system account.
   
• FTP (including SFTP and FTPS, and used for Covance)
  File Transfer Protocol: a standard network protocol used to transfer files from one host to another host over the Internet.
  elluminate supports both SFTP and FTPS.
  Covance data can be imported into elluminate via FTP and used to configure and map data in the Operational Data Repository (ODR).
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the data is imported.
  Username: Enter the username to be used to access the remote system.
  Password: Enter the password to be used to access the remote system.
  Remote Files: Enter the data files to import from the source system folder. Use the * as the wildcard. For example, *.xlsx imports all xlsx files from the selected folder.
  Remote Folder: Enter the folder for the source system folder.
   
• Medrio
  Web service connection to Medrio's EDC platform.
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the data is imported.
  API Key: Web Service key provided by the source system to define the study and authenticate access.
   
• Merge (Zelta)
  Web service connection to Merge’s (Zelta's) EDC platform.
  
  Name and Description: Confirm that both fields are entered appropriately.
  Address/URL: Enter the full URL from which the Merge (Zelta) data is imported.
  Username: Enter the username to access the Merge (Zelta) system.
  Password: Enter the password to access the Merge (Zelta) system.
  Environment: UAT or PROD, for example. If an environment is not specified, PROD is assumed.
  Remote Study ID: Enter the name of the remote study ID to be accessed in the Merge (Zelta) system.
   
• Nurocor MDR
  The Nurocor repository can be used to import and upload custom standards into the elluminate Standards module. Only available for system data sources.
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the data is imported.
  Library: Select from the version of Nurocor to be used.
  APIKey: Web Service key provided by the source system to define the study and authenticate access.
   
• ODBC
  This data source provides configuration options to establish connections to external Open Database Connectivity (ODBC) compliant SQL or Snowflake databases to import datasets for use within elluminate. 
  Configure this data source using the target server with the appropriate ODBC driver installed, relevant credentials, target database, and other options. Once configured, users can test the connection and configure import definitions to ingest data from the external SQL or Snowflake database.
  The same data source can be used to export data to the configured database, similar to the existing functionality where users can push files to an external file storage system like an sFTP server or Box directory. Datasets can be pushed through the configured ODBC data source to the target SQL or Snowflake database.
  
  Name and Description: Confirm they have been entered appropriately.
  Driver: Select from the drop-down: SQL Server or Snowflake.
  Server: Enter the server URL.
  UId: Enter the user ID to access the remote system.
  Pwd: Enter the password to access the remote system.
  Database: Enter the name of the database.
  Additional Options: Enter additional options details.
  Test Connection: Click to test the connection.
   
• Oracle Health Sciences InForm Discrepancy
  InForm Queries can be imported into elluminate and supports 2-way query integration.
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the Oracle InForm data is imported.
  Username: Enter the username to access the Oracle InForm system.
  Password: Enter the password to access the Oracle InForm system.
  Tenant Prefix: Enter the name of the tenant prefix.
  Remote Study: Enter the name of the remote study to be accessed in the Oracle Health Sciences InForm.
   
• PPD - Deprecated
  PPD data can be imported into elluminate and used to configure and map data in the Operational Data Repository (ODR) module.
  
  Name: Enter the name for this import. Any combination of letters, numbers, characters, or spaces can be used.
  Address/URL: Enter the entire URL from which the data is imported.
  Username: Enter the username to access the remote system.
  Password: Enter the password to access the remote system.
  Description: Enter a description of the data being imported.
   

• PPD Preclarus - Labs
  The PPD Preclarus—Labs data source enables users to import all lab results and samples from the remote study in the PPD Preclarus Lab Data Portal. This connection provides access to the most current lab data stored in the PPD system.
  
  Name: Enter the name for this data source. Any combination of letters, numbers, characters, and spaces can be used.
  Description: Enter an appropriate description for this data source. 
  Address/URL: Enter the complete URL from which the PPD Lab Data is imported.
  Username: Enter the username used to access the PPD Lab Data Portal.
  Password: Enter the password used to access the PPD Lab Data Portal. 
  Remote Study: Enter the name of the remote study to be accessed in the PPD Preclarus Lab Data Portal.
  Study Start Date: Enter the initial date when data becomes available for the study. This serves as a starting point for incremental data imports from the PPD Lab Data. 

  Note: The Preclarus PPD Data Source has been separated into two distinct data sources for Labs and Subjects. This data source retrieves up-to-date, detailed lab information from the PPD Preclarus Lab Data Portal.

• PPD Preclarus - Subjects
  The PPD Pleclarus—Subjects data source enables users to import subject demographics, site information, and visit data from the remote study's PPD Preclarus Lab Data Portal. This connection provides access to the most current datasets of all subject information stored within the PPD system.
  
  Name: Enter the name for this import. Any combination of letters, numbers, characters, or spaces can be used.
  Description: Enter an appropriate description for this import.
  Address/URL: Enter the complete URL from which the PPD Lab Data is imported.
  Username: Enter the username used to access the PPD Lab Data Portal.
  Password: Enter the password used to access the PPD Lab Data Portal.
  Remote Study: Enter the name of the remote study to be accessed in the PPD Preclarus Lab Data Portal.

  Note: The Preclarus PPD Data Source has been divided into two different data sources for Labs and Subjects. This data source retrieves subject demographic, site and visit data from the PPD Preclarus Lab Data Portal.

• Rave
  Accesses Rave’s Clinical, Operational, and audit data to import into elluminate
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the data is imported.
  Username: Enter the username to access the Rave system. The user must have both Read Only and Architect access.
  Password: Enter the password to access the Rave system.
  Environment: UAT or PROD, for example. If an environment is not specified, PROD is assumed.
  Remote Study: Enter the name of the remote study to be accessed in the Rave system.
   
• SAS Life Science Analytics Framework (LSAF)
  The SAS Life Science Analytics Framework (LSAF) API connector allows elluminate to import data files stored in repositories on the SAS LSAF platform. LSAF creates SAS datasets, which are stored in a repository accessible with the credentials used to connect to SAS LSAF. This import enables users to retrieve the latest LSAF dataset outputs for direct use in elluminate.
  
  Name: Enter an appropriate name for the data source up to 50 characters.
  Description: Optionally, enter a brief description. 
  Address/URL: Provide the full URL or API endpoint from which the data is imported.
  Username: Enter the username for an account that has access to the LSAF repositories.
  Password: Enter the password for the account accessing LSAF. 
  Remote Files: Enter the exact name(s) of the remote data files to be imported, separated by commas. Users may also employ the * multi-character and ? single-character as wildcards. 
  Remote Folder: Specify the full path to the folder in the LSAF environment containing the files to be accessed. Note that the path is case-sensitive and limited to 100 characters in length.
   
• Trial Master
  Data from OmniComm Trial Master EDC can be imported into elluminate and used in configuring and mapping data to be displayed and reviewed in Data Central and in the CDA Analytics application.
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the data is imported.
  User: Enter user (user@trialgroup)
  Password: Enter the password to access the Trial Master EDC system.
  Remote Study: Enter the name of the remote study to be accessed in the Trial Master EDC system.
   
• Veeva
  eTMF and operational data from Veeva can be imported into elluminate and used to configure and map data in the Operational Data Repository (ODR).
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the Veeva data is imported.
  Username: Enter the username to access the Veeva system.
  Password: Enter the password to access the Veeva system.
   
• Veeva CDMS
  Provides configuration options to establish connections to Veeva's Clinical Data Management System for importing operational data, enabling users to engage in 2-way querying with Veeva CDMS and access other beneficial tables like subject data, site data, forms, events, and more. This data source is used to create a corresponding import definition to import operational data for mapping, viewing in Data Central, and exporting from elluminate.
  
  Name and Description: Confirm they have been entered appropriately.
  Address/URL: Enter the entire URL from which the Veeva CDMS data is imported.
  Username: Enter the username to access the Veeva CDMS system.
  Password: Enter the password to access the Veeva CDMS system.
  Environment: Enter environment name.
  Remote Study: Enter the name of the remote study to be accessed in the Veeva CDMS system.

Note: After a data source has been defined, it can then be used in the Importer or Exporter modules.

Rave Integration Overview

As a clinical data repository, elluminate provides increasing value as more data is ingested. The elluminate importer supports three data import methods: manual file import, FTP server, and web service integration. Import methods may vary by vendor or source; however, web service integration is the recommended approach for high-volume and high-frequency sources such as EDC systems. This approach is especially applicable to Medidata Rave, where an out-of-the-box web service integration pulls data directly from Rave and supports near real-time reporting and analysis within elluminate. This integration also enables advanced elluminate modules, including Clinical Trials Operational Analytics and Data Central.

There are three sequential steps in establishing a web service integration between elluminate and Medidata Rave. Each step is explained in detail at the linked page.

Set Up Rave Web Service User Account 

Perform the following steps in Medidata Rave to setup a Rave user account to be used for the web service connection.

The user completing the following task must have access to the User Administration module within Rave and have experience in setting up and activating Rave user accounts.

1. Login to Rave, access User Administration and initiate the creation of a new user.
2. Complete the fields indicated below (in red) to create a generic user that is specific to the elluminate integration (such as "elluminate_integration") and not linked to an individual person. We recommend selecting a PIN that does not have a personal connection, as it is necessary to share this password while establishing and testing the web service integration.
   
    
3. Once complete, click the Update button to create the user and generate an Activation Code.
   
4. Copy the Activation Code.
5. Go the Rave URL in a separate Web Browser and activate the account
   
    
6. Click the Activate New Account link.
7. Click the Activate New Account link; copy the Activation Code and enter the PIN entered in Step 2 above.
   
8. Enter a password that is easily remembered. Once done, click the Save Password and Continue button.
   
9. The screen below appears if the password is accepted.  Click the Logout button in the upper-right corner.
   
10. Login to Rave as an Administrator. Access User Administration and change the "Password Expires" year to 9999, and click the Update button.
    
11. Go back to the account in Rave User Administration and update the following.  Assign the Study/Environment accordingly.  The role should have access to all fields with the study; typically, Power User.
    
     
12. Click the Update button after the Training section is done.

Performed in Rave, this establishes the credentials via which elluminate authenticates to pull the data from Rave.