• Use Cases

Use case 1 : Real-World IVD Clinical Evidence

Use case overview

This use case is described in both

• The MDIC publication where an IVD manufacturer intends to perform an observation study for a given test (i.e. IVD system & reagent) in order to get clinical performance insight.

• The draft FDA guideline in appendix B presenting exemples were RWE is used

This draft guidance contains explicit data requirements described in sections A - 1, Data Availability (lines 385 – 417) and B - 1, Reliability, lines 467 - 513.

Actors

IVD manufacturers

May also be a regulatory body or a medical laboratory rep.

Preconditions

Lab data

• from one or multiple medical laboratories are made available with a sufficient granularity to enable targeted analytics of the test and IVD system of interest.

• are harmonized and standardized across the data providers

Relevant patient data 

This use case is detailed in the following situation

Use Case1-A 

Use case overview

As illustrated in fig 2 of MDIC publication either of the two situation may be considered

• patients leftover samples are tested with a candidate test followed by a comparator test (e.g., gold standard) according to routine clinical practice. 

• patients leftover samples are tested with a candidate test and compared to clinicals status

Actors

IVD Manufacturer

Medical Laboratory

Data Hosting Service(s) - Identified RW Data Source

Preconditions

Lab performs test as part of routine clinical practice 

Patients have targeted condition/disease for the IVD 

Comparator test or clinical status must be available

Assumption

1. Candidate and Comparator Test are run for each patient sample leftover OR Candidate test is run for each patient sample leftover and patient clinical status is known

2. Populations are characterized and comparable

3. Data Sources are equivalent (i.e., the data is consistently represented and available)

4. Test Target is the same 

Flow

1. Candidate Test is run for each patient sample 

2. Comparator Test is run for each patient sample

3. Candidate and Comparator observations & results are sent to the Lab IS

4. Patient & lab data are transferred to a repository suitable for data analytics

5. Data analytics is performed across one or several repositories

Use Case 1-b - Assess Health Technologies

Use case overview

This use case is described in the MDIC publication where a regulatory body intends to perform an observation study for a given test (i.e. IVD system & reagent) .

Actors

Regulatory body

Preconditions

Lab data

• from multiple medical laboratories are made available with a sufficient granularity to enable targeted analytics of the test and IVD system of interest.

• are harmonized and standardized across the data providers

Relevant patient data are gathered from XXX

Flow

TBD

Use case 2 : Aggregate data from numerous data source

Use case overview

UC1 and 2 require that laboratory data and patient data be aggregated in a data repository while preserving data ownership, data privacy according to XXX

Actors

Medical Laboratory

Data Hosting Service(s) - Identified RW Data Source

Preconditions

• Standardized and harmonized data are produced at the data source and conveyed without losses across all the health care chain : IVD system, LIS, EHR, etc.

Alternatives

• Data are made available and aggregated in a centralized repository

• Data are made available through a network of federated databases, leaving the data under the control & ownership of the production site (i.e. hospital or laboratory)

Flow

TBD

Baseline high level requirements

Use Case

Use case overview

Patients are tested with a candidate test followed by a comparator test (e.g., gold standard) according to routine clinical practice.

Actors

IVD Manufacturer

Medical Laboratory

Data Hosting Service(s) - Identified RW Data Source

Preconditions

Lab performs test as part of routine clinical practice 

Patients have targeted condition/disease for the IVD 

Comparator test must be available 

Assumption

5. Candidate and Comparator Test are run for each patient sample 

6. Populations are characterized and comparable

7. Data Sources are equivalent (i.e., the data is consistently represented and available)

8. Test Target is the same 

Flow

6. Candidate Test is run for each patient sample 

7. Comparator Test is run for each patient sample

8. Candidate and Comparator observations & results are sent to the Lab IS

9. Patient & lab data are transferred to a repository suitable for data analytics

10. Data analytics is performed across one or several repositories

Baseline data high level requirements

(neither functional nor non-functional requirements)

*Jurisdictional requirements must be followed.  Those are not captured in this set of requirements.

Actors

Systems -  RW Data Sources

• Electronic health records (EHRs)

• Laboratory information systems (LISs)

• Administrative claims databases

• Hospital discharge databases

• Disease registries

• Medical device registry

• Image databases and repositories

• Patient-reported outcomes (PROs)

• Death indexes

• Public health databases (e.g., Centers for Disease Control and Prevention)

Other

• AI - Where do we put this

• Synthetic Data 

RWD/RWE - 

Use Cases in Regulatory decision making

Support premarket authorization

• In settings where clinical studies are impractical to conduct (e.g., rare diseases, long follow-up needed to achieve endpoint(s), cost)

• For filling important evidentiary gaps not typically addressed with traditional clinical studies

• When allowing sponsors to generate evidence in support of an effectiveness claim that is potentially useful to multiple stakeholders, regulators, patients, and the public

• For potentially reducing time and cost of evidence development for regulatory decisions

Premarket Use Cases

• Evidence to identify, demonstrate, or support the clinical validity of an IVD device

• Generating performance characteristics as clinical sensitivity and specificity

• Evidence to support an Investigational Device Exemption (IDE) submission for a significant risk IVD

Study Design Types

• Registry embedded

• Feasibility planning

• External Control of clinical study

• Pragmatic trial

• Clinical Performance studies - RWD is suitable for Observational Studies

• Examples

  • Virtual clinical performance study.  Patients are tested with a candidate test followed by a comparator test (e.g., gold standard) according to routine clinical practice.

  • Scheme for RWD aid in diagnosis and follow-up.  RWD are intended to demonstrate the claim extension of an “aid in diagnosis” IVD for risk assessment/prognosis of a given disease or target condition.

  • RWD for an AI system as a binary test. RWD are from routine clinical practice in which positive results will be referred for additional testing.

  • Virtual clinical study for an AI system as a test with three categories. In this system, the positive category is divided into two categories (medium positive and high positive).

  • New IVD test using AI/Machine Learning. RWD are “multiple individual tests,” and the virtual study is a “new IVD device using AI/machine learning.”

R&D Uses

• Site or patient identification – for enrollment in clinical studies

• Feasibility studies – to test protocols for new products or a potential new intended use

• Selection of historical control, comparator, or predicate device(s) for clinical study

• Quality improvements – surveillance of specific postmarket IVD performance factors

• Meeting design control requirements – for example, RWD could be used as design input for new product development

• New uses and expanded indications for IVD devices – safety and effectiveness of an assay outside of its intended use (i.e., “off-label” use), extending indications to other patient populations, tissues, or sample types; RWE can provide evidence for identifying and supporting new claims or otherwise expanding the labeling of an IVD

• Methods selection – use of RWD to simulate clinical trial results or outcomes of testing prior to implementing a costly clinical trial

• Review and comparison of safety and effectiveness endpoints – Objective Performance Criteria and Performance Goals derived from clinical studies and/or registries may be used

• Redefining current use models, for example, to define new scoring methods in histology or to define new cutpoints for existing assays

IVD Data Characteristics

• Inclusion of data from heterogeneous patients with a wide range of characteristics, experiences, comorbidities, and treatment protocols. 

• Target Patient population identified in the IFUs - should be included in the analysis; and any additional patient populations identified as to the IVDs performance 

• RWD may also include pre-analytical variables (e.g.,specimen collection containers, sample storage, and handling).

• Not all RWD are of appropriate quality

• RWE is fit for use in regulatory submissions

• Level of granularity needs to be captured/documented in study design

Can we define which IVD data in the RWD Sources will be acceptable for use in various use cases?

• Data Quality

• Validity

• Completeness

• Fit for Purpose

Consider data granularity

• Data lost in downstream workflow (e.g., LOINC codes)

• Data not exchanged from the originating system 

Regulatory requirements - 

• Reporting Results in US: ​https://www.fda.gov/regulatory-information/search-fda-guidance-documents/statistical-guidance-reporting-results-studies-evaluating-diagnostic-tests-guidance-industry-and-fda