European Epidemiological Forum

Real world data (RWD) has been described as data that are routinely collected within health systems on a patient’s health status, in a purely observational fashion and; real world evidence (RWE) presented as evidence derived from analysis of RWD. As the healthcare ecosystem around the world continues to face, pressures from an ageing population, an escalating drug development cost, an increased drug prices, limited/restricted reimbursement and/or push toward personalised/stratified medicine; pharmaceutical companies as well as many other stakeholders are looking to capitalise on the increased availability of RWD, powered by advances in information technology and epidemiological methods, to alleviate some of these pressures.

Advances in technology in particular, while increasing the availability of bigger and better data, has also facilitated faster computers with unprecedented storage power. The later has brought to life some of the advanced analytical tools such as ML/AI whose long-recognised promises, have for decades been hampered, among others, by computing power and limited data.

In the pharma industry, there is a call for increased use of RWE to help define, differentiate and deliver value across the drug life cycle. During this talk we will recall some of the key elements why the scientific community now believe ML/AI potentially has a critical role to play in drug development and drug commercialisation, this from the discovery phase with a better leverage of bigger and better omics and imagine data, right through the clinical development phase with better and innovative trial designs; to the commercialisation phase with an optimally customised healthcare, treatment and practices.

Digital health technologies – including those driven by artificial intelligence and machine learning – are providing new opportunities to operationalise previously untapped and rapidly growing sources of data for patient benefit. These new and potentially disruptive technologies are however posing challenges to regulators and health technology assessment bodies. How should the agencies assess safety, effectiveness and cost effectiveness of such technologies? Currently, such assessments are difficult because we lack guidance on what differentiates good from bad at the different stages of the healthcare-related ML/AI development pipeline, from design and data analysis, through to reporting, and evaluation of effectiveness/impact on health.

In this presentation we will look at some of the questions that healthcare decision makers should consider and NICE’s approach to evaluating new and emerging digital health technologies will be outlined.

Observational studies and real world evidence (RWE) in general are a unique resource that is difficult to ignore in our current approaches to better understand the advanced medications that we are developing. So far, statistical approaches and interpretation of observational data were largely based on techniques developed for traditional randomized clinical trials; and could be considered as an extension of the set of tools used in registrations trials.

Some RWE statistical methods such as propensity score matching were even developed in order to “simulate” a cohort randomization.

With the advent of AI/ML, for the first time, a set of tools never used in any other clinical research is coming on us, with a lot of promises but also some challenges that we will describe. AI/ML has the potential of being transformative, offering insights not accessible before with traditional analytical techniques.

However, the lack of transparency of algorithms and the “black box” ethos might impede the trust researchers could have in AI/ML.

This aspect coupled with the quality of data (garbage in, garbage out) is a barrier to further adoption.

A phenotype definition specifies cohorts of patients that exhibit certain phenotypic traits, such as the same diseases, characteristics, or set of co-morbidities. Electronic health record (EHR)-based phenotype definitions (also referred to as phenotyping algorithms) express the logic required to identify such a cohort from patient health records. Phenotype definitions can be represented in many forms, including narrative descriptions, pseudo-code, or in a standard structured format.

A high-quality definition is one that can be easily implemented across multiple organisations and datasets (portable); one that can be implemented accurately, to facilitate complementary research or validate existing results (reproducible); and one that is shown to effectively capture the disease or condition modelled (valid).

Phenotype libraries provide a platform through which a phenotype definition can be uploaded, stored, indexed, retrieved, and downloaded by users. Rather than just passively cataloguing definitions, these platforms can also impact the quality of the phenotype definitions they host.

One way to do this is for a library to facilitate parts of the phenotype development lifecycle often considered extraneous, including modelling, implementation and validation. For example, libraries that host definitions structured according to some set of standard models, and track the evolution of definitions under these models, often contain phenotypes that are clearer to understand and thus have the potential to be more reproducible.

We shall discuss a number of desiderata for the development of phenotype libraries, to ensure that future libraries are designed to maximise the quality of the phenotypes they contain.

Digital endpoints can be used based on data through various sources, such as apps, sensors, and wearables such as smart watches, for example, under the bring your own device model.

Digital therapeutics may be evaluated through randomized controlled trials (RCTs), pragmatic clinical trials (PCTs), or real-world data (RWD). The corresponding endpoints may be collected continuous in routine daily living and real-time activities, rather than settings such as doctors’ offices or hospitals. Digital endpoints (DTx) may be useful for remote patient monitoring and continuous tracking of patient responses and wellbeing through digitalization of patient diaries, electronic patient-reported outcomes (ePROs), mobility, medication use, and activities to generate real-world evidence (RWE) beyond RCTs. Therefore, they are useful in medical research via de-centralizing RCTs.

This presentation aims to: review the uses of digital endpoints and digital therapeutics in various types of studies; examine how cutting-edge digital innovation can potentially improve medication adherence; share some best practices by study type, RCTs, PCTs or RWD, for evidence-generation; discuss advantages and limitations.

Key words: Digital Endpoint; Digital Therapeutics; Randomized Controlled Trial; Pragmatic Clinical Trial; Real-World Data and Evidence; Electronic Patient-Reported Outcome.

A brief overview of methodology

1. Review the uses of digital endpoints and digital therapeutics in various types of studies
2. Examine how cutting-edge digital innovation can potentially improve medication adherence
3. Share some best practices by study type, RCTs, PCTs or RWE, for evidence-generation
4. Discuss the advantages and limitations of digital for remote monitoring and patient diversity

During the presentation the speaker with deal with the following topics:

• Increasingly complex payer landscape for access & reimbursement of new medicines.
• Increasingly complex innovative medicines (CAGT) with curative intent is “breaking” the model.
• Use of AI/ML by payers to integrate RWE and RWE/Real Time Data on conditional reimbursement models.
• Can we afford a cure? Paying for innovation – AI/ML will be the future to integrate value and access.
• How payers are using AI & ML to assess value and deploy innovative reimbursement models.

The CAPABLE (CAncer PAtient Better Life Experience) project (https://capable-project.eu/), funded in H2020 program, is realizing a novel system to improve the quality of life of cancer patients managed at home.

Project team is formed by experts in complementary fields, such as data- and knowledge-driven AI, data integration, telemedicine, and decision support, with the shared goal of fully exploiting Artificial Intelligence for cancer care and bringing the benefits right to patients’ homes. Within the CAPABLE framework, predictive models, based on both retrospective and prospective data and computer interpretable guidelines, are made available to oncologists.

Furthermore, CAPABLE’s Virtual Coach provides patients with decision support and lifestyle guidance to improve mental and physical wellbeing.

CAPABLE project offers the opportunity to show different aspects of the Real World Data lifecycle because:

• Data is collected both from Clinical Systems and Patient Generated Health Data sources (e.g., wearable sensors and smartphone applications).
• Data are organized and made available according to international standards (HL7 FHIR and OMOP CDM)
• Novel AI/ML methodologies are developed (multi-agent systems, knowledge-data integration and machine learning algorithms geared toward human behavior) to change the way cancer home care is delivered.

Risk prediction for Acute Coronary Syndromes (ACSs) represents an unmeet need in medicine. In particular, prediction of risk of death may be of help to tailor aggressive lifestyle interventions and close clinical and with imaging follow up. Moreover, these patients are themselves at increased risk of recurrent thrombotic events, but in the mean time, due to use of antiaggregant therapy they are exposed to elevated risk of bleedings. “Traditional” risk scores, derived with mainly with logistic regression, are derived from patients suffering both ACS and stable angina, consequently with limited reproducibility in every day clinical practice. Moreover, Artificial Intelligence (AI) and Machine Learning (ML), through exploration of non-linear complex relationships may help to capture other kind of associations.

In the PRAISE (PRedicting with Artificial Intelligence riSk aftEr ACS) study, 19826 patients were exploited to predict 1-year mortality, recurrent infarctions and major bleedings. Trials of four ML classifiers [Adaptive Boosting , Naive Bayes, KNN, Random Forest] were employed to generate four models for the prediction of each study outcome. The Adaptive Boosting, which performed superior to others, offered for 1-year all-cause death an AUC of 0.82 (95%CI 0.78-0.85) and 0.92 (95%CI 0.90-0.93). in the evaluation and the validation cohort, respectively. The PRAISE score for MI showed an AUC of 0.74 (95%CI 0.70-0.78) and 0.81 (95%CI 0.76-0.85) in the evaluation and the validation cohort, respectively. The PRAISE score for major bleeding showed an AUC of 0.70 (95%CI 0.66-0.75) in the evaluation cohort and 0.86 (95%CI 0.82-0.89). in the evaluation cohort and 0.86 (95%CI 0.82-0.89). in the validation cohort.