Mike's Blog

Translating discoveries from early human trials into effective real-world health solutions is central to the CTSA Program mission. As translational science continues to evolve, rigor and reproducibility remain essential research pillars. Real-world data (RWD) and the resulting real-world evidence (RWE) are increasingly important for advancing research across the translational spectrum, offering opportunities to improve healthcare delivery, accelerate innovation, and strengthen evidence generation.

The rapidly expanding ecosystem of health-related data, including electronic health records, genomic, wearable devices, ambient sensors, geolocation and other multimodal data has created unprecedented opportunities for translational research. At the same time, the complexity, scale, and variety of these data sources present significant challenges related to data quality, interoperability, governance, and reproducibility. Successfully leveraging RWD requires adaptable infrastructure, robust analytical methods, and collaborative approaches that can evolve alongside advances in technology and data science.

To address these opportunities and challenges, NCATS in collaboration with Stanford Medicine and CCOS hosted a Collaborative Workshop, bringing together investigators, clinicians, bioinformaticians, ethicists, patient advocates, government representatives, RWD companies and translational science leaders for a day of discussion and collaboration.

Near perfect weather, nicely organized agenda, great coordination, and attendance led to a highly interactive and scientifically productive day. Morning sessions focused on foundational topics including the applications of RWD and RWE, study design and methodological challenges, governance models, and the importance of transparency and adaptability in data-driven research. Speakers emphasized that scientific findings may evolve over time as data sources mature and analytical approaches improve, underscoring the importance of maintaining openness and flexibility in research practices. Discussions also highlighted the growing need for workforce development, interdisciplinary training, and support for future RWD and RWE experts.

Afternoon breakout sessions focused on four major themes: governance; risk and trust; cultural issues, team science & training; and data characteristics. Participants stressed the importance of involving all stakeholders including patients, communities, healthcare systems, researchers, and private sector partners throughout the research process. There was broad agreement that patient and community engagement should extend beyond participation to include meaningful input into future research priorities and oversight.

Team science and workforce sustainability emerged as key priorities. Participants discussed the need for academic incentives that recognize collaborative research efforts, along with mentorship and specialized support systems to guide investigators navigating the increasingly complex RWD landscape. Suggestions included the development of research navigators who can assist teams in designing scientifically rigorous and clinically meaningful studies.

Data quality and interoperability were also identified as critical areas for future progress. Attendees emphasized the importance of common data models, metadata standards, and shared data integration practices to improve reproducibility, reduce variability, and support regulatory and clinical applications of RWD.

This is where the conversation is now moving from discussion to action. Building on these priorities, NCATS recently announced the Real-World Evidence Innovation Challenge, a national effort designed to establish validated benchmarks for analytic performance across health systems. The Challenge will use the National Clinical Cohort Collaborative, or N3C, as a secure, multi-site environment to test whether analytic approaches can generalize across real-world clinical settings before they are relied upon more broadly.

The Challenge reflects a central lesson from the workshop: the future of RWD and RWE depends not only on access to data, but on confidence that analytic methods produce reliable results, across different environments. Through N3C, analytic approaches can be evaluated across multiple health systems while preserving the real-world variation that makes evidence generation both powerful and challenging. This creates an opportunity to measure where, why, and how performance changes across settings; helping the field move toward Good Algorithmic Practice (GAP) for real-world data. As the goal is not simply to create more models, but to ensure that models and methods can produce trustworthy evidence when applied across the health systems, patient populations, workflows, and data sources where they are ultimately intended to be used.

The workshop concluded with a shared commitment to building consensus guidelines, shared governance and sustainable frameworks that support rigorous, reproducible, and impactful RWD research. Through our collective efforts, and through initiatives such as the Real-World Evidence Innovation Challenge, the CTSA Program is uniquely positioned to leverage its collaborative infrastructure, resources, expertise, and multidisciplinary approach to accelerate large-scale, rigorous, reproducible, and meaningful RWD research.

The question now is not whether real-world data can help transform translational science. The question is whether we can build the standards, infrastructure, and partnerships needed to ensure that real-world data generate reliable evidence for the benefit of all. With the CTSA Program, N3C, and the broader translational science community working together, we are ready to make that vision a reality.

If you can't explain it to a six year old, you don't understand it yourself.
- Albert Einstein