A gathering of eight rural-state Clinical and Translational Science Awards (CTSA) institutions in October 2023 turned out to be especially fruitful for Jessica Presley, M.P.P., TRI senior director of evaluation, and her counterparts.
In fact, the connections she made at the Consortium of Rural States (CORES) annual meeting resulted in a significant collaborative project that will be presented at the annual meeting of the American Evaluation Association (AEA) on October 26, 2024 in Portland, Oregon.
In addition to University of Arkansas for Medical Sciences (UAMS), CORES members are the University of New Mexico Health Sciences Center, University of Kansas, University of Kentucky, University of Utah Health, University of Iowa, Pennsylvania State University and Dartmouth College.
Presley and other CORES evaluators will co-present: “Rural Health Impacts: Leveraging the Translational Science Benefits Model (TSBM) to Capture Larger Research Consortium Trends.”
The TSBM was developed in 2018 by Douglas A. Luke, Ph.D., at Washington University in St. Louis as a tool for measuring and showcasing the real-world impacts of research. The tool helps researchers and funding agencies assess the benefits of research beyond scientific presentations and manuscripts. By tracking societal benefits, the TSBM helps ensure that research expands beyond knowledge generation to societal impacts. The TSBM model has been widely adopted by evaluators across the CTSA consortium, including by Presley in her team’s evaluation of TRI.
The AEA presentation represents the completion of the first phase of the CORES project.
“We have compiled translational research activities that impact rural communities from our CORES institutions and coded them as a potential or realized clinical, community, economic, or policy impact according to the primary TSBM domains,” Presley said. “Our goal is to identify any common trends or gaps in TSBM impacts in our rural communities to either inform future research in our rural communities or adapt the TSBM model to ensure and improve alignment with rural communities to evaluate impact."
Presley will be joined at the meeting by colleagues Alex Jauregui-Dusseau, DH.Sc., and Shani Worrell, Ed.D. For TRI’s evaluation, Presley’s team is applying the TSBM approach to retrospectively analyze projects from past CTSA award cycles.
“Moving forward, we are asking TRI’s K12 and T32 trainees to identify the potential TSBM impacts their research could produce,” she said. “Identifying impacts early in the research process will allow researchers to be more intentional in working towards larger societal impacts. This will also allow evaluators to actively track indicators of impact throughout the research process.”
With more than 10 years of experience as a program evaluator, Presley said her three years as TRI’s senior director of evaluation have presented exciting new challenges that should benefit from collaboration with other rural-state CTSA evaluators.
In her team’s evaluation of nearly two dozen TRI programs and initiatives, her work includes determining how effective they are at:
- Preparing a diverse group of early career researchers to become independently funded researchers, implementation scientists, entrepreneurs and mentors
- Establishing and expanding community-based research that involves partnerships between UAMS-affiliated faculty and community-based organizations
- Expanding research and programs that benefit the health of rural populations
“The TRI’s scope is expansive and wide-ranging, so evaluation of the TRI is multi-faceted and can be challenging,” Presley said. Her fellow evaluators are in the same boat, which has inspired a strong spirit of collaboration.
“We are coming together and learning from each other and collaborating on approaches to address the unique challenges rural-state CTSAs face,” said Presley, who is pursuing a Ph.D. in interdisciplinary evaluation.
The evaluators have set up a repository for sharing numerous resources, including some of the survey instruments they are using.
Presley noted that TRI’s External Advisory Board last year praised TRI’s rural research efforts but wanted to know how TRI was measuring impact in rural communities. She learned that TRI wasn’t the only CTSA struggling with the question.
“We realized that it’s been a challenge for all of us as evaluators,” she said.
In response, the CORES evaluators began their collaborative TSBM project, which first maps all of the activities that the individual institutions are doing within rural populations and will determine the health and social benefits that resulted from their projects.
“We’re comparing and contrasting and doing a deep dive to see what conclusions we can draw from our rural research efforts,” Presley said. “For example, can we develop some best practices based on what we learn to help us broaden impact within our rural communities?”
