2026 Fall CTSA Program Annual Meeting

The National Center for Advancing Translational Science (NCATS) supports an NIH K12 award for career development in translational science. As part of the administration of this award, NCATS requires individual CTSA hubs to track K12 awardees long after they graduate from the program. At the University of Illinois Chicago’s (UIC) Center for Clinical and Translational Science (CCTS) efforts to track scholars after successful completion of the program have primarily included scholar self-report and direct scholar searches by K12 program and Evaluation & Tracking staff.
Utilizing the self-report survey, former scholars can report their promotion and tenure, publications, and successful grant applications through a survey distributed by the K12 program staff. However, the survey requires substantial burden for former scholars to complete, and the response rate across all former scholars is 11.9%. The other option CCTS staff have used to report scholar success is through direct searches of scholar activity utilizing databases such as PubMed, NIH RePORTER, and Flight Tracker alongside personal profiles such as LinkedIn and internal university data. Using CCTS staff to conduct this work shifts the burden away from scholars but represents a substantial financial investment (approximately .42 FTE) on the part of K12 program and Evaluation & Tracking staff.
To reduce the financial investment required to adequately track scholars for reporting purposes and enhance our ability to quickly and accurately identify scholar impact, we worked with UIC’s AIHealth4All team to build a human-in-the-loop AI agent. The agent goes to work assembling a portfolio of evidence for each scholar including key scholar profiles, media mentions, career milestones, successful grant applications, and one sentence Translational Science Benefits Model (TSBM) impact summaries sourced from the evidence assembled by the agent. CCTS staff reviews each portfolio, including TSBM impact summaries, for accuracy and accepts, edits, or rejects the findings of the agent. Each CCTS staff reviewer accepted or edited 81.7% of the agent's findings and spent a median of 14 minutes per scholar, replacing an estimated 15 hours of manual assembly per scholar. Inter-rater agreement was moderate (Cohen's kappa 0.43 on the usable-versus-reject decision). The agent assembled evidence that covered all four TSBM domains.
By utilizing the AI agent, we were able to reduce the average investment of human hours into K12 scholar tracking by 98% per scholar, leading to increased efficiency and redirected staff effort. In the future, we plan to expand the use of the agent within CCTS and test the agent with other CTSAs to ensure the agent is able to work with other programs.