Here’s an interesting offshoot of the ResearchScorecard database: NIH ScoreTracker, a data collection tool intended to help research institutions assess the impact of NIH-sponsored research.

In the specific case at hand, one of our clients needed to identify which grant proposals and publications benefited from services partly paid by funds from the NIH’s Centers for Clinical and Translational Science Awards (CTSA) program. For interesting social reasons, universities have historically been rather poor at tracking the impact of a funder’s support, even one as important as NIH.

For university service providers, this is an especially troublesome problem, because they often service many parts of an institution’s research effort but can’t easily point to specific contributions, for example, providing statistical consulting that eventually helps land a grant proposal.

This gets especially interesting for CTSA grant recipients, because NIH requires that they demonstrate that the funds are indeed impacting the biomedical research process, as is the program’s goal. Problem is, how does one do that? NIH doesn’t provide specific metrics, such that it is up to grantees to identify them, assuming they have the necessary data. In my experience, this is simply not the case, and yet the grantees must satisfy the NIH’s very reasonable request in some way. Interesting, non? Well, OK, perhaps only if you’re a data geek.

Our solution has been to develop a tool that automatically mails a group of researchers with a URL to a survey which is automatically populated by the ResearchScorecard database. NIH ScoreTracker makes it very quick and easy for the busy researcher to identify which publications and grants benefited from an institutional service, such as the services of a statistician paid by CTSA funds. It does so by boiling everything down to a Y/N format — the researcher never has to enter anything else (see screenshots).

Collecting impact for publications

Collecting impact for grants

It doesn’t get any simpler than that, and this is also true for the administrators seeking the data. All they need to do is identify the group of researchers they want to contact, and NIH ScoreTracker will automatically e-mail each researcher until they take the survey. In the absence of a response, NIH ScoreTracker will eventually conclude that an answer is not forthcoming, at which point the administrators are contacted so that they can use their charms to remedy the problem. This is all done automatically.

Beyond tracking successful grants and publications, which aren’t terribly informative measures of the impact of research upon medical practice, more innovative approaches can be envisioned, such as counting literature mentions of techniques and products derived from an institution’s research. For example, much of microarray technology was invented at Stanford University. ResearchScorecard’s database can answer question such as “what is the growth of mentions of microarray-based approaches in the literature?”, which can serve as a proxy for estimating the impact of that research on biomedical research, where impact means “changing the way research is done”. A more involved version of this analysis estimate the dollars spent on the technology over the years to generate an ROI figure. Watch this space for future tools of this nature.