The Launch Sequence

Validating and synthesizing science

AI can help validate scientific outputs and synthesize a research corpus too large for any one scientist to digest.

Why this is a top priority

The scientific corpus, including the literature, shared software tools, and datasets, long ago became far too large for any one scientist to digest, and the reproducibility crisis has sown doubts about its trustworthiness. AI will only compound these problems as it dramatically increases the number of scientific artifacts while also potentially introducing serious flaws due to hallucinations or by adopting p-hacking, data fabrication, or other research misconduct behaviors. Philanthropies and the federal government can help confront these problems by supporting new tools aimed at validating scientific outputs and synthesizing the research corpus for human comprehension.

Additionally, not all fields of science will benefit equally from productivity gains from AI. Many areas of cutting-edge research have little prior knowledge to work with or rely on custom experimental and analytical approaches that are not yet captured by AI models. The federal government and philanthropies should continue to support these areas of research while also working with those researchers to understand and address their unique challenges, possibly through novel AI-driven approaches that could improve their boutique workflows.

What should be done

Philanthropies and government agencies

  • Philanthropies and government agencies like NIH, NSF, and DOE should support “unstructured data labs”: These labs employ cameras, keystroke loggers, and other passive forms of observation to record scientists as they work. These data streams can then be used to extract the tacit knowledge that scientists deploy as they perform experiments. See full proposal.
  • Philanthropies and government agencies like NIH, NSF, and DOE should validate human and AI-driven scientific outputs: The government and nonprofits should support the development of AI tools that validate new scientific claims by automatically checking their data and analyses as they are made available. See full proposal.
  • Philanthropies and government agencies like NIH, NSF, and DOE should support self-driving labs that test AI-driven scientific outputs: These robotic systems automate the design, execution, and analysis of experiments, closing the loop between AI-generated hypotheses and real-world validation. One example is in materials science, where a national self-driving labs consortium could be convened by the Materials Genome Initiative, going on to serve as a coordinator for research agendas that deploys funding from existing ARPA-E and CHIPS R&D budgets. See full proposal.
  • Philanthropies and government agencies like NIH, NSF, and DOE should develop tools to synthesize the scientific literature and use these summaries to steer funding decisions: As AI increases the output of new scientific artifacts, AI tools can also help to synthesize the scientific corpus, helping to identify gaps or “slow” areas of science.

Get in touch

If you are a grantmaker, entrepreneur, or policymaker interested in accelerating progress in validating and synthesizing science, get in touch by emailing science-validation.tls@ifp.org. We are not able to reply to all requests, but we read them all.

You can join our talent database to work on validating and synthesizing science here. We will use this database if we ever headhunt for talent to found or be an early employee in organizations working on this issue.

Join our Launch Sequence mailing list here to receive new proposals, important updates, and occasional opportunities straight to your inbox.

Project proposals and strategy research