The hard parts of AI-assisted science

AI can already assist with a surprising amount of scientific work: search literature, analyze datasets, write code, and generate and test hypotheses. The harder problem is building systems scientists can steer as a research project unfolds, especially when new evidence changes which hypotheses to pursue, experiments to run, or questions to investigate next.

That challenge was the focus of an August 27 event at Ai2 marking an expansion of our work with the Paul G. Allen Research Center at Providence Swedish Cancer Institute. We’ve written separately about the cancer research that grew out of that collaboration. At the event, that work served as a starting point for a broader look at what today’s scientific AI still lacks.

Across the presentations and panel, five challenges kept resurfacing.

Majumder described scientific taste as the judgment that helps a researcher distinguish an interesting result from one that’s trivial, implausible, already understood, or unlikely to lead anywhere. Today’s AI-for-science systems don’t reliably make those distinctions on their own.