Teaching future scientists to interrogate AI tools for scientific discovery

AutoDiscovery, our AI agent for scientific research, analyzes datasets, proposes hypotheses, runs experiments to test them, and ranks the results by Bayesian surprise—the gap between what the model predicted and what the data showed. In a University of Washington classroom this spring, it surfaced a possible flaw in how battery aging is measured, an unexpected pattern in silicon polymers, and a recurring feature in proteins that respond to light.

But a surprising result is not necessarily an important one—or one that will withstand further testing. Students still had to scrutinize the evidence, compare the findings with published research, and decide whether each result reflected a genuine discovery, a coincidence, or a problem with the data.

Luna Yue Huang, an associate teaching professor of materials science and engineering at the UW, invited students to explore this way of working with AI through the GenAI for Science: Ai2–UW Materials Challenge. Twenty-five teams proposed a dataset and research question; ten were selected to spend several weeks working with AutoDiscovery.

A UW instructional team and Ai2 researchers reviewed the proposals based on the richness and scientific distinctiveness of the data, their potential to produce meaningful findings, and the overall diversity of topics represented. Ten projects were selected, with students from other proposals joining those teams. In some cases, instructors also helped narrow broad datasets to a more manageable scope aligned with the students’ research interests.

Students tested AutoDiscovery’s hypotheses against the available evidence, identified gaps in its reasoning, and assessed whether it meaningfully reduced the amount of manual data analysis required. AutoDiscovery's performance varied considerably—in some cases, it surfaced leads the students might not otherwise have explored.  

Here are four representative projects from the class: