Google-led Team Develops AI System to Write Expert-Level Scientific Software

Introduction of ERA

A research team at Google, co-led by Michael Brenner, a Catalyst Professor of Applied Mathematics and Physics at Harvard University, has developed an AI system designed to automate the creation of expert-level empirical software. The system, named Empirical Research Assistance (ERA), is intended to address a persistent bottleneck in the scientific discovery process: the slow and manual creation of software required to support computational experiments.

ERA is designed to create scientific software with the specific goal of maximizing a quality metric. Unlike standard code generation tools, the system is built to handle measurable research tasks across various scientific domains.

Technical Framework

The ERA system operates by combining Large Language Models (LLMs) with Tree Search (TS). This combination allows the system to automatically write, test, and refine scientific software. By utilizing tree search, the AI can explore different coding paths and iteratively improve the software based on the defined quality metrics.

The system creates expert-level scientific software with the goal of maximizing a quality metric.

Application and Performance

The system has been applied to a variety of complex scientific fields. Specifically, ERA has been used to write software for bioinformatics and COVID-19 forecasting. In these contexts, the system focuses on tasks where the output can be measured against a specific performance or quality benchmark.

Context in AI Science

The development of ERA occurs alongside a broader trend of automating the scientific process. Other organizations, including Sakana AI, are currently working toward the automation of the entire scientific workflow. Additionally, AI researchers are developing agent-based tools for scientific discovery that are intended to eventually integrate directly with physical laboratory equipment.

However, some analysis suggests that these advancements also highlight the inherent limits of current AI. Some observers note that while these systems are improving, language-based models alone may have fundamental limits when applied to the complexities of science.