At first glance, a recent batch of research papers produced by a prominent artificial intelligence lab at the University of British Columbia in Vancouver might not seem that notable. Featuring incremental improvements on existing algorithms and ideas, they read like the contents of a middling AI conference or journal.
But the research is, in fact, remarkable. That’s because it’s entirely the work of an “AI scientist” developed at the UBC lab together with researchers from the University of Oxford and a startup called Sakana AI.
The project demonstrates an early step toward what might prove a revolutionary trick: letting AI learn by inventing and exploring novel ideas. They’re just not super novel at the moment. Several papers describe tweaks for improving an image-generating technique known as diffusion modeling; another outlines an approach for speeding up learning in deep neural networks.
“These are not breakthrough ideas. They’re not wildly creative,” admits Jeff Clune, the professor who leads the UBC lab. “But they seem like pretty cool ideas that somebody might try.”
As amazing as today’s AI programs can be, they are limited by their need to consume human-generated training data. If AI programs can instead learn in an open-ended fashion, by experimenting and exploring “interesting” ideas, they might unlock capabilities that extend beyond anything humans have shown them.
Clune’s lab had previously developed AI programs designed to learn in this way. For example, one program called Omni tried to generate the behavior of virtual characters in several video-game-like environments, filing away the ones that seemed interesting and then iterating on them with new designs. These programs had previously required hand-coded instructions in order to define interestingness. Large language models, however, provide a way to let these programs identify what’s most intriguing. Another recent project from Clune’s lab used this approach to let AI programs dream up the code that allows virtual characters to do all sorts of things within a Roblox-like world.
The AI scientist is one example of Clune’s lab riffing on the possibilities. The program comes up with machine learning experiments, decides what seems most promising with the help of an LLM, then writes and runs the necessary code—rinse and repeat. Despite the underwhelming results, Clune says open-ended learning programs, as with language models themselves, could become much more capable as the computer power feeding them is ramped up.
“It feels like exploring a new continent or a new planet,” Clune says of the possibilities unlocked by LLMs. “We don’t know what we’re going to discover, but everywhere we turn, there’s something new.”
Tom Hope, an assistant professor at the Hebrew University of Jerusalem and a research scientist at the Allen Institute for AI (AI2), says the AI scientist, like LLMs, appears to be highly derivative and cannot be considered reliable. “None of the components are trustworthy right now,” he says.
Hope points out that efforts to automate elements of scientific discovery stretch back decades to the work of AI pioneers Allen Newell and Herbert Simon in the 1970s, and, later, the work of Pat
Langley at the Institute for the Study of Learning and Expertise. He also notes that several other research groups, including a team at AI2, have recently harnessed LLMs to help with generating hypotheses, writing papers, and reviewing research. “They captured the zeitgeist,” Hope says of the UBC team. “The direction is, of course, incredibly valuable, potentially.”
Whether LLM-based systems can ever come up with truly novel or breakthrough ideas also remains unclear. “That’s the trillion-dollar question,” Clune says.
Even without scientific breakthroughs, open-ended learning may be vital to developing more capable and useful AI systems in the here and now. A report posted this month by Air Street Capital, an investment firm, highlights the potential of Clune’s work to develop more powerful and reliable AI agents, or programs that autonomously perform useful tasks on computers. The big AI companies all seem to view agents as the next big thing.
This week, Clune’s lab revealed its latest open-ended learning project: an AI program that invents and builds AI agents. The AI-designed agents outperform human-designed agents in some tasks, such as math and reading comprehension. The next step will be devising ways to prevent such a system from generating agents that misbehave. “It’s potentially dangerous,” Clune says of this work. “We need to get it right, but I think it’s possible.”
Read the full article here
