Operating advanced scientific instruments with AI agents that learn on the job

Smarter Machines for Everyday Science

Modern scientific tools, from powerful X-ray microscopes to robot-run chemistry labs, can collect data faster than ever before. But running these machines still demands expert attention and careful setup, which limits who can use them and how quickly discoveries are made. This paper explores how a new generation of artificial intelligence "agents" can learn alongside human scientists to operate complex instruments more safely, flexibly, and efficiently.

From Button-Pushing to Learning Partners

Today’s advanced labs automate many routine steps, but true independence—planning experiments, interpreting images, and deciding what to do next—remains out of reach. The authors build on large language models, the same kind of AI that powers modern chatbots, and turn them into goal-driven agents that can write code, call software tools, and react to images and video. Instead of replacing scientists, these agents are designed to work in a “human in the loop” fashion: people describe what they want, provide corrections when needed, and the AI remembers those lessons for future use.

A Digital Co-Pilot for an X-Ray Microscope

The first test bed is a hard X-ray nanoprobe beamline, a machine that uses focused X-rays to map the structure of materials at the scale of billionths of a meter. Because of radiation hazards, no one can be in the room while it runs, and the entire setup is controlled by specialized software and scripts. The team connects several AI agents to this control system. One agent writes the commands needed to scan a sample; another reviews the code for safety and correctness; a vision agent looks at the resulting images to suggest where to zoom in for a closer look. The agents must not only translate plain-language requests into correct scan commands, but also read tiny bright spots in diffraction and fluorescence images and turn their locations into precise coordinates for new scans.

What the AI Got Right—and Where It Struggled

By comparing several leading vision-capable language models, the researchers find clear differences in performance. Some models are good at following text instructions and calling the right function, especially after they are shown examples and corrected by humans. Others are much better at visual reasoning, such as pinpointing isolated bright particles and avoiding crowded clusters when choosing where to scan next. The standout model in this study combines both strengths and shows the most consistent behavior across repeated trials. However, the authors also find that giving feedback mainly improves text-based skills like parameter selection; it does not magically fix weak visual understanding.

Teaching Robots to Run Their Own Experiments

The second test bed is an automated station for making thin polymer films, used in electronics and energy devices. Here, a robotic arm equipped with different grippers moves vials, substrates, pipette tips, and a blade-coating tool between various stations. The researchers strip away most high-level routines and give the agents only low-level commands and a map of where everything is. The AI must read a scientific paper to extract suitable coating conditions, plan a long chain of steps—like picking up a substrate, placing it in the coater, uncapping a vial, dispensing solution, and spreading it under controlled temperature and speed—and then generate and verify the robot code to carry this out. Human supervisors approve each new sequence, and their corrections are stored in a memory system so the agents can reuse successful patterns later.

Building Toward Self-Driving Laboratories

Across both the X-ray beamline and the robotic fabrication platform, the study shows that AI agents can learn on the job, gradually turning expert guidance into reusable know-how. With a structured memory and safety checks that limit what the AI is allowed to do, these systems can become more dependable over time while still leaving critical decisions in human hands. The authors emphasize that true fully automatic labs will require more work on calibration, long-term memory management, and handling tasks that still depend on human dexterity and intuition. Even so, this work points to a future in which scientists spend less time wrestling with complex controls and more time asking ambitious questions, as AI partners help run the instruments that turn ideas into discoveries.

Citation: Vriza, A., Prince, M.H., Zhou, T. et al. Operating advanced scientific instruments with AI agents that learn on the job. npj Comput Mater 12, 160 (2026). https://doi.org/10.1038/s41524-026-02005-0

Keywords: AI agents, self-driving laboratories, scientific instrumentation, robotic experiments, multimodal large language models