Accurate and fast event-based shape measurement of mixed reflectance scenes

Seeing Shiny and Matte Worlds Together

From phone face unlock to factory robots, many devices now rely on cameras that can sense depth, not just color. Yet these systems often stumble in the real world, where dull walls, glossy plastics, and mirror-like metals all sit side by side. This research presents a new way to capture the 3D shape of such mixed scenes quickly and with high accuracy, using a special “event-based” camera and a scanning laser so that even shiny and moving objects can be measured in detail.

Why Shiny Things Are So Hard to Measure

Most 3D cameras are built for one type of surface at a time. Techniques that project patterns and watch how they deform work well on matte objects, because light scatters in many directions and is easy to see from the camera. But they fail on mirrors and polished metal, where light bounces in just one direction like a billiard ball on a pool table. On the other side, methods that excel on mirrors, such as deflectometry using a big display as a known light source, struggle with partially shiny objects and require bulky, carefully calibrated setups. Real scenes, like car interiors or living rooms, mix all of these surface types, which makes today’s systems slow, fragile, or incomplete.

Event Cameras and Scanning Beams

The authors build a compact system from only two parts: an event-based camera and a laser that sweeps thin lines across the scene. Unlike a normal camera, which captures full images at fixed times, an event camera only reports pixels where the brightness changes, and it does so with microsecond timing. As horizontal and vertical laser lines sweep across the scene, the camera records a precise stream of “events” marking where and when the light moved. By combining these sweeps, the system can infer depth on matte areas through triangulation, similar in spirit to structured-light scanners but at higher speed and with better tolerance to glare and changing room lighting.

Turning Every Wall Into a Virtual Screen

The key idea of the work is to reuse the measured matte parts of the scene as a kind of virtual display for analyzing shiny objects. First, the system reconstructs the geometry of all diffuse surfaces using the direct reflections of the laser. Those surfaces then act as light sources: when the laser hits them, they scatter light toward nearby mirrors and glossy items, which in turn reflect light into the event camera. By comparing the timing and geometry of these two-bounce paths with what was already learned from the matte surfaces, the method can estimate the slopes and shapes of specular objects without needing any physical screen. In effect, “everything around becomes a screen,” and the coverage of shiny surfaces can be increased simply by adding or moving everyday diffuse objects.

Sorting Out the Many Ways Light Can Travel

To make this work, the system must separate different light paths that all mix together in the event stream. The authors use geometric rules, known as epipolar constraints, to decide whether a detected event likely comes from a single-bounce reflection suited for triangulation, a two-bounce path useful for specular shape recovery, or a more complicated multi-bounce or subsurface path that should be discarded. They also design an optimization procedure that refines the shape of shiny objects so that the inferred surface normals agree with the observed light directions. Tests with spheres, mirrors, balloons, and glossy toys show that depth errors stay below 0.6 millimeters on both matte and specular surfaces, and that the system can operate at around 14 3D frames per second for mixed scenes and up to 250 frames per second for purely diffuse scenes.

What This Means for Future 3D Cameras

This approach points toward a new class of depth sensors that can handle cluttered, reflective, and even moving environments with a single compact device. By combining event-based sensing with clever laser scanning and by treating surrounding walls and objects as virtual screens, the method bridges a long-standing gap between technologies tuned for either matte or mirror-like surfaces. While there are still limits, such as relying on some diffuse material in the scene and dealing with rare complex reflections, the results suggest practical paths for more reliable 3D vision in applications ranging from AR and VR headsets to robotic inspection of shiny industrial parts.

Citation: Dashpute, A., Wang, J., Taylor, J. et al. Accurate and fast event-based shape measurement of mixed reflectance scenes. Nat Commun 17, 4407 (2026). https://doi.org/10.1038/s41467-026-72254-6

Keywords: 3D imaging, event camera, mixed reflectance, specular surfaces, structured light