Upconversion optical entropy encoding for infrared complex-amplitude imaging

Seeing the Invisible Heat Around Us

From cars driving at night to tiny structures inside living cells, much of the world around us gives off invisible infrared light. Capturing detailed movies of this “heat light” would transform fields from autonomous driving to medical imaging, but today’s infrared cameras are expensive, power-hungry, and often slow. This paper introduces a new way to turn faint infrared scenes into sharp, video-rate pictures using ordinary visible-light camera chips and a dash of intelligent optics and AI.

Turning Heat Into Visible Light

Standard infrared cameras rely on specialized materials that must often be cooled to very low temperatures, which makes them bulky and costly. An appealing alternative is to “upconvert” infrared light into visible light that cheap silicon sensors can detect. Existing upconversion methods fall into two camps. Coherent techniques preserve the fine details of the light but demand powerful lasers and careful alignment. Incoherent methods, based on special glowing materials, are simpler and work with much weaker light, but they throw away the wave-like part of light known as phase. The new work combines the strengths of both worlds: it keeps the simplicity and sensitivity of glowing materials while regaining access to the hidden wave information.

Scrambling Light to Reveal More

The heart of the approach is an idea called optical entropy encoding. The researchers first pass the incoming infrared scene through a rough piece of ground glass that scatters the light into a random-looking speckle pattern. This “scrambler” acts like a complex code, mixing together both the brightness and the wave shape of the light. Next, a thin film containing lanthanide ions absorbs this speckled infrared light and re-emits it as visible light through a stepwise upconversion process. A standard silicon camera records only the intensity of this visible speckle, which by itself looks meaningless. However, because the scattering pattern is rich and complex, it secretly carries enough information about the original scene’s brightness and phase to be decoded later.

Letting a Neural Network Do the Decoding

The tricky part is turning the recorded speckle patterns back into a usable image of the scene. There is no simple formula linking the scrambled visible speckle to the original infrared light. Instead, the team trains a deep neural network, called S-ULRnet, to learn this connection from examples. They feed the network many pairs of known infrared patterns and their corresponding upconverted speckle images. Over time, the network learns to reconstruct both brightness and phase from a single snapshot. The authors also show that by tuning how strongly the ground glass scrambles the light—essentially increasing the “entropy” or information content of the speckle—they can significantly boost reconstruction accuracy.

Crisp Movies from Faint Infrared Signals

Once trained, the system performs impressively. It recovers detailed, 8-bit grayscale images of both brightness and phase at a video rate of 25 frames per second, all from a single camera exposure per frame. The setup can detect extremely weak infrared power, down to about 0.2 nanowatts per square micrometer—around a thousand times more sensitive than many conventional upconversion approaches. The team demonstrates real-time videos of natural scenes, moving number sequences, and even speed-limit road signs, which can then be accurately classified by a separate recognition network. This shows how the system could plug into practical tasks like autonomous driving or intelligent surveillance.

A New Pathway for Smart Infrared Vision

In plain terms, the researchers have built a smart translator that turns hard-to-detect infrared light into information-rich visible patterns, then lets AI read those patterns to reconstruct what the infrared scene looked like. Their method is fast, highly sensitive, and uses relatively simple hardware, making it attractive for applications ranging from medical diagnostics to environmental monitoring. Because the glowing materials used can respond to different infrared colors, the same concept could be extended to multiple wavelength bands and more advanced imaging modes. As a result, this work points toward future cameras that see heat and structure with remarkable detail, using affordable components and intelligent decoding.

Citation: Zhu, Sk., Pan, T., Tang, Cx. et al. Upconversion optical entropy encoding for infrared complex-amplitude imaging. Light Sci Appl 15, 158 (2026). https://doi.org/10.1038/s41377-026-02215-7

Keywords: infrared imaging, upconversion, speckle encoding, neural network sensing, short-wave infrared