ERS superpixel guided pigment identification and convolutional autoencoder unmixing in mural painting hyperspectral images

Why These Ancient Colors Still Matter

Ancient mural paintings are more than decoration; their colors record trade routes, religious ideas, and lost technologies. Yet many of these wall paintings are too fragile to sample directly, and centuries of exposure to light, moisture, and salts have altered their original hues. This study presents a new way to "read" the pigments in a famous Buddhist cave mural in China without touching the surface, combining advanced imaging and artificial intelligence to reveal what the artists actually used and how those materials have aged.

Looking at a Wall with Many Eyes

Instead of photographing the mural with ordinary cameras, the researchers used hyperspectral imaging, which captures hundreds of narrow color bands from visible light into the short-wave infrared. Each tiny patch of the wall yields a detailed color fingerprint linked to specific materials. They focused on a richly painted section of Cave 171 in the Kizil Grottoes, an early Buddhist site along the Silk Road whose murals were made with both mineral and organic pigments and have suffered from wind, water, and salt damage. To anchor their interpretations, the team also prepared traditional pigment samples on boards, measured their spectra in the lab, and checked their composition with techniques such as X‑ray fluorescence and Raman spectroscopy. This created a reference library of 26 traditional pigments against which the mural’s signals could be compared.

Grouping the Picture into Smart Patches

A major challenge is that each image pixel often contains mixtures of pigments, and aging can blur the usual spectral clues. Rather than treating every pixel separately, the authors used an approach called superpixel segmentation. First, they applied principal component analysis to simplify the hyperspectral data. Then they used Entropy Rate Superpixel Segmentation to carve the mural into small regions whose spectra are internally consistent and follow painted boundaries. For each such region, they averaged the spectra and applied a mathematical step called continuum removal that highlights subtle absorption dips linked to particular pigments. They then compared these enhanced regional spectra to their pigment library using a similarity test. By combining how often a pigment appeared, how much area it covered, and how closely its spectrum matched, they could robustly pick out the main pigments present.

Teaching a Neural Network to Unmix Colors

Identifying which pigments exist is only half the story; conservators also want to know how strongly each pigment is present at each point on the wall. To do this, the team turned to an unsupervised convolutional autoencoder, a type of neural network that learns to compress and reconstruct the spectral image. In their design, the compressed representation directly encodes the proportions of a small set of fundamental spectra, or "endmembers," across the mural. To guide this learning, they first used a geometric algorithm called N‑FINDR to find a set of representative spectral extremes in the data and used these as pseudo-endmembers. The network was trained to reconstruct the image while staying close to these reference spectra and preserving the detailed shape of pigment fingerprints, helping it distinguish very similar reds and other same-color families.

What the Wall Reveals About Its Colors

Using this combined strategy, the authors identified six principal pigments in Cave 171: the blue stone lapis lazuli, the copper green atacamite, the reds minium, iron oxide red, and lac, and the white mineral gypsum. Maps of pigment abundance showed how these materials are layered and mixed: gypsum appears both as a white paint and as an underlying ground layer; lapis lazuli and atacamite cluster in blue and green zones; and the three reds overlap but can still be teased apart by their subtle spectral signatures. The method even helped flag areas where the lead-based red minium has darkened into a different compound, pointing to ongoing deterioration that conservators must monitor.

From Hidden Fingerprints to Practical Care

For non-specialists, the key outcome is that the authors have devised a way to turn complex, invisible spectral information into clear maps of what pigments were used and how they have changed, all without sampling the mural. By grouping similar regions before identification and constraining the neural network with physically meaningful pigment fingerprints, the workflow delivers reliable, spatially coherent pigment maps even when colors are mixed or degraded. This provides conservators with a powerful, non-contact tool to document materials, detect fading and alteration, and plan targeted interventions, helping to preserve the stories carried by the ancient colors of sites like the Kizil Grottoes for future generations.

Citation: Chen, W., Zhang, X., Pan, X. et al. ERS superpixel guided pigment identification and convolutional autoencoder unmixing in mural painting hyperspectral images. npj Herit. Sci. 14, 177 (2026). https://doi.org/10.1038/s40494-026-02450-6

Keywords: hyperspectral imaging, mural conservation, pigment mapping, deep learning, cultural heritage