An improved semantic and sketch biconditional guided image inpainting model for Chinese landscape painting

Breathing New Digital Life into Fragile Masterpieces

Ancient Chinese landscape paintings, treasured for their misty mountains and flowing brushwork, are also fragile objects: centuries of handling, insects, and humidity leave them riddled with cracks, wormholes, and missing patches. Conservators must balance saving what remains with avoiding further damage. This study introduces an artificial intelligence (AI) method designed specifically to help restore these paintings in the digital realm, filling in losses while staying faithful to the artist’s original structure and style.

Why Old Paintings Need New Technology

Traditional Chinese landscapes follow two broad traditions: the Northern School, with its bold, towering peaks, and the Southern School, known for soft ink washes and empty space. Both rely on subtle variations of line and tone that are easily disrupted when the paper or silk deteriorates. Manual touch-ups are time‑consuming and irreversible; one wrong stroke can permanently alter a masterpiece. Earlier digital techniques either copied nearby pixels or relied on general‑purpose photo tools. They could patch holes in a photograph of a street, but they often failed on paintings, producing awkward rocks, broken tree trunks, or brushwork that simply “felt wrong” to experts.

How the New AI Learns Structure and Meaning

To overcome these limits, the authors designed a restoration system that looks at a painting in three complementary ways at once. First, it extracts a detailed “sketch” showing the strength of every line, from bold mountain ridges to faint texture strokes, using an edge‑detection network tuned to preserve delicate ink transitions. Second, it builds a color‑coded map of what each region represents—sky, water, rock, foliage—using an unsupervised semantic segmentation model. Third, it analyzes the partially masked image itself. These three streams are fused and fed into a Transformer, a powerful AI architecture originally developed for language, which predicts how the missing patches should look so that they fit with both the underlying structure and the overall scene.

Teaching the AI to Imitate Brushwork, Not Just Shapes

Matching the composition is only half the challenge; the fill‑in must also match the artist’s hand. The team therefore added a lightweight stylistic feature extraction module that focuses on the subtle qualities of brushwork and ink—how strokes taper, how texture builds on rock faces, how washes fade into blank paper. This module distills style‑relevant information from the visible parts of the painting and injects it at multiple stages as the system reconstructs the missing regions, nudging the output toward the same rhythm and tonality as the original. Training is guided by a composite objective that penalizes not only pixel‑level errors, but also mismatches in perceived structure, texture statistics, and overall style, encouraging results that look “right” to the human eye, not just to a calculator.

Putting the Method to the Test

To evaluate the approach, the researchers assembled a large dataset of nearly 5,000 high‑quality landscape paintings from public museum collections and a public benchmark, balancing Northern and Southern School works. They digitally applied irregular masks mimicking real damage—small losses, broad scrapes, and clustered wormholes—and compared their method with six leading inpainting systems, including widely used convolutional networks, Transformer models, and modern diffusion models. Across a range of damage levels, the new model consistently achieved higher scores on sharpness, structural similarity, and visual realism. Zoomed‑in comparisons showed smoother mountain contours, more convincing tree branches, and ink textures that blended seamlessly into the untouched areas. Human reviewers, including trained painters, also preferred its restorations, judging them more coherent in both structure and style.

What This Means for Cultural Heritage

In plain terms, the study shows that an AI system can be trained not just to “fill holes” in images, but to respect the inner logic and personality of traditional Chinese landscape painting. By combining line drawings, region meaning, and stylistic cues, the model reconstructs missing parts that look as though they could genuinely belong to the original scroll. While it cannot replace conservators, it offers museums and researchers a powerful, non‑invasive tool for visualizing possible restorations, planning treatments, and creating more complete digital surrogates of fragile works—helping these landscapes survive, and be studied, long after the paper has grown too delicate to unroll.

Citation: Cao, S., Mu, D., Zhang, Y. et al. An improved semantic and sketch biconditional guided image inpainting model for Chinese landscape painting. npj Herit. Sci. 14, 103 (2026). https://doi.org/10.1038/s40494-026-02371-4

Keywords: digital restoration, Chinese landscape painting, image inpainting, cultural heritage, art conservation