MRR-YOLO: an instance segmentation technique for ground-based cloud images

Why watching clouds matters for clean energy

Modern cities are increasingly powered by the Sun, yet something as familiar as a passing cloud can cause sudden dips and spikes in solar electricity. These swings not only waste potential energy but can also stress equipment and the power grid. This study introduces a new way to "see" and separate individual clouds in sky images so that solar plants can better predict when the light will fade or flare, making renewable power more reliable and easier to manage.

How clouds disrupt solar power

Solar panels and solar-thermal plants depend on steady sunlight. When clouds race across the sky, the amount of light hitting panels can change within seconds, leading to flickering voltages, power imbalances, and even equipment fatigue in high‑temperature receivers. Not all clouds have the same impact: thin, high clouds dim the Sun gently, while thick, low clouds can cast sharp, fast‑moving shadows. To operate solar plants safely and efficiently, engineers need to know not just whether clouds are present, but where each cloud is, what type it is, and how it is likely to move.

From simple masks to counting individual clouds

Traditionally, computers have treated the sky as a simple puzzle of "cloud" versus "no cloud." Older methods relied on brightness thresholds or color rules, which are easily confused by glare, haze, or poor image quality. More recent "semantic segmentation" approaches based on deep learning improved accuracy, but still merged all clouds of the same kind into one broad region. That means important details are lost: overlapping clouds, thin streaks, and small pockets are all blended together, limiting our ability to track cloud thickness, height, and motion from one image to the next.

A new eye on the sky: the MRR‑YOLO model

This work develops MRR‑YOLO, a new "instance segmentation" model that looks at a sky image and separates it into many distinct cloud pieces, each with its own outline. Built on the fast YOLOv8 framework, the model is adjusted specifically for the messy, irregular patterns found in real skies. Three added components play key roles. A multi‑scale attention block lets the network focus on cloud details of different sizes without becoming heavy to run. A channel‑reorganizing block makes it easier to pick out meaningful cloud signals from the background. An adaptive convolution block adjusts how it scans each part of the image, improving its ability to follow fuzzy, curved cloud edges. Together these changes help the system remain relatively lightweight while sharpening its view of complex cloud shapes.

Putting the model to the test

The researchers tested MRR‑YOLO on several well‑known cloud image collections, including thousands of ground‑based sky photos from different locations and conditions, as well as new pictures taken over Zhengzhou, China. They compared its performance with popular deep‑learning methods that either classify each pixel as cloud or sky, or try to find separate cloud objects. While traditional semantic methods sometimes scored higher on raw pixel accuracy, they could not tell one cloud object from another. Earlier instance‑based methods struggled with thin or scattered clouds and often confused sky and cloud regions. MRR‑YOLO showed more balanced behavior: it identified individual clouds more reliably across many cloud families—from wispy cirrus to towering storm clouds—while maintaining competitive overall accuracy and running fast enough for practical use.

What the findings mean for future solar power

By turning the sky into a set of clearly separated cloud objects rather than a single blurry mass, MRR‑YOLO opens the door to more precise solar forecasting. Each segmented cloud can be analyzed for size, shape, and likely motion, feeding into models that estimate when and how much sunlight will reach solar plants in the next minutes. That in turn can help grid operators smooth out power fluctuations, protect sensitive equipment, and make better use of clean energy. Although further work is needed to broaden the training data and streamline the model for small devices, this study shows that smarter cloud vision can be a practical tool for making solar power more stable and dependable.

Citation: Wan, Z., Su, J., Fan, B. et al. MRR-YOLO: an instance segmentation technique for ground-based cloud images. Sci Rep 16, 11711 (2026). https://doi.org/10.1038/s41598-026-46567-x

Keywords: solar energy, cloud detection, deep learning, image segmentation, sky cameras