Gastrointestinal polyp detection method based on the improved RT-DETR

Why catching tiny growths matters

Colorectal cancer often begins as small growths called polyps on the lining of the gut. Doctors use colonoscopies and other endoscopic exams to find and remove these polyps before they turn dangerous. Yet even skilled endoscopists can miss subtle or oddly shaped lesions, especially in noisy, fast-moving video. This study introduces an artificial intelligence (AI) system designed to act as an ultra-fast second set of eyes, spotting more polyps in real time without slowing down the procedure.

The challenge of seeing the unseen

Polyps come in many sizes and shapes, from tiny flat spots to more obvious bumps, and they can hide among folds, shadows, fluids, and glare inside the intestine. Commercial AI assistants already exist, but they sometimes struggle when images come from different cameras or when polyps are very small or low in contrast. Many research systems face a trade-off: if they are accurate, they tend to be slow; if they are fast enough for real-time video, they may overlook hard-to-see lesions. The authors focus on breaking this trade-off so that doctors can have both speed and sharper vision.

A smarter way to read endoscopy video

The team builds on a modern detection framework called RT-DETR-r18, which treats polyp finding a bit like translating a picture into a list of objects. They add three key upgrades aimed at the quirks of endoscopy images. The first upgrade, a detail-preserving module, is designed to hang on to fine surface textures of flat or distant polyps that standard algorithms tend to blur away when shrinking images for analysis. The second upgrade introduces an efficient attention mechanism: instead of examining every pixel pair in a heavy computation, it uses a leaner way of focusing on the most informative areas, helping the system ignore distractions such as bubbles, stool, or reflections. The third upgrade blends information from several scales, so the system can handle both close-up, high-detail views and tiny "visual dots" that represent lesions seen from farther away.

Putting the system to the test

To see how well their method works, the researchers trained and evaluated it on 1,611 labeled images from two different sources: standard colonoscopy and wireless capsule endoscopy. This mixture forces the AI to rely on true lesion features rather than quirks of any single device. They converted expert segmentation masks into tight bounding boxes to give the model precise examples of where polyps are. Performance was judged with common measures such as precision (avoiding false alarms), recall (avoiding misses), and average precision, along with the number of images processed per second. Across five independent runs, the improved system raised precision from 90.7% to 94.8% and recall from 84.0% to 89.9%, while boosting overall detection quality. Crucially, it still analyzed video at about 188 frames per second—far beyond the 30–60 frames per second typical for clinical endoscopy—so it can keep up with real procedures.

How it compares and where it fails

When stacked against popular object detectors from the YOLO family and stronger RT-DETR variants, the new method achieved the best balance of accuracy, tightness of polyp outlines, and computational cost. It produced cleaner detection results, with fewer oversized boxes and fewer missed lesions, especially in complex scenes. Still, the system is not perfect. It sometimes fails in very dark areas or where lesions are partly hidden by folds. It can also mistake bright reflections or bubbles for true polyps if they mimic the round, raised appearance of a growth. The authors suggest that adding information from neighboring video frames in the future could help filter out such fleeting artifacts and further stabilize the alerts.

What this means for patients and doctors

From a lay perspective, the study shows that AI can already scan endoscopy images much faster than a human while making fewer mistakes than current real-time detectors. By better preserving tiny details, focusing on meaningful regions, and handling objects at many visual scales, the proposed system finds more potential trouble spots without bogging down the exam. Although these results come from carefully curated image datasets rather than live colonoscopies, they point toward AI tools that could reduce the chance that an important polyp slips by unnoticed. The next step will be large-scale clinical trials to determine whether these technical gains translate into fewer missed cancers and more confident, efficient screening for patients.

Citation: Du, J., He, Z., Zhang, S. et al. Gastrointestinal polyp detection method based on the improved RT-DETR. Sci Rep 16, 7020 (2026). https://doi.org/10.1038/s41598-026-38617-1

Keywords: colonoscopy, polyp detection, medical AI, endoscopy imaging, real-time screening