DermaGPT a federated multimodal framework with a meta learned trust function for interpretable dermatology diagnostics

Why smarter skin checks matter

Skin problems affect billions of people, yet many communities have few or no dermatologists. That means suspicious moles or rashes can be misjudged or treated too late, especially in smaller clinics with limited technology. This study presents DermaGPT, an artificial intelligence system designed to help doctors spot common skin cancers and other lesions from photos, while also explaining its reasoning in plain language and protecting patient privacy.

A new kind of digital skin assistant

DermaGPT is built as a two-part assistant. First, a vision module looks at close-up photos of the skin, taken with ordinary smartphone cameras or dermatoscopes, and predicts which of 11 common lesion types it sees and whether it is likely benign or malignant. Second, a separate language module turns those predictions into patient-friendly explanations answering questions such as what the condition is, how serious it might be, and what treatments are usually considered. By separating “seeing” from “explaining,” the designers aim to keep the diagnostic core stable while allowing the explanation side to be improved or swapped out over time.

Designed for real-world clinics

Unlike many headline-grabbing medical AI systems that run only on large, expensive servers, DermaGPT is intentionally lightweight. Its vision backbone, adapted from a Google vision–language model, is fine-tuned in a way that changes only about one percent of its parameters. This makes it fast and affordable enough to run on modest graphics cards commonly available in hospitals. The authors trained the system on biopsy-confirmed images from four private clinics and then tested it on an independent public dataset from Stanford containing 4,452 images. On this external test, DermaGPT correctly identified the lesion type about 90 percent of the time and correctly distinguished benign from malignant lesions about 93 percent of the time.

Keeping data local and learning to trust each site

Because medical images are sensitive, DermaGPT is trained using federated learning: each hospital keeps its images on-site and only shares model updates, not raw pictures. However, hospitals differ in patient mix, camera quality, and skin tones, which can make a shared model less reliable. To address this, the authors added a meta-learned trust function that estimates how dependable each clinic’s updates are, based on measures like uncertainty, calibration, and signs of data shift. During training, updates from better-calibrated, more consistent sites are given higher weight, while noisier ones are down-weighted. This “trust-aware” scheme improved both accuracy and the reliability of the model’s confidence scores, especially at the most challenging site with more diverse skin types.

Explaining diagnoses in everyday language

For explanations, DermaGPT plugs its predictions into several large language models and compares how well they perform. It also uses an “advanced retrieval” module that pulls short passages from carefully curated online dermatology resources and feeds them to the language model as context. Four board-certified dermatologists scored the resulting answers on clarity, usefulness, factual grounding, and how likely they would be to use such a tool. Across all models, adding this retrieval step made explanations more informative and less prone to unsupported claims. One model, called DeepSeek-V3, stood out, producing the highest-rated explanations while using a relatively efficient architecture that activates only a subset of its neurons for each response.

Benefits, caveats, and what comes next

Overall, DermaGPT shows that it is possible to build a skin-diagnosis assistant that is fast, accurate, privacy-aware, and able to explain itself in human terms. It does not replace dermatologists; instead, it is meant to help non-specialists triage cases, support counseling, and extend expert-style guidance to clinics that lack specialists. The authors stress that some risks remain—such as confident explanations based on a wrong underlying diagnosis—and that more real-world trials are needed. They plan to expand the range of conditions, better cover rare diseases and darker skin tones, and add multilingual and self-monitoring features. If these challenges are met, systems like DermaGPT could help make high-quality skin care more accessible and consistent across very different healthcare settings.

Citation: Hashjin, N.M., Amiri, M.H. & Najafabadi, M.K. DermaGPT a federated multimodal framework with a meta learned trust function for interpretable dermatology diagnostics. Sci Rep 16, 7959 (2026). https://doi.org/10.1038/s41598-026-38715-0

Keywords: dermatology AI, skin cancer detection, federated learning, medical explainable AI, clinical decision support