Multimodal system for automated medical documentation and clinical decision support integrating contact center solutions

Helping doctors spend more time with patients

Many family doctors feel that paperwork steals time from face to face care. The Parrot AI project tackles this problem by building a smart helper that listens to medical visits, writes up the notes, and offers suggestions, while always keeping the doctor in charge. For patients, this could mean shorter waits, more focused conversations, and clearer records of each visit.

Why paperwork is such a burden

Digital medical records were meant to simplify care, but in practice doctors still type or click through most forms by hand. Studies suggest that nearly half of a standard appointment can be spent on documentation instead of direct contact. In Poland alone, primary care doctors handled more than 180 million consultations in 2023, so even small gains per visit matter. Existing digital tools tend to cover only narrow tasks, such as booking visits or dictating notes, and do not support the entire flow from first contact to diagnosis and treatment.

What the Parrot AI system does

The Parrot AI system is designed for internal medicine and pediatric clinics and links several tools into one workflow. First, phone and chat assistants handle appointment booking and ask simple questions about symptoms before the visit. During the consultation, speech recognition turns doctor patient talk into text. A language model trained on Polish conversations then scans the dialogue to spot key items such as symptoms, medicines, measurements, and test referrals, and uses them to fill much of the electronic form automatically. This gives the doctor a draft record to check instead of a blank screen to complete from scratch.

How the smart assistant supports decisions

Beyond writing notes, Parrot AI includes an expert module that looks at the collected symptoms and proposes likely diagnoses and treatment options. It relies on large teaching sets that link patterns of complaints to specific conditions and therapies, based on standard disease codes. Several machine learning methods were tested, with a technique called random forest performing best. For each patient, the doctor sees the top three suggested diagnoses along with possible prescriptions, tests, and sick leave. The physician can accept, change, or reject every suggestion, so medical judgment always remains with the human expert.

How well the system works so far

The researchers evaluated each building block separately. The language model that extracts information from conversations correctly identified medical details and their meaning in more than four out of five cases. The expert module reached similar effectiveness when matching symptom patterns to diagnoses and treatments, often above the levels reported for other medical support tools. Voice and chat assistants also handled most registration and pre interview tasks without the need for a human operator. These tests were based on thousands of real, anonymized Polish clinic visits, and the system worked in near real time.

Limits, safety, and next steps

The authors note that Parrot AI has been trained only on Polish and mostly on data from one region, so it does not yet cover all diseases or speaking styles. Background noise and varied clinic equipment can also affect accuracy. For now, the system has mainly been tested on past recordings and in controlled pilot use. To protect patients, every automated step is logged, and the doctor must confirm all diagnoses, referrals, and prescriptions before they become part of the record. Future work will expand the range of specialties, improve handling of rare conditions, and add clearer explanations of how the algorithms reach their conclusions.

What this could mean for everyday care

The study suggests that a well designed digital assistant can shoulder much of the routine typing and checking that weighs down medical visits, while still leaving final choices to clinicians. By combining call center style tools, speech recognition, language understanding, and an expert module, Parrot AI points toward clinics where visits are documented more consistently and doctors can devote more attention to listening and explaining. If further trials confirm the results, systems like this could become a common, mostly invisible helper in primary care offices.

Citation: Płaza, M., Płaza, M., Lucińska, M. et al. Multimodal system for automated medical documentation and clinical decision support integrating contact center solutions. Sci Rep 16, 15017 (2026). https://doi.org/10.1038/s41598-026-45879-2

Keywords: medical documentation, clinical decision support, natural language processing, contact center, primary care