MedGame: Storytelling Gamification Empowered by Large Language Models for Medical Education

Source: arXiv:2607.21570 · Published 2026-07-23 · By Qian Wu, Xinrong Zhou, Zizhan Ma, Kai Chen, Zheyao Gao, Xun Lin et al.

TL;DR

MedGame addresses the challenge in medical education of transforming static clinical case reports into interactive, decision-centered learning trajectories rather than isolated question-answering sessions. It introduces a novel dual-engine framework consisting of a Medical Narrative Designer, which restructures raw patient summaries into hierarchically organized clinical storylines with explicit states and decision nodes, and a Story Director, which converts these storylines into executable, dependency-aware multimodal orchestration plans rendered on an interactive platform. This approach leverages storytelling gamification to enable learners to actively participate in clinical reasoning through successive decisions that impact case progression.

MedGame Bench, the accompanying benchmark and evaluation protocol, comprises 5,000 real patient cases balanced across eight medical specialties. Experiments demonstrate that large commercial LLMs like Claude-Sonnet-4.5 and Gemini-3-Pro achieve near-perfect structural validity and better clinical and educational content quality compared to open-source LLMs prior to fine-tuning. Task-specific fine-tuning of open-source models substantially closes the performance gap, improving narrative structuring, decision design, and multimodal orchestration plan validity. A pilot student user study finds that learners perceive MedGame’s interactive storytelling games as more engaging and pedagogically useful than text-only case presentations.

Key findings

• Commercial frontier LLMs (Claude-Sonnet-4.5, Gemini-3-Pro) achieve 99.5%–100% Strict validation on Medical Narrative Generation, whereas open-source models like Qwen3-32B only reach 79.4% before fine-tuning (Table 2).
• On Story Direction, commercial models exceed 99% sample-wise structural validation while open-source models range from 56.5% to 80.3% without fine-tuning (Table 3).
• Task-specific fine-tuning of open-source LLMs improves strict validation on Medical Narrative Generation to >99.5% and boosts use of curated personas and clinical locations (e.g., Qwen3.5-27B∗ CSU score improved from 5.41 to 8.50).
• Fine-tuned Qwen3.5-27B∗ nearly matches or slightly surpasses Claude-Sonnet-4.5 on combined narrative content metrics, demonstrating effective knowledge transfer from commercial models.
• Expert-in-the-loop revisions on generated story drafts improve Medical Accuracy from ~6.2–6.4 to 7.9–8.1 (out of 10) and Educational Quality from ~6.3–6.4 to 7.7–8.0, highlighting persistent need for expert guidance.
• Learner feedback from a pilot student study indicates that MedGame’s interactive storytelling format is perceived as more engaging and pedagogically useful compared to text-only case presentations.
• Related-case overlap (similar patient variants) in the benchmark test splits does not significantly inflate performance, removing a potential confounder (p > 0.05).
• LLM-as-a-Judge scoring correlates strongly (r=0.81) with expert clinician scoring for Medical Narrative Generation and moderately (r=0.61) for Story Direction, supporting automated evaluation reliability.

Threat model

Not applicable; this work focuses on educational content generation and interactive storytelling in medical education rather than adversarial threat scenarios. The system assumes LLMs generate clinically plausible narratives but does not model malicious adversaries or attacks.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is not explicitly defined as MedGame focuses on educational content generation rather than security. The system assumes that clinical cases are static, complete clinical case reports from PubMed Central and that learners make decisions within these gamified scenarios. Fidelity to the source case and clinical accuracy are core constraints.

2. Data: The key data resource is MedGame Bench, a dataset of 5,000 patient case summaries derived from the PMC-Patients dataset. Cases are balanced equally across eight specialties: cardiology, endocrinology, gastroenterology, hematology/oncology, nephrology, neurology, respiratory/critical care, and rheumatology (625 each). The dataset is split into 4,000 training and 1,000 test cases, with about 41% of test cases having high-similarity variants in training. Data preprocessing involved extracting clinical text, parsing into hierarchical story components, and manually designing narrative elements.

3. Architecture / Algorithm: MedGame comprises two main components:

• Medical Narrative Designer: Given a raw patient case summary input, it leverages an LLM prompted with clinical exemplars, medical rubrics, and a strict Pydantic schema to produce a hierarchical clinical storyline with Acts (macro clinical phases), Scenes (intermediate clinical steps), and Decision Nodes (learner-facing clinical choices with feedback). This engine structures medical case information into a verifiable JSON schema suitable for downstream processing.
• Story Director: Takes the generated Clinical Storyline as input and builds a dependency-aware Directed Acyclic Graph (DAG) of multimodal generation primitives corresponding to visual, audio, video, and character generation tasks. Utilizing tool APIs for these multimodal assets, the director respects dependencies to maintain narrative and visual coherence across scenes (e.g., consistent character appearance) and executes the graph to render interactive game components.

4. Training Regime: Open-source LLMs (Qwen, Gemma, MedGemma) were fine-tuned on the 4,000 training cases of MedGame Bench using LoRA adaptation (Rank 32, Alpha 64) on hardware with 2 H100 GPUs. Fine-tuning employed instructive prompts structured around medical constraints and narrative schema. Commercial models were accessed via APIs without fine-tuning.

5. Evaluation Protocol: Two evaluation tracks mirror the two-stage generation:

• Medical Narrative Generation: Evaluated for structural validity (JSON parsing, schema compliance, logical constraints), story adaptation metrics (clinical content integration, persona and scene utilization, narrative quality), clinical accuracy (decision correctness, option design, and explanation accuracy), and educational quality (question and feedback quality).
• Story Direction: Evaluated for sample- and task-level schema validation and task reasonability (correct resource assignment, API type selection, and parameter content). Automated checks were combined with GPT-5.2 as LLM-as-a-Judge scoring each metric on 1-10 scales. Human expert review of subset outputs confirmed correlation with automated scores.

6. Reproducibility: The authors release the MedGame framework platform on GitHub. Benchmark dataset is publicly derived from PMC-Patients but requires linkage to original PubMed data. Fine-tuned weights for open-source models are not publicly detailed. Expert revision data is partially documented. Exact hyperparameters and seed strategies are reported but coarse.

Concrete example: For a patient presenting with severe abdominal pain and bloody diarrhea, the Medical Narrative Designer generated an interactive storyline organized into two Acts comprising multiple Scenes with learner questions such as “Which labs do you order?” The Story Director decomposed these into multimodal tasks including generating character portraits (patient, clinician), generating environment scenes (consultation room, radiology), and fusing character images with backgrounds in dependency order, producing an interactive game that allows the learner to explore clinical decision-making progressively.

Technical innovations

• Dual-engine architecture that decouples clinical narrative generation (Medical Narrative Designer) from multimodal execution planning (Story Director), inspired by film continuity scripts.
• Hierarchical Clinical Storyline representation with explicit Acts, Scenes, and Decision Nodes enabling structured, interactive medical case simulations.
• Dependency-aware multimodal orchestration framework that models generation primitives as a Directed Acyclic Graph to maintain narrative and visual coherence across interactive game stages.
• Construction of MedGame Bench, a 5,000-case, multi-specialty benchmark dataset with structured labeling and evaluation protocols targeting medical narrative generation and multimodal story direction.

Datasets

• MedGame Bench — 5,000 clinical case summaries — derived from PMC-Patients (public patient case reports from PubMed Central)

Baselines vs proposed

• Qwen3-32B (no fine-tuning): Strict Structure Validation = 79.4% vs fine-tuned Qwen3-32B* = 99.1%
• Gemma-3-27B (no fine-tuning): Strict Structure Validation = 58.7% vs fine-tuned Gemma-3-27B* = 98.9%
• Claude-Sonnet-4.5 (commercial): Strict Structure Validation = 99.5% vs Qwen3.5-27B* (fine-tuned open-source) = 99.5%
• Gemini-3-Pro (commercial): Story Direction Sample-wise validation = 99.2% vs Qwen3.5-27B* = 99.8%
• Open-source models improve CSU (character & scene usage) scores by ~3 points post fine-tuning (e.g., Qwen3-32B from 5.41 to 8.50).
• Expert re-annotation improves Medical Accuracy from ~6.4 to 8.1 and Educational Quality from ~6.4 to 7.95 on 10 sampled story drafts.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.21570.

Fig 1: MedGame at a glance. Given a static clinical

Fig 2 (page 1).

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 7 (page 1).

Fig 8 (page 1).

Limitations

• Current evaluation primarily on static case reports; dynamic multi-branch decision trees reflecting true clinical reasoning variability are discussed but not extensively evaluated.
• Model-generated medical content still requires expert review to ensure accuracy and appropriate pedagogy; expert-in-the-loop refinement significantly improves output quality.
• Open-source LLMs still underperform commercial frontier models before fine-tuning, indicating baseline capacity limitations.
• The pilot user study is limited in scale and scope; broader, longitudinal learner outcome evaluation is needed.
• Multimodal orchestration depends on underlying generation tools whose fidelity and bias are not systematically analyzed.
• Benchmark dataset overlaps partially with similar cases between train and test splits, potentially inflating performance; though no significant effect found, this remains a concern.

Open questions / follow-ons

• How to extend MedGame’s linear clinical storylines into complex, branching scenario graphs that better simulate real-world clinical decision uncertainty and variability?
• What are the longitudinal impacts of MedGame’s interactive storytelling format on medical learners’ retention, diagnostic skills, and clinical reasoning compared to traditional methods?
• Can multimodal orchestration plans be generalized to incorporate real-time learner inputs and dynamically adapt case progression or difficulty?
• How to better integrate expert feedback loops into model training and interactive scenarios to continuously improve clinical accuracy and educational effectiveness?

Why it matters for bot defense

MedGame’s approach to converting static medical cases into interactive, decision-driven storytelling games demonstrates the potential of large language models to scaffold complex learning trajectories structured around meaningful checkpoints where users make choices and receive feedback. For bot-defense and CAPTCHA developers, this work exemplifies how LLMs can be harnessed to generate multi-turn, dependency-aware scenario flows that require strategic user decisions rather than single-turn responses, aligning with evolving CAPTCHA designs that emphasize reasoning and continuous engagement.

Moreover, the Story Director’s use of dependency graphs to orchestrate multimodal asset generation with strict coherence constraints provides a useful architectural analogy for production systems that coordinate varied challenge modalities (e.g., visual puzzle elements combined with timed audio prompts). Finally, the detailed benchmarking and task-specific fine-tuning protocols illustrate how domain adaptation and structured evaluation frameworks can improve reliability and fidelity in complex LLM outputs, lessons applicable for CAPTCHA challenge curation and validation pipelines.

Cite

@article{arxiv2607_21570,
  title={ MedGame: Storytelling Gamification Empowered by Large Language Models for Medical Education },
  author={ Qian Wu and Xinrong Zhou and Zizhan Ma and Kai Chen and Zheyao Gao and Xun Lin and Hongqiu Wu and Longfei Gou and Yixiao Liu and Ann Sin Nga Lau and Qi Dou },
  journal={arXiv preprint arXiv:2607.21570},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.21570}
}

Read the full paper

• arXiv:2607.21570 (abstract)
• arXiv:2607.21570 (PDF)