SGA: Plug&Play Geometric Verification for Educational Video Synthesis

Source: arXiv:2607.18116 · Published 2026-07-20 · By Lopez Jhon, Hinojosa Carlos, Ghanem Bernard

TL;DR

This paper addresses the critical challenge of spatial correctness and visual legibility in educational video synthesis generated via code-centric pipelines powered by Large Language Models (LLMs). Existing frameworks such as Code2Video and TheoremExplainAgent generate syntactically valid animation scripts but suffer from geometric occlusions and layout errors that degrade readability and educational effectiveness. The authors propose the Symbolic Geometric Agent (SGA), a plug-and-play verification module that intercepts LLM-generated Manim code, extracts symbolic scene graphs through partial execution, and deterministically detects spatial conflicts using axis-aligned bounding box analysis. SGA then generates structured, actionable feedback for iterative refinement of the code, improving spatial layouts prior to rendering without requiring costly video synthesis or perceptual vision-language model (VLM) scoring.

They also introduce the Manim Visual Quality Score (MVQS), a rendering-free deterministic metric that evaluates spatial correctness, compositional balance, and pedagogical clarity in the scene graph. Experiments on the MMMC-Code benchmark over 117 instructional topics and four LLM backbones show SGA consistently improves MVQS by up to 16.1% relative over raw baselines, outperforming VLM-based critics in 7 of 8 pipeline and model combinations. Furthermore, SGA offers 6×–18× speedup per refinement iteration over expensive full rendering and multimodal evaluations. Qualitative results demonstrate that SGA effectively eliminates object overlaps, out-of-bounds placements, and occlusions that perceptual critics often miss. The work establishes a new, symbolic grounding paradigm for spatial verification in code-driven educational video generation.

Key findings

• SGA improves Manim Visual Quality Score (MVQS) by up to 16.1% relative over raw baselines on MMMC-Code benchmark using Code2Video + GPT-5.1 (MVQS = 73.11 vs 62.97).
• SGA consistently increases MVQS in 7 out of 8 LLM backbone × pipeline combinations tested (4 backbones, 2 pipelines).
• VLM-based aesthetic judges have limited sensitivity to geometric inconsistencies, compressing scores in a tight 10.5-point range versus 14.4 points of spread in MVQS (Fig 3).
• SGA achieves a 6×–18× per-iteration speedup in layout refinement compared to VLM critic methods that require full video rendering.
• Symbolic conflict detection uses deterministic axis-aligned bounding box (AABB) intersection analysis, enabling exact spatial conflict identification without relying on stochastic or pixel-based heuristics.
• Targeted line-level code refinement based on Minimum Separation Vectors (MSV) and canonical Manim fix templates improves layout without global regeneration.
• MVQS sub-scores for spatial correctness (Ssp), layout quality (Sly), and educational utility (Sed) combine multiple geometric and semantic criteria, with primary weight on spatial occlusions.
• SGA supports integration as a plug-and-play verification layer without modifying underlying LLM architectures or generation pipelines.

Threat model

The adversary in this setting is effectively the code-generation pipeline (the LLM) which may produce syntactically correct but spatially inconsistent animation scripts, resulting in layout conflicts such as overlapping objects or out-of-bounds elements. The adversary's capability includes generating arbitrary Manim code from natural language prompts without inherent geometric awareness. The model assumes the defender cannot control or pre-train the LLM to be geometry-aware but can intercept and refine outputs iteratively using symbolic verification. The framework cannot prevent semantic or pedagogical errors in content correctness, only spatial layout defects.

Methodology — deep read

The core threat addressed is spatial randomness and layout errors in code-generated educational animations, where the adversary is the code generation pipeline (LLM) producing syntactically correct but spatially inconsistent scene graphs.

The authors evaluate on the MMMC-Code benchmark, a dataset of 117 complex instructional topics in calculus, linear algebra, and physics, which require precise spatial arrangement and pedagogically meaningful animations. Four LLM backbones (GPT-5.1, GPT-5 mini, Claude 4.6 Sonnet, Gemini 3.0 Flash) are tested, using two code-centric educational video generation pipelines: TheoremExplainAgent and Code2Video.

SGA intercepts LLM-produced Manim Python scripts before rendering. It performs partial execution by instrumenting the 'construct()' method with fault-tolerant AST-level code wrapping to extract a symbolic scene graph representing visual objects (mobjects) along with semantic labels, spatial centroid and bounding boxes, source code grounding, and creation lines.

Spatial conflict detection proceeds via axis-aligned bounding box (AABB) intersection tests between pairs of objects, calculating intersection areas and asymmetric coverage ratios to classify occlusions as HIGH, MEDIUM, or LOW severity. Semantic and spatio-temporal filters suppress intentional occlusions such as decorative or animation-transient contacts. Persistent conflicts trigger calculation of Minimum Separation Vectors (MSV), the minimal axis-aligned displacement needed to remove overlaps.

The feedback compiler translates conflicts into structured reports containing MSVs and canonical Manim correction templates that suggest localized geometric fixes such as translation (.shift()), rescaling, or repositioning. Refinement prompts include +/-5 lines of source context around the conflicting code lines.

Refinement iterates for a fixed budget, with the LLM generating patched code versions guided by symbolic feedback, validated via AST parsing to ensure syntactic correctness.

The authors propose Manim Visual Quality Score (MVQS) computed deterministically on symbolic scene graphs without rendering, combining spatial correctness (object overlap, canvas boundary containment, minimum scale, separation), layout quality (density and distribution across grid), and pedagogical utility (labeling completeness and animation quality). Each component is weighted and normalized to range 0-1.

Evaluation quantifies spatial symbolic integrity via MVQS, perceptual quality via a Vision-Language Model (Gemini 3.0 Flash) scoring educational logic, aesthetics, layout flow, and visual clarity on rendered video frames, and computational efficiency.

Comparison between SGA, raw baselines, and VLM critic feedback is measured over the benchmark across LLMs and pipelines, reporting improvements in MVQS, sub-scores, and runtime reduction. Qualitative visual comparisons illustrate reductions in label collisions and occlusions.

No closed dataset restrictions reported; authors instrumented standard Python Manim runtime, and SGA is designed as a framework-agnostic plug-in.

An example refinement cycle: The initial Manim script is partially executed symbolically to extract scene graph. Detecting a high severity occlusion between two labels, SGA computes MSV displacement vector. The feedback compiler formulates a patch prompt with minimal context and MSV for the LLM. The LLM edits the initial script by shifting one label accordingly and outputs the revised script, verified for syntax. This process iterates until no conflicts or iteration limit.

Technical innovations

• Introduction of a plug-and-play symbolic geometric verification layer intercepting LLM-generated code for educational video synthesis, operating prior to rendering.
• Deterministic spatial conflict detection using axis-aligned bounding box (AABB) intersection on symbolic scene graphs extracted via partial execution of Manim code.
• Manim Visual Quality Score (MVQS), a novel deterministic, rendering-free metric evaluating spatial correctness, layout quality, and pedagogical clarity in educational animations.
• Structured feedback compiler converting geometric conflict metrics into precise line-level correction prompts for localized LLM refinement with guaranteed syntactic validity.
• Semantic and spatio-temporal filtering heuristics to distinguish intentional pedagogical occlusions from genuine visual conflicts.

Datasets

• MMMC-Code benchmark — 117 instructional topics across calculus, linear algebra, and physics — public benchmark introduced by Chen et al. (2025)

Baselines vs proposed

• Code2Video RAW + GPT-5.1: MVQS = 62.97 vs Code2Video + SGA: MVQS = 73.11 (+16.1% relative)
• TheoremExplainAgent RAW + Gemini 3.0 Flash: MVQS = 63.67 vs TheoremExplainAgent + SGA: MVQS = 67.14 (+5.4%)
• Code2Video + VLM Critic + GPT-5.1: MVQS = 64.54 vs Code2Video + SGA + GPT-5.1: MVQS = 73.11 (+13.3%)
• SGA improves Sed sub-score over VLM Critic by 15.0 points on TheoremExplainAgent and by 10.5 points on Code2Video
• Code2Video + SGA reduces refinement overhead runtime from 8.8 minutes (VLM critic) to 6.8 minutes (SGA), approximately 6× speedup at iteration level
• Across 8 backbone × pipeline configurations, SGA improves MVQS in 7, with one marginal drop of −1.01 points for TheoremExplainAgent + Claude 4.6 Sonnet

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.18116.

Fig 1: Current SOTA agents produce syntactically valid but visually occluded animations. (Left) TheoremEx-

Fig 2: illustrates the proposed Symbolic Geomet-

Fig 3 (page 2).

Fig 4 (page 2).

Fig 5 (page 2).

Fig 3: Perceptual vs. symbolic evaluation across

Fig 4: Comparison on Code2Video (Chen et al., 2025) scenes. Raw frames exhibit label-equation collisions and

Fig 5: Qualitative comparison on TheoremExplainAgent (Ku et al., 2025) frames. Raw output exhibits label

Limitations

• Current SGA implementation specializes in 2D geometric primitives within the Manim engine; extension to 3D scenes and depth sorting is non-trivial.
• SGA focuses on spatial geometric verification and does not address temporal synchronization issues such as aligning label appearance with shape drawing.
• MMMC-Code benchmark is limited to English language and mathematical educational content; robustness across diverse languages and pedagogies remains untested.
• MVQS metric has not yet been validated against human expert annotations; correlation with subjective quality assessments needs further study.
• Refinements depend on LLM capability to correctly interpret structured feedback and produce syntactically valid patches; failures require rollback.
• Semantic filtering heuristics to suppress intentional occlusions may have false positives/negatives affecting correction scope.

Open questions / follow-ons

• How can SGA be extended or adapted to handle 3D animations involving camera projections, depth ordering, and occlusion?
• What is the correlation between MVQS scores and human expert judgments of educational animation quality in diverse pedagogical contexts?
• Can symbolic verification techniques be integrated into the LLM training or prompting process to reduce the need for iterative refinement?
• How effective is SGA in multilingual or non-mathematical educational video synthesis pipelines, and what adaptations are required?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work offers a compelling example of how symbolic, deterministic verification can effectively identify and correct subtle spatial inconsistencies in code-generated visual content without expensive rendering or stochastic perceptual models. SGA's approach illustrates that embedding geometric reasoning directly into the generation and refinement loop improves robustness and efficiency, properties that are valuable for secure, fault-tolerant challenge generation or layout validation in CAPTCHA systems.

The introduction of a deterministic quality metric (MVQS) grounded in symbolic scene representation also highlights a pathway to evaluate layout integrity without relying solely on neural perceptual models, which may be vulnerable to adversarial or out-of-distribution failures. Applying similar symbolic verification and localized refinement pipelines to CAPTCHA or bot-detection image/video generation could enhance the controllability and verifiability of challenge content, potentially reducing false positives and improving user experience.

Cite

@article{arxiv2607_18116,
  title={ SGA: Plug&Play Geometric Verification for Educational Video Synthesis },
  author={ Lopez Jhon and Hinojosa Carlos and Ghanem Bernard },
  journal={arXiv preprint arXiv:2607.18116},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.18116}
}

Read the full paper

• arXiv:2607.18116 (abstract)
• arXiv:2607.18116 (PDF)