Simulating the Resident: Generating Executable Smart Home Schedules via LLM Personas

Source: arXiv:2607.08231 · Published 2026-07-09 · By Victor Jüttner, Xenia Wagner, Christoph Jahn, Erik Buchmann

TL;DR

This paper addresses the significant challenge faced by smart-home researchers in acquiring realistic, diverse, and privacy-preserving datasets of resident-device interactions. Current datasets from real homes are costly, limited in scale, and raise privacy concerns due to intrusive behavioral monitoring. The authors propose a novel approach that leverages large language models (LLMs) to generate synthetic yet behaviorally grounded resident personas and their daily routines. These personas interact with a configurable simulated smart home, producing detailed, temporally coherent, and schema-compliant device interaction schedules.

The main technical contribution is a multi-stage pipeline that transforms high-level contextual and socio-technical household parameters into executable JSON schedules that can directly drive physical smart-home testbeds. The pipeline includes persona initialization, narrative generation grounded in socio-technical context, strict JSON action extraction, and a memory update mechanism for longitudinal simulation. An initial proof-of-concept using OpenAI's GPT-5.4 demonstrates feasibility by simulating a morning routine of two residents in an 8-device smart home, generating plausible, timestamped device interactions respecting environmental and social constraints. This represents a promising privacy-conscious alternative to real-world data collection for HCI, security, and privacy research in smart homes.

Key findings

• The proposed multi-stage LLM pipeline successfully generates persona-consistent, temporally explicit, and schema-compliant device interaction schedules in JSON format suitable for execution on physical smart-home testbeds.
• The design framework configures simulated households using five socio-technical dimensions: occupational routines, simulation timeframe, household dynamics, device ecosystem, and environmental context, enabling varied and realistic simulations.
• In the proof-of-concept experiment, the pipeline accurately generated 06:00-10:00 morning routines for two residents (a work-from-home professional and an office worker) in a compact urban apartment equipped with 8 smart devices.
• Generated narratives and JSON schedules reflected realistic temporal patterns, such as clustered device use before departure times and adaptations to seasonal context (e.g., increased indoor lighting usage due to winter sunrise).
• The JSON outputs strictly adhered to the provided device schema, preventing hallucination or invalid commands, aided by a separated action extraction stage with deterministic post-validation.
• The approach enables capturing authentic network traffic and device state changes on physical testbeds without intrusive real household observation or reliance on prior real data.
• The pipeline supports modular multi-day simulations via a memory update stage that summarizes resident and environment state changes to maintain temporal and behavioral consistency.
• Unlike prior work that required real-world IoT traces as input or operated only in virtual environments, this method simulates residents fully from scratch and produces executable schedules with no prior real data dependency.

Threat model

n/a — The paper focuses on generating synthetic smart-home resident behavior data via LLMs for research use, rather than defending against or modeling adversaries. The implicit assumption is a non-adversarial simulation scenario where the primary goal is realistic and privacy-preserving data synthesis.

Methodology — deep read

1. Threat Model & Assumptions: The pipeline assumes a benign simulation scenario wherein the LLM generates synthetic smart-home resident personas and their device interaction schedules from scratch, with no adversarial interference or knowledge of real household data. The adversary is not modeled as the focus is on realistic data synthesis rather than security evaluation of the system itself.

2. Data: The framework uses no real resident data. Instead, a configurable design framework specifies five socio-technical household dimensions (occupational routines, simulation timeframe, household dynamics, device ecosystem, environmental context) to define simulation parameters. The proof-of-concept experiment instantiates two residents with distinct occupational schedules and an 8-device smart home, simulating a four-hour morning window.

3. Architecture / Algorithm: The core is a four-stage multi-stage LLM pipeline adapted from the "generative agents" approach [Park et al. 2023].

• Stage 1 initializes concise persona memory cards for each simulated resident, summarizing daily schedules, smart home habits, and device preferences mapped explicitly to a provided JSON device schema.
• Stage 2 generates a natural-language narrative day-plan for a specified time window, grounded in persona memory cards and household context. The narrative includes detailed timestamps and rationales for device interactions.
• Stage 3 extracts structured device-level actions from the narrative, enforcing schema compliance and JSON formatting via an action parser prompt and post-validation.
• Stage 4 (proposed for longitudinal use) updates persona memory cards based on executed actions, preserving state and habits over multi-day simulations.

4. Training Regime: The system uses a pretrained LLM (OpenAI GPT-5.4) in a zero-shot or few-shot prompting regime rather than classical training. Hyperparameters or seed strategies are not detailed.

5. Evaluation Protocol: The proof-of-concept evaluates the pipeline by running a single simulation window producing a morning routine. Qualitative assessments verify temporal coherence, persona consistency, social coordination, environmental context adaptation (e.g., lighting in winter), and device schema adherence. The output JSON is confirmed valid and executable. Detailed quantitative benchmarks, ablation studies, or adversarial tests are not performed.

6. Reproducibility: Code or weights are not publicly released. The demonstration uses GPT-5.4 (not public at time of writing). The method’s iterative LLM prompting stages and JSON schemas are documented in detail for replication.

Example Walkthrough: For resident Bob (9-to-5 office worker) in a winter apartment, Stage 1 generates a memory card specifying his preference for a bright, fast wake-up using the bedroom lamp with brightness 60 and temperature 4200K. Stage 2 produces a narrative placing this device interaction at 06:00 with explicit rationale. Stage 3 extracts this into a JSON action record with fields timestamp: "06:00", resident: "Bob", device: "Bedroom_Lamp", action: "turn_on", etc. This JSON can be executed on a testbed to replicate the behavior physically.

Technical innovations

• A design framework that models simulated smart-home households using five socio-technical dimensions to enable varied, realistic resident-device interaction scenarios.
• A novel multi-stage LLM pipeline that converts high-level household and persona descriptions into temporally explicit, natural-language narratives and then strict JSON executable schedules compliant with a predefined device schema.
• Separation of natural language narrative generation from structured action extraction, reducing hallucinations and ensuring schema adherence when producing executable device interaction schedules.
• Introduction of a memory update mechanism for longitudinal simulation, maintaining temporal consistency of resident preferences and environment state across consecutive simulation windows.
• Application of LLM-generated personas directly for executable smart-home testbed control, enabling physical device interaction data capture without requiring any real-world behavioral data as input.

Datasets

• Proof-of-concept simulation: 1 synthetic dataset with 2 residents, 8 smart home devices, simulated over 4 hours (06:00-10:00) in an urban winter apartment.

Baselines vs proposed

• Not applicable — no quantitative baselines or prior pipelines directly compared in the paper.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.08231.

Fig 5: shows an exemplary excerpt from this experiment using

Limitations

• Evaluation is limited to a single-window proof-of-concept; no extensive robustness or generalization across varied household configurations or longer timeframes yet conducted.
• The ecological validity of LLM-generated personas is unverified; uncertain if simulated routines realistically capture irregular, inconsistent, or forgetful resident behavior found in real homes.
• Physical execution on a live smart-home testbed is proposed but not yet demonstrated end-to-end; real hardware constraints remain unexplored.
• No adversarial or security robustness evaluation performed on simulated outputs or LLM behavior.
• Dependence on proprietary GPT-5.4 model may limit immediate reproducibility and introduce biases inherent to the language model.
• Pipeline currently focuses on generating benign resident behavior, lacking modeling of anomalous or malicious device interactions.

Open questions / follow-ons

• How well do LLM-generated personas capture the complexity and irregularities of real resident behavior over extended periods, including forgetfulness or spontaneous actions?
• How does the pipeline perform when scaling to more complex households with many residents and diverse, heterogeneous device ecosystems?
• Can the pipeline be effectively integrated and validated with live smart-home testbeds to assess real execution fidelity and network traffic realism?
• What methods can improve or validate the ecological validity of generated behaviors, perhaps via comparison to real-world smart home interaction datasets?

Why it matters for bot defense

Bot-defense and CAPTCHA practitioners can draw insights from this work primarily in the domain of synthetic behavior generation for IoT and smart-home contexts. The paper presents a novel method using LLMs to simulate realistic user-device interaction sequences that can be executed on physical testbeds, potentially enabling platform operators or researchers to generate rich, behaviorally grounded interaction traces without privacy-invasive real data collection. While not directly targeting CAPTCHA, the approach illuminates how LLM-driven personas can produce semantically coherent and temporally consistent activity traces, a concept relevant for generating synthetic human-like interaction data for behavior-based bot detection or anomaly detection in IoT scenarios. However, the work currently does not evaluate adversarial resilience or mimicry potential of LLM-simulated behaviors against automated attacks, indicating an area for further exploration before direct application to bot detection or CAPTCHA.

Cite

@article{arxiv2607_08231,
  title={ Simulating the Resident: Generating Executable Smart Home Schedules via LLM Personas },
  author={ Victor Jüttner and Xenia Wagner and Christoph Jahn and Erik Buchmann },
  journal={arXiv preprint arXiv:2607.08231},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.08231}
}

Read the full paper

• arXiv:2607.08231 (abstract)
• arXiv:2607.08231 (PDF)