SAT-Edge-Agent: Hardware-in-the-Loop Edge-Agent Orchestration for Onboard Satellite Intelligence
Source: arXiv:2608.03728 · Published 2026-08-04 · By Longji He, Jeto Xu
TL;DR
The paper addresses the challenge of integrating onboard satellite intelligence beyond raw neural network inference, focusing on the orchestration of local edge tools within a constrained satellite payload environment. The authors present SAT-Edge-Agent, a hardware-in-the-loop (HIL) end-to-end system that combines a local YOLO-style oriented object detector and an OpenAI-compatible language model service on a commercial off-the-shelf ARM-based heterogeneous edge SoC. The system coordinates mission intent input, local tool invocation, structured detection artifact generation with FAIR1M metadata, and optional language-based operator summaries. Two fixed FAIR1M workloads (single-image and serial two-image detection tasks) were repeatedly executed (20/20 successes each) to characterize latency and resource usage profiles.
Key results include mean Full-Agent latencies of approximately 29.4s and 60.9s respectively, with detector runtime only 2.5–3% of the total latency, highlighting that orchestration and language-service overhead dominate request duration. CPU utilization averaged around 20%, while the shared NPU accelerator load sampled at 100% reflects joint use by detector and language services rather than solely detector occupancy. The work establishes a publicly reproducible HIL boundary to study observable edge-agent orchestration workflows suitable for satellite intelligence workloads but stops short of assessing detector accuracy, calibrated power efficiency, or operational flight readiness.
Key findings
- Full-Agent mean latency for single-image FAIR1M workload: 29.353 ± 1.613 s (20/20 completions).
- Full-Agent mean latency for serial two-image FAIR1M workload: 60.937 ± 3.512 s (20/20 completions).
- YOLO-style detector mean latency: 861.386 ± 11.778 ms (single-image) and 1510.920 ± 18.180 ms (two-image), only ~2.93% and 2.48% of full-agent latency.
- Mean CPU utilization during workloads about 20.7% (single image) and 20.5% (two-image).
- NPU load field sampled every 200 ms averaged 100% across entire request but represents shared accelerator usage, not detector-only.
- Validated event timeline shows majority of latency occurs outside detector execution, with delays both before and after first visible language-model token.
- Structured detector artifacts contain oriented bounding boxes, confidence scores, and FAIR1M metadata-backed geographic target fields, enabling machine-readable mission products.
- Repeated runs and partial-failure scenarios confirm stable agent workflow execution and structured error handling.
Threat model
n/a; the paper focuses on onboard satellite edge computing system integration and latency characterization rather than adversarial threat mitigation or security.
Methodology — deep read
The paper adopts a system-level hardware-in-the-loop (HIL) approach to evaluate end-to-end orchestration of local satellite edge intelligence on a COTS heterogeneous ARM-based edge-SoC platform. The adversary or threat model is not security-focused; instead, it assumes constrained satellite onboard compute resources that must reliably execute local perception and language tasks under power and communication limits.
The dataset consists of FAIR1M remote-sensing images annotated with oriented bounding boxes (OBB) and metadata for aircraft and vehicles, which enables structured geographic mission artifacts. Two fixed workloads were defined: a single-image request using train__t_10144.jpg and a serial two-image request adding train__t_10175.jpg. Each workload was repeated 20 times for latency and resource profiling. Images and metadata conform to FAIR1M licensing; detection accuracy was not benchmarked.
The architecture includes these main components: (1) a browser-based React/Vite operator workspace providing streamed Server-Sent Events (SSE) for progressive result display; (2) a FastAPI backend agent implementing tool orchestration, prompt handling, state management, and SSE streaming; (3) a project-internal YOLO-style oriented object detection endpoint (YOLO26) that outputs structured OBB detections with confidence and geographic metadata; (4) a local OpenAI-compatible language model service that generates optional human-readable mission summaries from tool artifacts; (5) an evidence layer capturing sanitized runtime logs, structured JSON, and telemetry for reproducibility.
During each mission request, the operator workspace sends instructions to the backend agent, which decides if detection is needed and invokes the detector tool. Detected results with FAIR1M metadata are returned as authoritative machine-readable artifacts before optionally generating natural-language summaries. The entire flow is streamed via SSE for incremental operator updates. Failures (e.g., invalid media uploads) are handled with structured error events.
Training details of the detector and language model are not disclosed, as the contribution focuses on orchestration rather than model innovation. The detector uses internal project weights withheld for licensing reasons. The local LLM’s exact identity is deployment-dependent but exposes a standard OpenAI-compatible API.
Measurements include end-to-end latency from request arrival to SSE done event, detector execution time, CPU utilization sampled from OS counters, and NPU accelerator load sampled via devfreq/sysfs every 200 ms. A profiler-enabled subset of runs captured finer-grained timing intervals (start → tool → first visible token → done) to localize latency.
The authors release sanitized artifacts including timing CSVs, redacted JSON logs, SSE examples, and statistical scripts, defining a reproducibility boundary for future researchers. Private weights, exact hardware model, and onboard deployment remain internal. No cross-validation, adversarial evaluation, or accuracy benchmarking was performed. The evaluation characterizes system-level fixed workload latency repeatability and resource usage rather than broad mission generalization or operational readiness.
An example run: for a single-image request, the agent invokes the detector once, which returns 7 oriented aircraft detections with confidence and geographic coordinates parsed from FAIR1M metadata. The agent then produces an optional mission summary that is streamed incrementally via SSE to the operator browser. The full-agent latency averages ~29 s, with detector runtime under 1 s, illustrating that orchestration overhead dominates.
Overall, the methodology emphasizes establishing a reproducible HIL orchestration baseline for onboard satellite edge intelligence combining perception and language tools within constrained embedded hardware.
Technical innovations
- Introduction of a hardware-in-the-loop end-to-end edge-agent architecture integrating local oriented object detection and language-model reasoning on a COTS ARM-based heterogeneous SoC for satellite payloads.
- Use of a mission-layer agent workflow that separates machine-facing structured detection artifacts (with FAIR1M metadata) from optional operator-facing natural-language summaries via SSE streaming.
- Demonstration of fixed-workload repeatability and latency profiling that identify orchestration and language-service overhead as primary contributors to end-to-end request latency, distinct from detector execution time.
- Establishment of a reproducibility boundary providing public artifacts and service contracts while preserving private model weights and hardware details.
Datasets
- FAIR1M — Moderate size (tens of thousands of images) — Public remote sensing dataset with oriented bounding box annotations and geographic metadata
Baselines vs proposed
- No direct baseline comparison reported; experimental focus is on internal repeated fixed-workload latency and resource profiling rather than detector or language-model accuracy.
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2608.03728.
Fig 1: SAT-Edge-Agent system overview. The FastAPI Agent uses the local LLM service for reasoning and
Fig 2: HIL Agent workflow. The backend receives a user instruction, selects and invokes local tools, converts
Fig 3: Repeated HIL evidence for the two fixed FAIR1M workloads. Panel A defines the protocol. Panel B
Fig 4: Request-level latency distributions supplementing Figure 3 for the two fixed FAIR1M workloads (n = 20
Limitations
- Detector accuracy, precision, recall, or any quality metric is not evaluated or reported.
- Energy efficiency and calibrated power measurements are secondary, with plug-meter pilot runs unsynchronized and kept separate from timing experiments.
- The hardware platform is a commercial off-the-shelf edge SoC; no radiation-hardening or flight qualification is claimed.
- Workloads cover only two fixed FAIR1M images, limiting generalizability to broader satellite mission distributions.
- No adversarial robustness, fault tolerance under onboard anomalies, or watchdog/recovery behavior testing is reported.
- Measured latency includes multiple components but finer attribution among orchestration, streaming, rendering, and language-model inference remains imprecise due to instrumentation gaps.
Open questions / follow-ons
- How can the observed orchestration and language-service overheads be optimized to reduce end-to-end latency for time-critical satellite missions?
- What are the accuracy trade-offs and mission impacts when substituting different detector weights or language models within the common service contract framework?
- How does the system perform under realistic radiation-hardened flight hardware and environmental conditions?
- Can energy consumption and power usage per inference be calibrated and minimized through hardware-software co-design?
Why it matters for bot defense
For practitioners in bot-defense or CAPTCHA-related fields, the paper provides a valuable case study of how constrained hardware environments can orchestrate multiple AI services—perception and language models—within strict latency and resource budgets. Although the domain is satellite edge computing, the findings about orchestration overhead dominating raw model runtime, and the importance of clear service contracts separating machine-readable structured outputs from optional natural-language summaries, generalize to edge AI scenarios relevant for bot detection or challenge-response design. The detailed reproducibility boundary and evidence package illustrate best practices for bounding complexity when integrating multiple AI components. However, the lack of adversarial robustness and security evaluation means the work serves primarily as a system integration and latency profiling reference rather than a direct bot-defense technique.
Cite
@article{arxiv2608_03728,
title={ SAT-Edge-Agent: Hardware-in-the-Loop Edge-Agent Orchestration for Onboard Satellite Intelligence },
author={ Longji He and Jeto Xu },
journal={arXiv preprint arXiv:2608.03728},
year={ 2026 },
url={https://arxiv.org/abs/2608.03728}
}