Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection
Source: arXiv:2606.27314 · Published 2026-06-25 · By Hamid Reza Firoozfar, Mohammadsadegh Abolhasani, Reza Mousavi, Paul Jen-Hwa Hu
TL;DR
This paper addresses the challenge of detecting indirect linguistic encoding (ILE) in social media posts, which users employ to evade moderation by camouflaging sensitive meanings. Existing taxonomies of ILE tend to have narrow or ad hoc coverage focused on surface forms, limiting their usefulness for Large Language Model (LLM)-based detection. The authors propose a novel, comprehensive, mechanism-oriented taxonomy that classifies ILE by the underlying encoding and decoding pathways rather than communicative intent or topical content. This taxonomy includes 11 top-level mechanism classes and 33 fine-grained sub-classes derived through an iterative, rigorous development process that combined conceptual theory and empirical data. They compiled a dataset of 2,000 manually annotated posts from TikTok and Bluesky, labeled for ILE presence, spans, and mechanism classes, to evaluate taxonomy-guided prompting of three LLMs against four existing taxonomies and a no-taxonomy baseline. The proposed taxonomy consistently outperforms all baselines across document- and span-level metrics, improving accuracy by up to 4.7% and F1 by 5.4%, and notably excels on mechanism classes neglected by prior taxonomies. The work demonstrates that taxonomy comprehensiveness and fine granularity are critical for effective LLM detection of coded language and highlights compositional encodings where multiple mechanisms co-occur. This mechanism-level taxonomy offers a stable scaffold for addressing the rapidly evolving coded language challenge in content moderation pipelines.
Key findings
- The proposed mechanism-oriented taxonomy yields 4.7% higher document-level accuracy and 5.4% higher F1 compared to the best existing taxonomy (Leal-Arenas & Corizzo) on GPT-5.4.
- At span-level detection, the taxonomy achieves 66.2% F1 on GPT-5.4, improving 3.4% over the closest benchmark taxonomy.
- Across all three tested LLMs (GPT-5.4, Claude Sonnet 4.6, DeepSeek V4), the proposed taxonomy consistently achieves the highest document- and span-level F1 scores.
- Each of the 11 top-level categories contributes non-redundant information: removing any category degrades performance by 1.7% to 3.6% accuracy and 1.9% to 3.9% macro-F1.
- The largest recall improvements are on classes poorly covered before, e.g. Formal Encoding Systems (+62.2%), Morpho-Lexical Encoding (+24.2%), and Referential Alias Encoding (+20.8%).
- Approximately 15% of ILE instances exhibit compositionality — multiple encoding mechanisms combined within a single expression.
- Baseline supervised models achieve ≤68.4% accuracy, while unsupervised NLP baselines perform no better than random (≤58.6%), confirming superiority of taxonomy-guided LLM prompting.
- Two trained annotators achieved high agreement: Cohen’s κ = 0.85 for document-level presence, 0.79 for taxonomy class assignment, and 0.89 for token span boundaries.
Threat model
The adversary comprises social media users deploying indirect linguistic encodings to evade content moderation, surveillance, and automated detection. They can generate diverse surface forms including orthographic, phonetic, symbolic, and referential variants, sometimes combining multiple mechanisms. They do not possess unlimited cryptographic sophistication, but exploit shared community conventions and linguistic cues not explicitly captured by fixed lexicons. The adversary cannot easily defeat mechanism-level detection if coverage is sufficiently comprehensive but can adapt by inventing new encoding methods outside the taxonomy’s scope.
Methodology — deep read
The study begins with a clear threat model: social media users deliberately encode sensitive meanings indirectly to evade moderation and detection systems. The adversary is assumed to use diverse linguistic mechanisms but does not have infinite creativity — the taxonomy aims to capture recurring encoding pathways. The authors collected 2,000 English-language posts (1,400 TikTok captions from Mar-May 2026, 600 Bluesky posts from Oct 2025-Jan 2026) via stratified sampling skewed toward likely ILE content using seed terms and random samples. Data was cleaned to remove duplicates, bots, and non-English content.
Two trained annotators labeled data on three nested layers: (1) binary document-level ILE presence, (2) token-level spans of encoded expressions, and (3) classification of each span into 1+ mechanism classes and sub-classes from the proposed taxonomy. Inter-annotator agreement was measured with Cohen’s kappa, yielding high reliability (0.85 document-level, 0.79 mechanism-level).
The taxonomy development followed the iterative method of Nickerson et al. (2013), starting from a meta-characteristic focusing on the formal encoding/decoding pathways rather than intent or topic. Four iterations alternated conceptual-to-empirical (extracting categories from prior taxonomies and linguistic theory) and empirical-to-conceptual (discovering new categories from unclassified instances). The final taxonomy comprises 11 top-level mechanisms with 33 sub-classes, allowing overlapping assignments.
Three state-of-the-art LLMs (GPT-5.4, Claude Sonnet 4.6, DeepSeek V4 Flash) were prompted in few-shot mode with 4 examples. Six prompt variants included: the proposed taxonomy, four existing taxonomies, and a no-taxonomy baseline. Four non-LLM baselines (TF-IDF+LR, char-ngram+SVM, Word2Vec cosine, embedding centrality) contextualize results.
Evaluation considered document-level binary classification (accuracy, macro-precision, recall, F1) and span-level detection (precision, recall, F1) with soft matching based on token and character overlap (≥0.5 similarity threshold). Pairwise statistical significance tests used paired bootstrap resampling (10,000 iterations) and McNemar’s test for document accuracy.
An ablation study dropped each taxonomy class to measure impact on GPT-5.4 performance, confirming all categories provide non-redundant information with effect sizes up to 3.9% F1 loss. Per-class recall analyses revealed larger gains where prior taxonomies had sparse coverage.
The dataset and code are publicly released except TikTok text due to platform restrictions. The prompting method, exact templates, and evaluation scripts are detailed in appendices, supporting reproducibility.
One concrete example: the post encoding "pornstar" via three emojis simultaneously using iconic resemblance, attribute-based alias, and name-based pictorial mapping illustrates multi-mechanism compositionality the taxonomy can represent, compared against previous approaches that fail to capture this complexity.
Technical innovations
- A mechanism-oriented taxonomy of indirect linguistic encoding with 11 top-level classes and 33 fine-grained sub-classes, derived via iterative conceptual-to-empirical and empirical-to-conceptual refinement constrained by a formal encoding-decoding meta-characteristic.
- Explicit allowance for non-mutually-exclusive overlapping mechanisms within single encoded spans, enabling detection of compositional coded language.
- Incorporation of novel classes missing or underrepresented in prior taxonomies, such as Formal Encoding Systems and Acronymic Camouflage as a distinct sub-mechanism within Conventional Sign Reassignment.
- Demonstration that comprehensive, mechanism-oriented taxonomies serve as more effective scaffolds for LLM-based detection of ILE than surface-form or intent-based taxonomies.
- Development of a large manually annotated English dataset of 2,000 TikTok and Bluesky posts labeled for ILE presence, span, and detailed mechanism assignments to enable rigorous evaluation.
Datasets
- TikTok captions — 1,400 posts — proprietary, text withheld due to data-sharing agreements
- Bluesky posts — 600 posts — publicly released (text portion)
- Combined 2,000 posts manually annotated for ILE presence, span, and mechanism classes
Baselines vs proposed
- TF-IDF + Logistic Regression (Grouped CV): Accuracy = 0.681 vs Proposed Taxonomy GPT-5.4: 0.843
- Character N-Grams + Linear SVM (Grouped CV): Accuracy = 0.684 vs Proposed Taxonomy GPT-5.4: 0.843
- Word2Vec Cosine-to-Seed (Unsupervised): Accuracy = 0.515 vs Proposed Taxonomy GPT-5.4: 0.843
- Embedding Graph Centrality (Unsupervised): Accuracy = 0.586 vs Proposed Taxonomy GPT-5.4: 0.843
- Leal-Arenas & Corizzo taxonomy GPT-5.4: Document F1 = 0.796 vs Proposed Taxonomy GPT-5.4: 0.839
- Calhoun & Fawcett taxonomy GPT-5.4: Document F1 = 0.753 vs Proposed Taxonomy GPT-5.4: 0.839
- Fillies & Paschke taxonomy GPT-5.4: Document F1 = 0.782 vs Proposed Taxonomy GPT-5.4: 0.839
- No-taxonomy prompt GPT-5.4: Document F1 = 0.771 vs Proposed Taxonomy GPT-5.4: 0.839
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.27314.
Fig 1: An ILE example from TikTok. Left: the flagged term “pornstar” returns no results. Right: a creator
Fig 2: Overview of the taxonomy development process.
Fig 3: Overview of the proposed taxonomy: the eleven top-level mechanism classes and their correspond-
Limitations
- Focus exclusively on English-language posts limits generalization to languages with non-alphabetic scripts or richer morphology; some mechanism classes may not fully transfer.
- Dataset limited to two platforms (TikTok, Bluesky) with specific moderation regimes, possibly biasing observed ILE types and distributions.
- TikTok post texts cannot be publicly released due to platform restrictions, reducing full reproducibility for that portion.
- Evaluation lacks adversarial testing against adaptive encoders or out-of-distribution ILE unknown at annotation time.
- Framework does not address downstream intent or harm classification; mechanism detection alone cannot disambiguate benign versus malicious use.
- Handling of rapidly emerging new mechanisms remains an open methodological challenge beyond current fixed taxonomy schema.
Open questions / follow-ons
- How can mechanism-oriented taxonomies be dynamically updated or extended as new ILE mechanisms emerge rapidly in evolving online discourse?
- What are the best methods to integrate detected mechanisms with contextual signals (intent, harm, topic) for end-to-end content moderation to minimize over-blocking of benign discourse?
- Can this taxonomy and detection approach be effectively adapted to structurally different languages, particularly non-alphabetic or morphologically rich systems?
- How resilient is taxonomy-guided LLM prompting to adversarially crafted encodings explicitly designed to circumvent known mechanism classes?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners, this work highlights the importance of understanding and systematically categorizing the mechanisms behind obfuscated or coded linguistic content. Since automated systems increasingly rely on LLMs for content analysis, employing a comprehensive mechanism-oriented taxonomy as a prompt scaffold can significantly improve detection of disguised language that bots or malicious actors use to evade moderation. Beyond direct keyword matching or surface-form perturbation detection, such a taxonomy provides a structured, extensible way to capture evolving indirect encodings and their compositional use. This foundation can help design more robust behavioral or semantic CAPTCHAs that distinguish genuine human use of indirect language from adversaries using coded or obfuscated expressions. Moreover, the emphasis on separating encoding mechanisms from intent serves as a useful reminder that detection is only part of the challenge; effective defense also requires context-aware interpretation downstream. Overall, this work advocates that thorough mechanistic modeling of coded language improves LLM-based filtering pipelines that contribute to bot and adversarial content defense.
Cite
@article{arxiv2606_27314,
title={ Beyond Surface Forms: A Comprehensive, Mechanism-Oriented Taxonomy of Indirect Linguistic Encoding for LLM-Based Coded Language Detection },
author={ Hamid Reza Firoozfar and Mohammadsadegh Abolhasani and Reza Mousavi and Paul Jen-Hwa Hu },
journal={arXiv preprint arXiv:2606.27314},
year={ 2026 },
url={https://arxiv.org/abs/2606.27314}
}