The Implications of Linguistic Illegibility for LLM Security
Recorded: Sept. 18, 2026, 8 p.m.
| Original | Summarized |
[2609.02852] The Implications of Linguistic Illegibility for LLM Security
Skip to main content Search Log in Search arXiv Press Enter to search · Advanced search Computer Science > Machine Learning arXiv:2609.02852 (cs) [Submitted on 2 Sep 2026] Abstract:LLMs are trained to generate natural language. However, various strands of evidence indicate that an LLM's externalized linguistic outputs and mechanistically-extracted linguistic features can be an unreliable lens for understanding internal model computation. We introduce the term ``linguistic illegibility'' to broadly refer to scenarios in which an LLM's externalized or mechanistically-probed language artifacts fail to represent how the model actually thinks. We argue that the specter of linguistic illegibility is unavoidable for LLMs whose internal computations are not directly expressed via language, but rather math over activation spaces (with lossy translations between activation spaces and natural language happening at the bookends). If linguistic illegibility is always possible, then security mechanisms that rely on a model's linguistic self-reporting (e.g., chain-of-thought monitoring, constitutional self-critique, activation probing for linguistically-defined feature vectors) can never be completely sound; the model sandbox will always need isolation techniques whose guarantees do not depend on reading a model's linguistic state at all. We argue that observing a model's outputs using taint tracking is a promising approach for an effective sandbox: regardless of how a model linguistically self-reports, a taint tracking policy can define, a priori, various pieces of system state that should never be influenced by model-produced data. We also discuss several additional sandboxing mechanisms (e.g., robust virtualization, third-party auditing of sandboxing configurations) which collectively provide a critical floor beneath linguistic monitoring, and would have mitigated recent sandbox exploits by frontier models. Subjects: Machine Learning (cs.LG); Cryptography and Security (cs.CR) Cite as: Focus to learn more arXiv-issued DOI via DataCite (pending registration) Submission history From: James Mickens [view email] [v1]
Full-text links: View a PDF of the paper titled The Implications of Linguistic Illegibility for LLM Security, by James MickensView PDFHTML (experimental)TeX Source view license < prev | new Change to browse by: References & Citations NASA ADSGoogle Scholar export BibTeX citation BibTeX formatted citation loading... Data provided by: Bookmark
Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) Demos Demos Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?)
IArxiv recommender toggle IArxiv Recommender Author About arXivLabs arXivLabs: experimental projects with community collaborators Which authors of this paper are endorsers? | We gratefully acknowledge support from About Major funding support from |
The work introduces the concept of linguistic illegibility to describe situations where an LLM's externalized or mechanistically-probed language artifacts do not accurately reflect the model's internal computational processes. The authors argue that this linguistic illegibility is inherent for LLMs whose internal computations are fundamentally represented by mathematics over activation spaces, where the translation between these activation spaces and natural language involves lossy mappings at the boundaries. This fundamental disconnect between internal mathematical computation and external linguistic expression has critical implications for the security of large language models. If linguistic illegibility is a persistent possibility, security mechanisms that depend on a model's linguistic self-reporting, such as chain-of-thought monitoring, constitutional self-critique, or activation probing based on linguistically defined feature vectors, cannot be considered entirely sound. Consequently, a robust model sandbox must implement isolation techniques whose guarantees are independent of reading or interpreting the model's linguistic state. The authors propose an alternative approach for effective sandboxing by focusing on taint tracking of model outputs. They contend that regardless of how a model linguistically reports its state, a taint tracking policy can define specific segments of system state that must remain inviolable, preventing them from being influenced by model-produced data. Furthermore, the paper discusses several complementary sandboxing mechanisms, including robust virtualization and third-party auditing of sandboxing configurations, which collectively establish a necessary baseline layer beneath linguistic monitoring. The authors suggest that these combined approaches could have effectively mitigated recent sandbox exploits encountered by frontier models. |