Cache-to-Cache: Direct Semantic Communication Between Large Language Models
Recorded: Sept. 18, 2026, 8 p.m.
| Original | Summarized |
[2510.03215] Cache-to-Cache: Direct Semantic Communication Between Large Language Models
Skip to main content Search Log in Search arXiv Press Enter to search · Advanced search Computer Science > Computation and Language arXiv:2510.03215 (cs) [Submitted on 3 Oct 2025 (v1), last revised 2 Mar 2026 (this version, v2)] Abstract:Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at this https URL. Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG) ACM classes: Cite as: Focus to learn more arXiv-issued DOI via DataCite Submission history From: Tianyu Fu [view email] [v1]
Full-text links: View a PDF of the paper titled Cache-to-Cache: Direct Semantic Communication Between Large Language Models, by Tianyu Fu and 6 other authorsView PDFHTML (experimental)TeX Source < prev | new Change to browse by: References & Citations NASA ADSGoogle Scholar export BibTeX citation BibTeX formatted citation loading... Data provided by: Bookmark
Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer (What is the Explorer?) Connected Papers Toggle Connected Papers (What is Connected Papers?) Litmaps Toggle Litmaps (What is Litmaps?) scite.ai Toggle scite Smart Citations (What are Smart Citations?) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv (What is alphaXiv?) Links to Code Toggle CatalyzeX Code Finder for Papers (What is CatalyzeX?) DagsHub Toggle DagsHub (What is DagsHub?) GotitPub Toggle Gotit.pub (What is GotitPub?) Huggingface Toggle Hugging Face (What is Huggingface?) ScienceCast Toggle ScienceCast (What is ScienceCast?) Demos Demos Replicate Toggle Replicate (What is Replicate?) Spaces Toggle Hugging Face Spaces (What is Spaces?) Spaces Toggle TXYZ.AI (What is TXYZ.AI?) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower (What are Influence Flowers?) Core recommender toggle CORE Recommender (What is CORE?) Author About arXivLabs arXivLabs: experimental projects with community collaborators Which authors of this paper are endorsers? | We gratefully acknowledge support from About Major funding support from |
Multi-Large Language Model systems leverage the complementary strengths of diverse models to achieve performance and efficiency gains unattainable by single models. Current methods for communication between Large Language Models typically rely on text-based exchange, which necessitates transforming internal representations into output token sequences. This process results in the loss of rich semantic information and introduces generation latency due to token-by-token processing. Researchers investigate whether LLMs can communicate beyond the limitations of text. Oracle experiments indicate that enhancing the semantics within the Key-Value (KV) Cache can improve response quality without increasing the cache size, thereby positioning the KV-Cache as an effective medium for inter-model communication. Building upon this, the authors propose Cache-to-Cache (C2C), a novel paradigm designed for direct semantic communication between LLMs. The core of the C2C approach involves employing a neural network to project and fuse the KV-caches from the source model and the target model, facilitating direct semantic transfer. To optimize this communication, the method incorporates a learnable gating mechanism that selectively determines which layers in the target model benefit most from this cache communication. This mechanism allows the system to selectively focus the semantic transfer where it is most beneficial. By leveraging the deep and specialized semantics embedded in both models, C2C avoids the need for explicit intermediate text generation, which is characteristic of text communication methods. Empirical evaluations demonstrate the superiority of the C2C method. The experiments show that C2C achieves an average accuracy improvement ranging from 6.4 to 14.2% compared to individual models. Furthermore, C2C surpasses the text communication paradigm by approximately 3.1 to 5.4%, while simultaneously delivering an average speedup in latency of 2.5 times. This indicates that direct semantic communication via cache fusion is not only more effective in capturing shared knowledge but also significantly more efficient in terms of computational time. |