Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed from four SSDs

Recorded: Sept. 8, 2026, 9:09 p.m.

Original

Summarized

GitHub - argonautlabsai/deltafin: ARGODRIVE Deltafin: Kimi K3 (2.8T MoE) streamed from SSDs on Apple Silicon — fork of gavamedia/deltafin with the ARGODRIVE storage work and benchmark package · GitHub

Navigation MenuSign inAppearance settingsPlatformAI CODE CREATIONGitHub CopilotWrite better code with AIGitHub Copilot appDirect agents from issue to mergeMCP RegistryIntegrate external toolsDEVELOPER WORKFLOWSActionsAutomate any workflowCodespacesInstant dev environmentsIssuesPlan and track workCode ReviewManage code changesCode QualityEnforce quality at mergeAPPLICATION SECURITYGitHub Advanced SecurityFind and fix vulnerabilitiesCode securitySecure your code as you buildSecret protectionStop leaks before they startEXPLOREWhy GitHubDocumentationBlogChangelogMarketplaceView all featuresSolutionsBY COMPANY SIZEEnterprisesSmall and medium teamsStartupsNonprofitsBY USE CASEApp ModernizationDevSecOpsDevOpsCI/CDView all use casesBY INDUSTRYHealthcareFinancial servicesManufacturingGovernmentView all industriesView all solutionsResourcesEXPLORE BY TOPICAISoftware DevelopmentDevOpsSecurityView all topicsEXPLORE BY TYPECustomer storiesEvents & webinarsEbooks & reportsBusiness insightsGitHub SkillsSUPPORT & SERVICESDocumentationCustomer supportCommunity forumTrust centerPartnersView all resourcesOpen SourceCOMMUNITYGitHub SponsorsFund open source developersPROGRAMSSecurity LabMaintainer CommunityGitHub StarsArchive ProgramREPOSITORIESTopicsTrendingCollectionsEnterpriseENTERPRISE SOLUTIONSEnterprise platformAI-powered developer platformAVAILABLE ADD-ONSGitHub Advanced SecurityEnterprise-grade security featuresCopilot for BusinessEnterprise-grade AI featuresPremium SupportEnterprise-grade 24/7 supportPricingSearch/Sign inSign upAppearance settings

You signed in with another tab or window. Reload to refresh your session.
You signed out in another tab or window. Reload to refresh your session.
You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

Uh oh!

There was an error while loading. Please reload this page.

argonautlabsai

/

deltafin

Public

forked from gavamedia/deltafin

Notifications
You must be signed in to change notification settings

Fork
0

Star
3

Code

Pull requests
0

Actions

Projects

Security and quality
0

Insights

Additional navigation options

Code

Pull requests

Actions

Projects

Security and quality

Insights

mainBranchesTagsGo to fileCodeOpen more actions menu Latest commit History41 Commits41 CommitsFolders and filesNameNameLast commit messageLast commit date.github/workflows.github/workflows docsdocs k3-public-benchk3-public-bench nativenative research/studyresearch/study toolstools .gitignore.gitignore CREDITS.mdCREDITS.md Cargo.lockCargo.lock Cargo.tomlCargo.toml LICENSELICENSE README.mdREADME.md View all filesRepository files navigationREADMELicenseMore itemsARGODRIVE Deltafin
A fork of gavamedia/deltafin (MIT) running Kimi K3 from SSDs on Apple Silicon, with the ARGODRIVE storage work. The benchmark package, placement manifests and results are in k3-public-bench/; the measurement instruments are published separately as ARGODRIVE. Credits and what this fork changes: CREDITS.md. The upstream README follows.

____ _ _ __ _
| _ \ ___| | |_ __ _ / _(_)_ __
| | | |/ _ \ | __/ _` | |_| | '_ \
| |_| | __/ | || (_| | _| | | | |
|____/ \___|_|\__\__,_|_| |_|_| |_|

Run the full, never-pruned, 2.8-trillion-parameter Kimi K3 on consumer hardware, as "fast" as possible

Deltafin is a single native binary that runs full Kimi K3. Nothing pruned. Nothing skipped. K3 decides every token.
All 16 experts, every single token. No shortcuts, no "close enough." It's exactly what Moonshot shipped.
The quality rule is simple: K3 itself decides every token, and nobody else. Small draft models are allowed to guess ahead (that's where much of the speed comes from), but K3 checks every guess, and nothing reaches you without its official sign-off.
Latest Benchmarks on an M1 Max laptop

0.2901 token/s (3.447 s/token) — 1.9% higher throughput than last update

Historical M1 benchmarks:

0.2847 token/s (August 2, 2026) — 7.0% higher throughput
0.2660 token/s (July 30, 2026) — 102.9% higher throughput
0.1311 token/s (July 28, 2026) — 829.8% higher throughput
0.0141 token/s (July 27, 2026)

Mission Statement
Pure raw uncut K3 quality, as fast as possible. Speed must never come from reducing model quality. Deltafin keeps all 16 routed experts and the full K3 target as the sole authority for every single token.
Our goal is to squeeze out every last drop of efficiency possible when running a huge model like K3, with all options on the table... except for reducing quality.

But Why?!?
Deltafin is not a product pitch. It is an experiment in how far consumer hardware can be pushed, and what we can learn by attempting something so challenging.
Kimi K3 targets infrastructure on the scale of 16 nodes and roughly 4.8 TB of aggregate VRAM. That means the full 2.8T parameters and the 1M-token context window, with the expert bank never pruned. On any home setup, this is an extreme constraint. Every 1% improvement is very hard-won. But each gain can teach something.
Research and exploration is the point. That is our mission. Not everything has to be a "minimum viable product" to impress venture capitalists. If Deltafin helps make frontier models usable on a $15,000 home setup, instead of a $2,000,000 infrastructure like Kimi recommends, we believe that is worthwhile progress on our self-hosted AI journey. Plus everything learned along the way could even benefit other projects in unexpected ways.

“We choose to run the full 2.8-trillion-parameter model locally, and do the other things, not because they are easy, but because they are hard.”
— John F. Kennedy probably

Other projects appear to run full K3, somehow faster. But look closer: they've re-encoded K3's expert bank down to ~3 bits. Clever engineering toward a different goal: the smallest K3 that fits and is "close enough." Those weights are no longer the ones Moonshot released, and nobody, including them, has measured what those compromises cost.
Deltafin is the other experiment: every expert byte exactly as Moonshot shipped it, made as fast as physics allows.

1. New Installation
Deltafin installs almost everything it needs. See Requirements if you're missing anything.
# 1. Get it
git clone https://github.com/gavamedia/deltafin.git
cd deltafin

# 2. Build it
cargo build --locked --release

# 3. Download the FULL 1.7 TB K3 model to disk (optional, but fastest)
./target/release/deltafin setup --full
Or, if you don't have enough disk space:
# 3. Stream K3 as you use it (slower, but 215 GB to start)
./target/release/deltafin setup --stream
setup --stream installs the resident model and fetches exact experts on-demand only, initially running far more slowly when routes have no local cache yet. As you build up your cache over time, this can be a way to save space, storing only the parts of the model you use, running entirely off cache on disk.
Default DSpark (and optional Qwen)
The normal setup includes Inferact's Kimi-K3-DSpark. It takes 6.635 GiB on disk and approximately 4.49 GiB when admitted at runtime. Deltafin avoids materializing DSpark's redundant copy of K3's embedding. Chat and server requests use DSpark automatically when beneficial; any failures, insufficient headroom, or bad live economics simply leaves full K3 running by itself.
Qwen is a separate add-on for faster raw text continuation:
# 4. Optionally install qwen later
./target/release/deltafin setup-qwen
Qwen speeds up raw completion only: the small models guess what comes next, K3 checks the guess, and you get identical output in less time. That helps code autocomplete and other /v1/completions traffic, plus deltafin run --prompt ... — one measured 17-token completion ran 2.7× faster with the same output IDs.
This adds 4.337 GiB on disk, and because Qwen will not improve chat speed, we make it an optional add-on. You can add it to a fresh install with deltafin setup --full --include-qwen, or add it later with the command above.
2. Upgrading
From the Deltafin folder:
./target/release/deltafin upgrade
upgrade gets what you need, and rebuilds the binary. Models, converted weights, and caches are left alone. It never re-runs setup or re-downloads K3.

NOTE: Upgrading from our old python version? Inspect it first:
git status --short

# ⬆️ Continue only when that returns nothing

git pull --ff-only
cargo build --locked --release
./target/release/deltafin upgrade
Continue only when git status --short is empty. If it lists files, preserve or commit that work yourself, rather than allowing an upgrade procedure to guess. Existing model data remains in the same repository-root directories.

upgrade needs a clean, non-diverged branch, and it remembers how the binary was built, so an NVIDIA/CUDA build stays a CUDA build rather than quietly falling back to CPU. Anything unexpected safely stops the upgrade.
upgrade ignores build environment variables — it reuses whatever the binary was already built with. So to switch configuration (CPU to CUDA, say, or a moved LibTorch tree), run cargo build --locked --release yourself once with the new variables set; see Requirements. That becomes the recorded setup, and later upgrades keep it.
3. Use from the command line
# Chat: apply K3's audited chat template and stop at the model's end marker.
./target/release/deltafin run --chat \
--prompt "What are the three largest moons of Saturn?"

# Raw continuation: cap output because raw text has no chat end boundary.
./target/release/deltafin run \
--prompt "The capital of France is" --max-new 17

# Add cumulative throughput and native transaction statistics.
./target/release/deltafin run \
--prompt "The largest planet in our solar system is" --max-new 17 --stats
Without --stats, generated text streams normally instead of printing one diagnostic line per token. --max-new N limits new tokens; it does not alter the prompt or context. Chat output stops at K3's control boundary, while raw completion should normally use a bound.
Long conversations are far slower than short completions — prefill and cache grow with history, and startup prints the actual usable context bound.
4. Use through an OpenAI-compatible server
./target/release/deltafin serve --host 127.0.0.1 --port 8000
curl http://127.0.0.1:8000/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model":"deltafin-kimi-k3","stream":true,"messages":[{"role":"user","content":"Hello!"}]}'
The native server implements /v1/chat/completions, /v1/completions and /v1/models, including server-sent-event streaming. Point an OpenAI-compatible client at http://127.0.0.1:8000/v1 and use any non-empty local API key expected by that client.
The server implements a deliberately small, strictly-checked subset of the OpenAI API — text-only, one generation at a time, always greedy and reproducible — and refuses anything it cannot honor exactly with a normal OpenAI-shaped error instead of silently ignoring it. Growing chats automatically benefit from exact conversation-state reuse, draft-verified DSpark speedups and an exact-response memo; every accepted field, refusal rule and caching detail is in the server reference.
The default server response ceiling is one million tokens. Lower it with --max-tokens N when integrating clients, and raise client timeouts because full K3 responses are slow. Request JSON is bounded by --max-request-bytes (128 MiB by default). Keep the server on loopback unless you add your own authentication and network boundary.

Documentation

How the native runtime works — the one-binary in-process design: Rust core, C-ABI providers, router tracing, expert prefetch and native tokenization.
OpenAI-compatible server reference — exactly which API fields are accepted or refused, the automatic chat speedups, and the exact-response memo.
Health checks — read-only, network-free auditors that verify the runtime and each installed component after an install, upgrade or problem.
Native storage preparation — the default row-int8 resident spine that setup prepares for you, selecting the original BF16 explicitly, packing either into contiguous DFSP files, and lossless scale4 expert sidecars.
Performance reference — how to reproduce measurements with the benchmark harness, and the established M1 Max reference results.
Configuration — the few flags and environment variables that matter, and the quality guard behind them.
Supported platforms — what each host class runs (MPS/Metal, CUDA, native CPU) and the evidence status per platform.
Development reference material — why historical tools/*.py files remain in the tree as frozen reference material the native runtime never executes.

Credits
Deltafin exists because other people published amazing work:

Moonshot AI released K3's weights, architecture and readable model semantics.
Inferact released the unchanged K3-specific DSpark checkpoint; TorchSpec documents its training framework, and the vLLM team published K3 integration and recurrent/attention cache research. Deltafin's native runtime, verifier, scheduling and state transactions are its own.
GigaToken, by Marcel Rød, inspired Deltafin's automatic stable-order parallel tokenization path for large server histories.
Maurice Brown (trumb) contributed Linux, aarch64, x86 SIMD and NVIDIA findings plus DGX Spark measurements in pull request #2. Deltafin retained those findings behind reviewed capability and ABI gates.
colibri demonstrated aggressive MoE streaming and router-lookahead ideas. ds4 / DwarfStar provided especially clear prior art for exact expert streaming, cache ownership and correctness-first measurement.
Qwen supplies the optional 0.6B/1.7B proposal-only raw-completion models.
flash-linear-attention, PyTorch, llama.cpp/ggml, tiktoken and the broader local-model community supplied essential semantics and prior art.

Exact provenance and distribution boundaries are recorded in Third-party provenance and notices.
License
Deltafin's tracked project code is MIT. Kimi K3 weights, the DSpark checkpoint, optional Qwen checkpoints and all other third-party material retain their upstream terms. Deltafin is an independent project with no affiliation to Moonshot AI.
AboutARGODRIVE Deltafin: Kimi K3 (2.8T MoE) streamed from SSDs on Apple Silicon — fork of gavamedia/deltafin with the ARGODRIVE storage work and benchmark packageResourcesReadmeLicenseActivityCustom propertiesStars3 starsWatchers0 watchingForks0 forksReport repositoryReleasesPackagesContributorsLanguages

Footer

Footer navigation

Terms

Privacy

Security

Status

Community

Docs

Contact

Manage cookies

Do not share my personal information

You can’t perform that action at this time.

Deltafin is a fork of gavamedia/deltafin that incorporates ARGODRIVE storage work and benchmark packages, focusing on running the full 2.8 trillion-parameter Kimi K3 Mixture of Experts (MoE) model from SSDs on Apple Silicon hardware. The core philosophy behind Deltafin is to achieve the highest possible raw quality with the fastest possible execution speed, asserting that efficiency should never come at the expense of model quality. The project aims to experiment with pushing the limits of consumer hardware in running frontier models, challenging the assumption that such tasks necessitate massive infrastructure, by demonstrating that achieving this can be worthwhile progress for self-hosted AI research and implementation.

Deltafin insists on running the entire Kimi K3 model without any pruning or skipping of experts, ensuring that all sixteen experts contribute to every token generation, with K3 itself retaining ultimate authority over every output. This contrasts with other approaches where smaller K3 variants are derived by re-encoding the expert bank down to approximately three bits, a method that sacrifices fidelity for efficiency. Deltafin's objective is to maintain the exact weights released by Moonshot AI while optimizing inference speed across various platforms.

The implementation involves a native binary design and sophisticated resource management. Installation requires cloning the repository, building the application, and optionally downloading or streaming the complete 1.7 terabyte K3 model to disk. A key feature of the setup is its efficient handling of model components by avoiding the materialization of redundant copies of K3 embedding that are present in related frameworks like DSpark. The system manages runtime execution by ensuring full K3 operates independently if necessary due to insufficient headroom or operational failures.

The repository provides clear command-line interfaces for various use cases. Users can initiate chat sessions using K3's audited template, execute raw text continuation prompts with controls on the maximum new tokens generated, and gather cumulative throughput statistics. Furthermore, Deltafin implements an OpenAI-compatible server that adheres to standard API formats for chat completions and other endpoints, ensuring strict adherence to response rules and including detailed information on caching and conversation state reuse.

The project is built upon contributions from numerous related works, recognizing the foundational work of Moonshot AI for the K3 architecture and weights, Inferact for the DSpark checkpoint, TorchSpec for the training framework, and others contributing to stream processing and expert streaming mechanisms. The structure also encompasses extensive documentation detailing the native runtime design, performance references, configuration options, and provenance tracking, establishing Deltafin as an independent experiment in optimizing large language model execution on consumer-grade systems.