GitHub - General-Instinct/InstinctFlash: High-Performance Serving Runtime for Robotics Models · GitHub

Skip to content

Navigation MenuSign inAppearance settingsPlatformAI CODE CREATIONGitHub CopilotWrite better code with AIGitHub Copilot appDirect agents from issue to mergeMCP RegistryIntegrate external toolsDEVELOPER WORKFLOWSActionsAutomate any workflowCodespacesInstant dev environmentsIssuesPlan and track workCode ReviewManage code changesCode QualityEnforce quality at mergeAPPLICATION SECURITYGitHub Advanced SecurityFind and fix vulnerabilitiesCode securitySecure your code as you buildSecret protectionStop leaks before they startEXPLOREWhy GitHubDocumentationBlogChangelogMarketplaceView all featuresSolutionsBY COMPANY SIZEEnterprisesSmall and medium teamsStartupsNonprofitsBY USE CASEApp ModernizationDevSecOpsDevOpsCI/CDView all use casesBY INDUSTRYHealthcareFinancial servicesManufacturingGovernmentView all industriesView all solutionsResourcesEXPLORE BY TOPICAISoftware DevelopmentDevOpsSecurityView all topicsEXPLORE BY TYPECustomer storiesEvents & webinarsEbooks & reportsBusiness insightsGitHub SkillsSUPPORT & SERVICESDocumentationCustomer supportCommunity forumTrust centerPartnersView all resourcesOpen SourceCOMMUNITYGitHub SponsorsFund open source developersPROGRAMSSecurity LabMaintainer CommunityGitHub StarsArchive ProgramREPOSITORIESTopicsTrendingCollectionsEnterpriseENTERPRISE SOLUTIONSEnterprise platformAI-powered developer platformAVAILABLE ADD-ONSGitHub Advanced SecurityEnterprise-grade security featuresCopilot for BusinessEnterprise-grade AI featuresPremium SupportEnterprise-grade 24/7 supportPricingSearch/Sign inSign upAppearance settings

You signed in with another tab or window. Reload to refresh your session.
You signed out in another tab or window. Reload to refresh your session.
You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

General-Instinct

/

InstinctFlash

Public

Notifications
You must be signed in to change notification settings

Fork
3

Star
14

Code

Issues
0

Pull requests
4

Actions

Projects

Security and quality
0

Insights

Additional navigation options

Code

Issues

Pull requests

Actions

Projects

Security and quality

Insights

mainBranchesTagsGo to fileCodeOpen more actions menuLatest commit History278 Commits278 CommitsFolders and filesNameNameLast commit messageLast commit date.github/workflows.github/workflows  assetsassets  benchmarksbenchmarks  docsdocs  evaleval  examplesexamples  instinctflashinstinctflash  releaserelease  scriptsscripts  servingserving  teststests  toolstools  .gitignore.gitignore  .python-version.python-version  INSTALL.rstINSTALL.rst  LICENSELICENSE  README.mdREADME.md  REPRODUCE.rstREPRODUCE.rst  pyproject.tomlpyproject.toml  requirements-serving.txtrequirements-serving.txt  uv.lockuv.lock  View all filesRepository files navigationREADMEAGPL-3.0 licenseMore items

A high-performance serving framework for robotics models.

What's new 🔥

[2026/09/17] RTX 5090 support. Deploy on your workstation with the same Runtime API used on Jetson Thor. Setup · Reproduce.
[2026/09/16] RTX 4090 support. Desktop inference and WebSocket serving with dedicated installation profiles. Setup · Reproduce.
[2026/09/15] Full-source release. Eight robotics model families, acceleration kernels, and Python / WebSocket serving through one Runtime. Get started.
[2026/09/15] Jetson Thor benchmarks. Up to 33.78× speedup with LingBot-VA @2V/4A, using FP8 and fewer sampling steps. Results · Reproduce.

Results
Prediction p50 on Jetson Thor (ms), measured September 15, 2026.
We’ve seen up to 33.78× speedup with no observed loss in task performance in our real-robot tests.

Model
Acceleration line
PyTorch
InstinctFlash
Speedup

LingBot-VA
FP8 · 25V/50A
15506.32
2891.74
5.36×

↳ LingBot-VA
FP8 · 2V/4A
2071.29
459.10
4.51×

LingBot-VLA-4B
FP8
624.22
221.53
2.82×

LingBot-VLA-V2-6B
FP8
734.56
394.11
1.86×

Cosmos3 Edge DROID
NUMERIC · UniPC4 / CFG3
3393.78
1048.01
3.24×

Cosmos3 Nano DROID
NUMERIC · UniPC4 / CFG3
10184.68
4772.38
2.13×

pi05
FP8
408.58
51.85
7.88×

GR00T N1.7
BITEXACT
139.50
117.30
1.19×

DreamZero DROID
FP8 · 16 steps · dynamic cache
23563.08
11899.42
1.98×

VA measures early continuations; each row compares the same schedule.
The 33.78× headline includes 25V/50A → 2V/4A.
FP8 and sampling changes are optional.
Protocol and raw results · Native VA 2V/4A · Reproduction commands
Install
git clone https://github.com/General-Instinct/InstinctFlash && cd InstinctFlash
python3 -m venv .venv-core
source .venv-core/bin/activate
python -m pip install . uv==0.12.5
The Python 3.10+ core inspects checkpoints and plans without PyTorch or a GPU.
Inference uses a separate, pinned environment for each model family. For RTX 4090:
python3 scripts/bootstrap_vendor.py install pi05 --target rtx4090 \
--python python3.12 --root ~/ifl-pi05-4090 --ptxas /usr/local/cuda/bin/ptxas
source ~/ifl-pi05-4090/activate.sh
Use va, vla4, vla2, pi05, groot, edge, nano or dreamzero.
Edge and Nano use Python 3.13; the other families use Python 3.12.
The bootstrap installs the upstream source, compatibility patches, core and adapter.
Model weights are downloaded separately. See RTX 5090 setup,
RTX 4090 setup
or Jetson Thor setup, which selects --target jetson_thor
and uses the Thor CUDA backend build.
Load a model
Your fine-tuned checkpoint — the expected case. Point serve at the training output; it
detects the family, writes the small instinctflash.json declaration from what the checkpoint
itself proves, and starts serving. One command:
instinctflash serve /path/to/your/checkpoint
Anything the checkpoint cannot prove is asked for explicitly, never guessed. Once the
declaration exists (serve writes it on first run), the same directory also loads in Python:
from instinctflash import Runtime

runtime = Runtime.from_pretrained("/path/to/your/checkpoint")
A stock release — use its Hub id after installing the family's environment:
runtime = Runtime.from_pretrained("robbyant/lingbot-va-posttrain-robotwin")

family
model id

LingBot-VA (5B WAM)
robbyant/lingbot-va-posttrain-robotwin

LingBot-VLA-4B
robbyant/lingbot-vla-4b-posttrain-robotwin

LingBot-VLA-V2-6B
robbyant/lingbot-vla-v2-6b-robotwin

pi0.5
lerobot/pi05_base · lerobot/pi05_libero_finetuned_v044

GR00T-N1.7-3B
nvidia/GR00T-N1.7-3B

Cosmos3 policies
nvidia/Cosmos3-Edge-Policy-DROID · nvidia/Cosmos3-Nano-Policy-DROID

DreamZero
GEAR-Dreams/DreamZero-DROID

Fine-tunes reuse their family's adapter; quality is evaluated per checkpoint.
The same Runtime defaults to precision="native" with a BITEXACT transformation ceiling.
Use tier_ceiling="numeric" to allow numerical changes, or precision="fp8"
(CLI: --fp8) to explicitly enable FP8. Step schedules are selected separately.
See precision policy and
FP8 support and validation.
DreamZero's opt-in dynamic step cache requires
tier_ceiling="behavioral" with either precision. See the
Thor measurements.
Get actions
In process — this is the whole Python API:
with runtime.episode(prompt="put the bottle in the dustbin") as episode:
while not done:
result = episode.predict(observation)
action = result["action"]
observation is a dict in the model's own format; result["action"] contains its action array.
For LingBot-VA, pass executed_action=... when the controller changes a predicted action
chunk, so the next prediction uses the actions actually executed.
Over the network — the serve command above hosts the same runtime behind the
msgpack-over-websocket wire protocol the pi0/openpi ecosystem already speaks, so existing
robot-side clients connect unchanged (pip install openpi-client):
from openpi_client.websocket_client_policy import WebsocketClientPolicy

client = WebsocketClientPolicy("my-server", 8000)
result = client.infer(observation)
action = result["action"]
The prompt rides in the observation; a changed prompt starts a new episode, and a client can
say it explicitly with {"reset": True, ...}. Four flags cover the rest:

--serve.dry_run — preflight only: device, declaration, plan. No weights, no GPU.
--serve.smoke — load, produce one action, exit.
--serve.seed — seed native execution for paired comparisons; FP8 serving rejects this option.
--serve.viz — stream observations, actions and latency to a Rerun viewer.

The second verb, instinctflash validate <dir>, checks a checkpoint is publishable; given
--validate.teacher_outcomes/.student_outcomes/.margin it also certifies non-inferiority and
stamps the certificate into the package.
Benchmark acceleration and quantization
After the vendor and auxiliary-asset preparation, reproduce paired
eager/default/selected Runtime measurements with the included inputs and fixed
checkpoint revision. Thor also requires its native backend.
Keep the model and asset environments activated. For RTX 4090:
python -I -m benchmarks.regression.reproduce prepare --target rtx4090 \
--model pi05 --mode fp8 --output pi05-inputs
python -I -m benchmarks.regression.reproduce run --prepared pi05-inputs --output pi05-results
python -I -m benchmarks.regression.serve_smoke --prepared pi05-inputs --output pi05-serving
run writes checked JSON/CSV reports and full action arrays. serve_smoke tests
the actual CLI and WebSocket pipeline across two episodes. Use --mode native
for default precision; FP8, numerical compilation and changed schedules are
explicit selections. Reproduction guide.
For additional framework comparisons, use the pinned comparison recipes.
Compare original and optimized models with instinctflash eval. Reports separate
latency, action agreement and simulator task success.
instinctflash eval adapters
instinctflash eval coverage --run /path/to/run
instinctflash eval --registry plan.registry.json report --run /path/to/run
See the evaluation guide to create and run
paired LIBERO / RoboTwin experiments, or benchmark details
for acceleration and quantization protocols. Results: simulator screening
and repeatability, checkpoints and edge latency.
The expanded V2 evaluation binds
latency and quality evidence to execution profiles and checks explicit control budgets.
The native qualification workflow
adds fresh-start admission, retained failures and checkpoint-specific evidence for each device.
LingBot-VA Hub IDs retain native step counts; 2V/4A requires an explicit nfe selection.
The September 9 Thor comparison
separates native acceleration, FP8 Runtime gains and paired task outcomes;
historical engine controls isolate additional implementation effects.
Shared BF16 fusion provides an opt-in NUMERIC path, with per-model compatibility and paired Thor regression results.
Shared tensor caching and prefill separation extend native Cosmos optimization to Edge and Nano; exact caching and NUMERIC compilation remain separate options.
Framework overview
InstinctFlash keeps model declarations, optimization planning, runtime execution, and evidence in
one inspectable path, whether it is called from Python or the command line.
Architecture
A checkpoint carries a short declaration of what it is. The runtime reads the declaration, decides
which optimizations are provably valid for those weights, applies them, and shows its work:
checkpoint ─▶ adapter ─▶ planner ─▶ engine passes ─▶ actions
declares what decides what apply and measure
the model is is valid (no GPU, each optimization
no weights needed)

Optimization is organized in six layers, by what each one changes:

layer

changes

1
MODEL
what is computed — distillation, step reduction, checkpoint compression (InstinctCompress, instinct-pdd)

2
GRAPH
when work is issued — prefill extraction, CUDA-graph capture, memory planning

3
CACHE
what is recomputed — KV reuse, cross-attention and episode caches

4
ATTENTION
how tokens mix — FlashAttention, hybrid and linear attention

5
KERNEL
how a kernel is written — backend and layout dispatch, fusion

6
HARDWARE
what it executes on — fp8/int8, TensorRT, Jetson-class edge devices (serving/)

Layer 1 changes the weights and produces a checkpoint; it lives in the companion repos. Layers
2–6 change how the weights execute and produce a plan; they are the runtime in this repo. The
layers are not a priority order — the runtime measures where the time actually goes and starts
there.
Add a model
To add your own model family, declare an instinctflash.adapters entry point and pip install
your package — see examples/external_plugin/.
Roadmap

Few-step distillation, when needed — only after native optimizations miss a declared
edge control budget; compare each student with its teacher and the matched untrained
schedule using paired closed-loop evaluation.
LingBot-VA on the edge engine — native and FP8 serving on Jetson Thor,
with paired inference and WebSocket checks for full and 2V/4A schedules.
Attention upgrades — a faster NUMERIC-tier attention arm beside the BITEXACT default
for pi05-class models; hybrid and linear attention for long-context world models.
Cosmos3 and DreamZero on Thor — pinned installation, paired inference
and installed CLI/WebSocket checks; task quality remains a separate evaluation.
Device-specific serving defaults — measure each family and operating point;
select a verified path within the caller's precision constraints. LingBot-VLA-V2
native Thor capture and LingBot-VA saturation profiling are complete; selective
VA action capture showed no speedup and stays experimental.
Results and evidence.
Execution-bound budget selection is available; the expanded V2 H100
evaluation remains SCREEN.

AboutHigh-Performance Serving Runtime for Robotics Modelsgeneral-instinct.com/ResourcesReadmeAGPL-3.0 licenseActivityCustom propertiesStars14 starsWatchers0 watchingForks3 forksReport repositoryReleasesPackagesContributorsLanguages

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms

Privacy

Security

Status

Community

Docs

Contact

Manage cookies

Do not share my personal information

You can’t perform that action at this time.

The InstinctFlash project establishes a high-performance serving runtime specifically designed for robotics models. This framework consolidates model declarations, optimization planning, runtime execution, and supporting evidence into a single inspectable path, accessible via both Python and command-line interfaces. The core architecture relies on a checkpoint carrying a declaration of its properties, which the runtime then uses to determine valid optimizations, apply them, and measure performance. This process is structured across six interdependent layers that govern the execution: Model, Graph, Cache, Attention, Kernel, and Hardware. The model layer defines what is computed (such as distillation or compression), the graph layer manages when work is issued (like prefill extraction), the cache layer handles recomputations (like KV reuse), the attention layer dictates how tokens mix (implementing FlashAttention), the kernel layer specifies kernel implementation (backend dispatch and fusion), and the hardware layer defines the execution environment (like FP8 or TensorRT deployment on edge devices). The runtime dynamically measures time spent across these layers to initiate optimizations from the most time-consuming aspects.

The framework supports various robotics model families, including LingBot-VA, LingBot-VLA-4B, LingBot-VLA-V2-6B, pi05, GR00T-N1.7-3B, Cosmos3 policies, and DreamZero. The system allows users to define precision and scheduling explicitly, with options to enable FP8 or numerical transformations, and specific mechanisms, like the opt-in dynamic step cache in DreamZero, which requires specific ceiling settings. The system integrates with existing protocols by hosting the runtime via a msgpack-over-websocket wire protocol, allowing existing robot-side clients to connect seamlessly.

The framework offers comprehensive operational capabilities through a Python API where users can execute episodes by passing observations to the model and receiving actions. This process supports features like passing executed actions when a controller changes a predicted action chunk to ensure sequential predictions are based on executed results. For network operations, clients can infer results directly, and there are specific command-line flags for serving, such as dry runs, smoke tests that produce one action, seeding native execution for comparisons, and visualization for streaming observations and latency. Furthermore, the framework includes validation tools, such as instinctflash validate, which can certify a checkpoint's publishability by checking against pre-defined outcomes and certifying non-inferiority.

Performance gains are demonstrated through benchmarks, notably showing speedups on hardware like Jetson Thor, where speedups reaching up to 33.78 times were observed for specific model execution, achieved through utilizing FP8 and fewer sampling steps. The system provides tools for reproducing these measurements and comparing optimized models against original ones, allowing for the separate evaluation of latency, action agreement, and simulator task success rates. The framework supports advanced evaluations by binding latency and quality evidence to execution profiles, and features a native qualification workflow that incorporates fresh-start admission and checkpoint-specific evidence for different devices. The architecture facilitates shared BF16 fusion paths and tensor caching to extend native Cosmos optimizations to edge and nano devices, though exact caching and numerical compilation remain separate configuration options. The roadmap outlines future directions, including few-step distillation, attention upgrades for long-context models, and the integration of Cosmos3 and DreamZero on Thor hardware.