Shapelearn Qwen 3.8 27B (13.1 GB VRAM)
Recorded: Sept. 18, 2026, 4 a.m.
| Original | Summarized |
ShapeLearn-Lite Held Up. ShapeLearn Did Better: Qwen 3.8 27B Skip to content Home Talk to us ↗ ShapeLearn-Lite Held Up. ShapeLearn Did Better:
We were a little impatient. Qwen 3.8 27B was released on August 14, 2026. Four days later, on August 18, we published our first set of GGUFs. We called them ShapeLearn-Lite for a reason: they were produced using a much smaller optimization budget, fewer checks, and much less waiting. Now the full ShapeLearn models are done, and we have benchmarked them alongside the original Lite set and competing quants. The good news: ShapeLearn-Lite held up pretty well. We will come back to that later in “ShapeLearn-Lite, in retrospect”. The better news: the full ShapeLearn models are even better. Try Qwen3.8-27B (GGUF) Quick start with llama.cpp The MTP draft head is bundled in every GGUF. DFlash2 uses a separate 1.1 GB draft model. Both commands use GPU-5. Swap the tag for any other model in the release. MTP Embedded draft. Works with image inputs. DFlash2 External draft. Fastest option, text only. DFlash2 needs llama.cpp b10658 or newer. Ready-to-run commands for every model, with the recommended sampling settings, are in the run tool and on the model card. TL;DR Full ShapeLearn moves the measured quality-speed frontier beyond Lite. All five models in the new release sit on the frontier in each of our six GPU comparisons. Full ShapeLearn moves the frontier We are releasing the full ShapeLearn run for Qwen 3.8 27B. Within this release, larger models yield higher aggregate scores, while smaller models deliver higher throughput. That ordering holds across all six GPUs tested. Because this is a dense model and memory transfers are the bottleneck, lower BPW translates more directly into higher TPS than it does for MoEs. The per-GPU comparisons also include AtomicChat, Bartowski, ISTA-DASLab, and Unsloth Dynamic v3. Bartowski’s latest models were released after our testing and are not included. Full ShapeLearn is labelled ByteShape in the figures. All five ShapeLearn models remain on the measured frontier, with GPU-5 achieving the highest aggregate score among the plotted quants. Other teams also contribute competitive points. Notably, ISTA-DASLab’s excellent model (the yellow “d” on the graph below) also sits on the frontier. By “frontier,” we mean that no other plotted model is both faster and more accurate. RTX PRO 6000, the GPU with the most memory, lets us show the full range of models tested. RTX Pro 6000: tokens per second vs quality (full ShapeLearn and competing quants) RTX Pro 6000: tokens per second vs quality (full ShapeLearn and competing quants) Show Legend #ModelAccTPSBPW ByteShape GPU-5 is our default wherever you can fit it: it reaches 90.4 tok/s at 99.63% of the BF16 baseline. The RTX 5090 tells a similar story, leading to the same recommendations. RTX 5090: tokens per second vs quality (full ShapeLearn and competing quants) RTX 5090: tokens per second vs quality (full ShapeLearn and competing quants) Show Legend #ModelAccTPSBPW ByteShape Once again GPU-5 is our default choice, reaching 93.7 tok/s. Choose GPU-4 for slightly more context length or slightly better TPS. Both 24 GB cards fit all five ShapeLearn models. We plot them separately because their throughput differs, but the ordering is the same on both. The RTX 4090 keeps the same pattern: GPU-5 is the default, reaching 59.2 tok/s. RTX 4090: tokens per second vs quality (full ShapeLearn and competing quants) RTX 4090: tokens per second vs quality (full ShapeLearn and competing quants) Show Legend #ModelAccTPSBPW ByteShape RTX 3090 Older, but still fast in these measurements. RTX 3090: tokens per second vs quality (full ShapeLearn and competing quants) RTX 3090: tokens per second vs quality (full ShapeLearn and competing quants) Show Legend #ModelAccTPSBPW ByteShape GPU-4 reaches 49.5 tok/s, compared with 45.8 tok/s for GPU-5. Moving to the larger model costs about 7.5% in throughput, while the aggregate score rises from 98.72% to 99.63% of BF16. That makes GPU-5 the default here as well. With a tighter VRAM budget, the pragmatic choice is to leave room for the context you need, not just the model weights. These plots contain fewer competing configurations, but all five ShapeLearn models are represented. On the RTX 4080, GPU-4 reaches 52.4 tok/s, while GPU-5 reaches 45.7 tok/s. RTX 4080: tokens per second vs quality (full ShapeLearn and competing quants) RTX 4080: tokens per second vs quality (full ShapeLearn and competing quants) Show Legend #ModelAccTPSBPW ByteShape RTX 5060 Ti On the RTX 5060 Ti, the corresponding figures are 33.1 tok/s and 29.1 tok/s. RTX 5060 Ti: tokens per second vs quality (full ShapeLearn and competing quants) RTX 5060 Ti: tokens per second vs quality (full ShapeLearn and competing quants) Show Legend #ModelAccTPSBPW ByteShape GPU-5 remains the default on both cards when the model, KV cache, and runtime buffers fit within your memory budget. When they do not, GPU-4 is still very competitive: almost 99% of BF16 at a much smaller size, and faster. A model appearing in these measurements does not establish that every context length or serving configuration will fit. ShapeLearn-Lite, in retrospect ShapeLearn-Lite uses a smaller optimization budget than full ShapeLearn. It let us get Qwen 3.8 27B onto 12 GB to 24 GB GPUs within a few days. We released after targeted sanity checks and started the full evaluation afterwards. The full ShapeLearn models were ready before the benchmarking was finished. Evaluating both sets, along with the competing models, is what took most of the time. Then Unsloth released its Dynamic v3 models. At similar sizes, several had lower KLD than Lite in our measurements. On KLD alone, Lite looked less competitive. KLD measures divergence between a quantized model’s predicted token distributions and the BF16 reference under a particular evaluation setup. It is useful for diagnosing substantial changes, but lower divergence does not automatically mean better task performance. We measure KLD on a dataset of about 5 million tokens of prompt and response pairs, drawn from several benchmarks, including long-context and agentic tasks. We also changed how KLD is computed, so that it is closer to what we expect KLD to measure: KLD is measured on response tokens only, not on prompt tokens. We do not want to measure how well a model can generate prompts. KL divergence versus model size for ShapeLearn-Lite and Unsloth KL divergence versus model size for ShapeLearn-Lite and Unsloth Show Legend #ModelKLDSize (GB)BPW ShapeLearn-Lite For example, Unsloth’s UD-IQ3_S (vii) has about 20% lower KLD than the similarly sized smallest Lite model (Lite-1): 0.028759 versus 0.035875. Yet its aggregate benchmark score is lower: 95.55% versus 97.33% of BF16. If lower KLD were sufficient to rank these models by task performance, the benchmark ordering should have followed it. It did not. The point is not that KLD is useless. It is that a fidelity ranking is not a task-performance ranking. This is the distinction explored in our KLD evaluation blog. Our related paper on KLD and quantization fidelity metrics was also recently accepted to the EMNLP Industry Track. Naturally, we made more plots. Here, we show the RTX Pro 6000 because it can accommodate the full comparison. Each model’s benchmark score is reused across the GPU plots; the measured throughput and the set of displayed models change. RTX Pro 6000: ShapeLearn, ShapeLearn-Lite, and Unsloth Dynamic v3 RTX Pro 6000: ShapeLearn, ShapeLearn-Lite, and Unsloth Dynamic v3 Show Legend #ModelAccTPSBPW ShapeLearn (this release) Leaving the full ShapeLearn models aside for a moment, three of the six ShapeLearn-Lite models sit on the Lite-versus-Unsloth frontier: the three smallest Lite models, the lighter orange bubbles labelled 1-3. Of the twelve Unsloth v3 models shown, three also sit on that frontier: UD-IQ2_S (A), UD-Q2_K_XL (B), and UD-IQ4_XS (F). UD-IQ4_XS (F) is a strong higher-quality point, while Lite earns its places in the middle of the range. Add the five full ShapeLearn models back in (the darker orange bubbles), and they take over the entire frontier. Lite was never meant to be the final result. It still held its own where it mattered. Speculative Decoding We also evaluated MTP and DFlash2 with the new models, using 3 draft tokens for MTP and 7 draft tokens for DFlash2. Both methods increased throughput for all five ShapeLearn models on all six GPUs tested. DFlash2 was faster than MTP in almost all cases. Across the full lineup, DFlash2 reached 1.34-2.10x the baseline next-token prediction (NTP) throughput, while MTP reached 1.28-1.66x. We measured with the sampling parameters Qwen recommends for thinking mode, over a diverse set of agentic coding, mathematics, and general-knowledge requests. The speedups would likely be larger under greedy decoding, but temperature-based sampling better reflects real usage. The figure below shows NTP, MTP, and DFlash2 throughput for each GPU. The quality axis is the target-model benchmark score reported above. These plots do not independently establish quality equivalence between decoding methods. Tokens per second vs quality (NTP vs MTP vs DFlash2), one panel per GPU. MTP uses 3 draft tokens, DFlash2 uses 7. Tokens per second vs quality (NTP vs MTP vs DFlash2), one panel per GPU. MTP uses 3 draft tokens, DFlash2 uses 7. Show Legend #ModelAccNTP TPSMTP TPSDFlash2 TPSBPW RTX Pro 6000 (96 GB) There is also a memory tradeoff between the two approaches. The embedded quantized MTP weights add only about 250 MB to the model, and if MTP is not used, these weights are not loaded into GPU memory. In comparison, the 4-bit DFlash2 draft model is about 1.1 GB, so enabling DFlash2 requires roughly 1.1 GB of additional GPU memory. Packaging MTP as a separate GGUF file would largely eliminate this advantage. The standalone model would need its own MTP embedding and output layers, which are by far its largest tensors, bringing its memory footprint to roughly 1 GB as well. In addition, DFlash2 in llama.cpp currently does not support image inputs, which is an important consideration for multimodal use cases. Benchmarking Methodology We evaluate all reported models across a set of instruct and thinking benchmarks. Instruct benchmarks: GSM8K for math Thinking benchmarks: ACEBench for tool use and agentic tasks For the thinking benchmarks, we used Qwen 3.8’s medium thinking setting. For each benchmark, the score of a quantized model is normalized by the score of the corresponding BF16 model. The overall reported score is the average of these normalized benchmark scores. Our LiveCodeBench V6* evaluation includes problems from January 1, 2024 onward, excluding the 2023 problems. We found the 2023 problems to be relatively easy for current models, with most models achieving very high scores on them. As a result, they provide limited discrimination between models while adding substantial evaluation time. For BFCL V4*, we evaluate the following eight subsets: live_simple All evaluations were run with llama.cpp b10430. For both instruct and thinking experiments, we use the sampling parameters recommended by Qwen for the corresponding mode. Conclusion ShapeLearn-Lite did what it was designed to do. It got useful Qwen 3.8 27B quants onto 12 to 24 GB GPUs quickly, and it held up better than its KLD ranking suggested. Full ShapeLearn goes further. It improves the measured quality-speed trade-offs over Lite and contributes five frontier models across all six tested GPUs. GPU-5 is our default recommendation wherever it fits, reaching 99.63% of BF16’s aggregate benchmark score. When memory is tight, GPU-4 is still very competitive: almost 99% of BF16 at a much smaller size, and faster. KLD remains useful, but it is not a task-performance leaderboard. Fidelity metrics tell us how much the model’s distributions changed under a particular measurement. Benchmarks tell us whether those changes matter on the tasks we tested. We were impatient. This time, it worked out pretty well. AI models, shaped for real hardware. Products All models Technology How it works Resources Blog Company About Connect Hugging Face X © 2026 ByteShape. All rights reserved. × |
The investigation focused on comparing the performance of full ShapeLearn models against their Lite counterparts, utilizing the Qwen 3.8 27B model across various hardware configurations to explore the quality-speed frontier in large language model deployment. The initial release of ShapeLearn-Lite successfully adapted the Qwen 3.8 27B model onto smaller memory systems, demonstrating that it could achieve deployment on 12 to 24 gigabyte GPUs quickly, performing reasonably well relative to its KLD ranking, which measured the divergence between quantized predictions and the baseline. However, the full ShapeLearn models, resulting from a more extensive optimization process, demonstrated superior performance and established a new frontier across all six tested GPUs, with larger models yielding higher aggregate scores. The analysis differentiated between the quality metrics derived from KLD and the performance metrics derived from task-specific benchmarks. KLD was determined to be a measure of quantization fidelity—how much the model's token distributions shifted during quantization—and is useful for diagnosing changes, but it does not automatically translate to superior task performance. Task performance is better assessed through benchmarks, which evaluate models on instruction following, mathematics, general knowledge, coding, and agentic tasks, including GSM8K, MMLU, and ACEBench. The authors noted that while some Unsloth models showed lower KLD than Lite models, their aggregate benchmark scores were not necessarily higher, highlighting that a fidelity ranking is not equivalent to a task-performance ranking. The full ShapeLearn models consistently occupied the measured frontier across the six GPUs, with GPU-5 being the recommended default, achieving 99.63% of the BF16 aggregate benchmark score. When memory constraints dictated a different choice, GPU-4 remained highly competitive, offering nearly 99% of the BF16 score at a smaller model size and greater speed. This pattern was replicated across various hardware platforms, including the RTX Pro 6000, RTX 5090, RTX 4090, RTX 3090, RTX 4080, and RTX 5060 Ti. On these systems, GPU-5 was generally the default recommendation, while GPU-4 offered a trade-off, particularly on systems with limited VRAM, such as the 16 gigabyte RTX 4080, where GPU-4 achieved higher tokens per second. Speculative Decoding methods, specifically MTP and DFlash2, were evaluated to increase throughput across the tested models and GPUs. DFlash2 was generally faster than MTP, increasing throughput by a factor of 1.34 to 2.10 times the baseline next-token prediction throughput when using 7 draft tokens. However, MTP was sometimes preferred when multimodal support or managing VRAM was a primary concern. The choice between methods depends on the specific application, as DFlash2 does not natively support image inputs within llama.cpp, whereas MTP incorporates an embedded draft head that works with image inputs. Memory considerations also presented a trade-off between the two optimization approaches. The embedded quantized MTP weights added only about 250 megabytes, whereas enabling DFlash2 requires approximately 1.1 gigabytes of additional GPU memory for the draft model. Furthermore, packaging MTP as a separate GGUF file would negate the memory advantage gained by embedding, as the required embedding and output layers would bring the footprint closer to one gigabyte for the standalone model. Despite these memory considerations, the evaluation confirmed that utilizing these speculative decoding techniques substantially increased throughput for all ShapeLearn models across all GPU setups. |