Cresci, from Aofei Temple, QbitAI | WeChat Official Account QbitAI
Just one week after DSpark went open source, it was already integrated into Apple computers.
The ported version is called mlx-dspark, running the Gemma-4 12B and Qwen3-4B models.
After installation, these two models increased generation speed on Mac by 1.6x and 1.4x, respectively.
More difficultly, it achieved something that most ported versions cannot—outputting byte-for-byte identical results to the original model, with not a single byte different.
In other words, speed was gained without compromising quality.
The person who did it is Abdur Rahim, an engineer who tinkers with open-source projects in his spare time—he single-handedly created the first native Mac version of DSpark since its open-source release.
Run large models on Mac computers with 60% faster performance
Regarding DeepSeek's open-source DSpark released on June 27, the official figures indicate a speed improvement of 60% to 85% in server-side scenarios.
However, at the time, this technology was only implemented on data center GPUs and had no version adapted for Apple chips.
mlx-dspark is the first native Apple Silicon version of this technology.
DSpark’s approach is to assign a smaller model to assist the target model: the smaller model generates several candidate words at once, and the target model then verifies them all in one go—accepting the correct ones and sending the incorrect ones back for another guess.
The cost of this step differs between data centers and Apple computers.
On the GPU in the data center, verifying a batch of candidate words is like chartering a vehicle—whether you’re transporting a few people or many, the price is fixed. Decoding is inherently memory-bound, so checking a few additional words adds almost no extra time.
Apple chips are more like taxi meters—the more candidate words verified, the more the meter increases.
Rahim tested and found that Gemma-4 12B requires approximately 14 milliseconds more per additional token pair. He formulated this into a cost model and concluded that the speed ceiling on Apple chips is around 2.2x.
In summary, Rahim transferred this small auxiliary model from a Hugging Face checkpoint and adapted it for use with the target models Gemma-4 12B and Qwen3-4B.
He also rebuilt the verification process within the MLX framework, quantizing the weights to 4-bit.
As a result, on the M4 Pro, compared to Apple's official MLX tool, Gemma-4 12B's generation speed increased from 18.4 tok/s to approximately 30 tok/s, about 1.6 times faster; Qwen3-4B increased from 52.9 tok/s to approximately 73 tok/s, about 1.4 times faster.
Additionally, in mlx-dspark, Rahim did something that most porting efforts did not do.
The ported version can also accurately reproduce the original.
Most versions that deploy large models locally only support greedy decoding, which means selecting the word with the highest probability at each step.
Rahim implemented the temperature sampling method described in the DSpark paper within mlx-dspark: the draft model generates candidate tokens with an acceptance probability of min(1, p/q), and tokens that are rejected are resampled from the residual.
He verified it himself: the output generated by this process is strictly equal to the exact distribution that the target model would produce at the same temperature—not an approximate or diluted version.
Most speculative decodings only implement the greedy version because verifying the correctness of the greedy mode is straightforward—it simply requires a character-by-character comparison.
Rahim took an extra step by verifying the output distribution from the sampling mode to ensure it hadn't been distorted.
The accuracy level required for the target model was a pitfall he discovered through trial and error.
When a small model is paired with a base version of the target model that hasn’t been instruction-tuned, only 47% of the generated candidate words pass verification; switching to the corresponding instruction-tuned version increases this rate to 82%.
He also tested replacing the target model with BF16 precision and found that the cost increased more than the success rate, resulting in slower performance; therefore, keeping the target model at 8-bit by default is the most cost-effective choice.
The small model responsible for pre-screening candidate terms uses a different set of precision metrics.
The draft model itself was compressed by him; after 4-bit quantization, it is only 1.8 GB, making it effortless to load into memory while maintaining lossless performance.
As a result, DSpark not only achieved acceleration but also successfully replicated the 16% to 18% acceptance rate improvement mentioned in the paper on-device.
DFlash has also joined in, making code tasks faster.
After the tweet was posted, a comment appeared from Jian Chen, one of the authors of the DFlash paper, asking if they could try their team's model.
DFlash is another speculative decoding scheme proposed in a paper released by Z-Lab in May this year, led by author team head Zhijian Liu, Assistant Professor at UCSD and a research scientist at NVIDIA.
DFlash's approach differs from DSpark: it denoises an entire block of 16 tokens in a single parallel "block diffusion," rather than guessing step-by-step with dependencies as DSpark does.
Rahim acted quickly.
He used a migration script he wrote himself to connect the gemma4-12B-it-DFlash model released by z-lab to the Gemma-4 target model in mlx-vlm, and ran a head-to-head comparison on the same Mac against his recently tested DSpark.
On code and mathematical tasks, DFlash achieves a decoding length of 5.95 to 6.20 with a speed of approximately 36 tok/s, representing about a 2.1x improvement over DSpark.
However, DFlash outputs a full block of 16 tokens at once, but the target model may not accept all of them; only a portion typically passes verification. The industry refers to this as the "acceptance length," and it’s not always possible to fill all 16 slots.
Therefore, in scenarios like open chat where content is unpredictable, the input length cannot be maximized, blocks cannot be fully filled, and the advantages of DFlash cannot be fully realized.
The Markov header in DSpark was specifically designed to address this same issue: when generating entire blocks of words in parallel, the later positions are computed independently, making them prone to inconsistency. The Markov header introduces a layer of dependency between these positions to specifically correct this problem.
As a result, in chat scenarios, DSpark is faster than DFlash.
The updated mlx-dspark v0.0.3 officially integrates the original z-lab DFlash into the package and adds a parameter to manually shorten the effective block length of DFlash—using shorter blocks for chat scenarios and full 16-byte blocks for code and math scenarios.
After this, the same Mac and the same package can handle both chat and coding/math tasks without needing to switch back and forth between the DSpark and DFlash projects.
Rahim said in a tweet that the same approach should also work on the larger Qwen3-8B and 14B draft models.
Reference links: [1] https://x.com/_ARahim_/status/2072021710602432577 [2] https://github.com/ARahim3/mlx-dspark
This article is from the WeChat official account "Quantum Bit," authored by: Focused on Frontier Technology