Technical Deep Dive
MooreThreads' MT-FlashMLA inherits the core innovation of DeepSeek's original: exploiting the mathematical structure of multi-head latent attention (MLA) to compress the KV cache. In standard multi-head attention (MHA), each head stores a full key and value vector, leading to a memory footprint that scales linearly with sequence length and number of heads. MLA introduces a low-rank projection: instead of storing full K and V, it stores a latent vector of much smaller dimension (typically 1/8 to 1/4 the size), then reconstructs the full K and V on-the-fly during attention computation. This reduces memory bandwidth from O(n * d * h) to O(n * r), where r is the latent dimension.
DeepSeek's FlashMLA implementation on NVIDIA hardware achieves this through three key techniques:
1. Tiled KV Cache Management: Splits the KV cache into tiles that fit in shared memory, reducing global memory reads.
2. Fused Kernel Design: Combines the latent-to-full projection, attention score computation, and softmax into a single kernel, minimizing kernel launch overhead.
3. Tensor Core Utilization: Leverages NVIDIA's Tensor Cores for the low-rank matrix multiplications, achieving peak throughput on H100.
MooreThreads' challenge is that its MTT S80 GPU uses a fundamentally different architecture. The S80 has 4096 MUSA cores organized into 128 clusters, each with 16 KB shared memory per cluster — far smaller than the H100's 228 KB per SM. This means the tiling strategy must be completely reworked. Additionally, MooreThreads lacks Tensor Core equivalents; its MUSA cores are general-purpose SIMT units, so the low-rank projections must be implemented via standard matrix multiply instructions, which are 3-5x slower than NVIDIA's specialized hardware.
Benchmark Data (Estimated vs. NVIDIA H100)
| Metric | DeepSeek FlashMLA (H100) | MT-FlashMLA (MTT S80, estimated) | Difference |
|---|---|---|---|
| Peak Memory Bandwidth Utilization | 95% (3.35 TB/s) | ~60% (0.6 TB/s of 1.0 TB/s) | -35% efficiency |
| KV Cache Compression Ratio | 4x (latent dim 512 vs 2048) | 4x (same algorithm) | Same |
| Throughput (tokens/sec, 7B model) | 12,000 | ~3,500 | -71% |
| Latency (first token, 2K context) | 45 ms | ~180 ms | -75% |
| Power Efficiency (tokens/Watt) | 240 | ~70 | -71% |
*Data Takeaway: Even if MooreThreads achieves perfect algorithmic parity, the hardware gap in memory bandwidth and compute density means MT-FlashMLA will likely deliver 60-75% lower throughput than the NVIDIA version. The compression ratio is identical because it's a mathematical property of MLA, not hardware-dependent.*
A critical open question is whether MooreThreads has implemented the fused kernel approach. The original FlashMLA GitHub repository (deepseek-ai/FlashMLA, 2.3k stars) uses custom CUDA assembly for the fused kernel. MT-FlashMLA's codebase, as of April 2026, shows a pure MUSA C++ implementation without assembly-level tuning. This will likely result in higher kernel launch overhead and lower occupancy, further degrading performance.
Key Takeaway: MT-FlashMLA is a faithful algorithmic port, but the hardware limitations of MooreThreads' GPUs — particularly smaller shared memory, lack of Tensor Cores, and immature compiler stack — mean it cannot match NVIDIA's performance. The real test will be whether it can achieve even 50% of H100 throughput, which would still be a significant achievement for domestic hardware.
Key Players & Case Studies
MooreThreads (Beijing, China) was founded in 2020 by former NVIDIA and AMD engineers. Its MTT S80 GPU, released in 2022, was initially targeted at gaming and graphics, but the company pivoted to AI inference after U.S. export controls blocked NVIDIA A100/H100 sales to China. The S80 has 22 billion transistors, 4096 MUSA cores, and 32 GB GDDR6X memory. However, its software stack (MUSA SDK) has been criticized for poor documentation, limited operator coverage, and buggy drivers. The FlashMLA fork is part of a broader effort to port popular AI libraries — including PyTorch, TensorFlow, and vLLM — to MUSA.
DeepSeek (Hangzhou, China) is the original creator of FlashMLA and the MLA architecture. DeepSeek's V2 model (236B parameters, Mixture-of-Experts) was the first to demonstrate MLA's benefits at scale, achieving 2x inference speedup over comparable dense models. DeepSeek open-sourced FlashMLA under a permissive license, and the project has been adopted by several inference frameworks, including vLLM and TGI. DeepSeek's motivation is ecosystem growth: by making MLA efficient on NVIDIA hardware, they lower the barrier for others to use their model architecture.
Competing Solutions for Chinese Hardware:
| Solution | Target Hardware | MLA Support | Maturity | Performance (vs. H100) |
|---|---|---|---|---|
| MT-FlashMLA | MooreThreads MTT S80/S3000 | Yes | Early (16 stars) | Unknown (est. 25-40%) |
| Hygon DCU FlashAttention | Hygon DCU (AMD MI250 derivative) | No (standard MHA only) | Production | ~50% |
| Biren BR100 custom kernels | Biren BR100 | No | Beta | ~30% |
| Cambricon MLU370 FlashAttention | Cambricon MLU370 | No | Production | ~35% |
*Data Takeaway: MT-FlashMLA is the only domestic solution specifically targeting MLA, giving it a unique advantage for DeepSeek model inference. However, all Chinese GPU solutions lag NVIDIA by at least 50-75% in raw performance, and none have achieved production-grade MLA support.*
Case Study: ByteDance's Deployment ByteDance, which uses DeepSeek V2 for internal content moderation and recommendation systems, initially attempted to run MLA inference on MooreThreads hardware. Internal tests in Q3 2025 showed that without optimized kernels, throughput was only 8% of H100. ByteDance engineers then contributed a prototype MUSA kernel for FlashMLA, which MooreThreads incorporated into MT-FlashMLA. This suggests the fork is not just a MooreThreads effort but a collaborative industry push to make domestic hardware viable for high-throughput inference.
Key Takeaway: The success of MT-FlashMLA depends on whether MooreThreads can attract contributions from major Chinese AI companies like ByteDance, Alibaba, and Baidu, which have the engineering resources to optimize kernels. Without their buy-in, the fork will remain a niche project.
Industry Impact & Market Dynamics
The Chinese AI chip market is projected to grow from $5.2 billion in 2025 to $12.8 billion by 2028 (CAGR 25%), driven entirely by domestic procurement mandates. U.S. export controls (October 2022, October 2023, and May 2025 updates) have effectively banned NVIDIA A100/H100/B200 sales to China, forcing companies to adopt domestic alternatives. However, software compatibility remains the biggest bottleneck.
Market Data: Chinese AI Inference GPU Shipments (2025-2028)
| Year | NVIDIA (smuggled/stockpiled) | MooreThreads | Hygon | Cambricon | Others | Total |
|---|---|---|---|---|---|---|
| 2025 | 45,000 | 8,000 | 12,000 | 15,000 | 5,000 | 85,000 |
| 2026 | 20,000 | 25,000 | 18,000 | 20,000 | 10,000 | 93,000 |
| 2027 | 5,000 | 45,000 | 22,000 | 25,000 | 18,000 | 115,000 |
| 2028 | 0 | 70,000 | 30,000 | 35,000 | 25,000 | 160,000 |
*Data Takeaway: MooreThreads is projected to capture 44% of the Chinese inference GPU market by 2028, but only if its software ecosystem matures. MT-FlashMLA is a critical piece of that puzzle — without it, MooreThreads cannot serve the growing demand for DeepSeek model inference, which already accounts for 30% of Chinese LLM deployments.*
Business Model Implications: MooreThreads is currently selling MTT S80 GPUs at a loss ($1,200 per unit, estimated cost $1,800) to gain market share. The company plans to recoup losses through software licensing and cloud inference services. MT-FlashMLA, as open-source software, does not directly generate revenue, but it increases the value proposition of MooreThreads hardware. If MT-FlashMLA achieves even 50% of H100 performance, MooreThreads could charge a 30% premium on its GPUs, improving margins from -50% to -10%.
Key Takeaway: MT-FlashMLA is not just a technical project; it is a strategic asset in China's race for AI hardware self-sufficiency. Its success or failure will directly impact MooreThreads' market share and, by extension, the entire Chinese AI inference ecosystem.
Risks, Limitations & Open Questions
1. Lack of Independent Benchmarks: The most glaring issue is the absence of any third-party performance data. The GitHub repository has 16 stars and no issues or pull requests from external contributors. Without benchmarks on standard workloads (e.g., DeepSeek V2 inference at 2K/4K/8K context lengths), the project is essentially unvalidated. MooreThreads should publish MLPerf Inference results or at minimum, a reproducible benchmark script.
2. Driver and Runtime Instability: MooreThreads' MUSA driver has a history of crashes and memory leaks. The FlashMLA kernel is particularly sensitive to memory alignment and synchronization issues. Early testers report that MT-FlashMLA crashes on sequences longer than 4K tokens due to a bug in the tiling logic. Until these issues are resolved, the library is unusable for production.
3. Fragmented Chinese Software Ecosystem: Unlike NVIDIA's unified CUDA platform, Chinese GPU vendors each have their own SDK (MUSA, HygonDCU, BirenSDK, CambriconNeuware). This fragmentation means that optimizations for MooreThreads do not benefit other domestic hardware, and vice versa. A single, community-maintained abstraction layer (like OpenAI's Triton) would be more efficient, but no such effort exists for Chinese GPUs.
4. Intellectual Property Concerns: DeepSeek's FlashMLA is licensed under Apache 2.0, so the fork is legally permissible. However, if MooreThreads modifies the kernel to use proprietary MUSA instructions, they may create a derivative work that cannot be easily merged back upstream. This could lead to a permanent fork, fragmenting the MLA optimization ecosystem.
5. Ethical and Geopolitical Risks: The Chinese government may mandate that all AI inference for sensitive applications (e.g., military, surveillance) use domestic hardware. MT-FlashMLA could be used to accelerate models for these purposes, raising ethical concerns for the open-source community. Foreign contributors may be hesitant to contribute to a project that could be used in ways they disagree with.
Key Takeaway: The biggest risk is not technical failure but irrelevance. If MooreThreads cannot deliver a stable, performant implementation within 6 months, Chinese AI companies will continue to rely on stockpiled NVIDIA hardware or switch to alternative architectures (e.g., Hygon DCU). The window of opportunity is narrow.
AINews Verdict & Predictions
Verdict: MT-FlashMLA is a necessary but insufficient step toward Chinese GPU viability for MLA inference. The algorithmic port is sound, but the hardware and software gaps are so large that the current implementation is unlikely to achieve production-grade performance. We rate the project as 'Promising but Unproven' — it addresses a real need, but the lack of benchmarks and stability issues prevent any meaningful assessment.
Predictions:
1. By Q3 2026, MooreThreads will publish benchmark results showing MT-FlashMLA achieving 35-40% of H100 throughput on DeepSeek V2 inference at 4K context. This will be hailed as a 'breakthrough' in Chinese media but will be met with skepticism internationally due to the lack of standardized benchmarks.
2. By Q1 2027, at least one major Chinese cloud provider (Alibaba Cloud or Tencent Cloud) will offer MT-FlashMLA-optimized inference instances, but pricing will be 60% of NVIDIA instances for 40% of the performance, making it uneconomical for most workloads.
3. By 2028, MooreThreads will either acquire or partner with a software startup to build a unified Chinese GPU kernel library (similar to NVIDIA's cuDNN), rendering MT-FlashMLA obsolete as a standalone project. The fork will be merged into this larger library.
4. Wildcard: If U.S. export controls tighten further to block even stockpiled NVIDIA chips, Chinese companies will have no choice but to use MT-FlashMLA regardless of performance. In that scenario, the project becomes critical infrastructure, and MooreThreads will receive massive government funding to accelerate development.
What to Watch Next:
- The GitHub star count and issue tracker activity. If stars exceed 500 within 3 months, it indicates genuine community interest.
- Any mention of MT-FlashMLA in Chinese government procurement documents or AI industry white papers.
- Whether DeepSeek itself acknowledges or contributes to the fork. If DeepSeek's engineers submit pull requests, it signals validation.
Final Thought: MT-FlashMLA is a microcosm of the broader Chinese AI hardware challenge: the hardware is catching up, but the software ecosystem is years behind. This fork is a bet that software can close the gap faster than hardware can. We are skeptical, but we hope to be proven wrong.