DeepSeek open-sources its inference framework; social media estimates gross margin exceeds 95%.

iconMetaEra
Share
AI summary iconSummary
DeepSeek has open-sourced its internal inference framework, DeepInfer, built on MetaEra, and disclosed that its V4.1-Flash model generates $38 in revenue per GPU hour. Analysts on social media, using assumptions about cache hit rates and throughput, estimate DeepSeek’s inference gross margins to range between 84.2% and 96.1%. This has sparked on-chain discussions about AI and crypto news, as well as the profitability of major models from OpenAI and Anthropic, with some suggesting their API gross margins could also reach 95%.
DeepSeek has open-sourced its underlying inference stack, DeepInfer, and revealed that revenue per GPU hour reaches $38. Overseas social media platforms have calculated that its single-GPU hardware gross margin may be as high as 84% to 96%. This estimation has further sparked industry-wide discussion, with analysts suggesting that the API gross margins for mainstream models from OpenAI and Anthropic could also reach 95%.

Author: Long Yue

Source: Wall Street Journal

DeepSeek's open-source inference framework has shifted the industry discussion on large models from model capabilities to inference costs.

On October 20, DeepSeek announced the open-sourcing of its internal inference framework, DeepInfer, and deepseek-recipe for token encoding and decoding. DeepSeek stated that DeepInfer powers large-scale inference for DeepSeek V4.1-Flash, aiming to reduce latency while increasing throughput. The framework supports NVIDIA GPUs and Huawei Ascend NPUs and is licensed under Apache-2.0.

One context behind the attention generated by this open-source release is the inference service revenue disclosed by DeepSeek. According to the deployment Q&A posted by DeepSeek on GitHub, the V4.1-Flash cluster processes 15.5 trillion tokens per day, yielding overall revenue of $38 per GPU hour when calculated at off-peak electricity rates.

Based on assumptions such as cache rates, throughput, and pricing, social media estimates that its inference gross margin could reach as high as 84.2%–96.1%. Analysts project that the API gross margins for mainstream models from OpenAI and Anthropic could also reach around 95%.

This disclosure provides a window into analyzing the economics of the business. However, the revenue figures alone do not indicate profitability. To determine the gross profit corresponding to a $38/hour revenue, you need to account for costs such as hardware depreciation, energy, network, caching storage, and operations.

Social media analysis suggests a gross margin exceeding 96%. Zephyr, a海外科技投资研究员, stated on social platform X that DeepSeek has effectively “revealed the core secret behind its business model.” He estimated that DeepSeek’s inference gross margin could range from 84.2% to 96.1%, revising his earlier estimate of 87.76% to 96.94%.

He established a typical baseline model: assuming that in actual production, 97% of inputs hit the key-value cache, 2% of inputs miss the cache, and outputs account for 1%.

  • Cached Input: Accounts for 97% of the total token volume (extremely low unit price, negligible computational load);
  • Uncached Input / Prefill: Accounts for 2% of total tokens;
  • Output: Represents 1% of the total token supply.

Under this traffic structure, Zephyr calculates revenue based on the official published single-GPU peak throughput (80,000 tokens per second for prefill and 16,000 tokens per second for decoding). According to its model, when a single GPU operates at full capacity, it can theoretically process the corresponding token combinations and generate up to approximately $49 in gross revenue per hour.

Based on the current prevailing rental costs in the overseas public cloud market, where a top-tier GPU (such as the NVIDIA H100 or similar next-generation hardware) typically ranges from $1.5 to $4 per hour, the hardware-level inference gross margin formula is:

Under extreme test conditions (earning $49 per hour, with mining hardware costs ranging from $1.5 to $6):

Zephyr stated that, under specific ideal workloads, the inference gross margin for this framework ranges between 87.76% and 96.94%. He explicitly wrote in his tweet: "NVIDIA, SGLang, and vLLM's public performance are directly crushed."

Subsequently, Zephyr adjusted the estimation parameters based on the actual operational details disclosed by DeepSeek. He added: “These extreme gross margins are based on ideal scenarios where the service provider generates approximately $49 per hour. DeepSeek has officially stated that its actual revenue per card per hour during off-peak times is around $38, and production environments achieve approximately 80% of benchmark performance. Re-evaluating with these updated figures, the actual operating gross margin falls between 84.2% and 96.1%.”

In the current overseas cloud services market, the comprehensive cost of renting a single top-tier GPU typically ranges from $2 to $4 per hour. The significant gap between the single-card hourly output value of $38 and the unit computing cost forms the core basis for calculating the gross profit margin.

Industry-wide analysis: Following the release of the gross margin estimates for leading AI labs, a chain of discussions quickly emerged across the industry regarding the true profitability of top AI laboratories.

Tech investment observer Alex Corrino, while sharing the above calculation, asked: “What if I told you that the API gross profit margins of all top AI labs are likely above 80%?”

Zephyr immediately cited the tweet and offered a more aggressive assessment: “For Anthropic and OpenAI, this figure is actually 95% (applicable to mainstream flagship models such as Sol, Fable, Sonnet, and Astra).”

Is Anthropic making huge profits? Under the prevailing assumption that the entire industry enjoys high gross margins, investment analyst Scott further applies this logic to the end-user "subscription" economic model to uncover the real strengths of leading overseas players.

According to real-world testing by research firm SemiAnalysis, Anthropic provides its premium subscribers with a usable token limit for “Claude Max” that is 58.6 times the monetary value of its official API pricing. This led outsiders to initially believe that high-volume subscribers would severely erode platform profits, resulting in losses.

Scott provided the reverse calculation formula:

  1. Set baseline assumptions: Assume Anthropic's API hardware gross margin is also as high as 99% (cost rate of only 1%).
  2. Cost reverse-engineering for subscription usage at maximum capacity: When a user pays 1 unit of subscription fee and fully consumes the equivalent of 58.6 units of API-listed compute credits, the underlying hardware cost borne by Anthropic is:
Actual hardware cost = 58.6 × 1% = 0.586 per unit  
Actual hardware cost = 58.6 × 1% = 0.586 per unit
  1. Derive the subscription gross margin under extreme conditions:
Claude Max's extreme gross margin = 1 - 0.5861 = 41.4%  
Claude Max's extreme gross margin = 1 - 0.586 = 41.4%

Scott analyzes this as:

If DeepSeek’s gross margin reaches over 95%, and assuming Anthropic’s API gross margin is 99%, SemiAnalysis’s testing shows that Claude Max provides up to 58.6 times the usage allowance of the API’s list price. This means that even if all users fully exhausted their quotas, Anthropic would still maintain a gross margin of approximately 41% under its subscription model. In reality, the vast majority of users never come close to hitting their weekly limits.

He noted: "In fact, as long as Anthropic's API profit margin remains above 98.3%, its subscription business can break even even if users fully utilize their limits. Anthropic is currently in a solidly profitable position."

The tech analysis account Teortaxes also commented: “In the face of the proprietary internal architecture developed by the flagship lab, all public benchmarks on the market are insignificant. This is the result of serious engineering—AI inference is essentially a money-printing machine. You can imagine how anxious Dario (CEO of Anthropic) must be right now.”

Technical Path Divergence: Software Refactoring vs. Dedicated Chips

As the true cost structure of reasoning becomes apparent, the industry is reevaluating various technical approaches to reduce costs.

In terms of hardware architecture and unit throughput cost, this solution has been compared with Groq, a chip manufacturer known for ultra-low latency.

  • Industry observers note that LPU custom hardware architectures, exemplified by Groq, rely on onboard SRAM to achieve ultra-fast interactive generation; however, their chip memory capacity is extremely limited, requiring massive stacking of hardware boards and extensive node interconnects to support mainstream large models, resulting in high overall hardware depreciation and procurement barriers per token.
  • In contrast, DeepInfer demonstrates an alternative ledger logic for general-purpose chip pathways: by purely leveraging software scheduling to maximize concurrency and throughput on general-purpose GPUs and commercial NPUs such as Ascend, it achieves greater profit margins for scalable API services at a lower overall hardware cost.

However, several industry observers pointed out during the discussion that the aforementioned 80% to 95% gross margin calculation primarily focuses on the direct accounting difference between token price and single-unit hardware consumption. This narrow measurement does not account for upfront R&D expenses amounting to hundreds of millions of dollars for foundational models, high-speed bandwidth network costs across global nodes, or the substantial salaries of system engineering teams.

Next, it needs to be verified whether the open-source implementation can consistently improve throughput and reduce the cost per effective output token across different hardware and real-world workloads. For DeepSeek itself, external parties also require more data on costs and resource utilization to determine whether the high gross margins estimated on social media closely reflect actual operational performance.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.