DeepSeek's surprisingly inexpensive AI model, DeepSeek V3, has shaken the AI market, causing significant drops in NVIDIA's stock price. While DeepSeek claims a mere $6 million training cost, a closer look reveals a far more substantial investment.
Image: ensigame.com
DeepSeek V3's innovative architecture is key to its performance. It utilizes:
- Multi-token Prediction (MTP): Predicts multiple words simultaneously, boosting accuracy and efficiency.
- Mixture of Experts (MoE): Employs 256 neural networks, activating eight for each processing task, accelerating training and improving performance.
- Multi-head Latent Attention (MLA): Repeatedly extracts key details, minimizing information loss and enhancing nuance understanding.
Image: ensigame.com
However, SemiAnalysis uncovered DeepSeek's use of approximately 50,000 Nvidia Hopper GPUs—a significant investment totaling roughly $1.6 billion in servers and $944 million in operational costs. This contrasts sharply with the publicized $6 million pre-training cost, which omits research, refinement, data processing, and infrastructure.
DeepSeek, a subsidiary of High-Flyer, a Chinese hedge fund, owns its data centers, providing control and faster innovation. Its self-funded status enhances agility. The company attracts top talent, with some researchers earning over $1.3 million annually, primarily from Chinese universities.
Image: ensigame.com
While DeepSeek's $500 million+ investment in AI development is substantial, its lean structure allows for efficient innovation. The $6 million figure is misleading, representing only a fraction of the total cost. Despite this, DeepSeek's model training costs are significantly lower than competitors, such as the estimated $100 million for ChatGPT4o, highlighting its cost-effectiveness relative to industry giants.
Image: ensigame.com
DeepSeek's success showcases the potential of well-funded independent AI companies to compete effectively. However, its achievements are rooted in substantial investment, technological advancements, and a strong team, not a revolutionary budget.
