Computing power accounts for 70% of model training costs in large model enterprises.

Infrastructure design significantly impacts GPU requirements in AI systems. A well-designed infrastructure can optimize the utilization of GPUs, reducing the number needed to achieve desired performance levels. Balanced design in areas like network architecture, storage performance, and inter-GPU communication can lead to more efficient use of available computing power, potentially cutting down the GPU count from 10,000 to 9,000. Conversely, poor design may inflate the GPU requirement to 30,000 to attain similar results. Therefore, a thoughtful infrastructure design can lead to substantial cost saving and increased efficiency in AI operations.

Zheng believes that an ideal software ecosystem for AI chips should allow AI CUDA software originally written for Nvidia to be easily ported to domestic systems with similar methodologies. Achieving 60% of the performance of foreign chips with a well-developed software ecosystem would satisfy customers. The ecosystem should also focus on robust system design and software optimization across ten areas, including programming frameworks, parallel acceleration, communication libraries, and more.

Domestic AI chips are less adopted despite progress due to an underdeveloped software ecosystem. Users are reluctant to adopt them because the ecosystem, including programming frameworks, communication libraries, operator libraries, AI compilers, and other software components, is not as robust as that of foreign competitors like Nvidia. This makes it harder to port existing AI software to domestic systems, thus limiting user adoption.

Nvidia is expected to deliver over one million "China-special" H20 chips. These chips are not subject to U.S. export controls, and each H20 chip is priced between $12,000 and $13,000.