Computing power accounts for 70% of model training costs in large model enterprises.

GPT-4's monthly development cost using Nvidia GPUs is $2 million. This cost is attributed to the use of 800 Nvidia A100 GPUs for training the model.

Infrastructure design significantly impacts GPU requirements in AI systems. A well-designed infrastructure can optimize the utilization of GPUs, reducing the number needed to achieve desired performance levels. Balanced design in areas like network architecture, storage performance, and inter-GPU communication can lead to more efficient use of available computing power, potentially cutting down the GPU count from 10,000 to 9,000. Conversely, poor design may inflate the GPU requirement to 30,000 to attain similar results. Therefore, a thoughtful infrastructure design can lead to substantial cost saving and increased efficiency in AI operations.

Zheng believes that an ideal software ecosystem for AI chips should allow AI CUDA software originally written for Nvidia to be easily ported to domestic systems with similar methodologies. Achieving 60% of the performance of foreign chips with a well-developed software ecosystem would satisfy customers. The ecosystem should also focus on robust system design and software optimization across ten areas, including programming frameworks, parallel acceleration, communication libraries, and more.

Domestic AI chips are less adopted despite progress due to an underdeveloped software ecosystem. Users are reluctant to adopt them because the ecosystem, including programming frameworks, communication libraries, operator libraries, AI compilers, and other software components, is not as robust as that of foreign competitors like Nvidia. This makes it harder to port existing AI software to domestic systems, thus limiting user adoption.