Huawei is seeking to grab a larger share of the Chinese market for artificial intelligence chips dominated by Nvidia, by helping local companies adopt their rival silicon for so-called “inference” tasks.
Leading AI companies in China rely on graphic processing units (GPUs) made by Nvidia to “train” large language models, with the $3.4tn US chipmaker’s products seen as vital to developing the technology.
Instead of challenging Nvidia in training, Huawei is positioning its latest Ascend AI processors as the hardware of choice for Chinese groups running “inference”, the computation undertaken by LLMs to generate a response to a prompt.
The Chinese tech giant is betting that inference will be a bigger future source of demand if the pace of model training slows and AI applications such as chatbots become more widespread.
“Training is important, but it only occurs a few times,” said Georgios Zacharopoulos, a senior AI researcher working on inference acceleration at Huawei’s Zurich lab. “Huawei is mostly focused on inference, which ultimately will serve more customers.”
It is focused on the technically less challenging but potentially lucrative path of retrofitting AI models trained on Nvidia products to run on Ascend chips, according to company employees and Ascend customers. Since Nvidia GPUs and Ascend operate on different software, Huawei is helping companies use another software tool to make the two systems compatible.
Huawei’s push comes with top-down government support. Chinese officials have urged local tech giants to buy more of Huawei’s AI chips and shift away from Nvidia.
One person familiar with Nvidia’s operations in China said Huawei was seen internally as the most serious competitor in the country, adding its chip design capability was “advanced”.
Washington has sought to curb Beijing’s development of AI with export controls aimed at hampering the development of sensitive technologies in China.
Unlike their US rivals such as OpenAI and Google, companies are unable to access the most cutting-edge GPUs in China. But even though Chinese groups are only able to acquire Nvidia’s less H20 chips tailored to meet export controls, the less powerful GPUs remain in high demand as they are considered better than local alternatives.
Analysts and Huawei researchers said Ascend was not yet ready to replace Nvidia for model training because of technical issues, such as a breakdown in the ways the chips interact with each other inside a wider “cluster” of AI chips when training ever larger models.
“While the Ascend chips perform well on a per-chip basis, there is a bottleneck with the inter-chip connectivity,” said Lin Qingyuan, Bernstein’s China semiconductor analyst. “When training a big model, you must break it into smaller tasks. If one chip fails, the software needs to figure out a way for the other chips to take over without delay.”
The other challenge for Huawei is convincing developers to switch from Nvidia’s Cuda software, renowned as the company’s “secret sauce” for being easy to use for developers and capable of vastly accelerating data processing.
But Huawei’s soon-to-be-released and updated version of its AI chip, the Ascend 910C, is also expected to address these concerns. “We’re expecting this new generation of hardware to come with improved software that makes it more accessible for developers,” said a Huawei employee, who declined to be named.
Huawei and Nvidia face stiff competition. Chinese internet group Baidu and chip designer Cambricon have made strides in AI chip development. Meanwhile, in the US, Amazon and Microsoft are also betting they can grab more market share in chips for inference as AI applications become more widespread.
Estimates from SemiAnalysis, a chip consultancy, suggest Nvidia made $12bn on sales in China last year by delivering 1mn of its H20 chips in the country, selling twice as many AI Chips as Huawei with its Ascend 910B.
“Nvidia’s China-specific H20 GPUs make up the majority of AI chips sold in China. But the lead is narrowing quickly as Huawei increases manufacturing capacity,” said Dylan Patel, chief analyst at SemiAnalysis.
Industry insiders cautioned that Huawei’s AI chip push was also being constrained by insufficient supply, with two prospective customers telling the Financial Times that they were unable to secure the chips.
Huawei did not respond to a request for comment. Nvidia declined to comment.
Analysts have said that Huawei’s manufacturing is likely facing challenges due to US export controls that have left Chinese fabs dependent on outdated chip manufacturing equipment.
The focus on inference also points to an evolving dynamic in Chinese AI that differs from the US. Washington’s export controls mean Chinese AI players are not engaged in the same race as Silicon Valley rivals Meta, Elon Musk’s x.AI and OpenAI to build large mega-clusters of Nvidia’s most advanced GPUs.
“Chinese companies are playing a different game. They are paying much more attention to inference than the US because it is possible to make big efficiency gains even with less powerful chips, which also means they can achieve commercialisation faster,” said Bernstein analyst Lin.
Chinese companies are betting that they can stay competitive on AI by driving down the cost of inference, which in turn makes it cheaper to run AI applications, he said.
Last month, Hangzhou and Beijing-based start-up DeepSeek released its V3 model, which garnered attention because of its low training and inference costs compared to comparative models in the US.
The company proposed a new way for an AI model to selectively focus on specific parts of input data as a way of reducing the costs of running the model. It also used the “mixture of experts” technique popular with other Chinese AI start-ups, which also helps to speed up inference as only part of the model is used to generate a response.
DeepSeek said that Huawei had successfully adapted V3 to Ascend, providing detailed instructions for developers on how to use the chip. The FT has previously reported that Huawei had dispatched engineers to help customers migrate from Nvidia to Ascend.
Additional reporting by Zijing Wu in Hong Kong
Read the full article here
