Inside China's underground market for high-end Nvidia AI chips | Tech News

Inside China's underground market for high-end Nvidia AI chips

Psst! Where can a Chinese buyer purchase top-end Nvidia AI chips in the wake of U.S. sanctions?

By:REUTERS
| Updated on: Jun 20 2023, 23:32 IST
5 fastest supercomputers in the world: NVIDIA DGX GH200, Frontier system, and more
Nvidia
1/6 The NVIDIA DGX GH200 AI Supercomputer combines Grace CPU and Hopper H100 GPU, designed particularly for AI models with an output of 1 Exaflop/s. NVIDIA says that this supercomputer will be able to churn out AI models within weeks, instead of months. It should be noted that this supercomputer hasn’t yet been ranked by the Top500 Project, an organization that lists the most powerful and fastest supercomputers in the world. (NVIDIA)
image caption
2/6 In the list of the world’s fastest supercomputers that was released recently, at the first position is the Frontier system which is kept at the Oak Ridge National Laboratory in the USA. With 8,699,904 cores, the computer clocked a Rmax PFlops of 1,194, which is more than double the system which came in second. The system consumes a power of 22,703 kW to run. (ORNL)
image caption
3/6 The second position has been taken by Supercomputer Fugaku which has been kept at the RIKEN Center for Computational Science in Japan. With 7,630,848 cores, it was able to clock a Rmax PFlops of 442.01. It held the top position from June 2020 to November 2021 but was dethroned by Frontier in the next list. (RIKEN)
image caption
4/6 Lumi gets the third position. The supercomputer is kept at EuroHPC/CSC in Finland. The system entered the list for the first time in June 2022 and directly made its place in the third slot, which it has still managed to hold on to. It has managed to secure an HPL score of 309.1 Pflops. (LUMI)
image caption
5/6 The fourth position has been taken by Leonardo and is kept at EuroHPC/CINECA in Italy. It reached the fourth position six months ago after receiving an upgrade and now boasts of an HPL score of 239 Pflops.  (Leonardo)
image caption
6/6 Summit, an IBM Power System, has been placed in the fifth position. It is also placed at the Oak Ridge National Laboratory in the USA. With 2,414,592 cores, it has registered an HPL score of 148.60 PFlops. (IBM)
Nvidia
View all Images
While buying or selling high-end U.S. chips is not illegal in China, U.S. export restrictions have created a de facto underground market. (REUTERS)

Psst! Where can a Chinese buyer purchase top-end Nvidia AI chips in the wake of U.S. sanctions?

Visiting the famed Huaqiangbei electronics area in the southern Chinese city of Shenzhen is a good bet - in particular, the SEG Plaza skyscraper whose first 10 floors are crammed with shops selling everything from camera parts to drones. The chips are not advertised but asking discreetly works.

They don't come cheap. Two vendors there, who spoke with Reuters in person on condition of anonymity, said they could provide small numbers of A100 artificial intelligence chips made by the U.S. chip designer, pricing them at $20,000 a piece - double the usual price.

While buying or selling high-end U.S. chips is not illegal in China, U.S. export restrictions have created a de facto underground market with vendors keen not to draw scrutiny from either U.S. or Chinese authorities.

President Joe Biden's administration in September ordered Nvidia to stop exporting its two most advanced chips - the A100 and the recently developed H100 - to mainland China and Hong Kong, part of efforts to stymie Chinese AI and supercomputing development amid intensifying political and trade tensions. That was then followed up with an array of semiconductor-related export controls.

But, as AI booms across the globe after the runaway success of OpenAI's ChatGPT, demand for high-end chips has rocketed, particularly for Nvidia's microprocessors which are widely regarded as the best at handling machine-learning tasks.

"We are talking with two vendors now to get some," said Ivan Lau, co-founder of Hong Kong's Pantheon Lab who is trying to purchase 2-4 new A100 cards to run the startup's latest AI models.

Those vendors, who bought the chips outside the U.S., were quoting HK$150,000 ($19,150) per card, he said, adding: "They told us straight up that there will be no warranty or support."

Reuters spoke with 10 vendors in Hong Kong and mainland China who described being able to easily procure small numbers of A100s. Their information highlighted both intense demand in China for the chips and the relative ease with which Washington's sanctions can be circumvented for small-batch transactions.

Reuters was not able to estimate overall volumes of Nvidia A100 and H100 chips flowing into China or learn to what extent the transactions taking place go towards satisfying demand.

Buyers are typically app developers, startups, researchers or gamers, the vendors said, declining to be identified because the imports contravene U.S. trade restrictions. One vendor said buyers also included Chinese local authorities.

Nvidia said in a statement to Reuters it did not allow exports of the A100 or H100 to China, instead providing reduced-capability substitutes that comply with U.S. law.

"If we receive information that a customer is breaching their agreement with us and exporting restricted products in violation of the law, we would take immediate and appropriate action," the statement said.

The U.S. Department of Commerce, China's State Council Information Office and China's industry ministry did not respond to requests for comment.

Nvidia said in September that $400 million in sales during its third quarter could be lost if Chinese firms decided not to buy alternative Nvidia products.

Its new China-tailored slower variants - the A800 and H800 - developed to cushion that impact are now being bought by large Chinese tech firms such as Tencent Holdings and Alibaba, which have deep pockets to purchase huge quantities.

OFFERINGS ONLINE

The Chinese vendors said they procured the chips primarily in two ways: snatching up excess stock that finds its way to the market after Nvidia ships large quantities to big U.S. firms, or importing through companies locally incorporated in places such as India, Taiwan and Singapore.

This means the quantities they can secure are small, far from what's needed to build a sophisticated AI large language model from scratch.

A model similar to OpenAI's GPT would require more than 30,000 Nvidia A100 cards, according to research firm TrendForce. But a handful can run complex machine-learning tasks and enhance existing AI models.

According to an electronics procurement website that listed some 40 sellers of A100s, most were located in the Huaqiangbei electronics area. But listings for A100s could also be found on Alibaba's Taobao e-commerce site, on Xiaohongshu which is similar to Instagram, as well as on Douyin, the Chinese version of TikTok.

Alibaba, Xiaohongshu and Douyin-owner ByteDance did not respond to requests for comment.

Some of the vendors cautioned that fraud had become common with refurbished chips being passed off as A100s.

Nvidia's more advanced H100 chips, only on the market since March, appear much harder to come by.

Vinci Chow, a lecturer in economics at the Chinese University of Hong Kong whose department has procured four A100 cards from local vendors for research purposes, said he had been told some packs of eight H100 chips were available for purchase.But only one of the 10 vendors Reuters spoke with said they could procure H100s.

The U.S. is likely not too bothered about small transactions of the chips, said Charlie Chai, a Shanghai-based analyst at 86Research.

"Only if/when China poses a greater threat following significant catch-ups will we see more strict enforcement," he said.

He added the premiums currently commanded by Chinese vendors for A100 and H100 chips could collapse in the future as many of the Chinese AI startups that were driving purchases would eventually withdraw from the market.

($1 = 7.8307 Hong Kong dollars)

Follow HT Tech for the latest tech news and reviews , also keep up with us on Whatsapp channel,Twitter, Facebook, Google News, and Instagram. For our latest videos, subscribe to our YouTube channel.

First Published Date: 20 Jun, 23:32 IST
NEXT ARTICLE BEGINS