Microsoft Unveils Its First Custom-Designed AI, Cloud Chips

Microsoft seeks to reduce dependence on outside suppliers in two key growth areas.

A close-up view of the probe station that is being used to test the Azure Cobalt chip at Microsoft's lab in Redmond, Washington. (via REUTERS)

Microsoft Corp. unveiled its first homegrown artificial intelligence chip and cloud-computing processor in an attempt to take more control of its technology and ramp up its offerings in the increasingly competitive market for AI computing. The company also announced new software that lets clients design their own AI assistants.

The Maia 100 chip, announced at the company's annual Ignite conference in Seattle on Wednesday, will provide Microsoft Azure cloud customers with a new way run AI programs that generate content. Microsoft is already testing the chip with its Bing and Office AI products, said Rani Borkar, a vice president who oversees Azure's chip unit. Microsoft's main AI partner, ChatGPT maker OpenAI, is also testing the processor. Both Maia and the server chip, Cobalt, will debut in some Microsoft data centers early next year. 

Microsoft's multi-year investment shows how critical chips have become to gaining an edge in both AI and the cloud. Making them in-house lets companies wring performance and price benefits from the hardware. The initiative also could insulate Microsoft from becoming overly dependent on any one supplier, a vulnerability currently underscored by the industrywide scramble for Nvidia Corp.'s AI chips. Microsoft's push into processors follows similar moves by cloud rivals. Amazon.com Inc. acquired a chip maker in 2015 and sells services built on several kinds of cloud and AI chips. Google began letting customers use its AI accelerator processors in 2018. 

Not sure which
laptop to buy?

For a company of Microsoft's scale, “it's important to optimize and integrate” every element of its hardware to provide the best performance and avoid supply-chain bottlenecks, Borkar said in an interview.  “And really, at the end of the day, to give customers the infrastructure choice.”

Microsoft will also sell customers services based on Nvidia's latest H200 chip and Advanced Micro Devices Inc.'s MI300X processor, both intended for AI tasks, some time next year. Still, the industry seems to be embarking on a lasting shift toward in-house chips. The transition is particularly bad news for Intel Corp., whose own AI chip efforts are running behind. Meanwhile, with Cobalt, Microsoft is joining efforts by Amazon and AMD to grab share in the server chip market, which Intel currently dominates.

Maia is designed to help AI systems more quickly process the massive amount of data required to do such tasks as recognize speech and images. Azure Cobalt is a central processing unit that will come with 128 computing cores—or mini processors—putting it in the same league as products from Intel and AMD. The more cores the better because they can quickly divide work into small tasks and do them all at once. Cobalt also uses Arm Holdings Plc designs, which proponents say are inherently more efficient because they were developed from designs used in battery-powered devices like smartphones. Both chips will be manufactured by Taiwan Semiconductor Manufacturing Co.

Microsoft has engaged in some chip customization before, including working with AMD to design Xbox processors and  developing specialized chips for the HoloLens goggles and Xbox's Kinect motion controller. But Maia and Cobalt are the biggest and most general-purpose efforts yet—ambitious moves in a tough and expensive industry to crack.

The old joke about Microsoft is that it doesn't get things right until version 3.0, but in the chip space that's typically the case for every vendor. Borkar, who spent 27 years at Intel, said she's confident Microsoft's first efforts more than meet the mark. “We are going to deploy these next year,” she said.

The company also announced Copilot Studio, software that lets clients customize AI assistant software from Microsoft or build their own AI assistants from scratch. Customers can also design ways for Microsoft's copilot software to show up in their own existing apps. 

“Customers were very consistent in their feedback,  like, if I just want to get supply chain data in a copilot experience don't make me go build a whole other bot,” said Charles Lamanna, a Microsoft vice president. 

In general, Microsoft said it's integrating its various AI copilots, as well as the Bing Chat AI features, into one piece of software that lets users access all of them. 

The company also announced the following:

• a new lower price for clients who use both the Microsoft 365 Copilot product for Office software and the sales copilot, which normally cost a combined $70 a month. Now users can get both for $50 a month, Lamanna said. 
• new AI tools for customer service representatives, part of its Copilot products.
• Copilot for Azure, an AI assistant designed to help IT administrators troubleshoot apps, operating infrastructure and more.
• a Microsoft-run survey that showed wide satisfaction with its new corporate AI tools. For example, 77% of users said once they use it, they don't want to go back to what they used before.