New benchmark tests speed of systems training ChatGPT-like chatbots – Reuters
San Francisco, June 27 (Reuters) – MLCommons, a group that develops benchmark tests for artificial intelligence (AI) technology, on Tuesday unveiled results for a new test that determines system speeds when training algorithms used for chatbots like ChatGPT – and Nvidia (NVDA.O) came out on top.
The MLPerf benchmark is based on GPT-3, an AI model used to train ChatGPT, the viral chatbot developed by OpenAI and backed by Microsoft (MSFT.O). However, because the model is huge, the benchmark only uses a representative portion.
"This was our most expensive benchmark so far," MLCommons Executive Director David Kanter told Reuters. "We spent over 600K hours of accelerator compute time to develop it, plus some fantastically talented engineers."
Kanter declined to disclose the cost of development, only saying it was in the millions of dollars.
Only two chip firms – Nvidia and Intel's (INTC.O) Habana Labs – submitted results for the benchmark, with the fastest time coming from systems using the latest H100 chip from Nvidia, the uncontested leader in hardware for training AI.
Nvidia's largest system submitted in partnership with AI cloud startup CoreWeave used 3,584 H100 chips, resulting in a training time of 10.94 minutes. Habana Labs, an AI chip company acquired by Intel, ran the benchmark in 311.945 minutes with a much smaller system equipped with 384 Gaudi2 chips.
Generally, more chips and a bigger system mean faster training.
Intel's Jordan Plawner, senior director of AI Products, said the results demonstrated the potential of Gaudi2, which will have a software update in September to boost speed.
"You will get a 1.5X to 2X speed up on the Habana results. So that's when we see Habana Gaudi2 being really competitive and lower priced than H100," Plawner told Reuters.
Plawner declined to say how much a Gaudi2 chip costs, but said the industry needs a second supplier of chips for AI training, and the MLPerf results show Intel can fill that need.
Our Standards: The Thomson Reuters Trust Principles.
Thomson Reuters
Reports on global trends in computing from covering semiconductors and tools to manufacture them to quantum computing. Has 27 years of experience reporting from South Korea, China, and the U.S. and previously worked at the Asian Wall Street Journal, Dow Jones Newswires and Reuters TV. In her free time, she studies math and physics with the goal of grasping quantum physics.
The massive rally in Apple's shares is forcing some fund managers to revisit a thorny dilemma: they may not own enough of the stock.
Reuters, the news and media division of Thomson Reuters, is the world’s largest multimedia news provider, reaching billions of people worldwide every day. Reuters provides business, financial, national and international news to professionals via desktop terminals, the world's media organizations, industry events and directly to consumers.
Build the strongest argument relying on authoritative content, attorney-editor expertise, and industry defining technology.
The most comprehensive solution to manage all your complex and ever-expanding tax and compliance needs.
The industry leader for online information for tax, accounting and finance professionals.
Access unmatched financial data, news and content in a highly-customised workflow experience on desktop, web and mobile.
Browse an unrivalled portfolio of real-time and historical market data and insights from worldwide sources and experts.
Screen for heightened risk individual and entities globally to help uncover hidden risks in business relationships and human networks.
All quotes delayed a minimum of 15 minutes. See here for a complete list of exchanges and delays.
© 2023 Reuters. All rights reserved
