Commentary and a selection of the most important recent news, articles, and papers about AI Chipsets.
Today’s Brief Commentary
It’s been a while since I dedicated an entire newsletter to AI accelerator chips. Announcements from vendors tend to come in bursts around events such as the Hot Chips conference.
It’s hard to find any news of a new chipset that doesn’t mention NVIDIA. This makes sense, given their immense market share and revenue. It gives a point of reference for understanding why the chipset is faster, more energy efficient, better for training or inference, or is a superior choice for an application area such as Edge computing.
Sometimes, it seems like the GPU is the AI hammer, always looking for a nail to hit. While the “G” in “GPU” is for “graphics” for historical reasons, it could almost stand for “general” in many approaches to AI today. I believe there will always be a big market for these chips, but I’m particularly keen on following the new inferencing accelerators. Not all of them will survive; we’ve already seen some pivots and fire sales. This is the way of startups, though the cost of failed hardware companies can be hundreds of millions of dollars.
As one of today’s linked articles says, it could be a decade before we see serious competition to NVIDIA’s market dominance. For users, that’s okay. The field of AI began in the 1950s, and it will be unrecognizable to us in twenty-five years. Enjoy the ride, but don’t ignore the guardrails.
General News, Articles, and Analyses
NETL-Led Team Nets DOE Award To Advance Cerebras Wafer-Scale Engine Computer Chip for Energy Research Simulations | US Department of Energy
https://netl.doe.gov/node/14239
(Thursday, October 17, 2024) “NETL and partner Cerebras Systems of Sunnyvale, California, have been awarded $8 million by the U.S. Department of Energy (DOE) to advance the study of scientific phenomena using the Cerebras Wafer-Scale Engine (WSE).
The third generation WSE-3, which powers Cerebras’ CS-3 AI supercomputer, is an innovation that is designed to enable energy-efficient, extreme-speed scientific simulations, expand NETL’s computational modeling capabilities, reduce its carbon footprint, and evolve DOE’s leadership in traditional high-performance computing and artificial intelligence in a whole new approach.”
ECE startup MemryX releases MX3 Edge AI Accelerator | University of Michigan
https://ece.engin.umich.edu/stories/ece-startup-memryx-releases-mx3-edge-ai-accelerator
Author: Mena Davidson
(Tuesday, October 22, 2024) “Until now, there have been few good options for doing this type of efficient and local AI analysis. Instead, companies have been using integrated processors, which do not offer the performance for AI applications, or relying on NVIDIA chips, which are designed for graphics and data centers. These chips require a lot of power, are expensive, and are sometimes physically unable to fit the systems users are trying to deploy. Other solutions for local AI inference require engineers to retrain the models prior to running them locally, because the chips aren’t flexible enough to run the high-precision models trained on the cloud. This process can take months and requires dozens of software engineers, making these chips very challenging to use for edge applications. This is where the MX3 chips come in.”
AMD Will Need Another Decade To Try To Pass Nvidia
https://www.nextplatform.com/2024/10/30/amd-will-need-another-decade-to-try-to-pass-nvidia/
Author: Timothy Prickett Morgan
(Wednesday, October 30, 2024) “This is but a fraction of what Nvidia will do in terms of revenues and volumes. But, AMD is going to turn in the best year in its history, too. It will take a long time – and perhaps a major and highly unlikely screwup by Nvidia to let AMD catch it. Nvidia is not Intel, which let AMD catch it once with the Itanium debacle and then again with the foundry debacle. Nvidia co-founder and chief executive officer Jensen Huang, who is a distant cousin to Lisa Su, is a driven visionary and does not need to be paranoid to survive. Nvidia helped create the next wave of computing and is benefitting from first mover advantages, including massive revenue and profit streams.”
NVIDIA Sets Conference Call for Third-Quarter Financial Results | NVIDIA Newsroom
(Wednesday, October 30, 2024) “NVIDIA will host a conference call on Wednesday, November 20, at 2 p.m. PT (5 p.m. ET) to discuss its financial results for the third quarter of fiscal year 2025, which ended October 27, 2024.”
NextSilicon takes on Nvidia with Maverick-2 chip, secures tens of millions in sales
https://www.calcalistech.com/ctechnews/article/bk5eooyz1g
Author: Meir Orbach
(Wednesday, October 30, 2024) “Israeli chip unicorn NextSilicon has launched a new supercomputer chip, the Maverick-2, positioned as a direct competitor to Nvidia‘s similar offerings. Currently, only a handful of companies operate in this field, and NextSilicon is a leading startup among them. Calcalist has learned that the company has already received orders totaling tens of millions of dollars for its new chip.”
New York State Chosen as National Center for Chip Manufacturing Research
Authors: Ana Swanson and Madeleine Ngo
Commentary: Upstate New York is expanding its US leadership position in semiconductors, in addition to its world-class photonics.
(Thursday, October 31, 2024) “Albany will serve as one major hub of a bigger organization, the National Semiconductor Technology Center, which will focus on computer chip research and development, Mr. Schumer’s office said. The center is a key part of the Biden administration’s efforts to revitalize American high-tech manufacturing and lessen the country’s dependence on foreign sources of technology.
The New York site will focus on research into the complex machinery that is necessary to manufacture chips. The locations of the other two hubs, which will focus on how chips are designed and packaged together, will be announced later.”
Intel’s Gaudi AI chips are far behind Nvidia and AMD, won’t even hit $500M goal | The Verge
https://www.theverge.com/2024/10/31/24284860/intel-gaudi-wont-meet-500-million-goal
Author: Sean Hollister
(Thursday, October 31, 2024) “Nvidia made a fortune on the AI boom. AMD’s rival AI chip became the fastest ramping product in its history, already pulling in $1 billion per quarter and inspiring AMD to remake itself as an AI company too. But Intel, which suggested it would pull in $1 billion, even $2 billion on the back of AI in 2024, now says it won’t even meet its more modest $500 million goal for its Gaudi AI accelerator this year.”
Related Articles and Papers
Biden-Harris Administration Launches NSTC Workforce Center of Excellence with Expected $250 Million Investment | NIST
(Wednesday, September 25, 2024) “To address this, the WCoE, a critical part of CHIPS for America’s workforce development efforts, will bring together stakeholders from across the private sector, government, non-profits, training providers, community and technical colleges, universities, and labor organizations to develop innovative solutions to the industry’s workforce challenges, accelerate best practices, promote good jobs, and strengthen recruitment and training of the next generation of semiconductor researchers, engineers, and technicians.
The Department of Commerce’s expected $250 million investment into the WCoE over ten years will advance President Biden and Vice President Harris’s goals of driving innovation in the United States and supporting workers in securing good-paying semiconductor jobs. As part of today’s announcement, Natcast, the non-profit entity designated to operate the NSTC by the Department of Commerce, announced more than $11 million in anticipated awards across more than a dozen states and nationally through the NSTC Workforce Partner Alliance (WFPA) program to further support workforce development efforts.”
What is an AI accelerator? | IBM
https://www.ibm.com/think/topics/ai-accelerator
“An artificial intelligence (AI) accelerator, also known as an AI chip, deep learning processor or neural processing unit (NPU), is a hardware accelerator that is built to speed AI neural networks, deep learning and machine learning.”