The good, the bad, and the ugly of supercomputing 23 or close to it

The good, the bad, and the ugly of supercomputing 23 or close to it

This year’s event was as much about artificial intelligence as it was about high-performance computing. The only booths that didn’t talk about AI were, well, nobody. Everyone has been touting the miracles of AI, from central processing units (CPUs) to accelerators to systems companies to networking vendors to storage to the cloud to water cooling systems to the US Department of Energy and the US Department of Defense. And then BOD animation.

The 23rd Supercomputing Conference in Denver is a wrap, with some extracurricular activities thanks to Microsoft and Here’s a summary of the good, the bad, and the ugly.

The good

Nvidia was everywhere and nowhere

The traditional big green booth was absent at SC’23. They didn’t need their own suite because almost every system vendor offered an H100-based server.

Nvidia has made some announcements, of course. Most impressive was Europe’s first Exaflop monster at the Julich Supercomputing Center in Germany, which the company touts will be the world’s fastest AI system. “Jupiter” is the first system to use the Grace Hopper 200 (GH200) with additional HBM capacity and bandwidth. Based on Eviden’s BullSequana Nvidia Quantum-2 InfiniBand Networking platform. The system also includes a “cCuster Module” equipped with the new European ARM CPUs from SiPearl, supplied by the German company ParTec. SiPearl promises a massive memory data rate of up to 0.5 bytes per flip using the Rhea CPU, which is approximately five times that of the GPU, providing high efficiency for complex data-intensive applications.

Nvidia also announced (at Microsoft Ignite) AI Foundry services on Azure, with the core models and Enterprise AI software suite now available on Microsoft Azure. SAP, Amdocs, and Getty Images were among the first companies to build custom LLM courses and deploy those models using Enterprise AI software.

Cerebras continues to gain momentum, with G42 alongside it

As a follower of Cambrian-AI, you know that we rank very highly in Cerebras’ systems, and have been since it came out of stealth. It remains among the few companies with a long-standing discrimination against the green tide.

I spent a few minutes with CEO Andrew Feldman at the show, and he remained typically enthusiastic, especially since Cerebras is the only AI hardware startup with hundreds of millions in revenue. In addition to UAE-based G42, Cerebras includes GlaxoSmithKline, Total, AstraZeneca, Argonne National Laboratories, EPCC, the Pittsburgh Supercomputing Center, Inverness, the National Energy Technology Laboratory, the Leibniz Supercomputing Center, NCSA, and Lawrence Livermore Laboratories. National, Cerebras Corporation. An unnamed major financial services organization.

AMD is about to announce the MI300, but MS Maia might steal the show

But you couldn’t tell it wasn’t available yet while walking around the show floor. Microsoft, HPE Cray and others have talked about the upcoming MI300 family. I won’t spoil the news, which will be released on December 6, but at the kiosks and at MS Ignite, the news has been front and center with massive anticipation.

Micron Technologies: A better HBM mousetrap?

Micron, the only remaining U.S. memory company, was showing off its version of the HBM3e, which they say has more bandwidth and memory capacity than its Korean rivals, Samsung and SK Hynix. While talking to company representatives, I get the impression that a huge person is lining up to place orders. See our review here.

Microsoft Maia: Can SRAM make up for HBM’s deficit?

Also at Ignite, which ran in conjunction with Supercomputing 23, Satya Nadella announced the in-house Maia we covered on Forbes last week. This seems like a good start, but I’m surprised by the small amount of HBM. They sure got the memo that GPT4 needs a lot of fast memory, right? I’m sure they will fix this issue soon, but AMD and Nvidia are not standing still. The Maia 100 only has 64GB of HBM but a ton of SRAM. Standards please? I think Microsoft’s designers know more about how LLM performs with this combination of memory than I do.

Grok and Samba Nova find their groove.

The emergence, or explosion, of large language models has given two prominent startups, Groq Inc. And SambaNova Systems, a reason to brag. These two companies were working on next-generation silicon, and their booths were packed with interested scientists wanting access. Since both startups have adopted the AI-as-a-service business model, they can accommodate interested data scientists without installing massive hardware locally. Honestly, I was very skeptical of both companies until I saw their demos and spoke with company leadership at SC’23.

Groq Inc. showed World’s fastest inference performance of Llama 2 70B – competitive with GPT-3. To celebrate their record-breaking performance, Groq brought a cute and cuddly llama named Bunny to the SC23 event in front of the Convention Center. The company’s demo was nothing short of amazing, showing what appears to be at least a 10X performance advantage over Nvidia GPUs in running GPT-3 inference queries. Standards please!

(Tags for translation)Satya Nadella

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *