On November 13th local time, NVIDIA released its latest AI chip H200 Tensor Core GPU, equipped with 141 GB of graphics memory, capable of processing massive amounts of data for generative AI and high-performance computing workloads. According to official disclosure, compared to the previous generation H100 chip of the Nvidia H series, the capacity has almost doubled and the performance has improved by up to 60% to 90%. The previous generation H100 chip, which is the chip used by OpenAI, the world's most highly regarded artificial intelligence startup, to train the large language model GPT-4.
Broadly speaking, any processor that can run AI algorithms can be referred to as an AI chip, including GPU, FPGA, ASIC, etc. In a narrow sense, AI chips are designed for AI algorithms, often sacrificing a certain degree of universality in exchange for improving the specialized performance of AI algorithms.
According to New Intelligence, the Nvidia H200 is the first GPU to use the HBM3e extended version of high bandwidth storage, with a graphics memory of up to 141GB, nearly doubling compared to the 80GB of the previous generation H100. H200 and H100 are based on the same Hopper architecture, which can accelerate LLM and other deep learning models based on Transformer architecture.
The performance improvement of H200 is mainly reflected in the inference performance of large models, and its processing speed of Llama270B large language model is almost twice that of H100. The performance of H200 processing high-performance computing (HPC) applications has also improved by nearly 20% compared to the previous generation.
The bandwidth of graphics memory is crucial for HPC applications, enabling faster data transfer and reducing complex processing bottlenecks. For memory intensive HPC applications such as simulation, scientific research, and artificial intelligence, the higher memory bandwidth of H200 ensures efficient access and operation of data, and the time to obtain results can be up to 110 times faster compared to CPUs.
In addition, according to NVIDIA's official website, H200 has also improved energy efficiency, reducing energy consumption by approximately 50% compared to H100.
At present, NVIDIA's global partner server manufacturer ecosystem includes Huaqing Rack, Asus, Dell Technology, Eviden, Gigabyte, Huihe, Ingra Technology, Lenovo, QCT, Supermicro, Wistron, and Weiying Technology, among others, which can directly update their existing systems using H200. In addition to NVIDIA's own investments in CoreWeave, Lambda, and Vultr, cloud service providers such as Amazon Network Services, Google Cloud, Microsoft Azure, and Oracle Cloud will deploy the first batch of H200 chips starting next year.
On the day of the H200 chip release, NVIDIA's stock rose 0.59%, with an intraday gain of 1.4%. The official website of Nvidia has not yet disclosed the selling price of the H200 chip. According to CNBC, the previous generation chip H100 began mass production earlier this year, with a price of approximately $25000 (approximately 182000 RMB) per GPU.
According to New Intelligence, NVIDIA's supercomputer GH200, which integrates H100 GPU and Grace CPU, will provide a total of approximately 200 Exaflops of AI computing power to supercomputing centers around the world with the support of the new generation H200 chip. The Nvidia GH200 will be used to build supercomputers in major supercomputing centers around the world, such as the Ulrich Supercomputing Center in Germany, the Japan Advanced High Performance Computing Joint Center, the Texas Advanced Computing Center, and the National Supercomputing Application Center in the United States.
According to foreign media reports, industry experts have stated that AMD's upcoming MI300 accelerator is expected to become a competitive AI product. AMD will hold a keynote speech on December 6th on "Promoting Artificial Intelligence", unveiling the Instinct accelerator series codenamed MI300.
The development and cross-border sales of AI chips are also at the center of the international political vortex. In October of this year, the United States tightened its export of AI chips to China. The US government required Nvidia to obtain a license before selling chips such as A100 to Chinese customers, which hindered Nvidia and Intel's presence in the Chinese market.
At present, domestic AI chip related manufacturers mainly include Cambrian, Jingjiawei, Yuntian Lifei, Hengshuo Co., Ltd., Haiguang Information, Fudan Microelectronics, Anlu Technology, Lanqi Technology, Hangyu Micro, Guoxin Technology, Ziguang Guowei, Guoke Micro, etc. Among them, Cambrian ASIC chips are the leading domestic AI chips, while Jingjiawei GPU chips are the leading domestic GPU chips.
In addition, other Chinese technology giants and startups have also established a certain layout in the AI chip field.