Nvidia Cup B200 Chip: Moore's Law Fails, Multi Card Interconnection Wins the King
Ty奇葩罗牛山831
发表于 2024-3-19 21:39:12
1194
0
0
On the early morning of March 19th Beijing time, at the NVIDIA GTC (GPU Technology Conference), NVIDIA CEO Huang Renxun announced the successor of Hopper architecture chips - Blackwell architecture B200 chips. At present, there is a high demand for Nvidia Hopper architecture chips H100 and GH200 Grace Hopper superchips, providing computing power for many of the world's most powerful supercomputing centers, while B200 will provide further intergenerational leap in computing power.
The B200 chip of Blackwell architecture is not a traditional single GPU. On the contrary, it consists of two tightly coupled chips, although according to Nvidia, they do act as a unified CUDA GPU. These two chips are connected through a 10 TB/s NV-HBI (Nvidia High Bandwidth Interface) connection to ensure that they can function as a single, completely identical chip.
Multi card interconnection is the key to improving B200 computing power. The GB200, which combines two GPUs with a single Grace CPU, can provide 30 times the performance for inference work in large language models while potentially significantly improving efficiency. Nvidia claims that compared to H100, B200 can reduce the computational cost and energy consumption of generative AI by up to 25 times.
The improvement of NVIDIA AI chip performance in terms of computing power mainly relies on data accuracy. From FP64, FP32, FP16, FP8 to the current B200 chip FP4, the maximum theoretical computational cost of FP4 is 20 petaflops (data accuracy unit). FP4 is twice the performance of FP8, and the advantage of FP4 is that it increases bandwidth by using 4 bits instead of 8 bits for each neuron, doubling computation, bandwidth, and model size. If B200 is converted to FP8 and compared with H100 in the same category, theoretically B200 only provides 2.5 times more computing power than H100, and a large part of the computing power improvement of B200 comes from the interconnection of the two chips.
The Moore's Law of the CPU era (the number of transistors that can be accommodated on an integrated circuit doubles approximately every 18 months) has entered its twilight years. TSMC's breakthrough in the 3nm process has not brought about a generational improvement in chip performance. In September 2023, the Apple A17 Pro was launched, using TSMC's first 3nm process chip, but with only a 10% improvement in CPU performance. Moreover, the development of advanced process chips is costly. According to the Far East Research Institute, TSMC's wafer foundry prices in 2023 have increased by approximately 16% (advanced process) to 34% (mature process) compared to two years ago.
Besides Apple, another major chip customer of TSMC is NVIDIA - NVIDIA's hard currency AI chip H100 adopts TSMC's N4 (5nm) process and utilizes TSMC's advanced CoWoS packaging capacity.
Moore's Law is invalid, and Huang Renxun's Huang's Law states that the efficiency of GPUs will more than double every two years, and innovation is not just about chips, but the entire stack.
Nvidia continues to move towards multi card interconnection. Since the improvement of 3nm chips is limited, Nvidia's B200 chooses to place two 4nm chips side by side and form a super large chip with over 200 billion transistors through high-speed on-chip interconnection. At NVIDIA GTC, Huang Renxun briefly mentioned the performance of the chip itself, with a focus on the DGX system.
In terms of multi card interconnection, Nvidia's NVLink and NVSwitch technologies are its moat. NVLINK is a peer-to-peer high-speed interconnect technology that can directly connect multiple GPUs to form a high-performance computing cluster or deep learning system. In addition, NVLink introduces the concept of unified memory, supporting memory pools between connected GPUs, which is a crucial feature for tasks that require large datasets.
NVSwitch is a high-speed switch technology that can directly connect multiple GPUs and CPUs to form a high-performance computing system.
With the support of NVLink Switch, Nvidia miraculously connected 72 B200s together, ultimately becoming the "new generation computing unit" GB200 NVL72. A "computing unit" cabinet like this has an FP8 precision training computing power of up to 720 PFlops, approaching a DGX SuperPod supercomputer cluster (1000 PFlops) in the H100 era.
Nvidia has revealed that this brand new chip will be launched later in 2024. Currently, Amazon, Dell, Google, Meta, Microsoft, OpenAI, and Tesla have all planned to use Blackwell GPUs.
The method of packaging and wholesale card sales also meets the card usage needs of large model companies. Packaging multiple GPUs together into a data center is more in line with the purchasing methods of large model companies and cloud service providers. According to Nvidia's 2023 financial report, 40% of Nvidia's data center business revenue comes from large-scale data centers and cloud service providers.
As of the closing of the US stock market on March 18th Eastern Time, Nvidia's stock price was $884.550, with a total market value of $2.21 trillion.
CandyLake.com 系信息发布平台,仅提供信息存储空间服务。
声明:该文观点仅代表作者本人,本文不代表CandyLake.com立场,且不构成建议,请谨慎对待。
声明:该文观点仅代表作者本人,本文不代表CandyLake.com立场,且不构成建议,请谨慎对待。
猜你喜欢
- ASML's' Big Thunder ', Intel and Samsung' Blame '? NVIDIA and TSMC are in internal conflict! Tech giants' earnings season is not calm
- NVIDIA GPU租赁价格腰斩!H100和RTX 4090双双猛跌50%
- NVIDIA GPU rental prices halved! H100 and RTX 4090 both plummet by 50%
- NVIDIA GPUレンタル価格が足踏み!H 100とRTX 4090がともに50%急落
- NVIDIA GPU 임대 가격 요절!H100 및 RTX 4090 모두 50% 급락
- Only one step away from reaching the summit of the world! Nvidia's market value has exceeded $3.5 trillion, and Wall Street continues to be bullish
- Did you miss out on Nvidia in the AI craze? Hedge fund tycoon shouts: buy this' bargain '!
- Nvidia's new generation AI chip GB200 orders explode, H100 chip hits cold
- ParTec files patent infringement lawsuit against Nvidia in Munich
- Nvidia's intraday market value surpasses Apple again, and the battle for the top spot in the US stock market is becoming increasingly fierce
-
【科技记者古尔曼:苹果计划于12月第一周发布iOS 18.2系统更新 带来更多人工智能功能】科技记者古尔曼透露,苹果计划于12月第一周发布iOS 18.2系统更新。iOS 18.2将为iPhone 15 Pro机型和所有iPhone 16机型带来更多 ...
- cristianna
- 昨天 17:32
- 支持
- 反对
- 回复
- 收藏
-
为期超七周的大罢工终于落下帷幕。 当地时间11月4日,波音美国西海岸工厂工人们就改进后的合同提案投票。 随后,代表着波音超过33000名西雅图地区机械师的IAM工会经表决,以59%的同意票决定接纳波音提 ...
- cristianna
- 2 小时前
- 支持
- 反对
- 回复
- 收藏
-
近日,爱立信中国区总裁方迎在接受《经济参考报》记者采访时表示,5G技术在全球范围内得到了迅速发展,但面临商业潜力未能充分挖掘、网络运营难度较以往更高两大挑战。因此,运营商在继续5G网络部署的同时,应关 ...
- blueskybb
- 昨天 15:05
- 支持
- 反对
- 回复
- 收藏
-
“新四化”的时代浪潮下,新能源汽车行业百家争鸣。伴随着自主品牌不断崛起,合资品牌当下的生存状况备受外界关注,如何打好电动化时代的突围战,成为合资品牌的新课题。 作为国内合资车企的代表之一,上汽 ...
- mbgg2797
- 5 小时前
- 支持
- 反对
- 回复
- 收藏