On March 14th, OpenAI founder Sam Altman invested in the chip startup Cerebras Systems, which launched the latest Wafer Scale Engine 3 chip (also known as WSE-3 chip). The single area of this chip reaches about 46225 square millimeters, which is more than 50 times the usual chip area and even larger than the area of a book.
The company claims that the WSE-3 chip is specifically designed to train the industry's largest AI model, and this new device is built using TSMC's 5-nanometer process, containing 4 trillion transistors and 900000 AI cores. In contrast, the Nvidia H100 chip contains 80 billion transistors.
It is worth mentioning that when introducing the performance parameters of the WSE-3 chip, Cerebras Systems fully benchmarks this product against the Nvidia H100. The company's introduction page shows that in terms of artificial intelligence training acceleration, the performance of this chip is 8 times that of the H100& Amp; Quota; You may have seen Cerebras stating that its platform is easier to use than NVIDIA's platform. One important reason for this situation is the way Cerebras stores weights and activates them, and it does not need to scale to multiple GPUs in the system, and then to multiple GPU servers in the cluster& Amp; Quota; The company was introduced in a press release.
&Amp; Quota; Da& Quota; Make miracles with force
According to public information, Cerebras Systems is an American AI chip unicorn company founded in 2016. The company is mainly committed to changing the core architecture of chip computing, improving chip integration, simplifying cluster connections, and researching related fields, as well as developing chip products for deep learning. The latest financing for Cerebras Systems occurred in 2021, led by Alphawave Ventures and Abu Dhabi Growth Fund (ADG), with a financing amount of $250 million. As of this round of financing, Cerebras has a total financing amount of 720 million US dollars and a company valuation of over 4 billion US dollars.
Equally noteworthy is the investment team of Cerebras Systems, which can be described as& Quota; Big shots; Quota; Gathering, including OpenAI founder Sam Altman, former AMD CTO Fred Weber, as well as several star investment institutions such as Benchmark, Coat Management, and Eclipse Ventures.
Cerebras Systems' unconventional style has always been well-known in the industry - the chips produced by the company have always been; Quota; Giant; Quota; Famous. In August 2019, Cerebras Systems released the first chip of the Wafer Scale Engine (WSE) series, which integrates 1.2 trillion transistors and 400000 AI cores. When the manufacturing process of common chip manufacturers still involves cutting a wafer from a silicon ingot and dividing it into hundreds of independent chips, WES chooses to directly make the entire wafer into one chip, which also results in the area of a single chip being the same as the rectangular area that can be cut from a 12 inch wafer.
In April 2021, Cerebras Systems launched WSE-2. According to official data, the second-generation chip has 2.6 trillion transistors and 850000 cores, which is more than twice the number of transistors, cores, memory, and other components compared to the first generation. The core area has already reached 46225 square millimeters, which is 56 times the largest GPU core area at that time. In 2022, the chips produced by Cerebras Systems were collected by the Silicon Valley Museum of Computer History.
In the product introduction of WSE-3 chips, the company pointed out that in today's cluster, there may be tens of thousands of GPUs or AI accelerators to handle a problem. Reducing the number of chips by more than 50 times can reduce interconnect and network costs as well as power consumption. In NVIDIA GPU clusters with Infiniband, Ethernet, PCIe, and NVLink switches, a significant amount of power and cost is spent on reconnecting chips. Cerebras Systems solves this problem by keeping the entire chip together.
And large chips are also an important reason for Cerebras Systems to obtain investment. Industry analysts told First Financial reporters that such chips provide a possible alternative to Nvidia systems. At the same time, larger chips have more transistors, which means greater computing power and memory advantages.
Will the market pay for it?
Although Cerebras Systems has stated that they have seen strong market interest in this chip and have reported a large backlog of orders from various departments, including businesses, governments, and international cloud. However, the above industry insiders analyzed to First Financial reporters that Cerebras Systems' products are still outside the focus of the mainstream market. At the same time, if such a huge chip is to be applied in data centers, it must consider various issues such as power consumption, hardware supporting facilities, software architecture, etc. For mainstream market demand, this product is not a suitable solution.
Previously, Ultraman stated that his neural mimicry method can significantly reduce AI development costs and provide assistance in achieving truly universal artificial intelligence. As OpenAI is facing a shortage of computing resources in recent times, the call for OpenAI to acquire Cerebras Systems is also increasing.
Jim McGregor, Chief Analyst at Tirias Research, believes that compared to Nvidia, Cerebras is more like a niche platform, and the breadth of the two cannot be compared. From the AGI goals that OpenAI is committed to achieving, although WSE is a wafer level chip, there are also situations that cannot be handled. Million dollar AI supercomputers, as well as Nvidia's SuprPod and Selene, are more suitable alternatives.