On December 7th, Caixin News Agency reported that US technology giant Google recently announced the launch of its largest and most powerful AI intelligent model, the Gemini.
The Gemini model released by Google this time can achieve multimodality and significantly improve performance. Gemini is a multimodal model built on Transformer decoder, which can process information in different forms of content such as video, audio, and text. The latest Gemini model is able to perform more complex reasoning and understand finer information compared to previous technologies. It can extract key points from hundreds of thousands of documents by reading, filtering, and understanding information, which will help achieve new breakthroughs in many fields from science to finance.
The Gemini model can be divided into three versions based on its size: Gemini Ultra, Gemini Pro, and Gemini Nano, all of which support contextual 32K understanding. Among them:
1) The Ultra version is the most powerful version and can demonstrate the highest efficiency in the corresponding TPU infrastructure. In multiple tests, the performance of the Ultra version exceeds GPT4V;
2) The Pro version is a cost-effective optimized version with strong capabilities in reasoning, multimodality, and other aspects. It has good scalability and can complete pre training within a few weeks. In multiple tests, it is second only to GPT4V and stronger than mainstream large models such as PaLM2, Claude2, LLaMA2, and GPT3.5;
3) Nano: It is a 4-bit model distilled from other models, with two versions: 1.8B and 3.25B, targeting low memory and high memory devices respectively, and supporting local deployment
The Gemini model, as the first multimodal model released by Google and globally, supports cloud and edge testing. According to relevant test data, Gemini Ultra outperforms human expert models in MMLU (Massive Multi tasking Language Understanding), with performance surpassing GPT-4 in multiple tasks when compared horizontally.
Minsheng Securities stated that by evaluating the Gemini model family in over 50 benchmark tests, as the model size increases, the Gemini model family continues to improve its quality in reasoning, mathematics/science, and long texts. Among all six abilities, Gemini Ultra is the best model. As the second largest model in the Gemini model family, Gemini Pro is also highly competitive in performance and more efficient in providing services.
Minsheng Securities pointed out that the Gemini training process can also innovate infrastructure, algorithms, and datasets;
In terms of infrastructure: Gemini is trained by Google TPUV5e and TPUV4, and has demonstrated engineering innovation during the training process. For example, by connecting 4096 TPUV4 chips to a dedicated optical switch, the 4x4x4 chip cube can be dynamically reconfigured as a super node of any 3D ring topology structure in about 10 seconds, and targeted deployment of Gemini Ultra and thermal maintenance functions. In response to the high inter chip interconnection speed required for the Ultra version, Google has applied multiple patented technologies such as OCS optical switching, but the final speed is not yet provided in the article.
In terms of algorithms, techniques such as single control algorithms and XLA compilers are used to optimize the training process, and stable training is achieved by preventing SDC and other issues.
In terms of dataset, Gemini training and inference speed are improved through word segmentation technology, and a series of filtering methods are used to ensure the high quality of the data used for training
The latest version of Google's computing chip TPU v5p has been released simultaneously. TPU v5p is an improvement of the previous TPU v4 version. Compared with TPU v4, TPU v5p has twice the floating-point performance and trains large language models 2.8 times faster. CITIC Securities believes that the official release of the multimodal Gemini model can expand the application scenarios and bring about continuous upgrades in computing power demand. Minsheng Securities continues to be optimistic about the future prospects of the AI industry and believes that the release of models such as GPT-5 will also bring more catalysis.
CITIC Securities stated that in the current search scenario, Gemini can reduce latency by approximately 40%. For the entire industry, the promotion of Google's productization and commercialization will also bring about overall changes. At the same time, with the launch of models such as GPT-5, it is expected to see: 1) the increase in computing power demand brought by multimodal models; 2) More and more AI scenarios and products are emerging.
The release of Gemini will further bring more expectations for multimodal models, which will drive an increase in computing power demand for the industry; In the medium to long term, it is expected that the upgrade of multimodal models will enrich the usage scenarios of related products, coupled with cost optimization brought about by hardware upgrades and algorithm optimization. The progress of 2C products is worth looking forward to.
CITIC Securities stated that it remains optimistic about the long-term impact and changes of this round of generative AI on the technology industry, and continues to focus on leading manufacturers in areas such as computing power, algorithms, data, and applications.