"Edge computing and machine vision technologies are quite mature, and more attention and resources may be invested in the development and application of big language models and AI in the future. AI models may gradually shift from the cloud to the edge. We can foresee a huge wave of innovation brewing. At present, there are many mature solutions on the market, and we are at a critical moment of this transformation During the 2024 Intel Network and edge computing Industry Conference, Sachin Katti, senior vice president and general manager of Intel Network and Edge Business Unit, said in an interview with the Associated Press.
Recently, reporters from Caixin News Agency have learned from various sources in the industry that the penetration of artificial intelligence in the edge field will become a key trend from the perspectives of computing power, edge reasoning ability, and cost. Multiple giants have entered this field, and many mature solutions have emerged. Edge AI is thriving, and the global edge market size may reach billions of US dollars by 2030.
Targeting edge AI, mature solutions emerge
According to the latest information from Intel, the edge business of Intel in China maintained year-on-year growth in the first half of this year, and the company expects this trend to continue. Edge AI has always been a focus of Intel's attention over the years, with deployment areas involving retail, pharmaceuticals, logistics, agriculture, transportation, manufacturing, and more.
Sachin Katti told Caixin reporters, "The edge AI we are talking about now has gradually progressed from machine vision at the edge to applications such as big language models, edge applications, and generative AI. Therefore, Intel needs to continuously provide relevant capabilities to accelerate the deployment of generative AI and big language models at the edge. We are no longer limited to hardware supply, but have also expanded to multiple dimensions such as application layer, software, and customer service support
(Photo by Caixin reporter)
During this year's MWC, Intel announced that it has completed over 90000 actual deployments on the edge end. Sachin Katti stated at the conference that Intel will adopt an open, modular, and unified platform centered approach to accelerate the deployment process of edge Al solutions. On the hardware side, we will provide GPU and NPU solutions with sufficient memory, and many models such as Llama 2, Llama 3, etc. can run locally In terms of ecology, as of now, Intel Network and Edge Business Unit has established cooperative relationships with more than 500 OEM/ODM and more than 150 ISVs in China.
In fact, not only Intel, but also giants such as AMD, Nvidia, Qualcomm, as well as A-share listed companies such as Yuntian Lifei (688343. SH), Zhongke Chuangda (300496. SZ), and Guoke Microelectronics (300672. SZ) have taken actions targeting the edge end.
At the same time, Caixin News Agency reporters noticed that A-share manufacturers such as Robot (300024. SZ), Shenxin (300454. SZ), Digital Video (300079. SZ), and Zhongke Chuangda have all demonstrated their edge AI achievements in the exhibition area.
The reporter from Caixin News Agency noticed a case where data annotation was done through automated means during the model training process in the exhibition area. The booth staff told reporters that the CNN edge training and promotion solution based on Intel Core processors and Intel Ruixuan independent graphics card ARC770 has achieved automatic image annotation and MLFlow based model automatic retraining with the help of visual large models. Previously, training models required finding algorithm companies and labeling needed to be done by humans. Now, machine labeling can be used, greatly reducing training and labor costs. This type of training push solution is gradually emerging in the industry
Intel's latest 3D virtual cyborg "Xiaoying" is also on display this time. The booth staff said that "Xiaoying" is also based on Core processors and ARC770 graphics cards, fully deployed at the edge, and locally integrated with a large language model and RAG technology. It is reported that he is capable of fulfilling roles such as customer service in business halls, shopping guides in shopping malls, and museum guides.
What are the reasons for the emergence of AI agents in edge deployment?
From the perspective of industry development, Intel has made a three-stage summary of the development of AIGC and big language models: the first stage is the Age of AI Co Pilots, the second stage is the Age of AI Agents, and the third stage is the Age of AI Functions.
From the perspective of China, our innovation speed is very fast, and many AI Agent functions may have already seen signs of development at present. Cases related to AI Agents have actually slowly emerged around us, "said Zhang Yu, Chief Technology Officer of Intel's Network and Edge Business Unit in China and Senior Chief AI Engineer of Intel, to Caixin reporters.
He gave an example that telecom enterprise network security and network operation products can already analyze network log files based on large models, so that network managers can take timely measures based on the analysis results. Sachin Katti stated that he expects to see more AI agents emerge in the next one to two years.
Intel executives mentioned at the conference that the size of China's generative artificial intelligence market is expected to reach $3.3 billion this year. According to Gartner's forecast, by 2026, 80% of global enterprises will use generative AI, and 50% of global edge deployments will include AI. According to a report by STL Partners, the global edge services market is expected to reach $445 billion by 2030, with AI being the largest edge workload.
What are the actual reasons driving enterprises to deploy on the edge side as the edge AI market continues to heat up?
One is the security of data. Is it more reasonable to safely store all data in the cloud or at the edge? Secondly, with the increasing amount of edge data, the entire transmission bandwidth is a problem. Although China is the world's leading in infrastructure transmission bandwidth construction, when a large amount of data is generated at the edge, it may still trigger network storms. We still need to further optimize network management and data transmission strategies. Thirdly, real-time performance. Many things can only be solved at the edge to meet real-time requirements. "Guo Wei, Vice President of Intel's Marketing Group and General Manager of Intel China Network and Edge and Channel Data Center Business Unit, analyzed three reasons to Caixin reporters.
Sachin Katti told Caixin reporters that currently AI mainly operates in the cloud, but as edge devices generate a large amount of data locally, the cost of transmitting all data to the cloud has become quite high. Some factories may not be able to afford the cost of training large-scale industry level models, as the number of parameters for these models may reach trillions. Therefore, they often choose to train medium-sized models and make customized adjustments based on their own data
According to a reporter from Caixin News Agency, the current training, inference, calling, and deployment of large models are all completed in the cloud, which places extremely high demands on computing power (cluster) scale, network stability, energy efficiency, and other aspects. As large models enter the stage of competition and landing, it is required that all analysis and processing be completed locally.
Zhang Yu believes that "the characteristic of the edge is fragmentation. Different users have different requirements for computing power and performance. Some performance requirements are very high and must be supported by a cluster, while others may be supported by a few small devices, with a very large span. The edge often ultimately deploys business rather than pursuing a technical solution
