What do Zhejiang practitioners think of AI’s ＂epic upgrade＂ and the big model ＂a hundred battles＂？
Chao journalist Jin Chunhua
Source: vision china
These two days are probably the moment when we witness the great development of AI.
On March 17th, local time, in the early morning of 18th Beijing time, xAI Company, a subsidiary of Musk, announced the official open source of its large model Grok-1. There are two points that attract people’s attention: first, the parameters reach 314 billion, far exceeding the 175 billion of OpenAI GPT-3.5, and it is the largest open source large language model so far; Second, it is a mixed expert model. Simply put, it is like bringing together "experts" from various fields, distributing tasks to experts from different fields, and finally summarizing conclusions to improve efficiency.
Just as the "smoke" generated by this news has not dispersed, some media broke the news. OpenAI CEO Altman publicly stated for the first time that GPT-5 will achieve an epic performance jump. In terms of hardware, NVIDIA once again launched "super chips"-Blackwell B200 GPU and GB 200; B200 can support five H100, 30 times the reasoning speed, and can train a large model with trillions of parameters.
This year, we may witness another historic upgrade of AI.
The picture comes from the Internet.
The big model of "love irony"&the big model of "crushing"
Let’s start with Musk’s Grok-1.
In all kinds of large models at home and abroad, Grok, like Musk himself, has its own traffic. In March last year, Musk registered xAI. The team’s 12 start-ups are from DeepMind, OpenAI, Google Research, Microsoft Research, Tesla, University of Toronto, etc. It is reported that they have participated in DeepMind’s Alpha Code and OpenAI’s GPT-3.5 and GPT-4 projects. So, at first, everyone believed that it was for GPT.
XAI released Grok more than half a year after the establishment of the company. The industry evaluation focuses on two points: after only two months of training, some abilities surpass GPT3.5；; And, intentionally or unintentionally, this is an ironic chat robot.
The characteristics of irony are very consistent with Musk’s manner. One of the founders of OpenAI has repeatedly publicly satirized OpenAI that it is not "open" at all, and there is no open source model. Not long ago, OpenAI was brought to court for a complaint. Then, after the appetite of all parties, I opened up my own big model-but it was Grok-1, not the latest version of Grok-1.5. Readers familiar with the industry know that the big model is half a generation behind, and it will be far behind.
When OpenAI CEO Altman attended an event in Silicon Valley, he revealed the details about GPT-5. The picture comes from the Internet.
I don’t know if it’s really because Musk released the "Challenge Book" that OpenAI "met": According to media reports, Altman said at an event in Silicon Valley that the performance of GPT-5 will be improved beyond expectations, and even said that startups that underestimate the capabilities of GPT-5 will be crushed by the next generation model.
For the open source of Grok-1, domestic industry insiders are "bearish". The reporter interviewed several big model experts from big factories, and some said that they should test them first. Some say that trying other big models is not their main business; Some also expressed their affirmation, feeling that it will promote the overall ecological construction and ecological diversification of the large model, and "multiple directions are always good".
In contrast, the attention of GPT-5 is obviously higher.
"GPT-5 may bring innovation and improvement in many aspects, such as better ability to understand and generate languages, wider knowledge coverage, more efficient learning algorithms, stronger logical reasoning ability and lower misunderstanding rate." Dr. Wu Di from School of Electronics and Information Engineering, Tongji University said that every time the GPT series model is upgraded, OpenAI strives to break through the existing limitations, including improving the generalization ability of the model, reducing the generation of deviation and misleading information, and enhancing the naturalness of interaction and user experience. The epic performance jump may also include improved security and reliability to ensure the responsible use of AI technology and reduce potential negative impacts.
NVIDIA released Blackwell architecture GPU, which greatly improved the performance of AI computing. The picture comes from the Internet.
Open source or closed source is not simple.
"It is not as simple as everyone thinks that enterprises choose open source or closed source." Kong Jun, general manager of Zhejiang Big Data Trading Center, told the reporter. Generally speaking, enterprises should have enough technical confidence in open source, and the threshold is still quite high. For example, ali tong Yiqianwen Big Model won six full marks in IDC’s "AI Big Model Technical Capability Assessment Test".
Choosing open source has many considerations such as creating ecology and obtaining data. Some experts said that open source does not necessarily mean doing public welfare, but also commercial considerations. Perhaps its commercial interests are relatively long-term, and at this stage, it is necessary to expand ecology and expand influence.
According to the observation of research institutions, the relevant large model manufacturers are very flexible in the choice of open source or closed source: at present, there are GPT-3.5 and GPT-4 of OpenAI in foreign countries with completely closed source, and there are large models of Wenxin of Baidu in China; First close the source and then open the source, there is Alibaba Cloud’s general question; There are also open source first and then closed source, such as Baichuan-7B and Baichuan-13B of Baichuan Intelligent.
"Three elements of the big model, data, computing power and algorithm. Open source may invite all parties with data to try in an open and cooperative mode because of insufficient data. " Kong Jun said that this is also a way to use data, or it has certain reference value for promoting the marketization of data elements in China.
On Monday, local time, NVIDIA held its annual developer conference, and Huang Renxun announced a new generation of GPU architecture Blackwell, which supports the construction of real-time generative AI. According to industry analysts, NVIDIA’s new structure cannot be separated from the support of open source. In May, 2022, after being scolded by the developers for ten years, NVIDIA finally promoted the open source of GPU computing, attracting a large number of R&D personnel to participate in innovation. The widely used deep learning frames such as PyTorch and TensorFlow were deeply bound with the NVIDIA GPU, which also accumulated a lot of optimization experience for NVIDIA’s own GPU products.
More importantly, after this benign ecological interaction is established, the market will also open. It is reported that at present, NVIDIA’s high-end GPU has occupied 88% of the global market share, with a market value of about 2.2 trillion US dollars, second only to Microsoft and Apple.
"Open source of large models will become a trend, and a pattern of open source of basic versions and closed source of high-level versions will be formed." Zhang Wei, a professor at the School of Software of Zhejiang University and deputy director of the Institute of Complex Systems and Data Engineering, told reporters that open source can be understood as a high school student with certain basic knowledge and strong room for expansion, and then he enters the closed-source university study stage and chooses specific directions such as literature, science and medicine, that is, scene segmentation and specialization.
Based on the development trends at home and abroad at this stage, Zhang Wei suggested that on the one hand, the infrastructure such as computing power, algorithms and data is really important, on the other hand, China’s industrial categories and industrial chains are relatively complete, and the manufacturing industry is relatively developed, which is also a big advantage; Domestic large model manufacturers and start-up companies can focus on segmentation and strive to be ahead in the industry.
The European Parliament formally voted to pass and approve the European Union’s "Artificial Intelligence Act", which indicates that the European Union has cleared the last obstacle of legislative supervision of artificial intelligence. The picture comes from the Internet.
Foundation, foundation or foundation?
In the face of this "epic upgrade" of AI software and hardware, relevant people have once again emphasized the importance of the foundation.
In Zhang Wei’s view, although there are hundreds of big models, with the development of the industry, there won’t be too many truly basic general big models, just like the market competition process in bike-sharing in those years. This also puts forward higher requirements for large companies, especially large companies, to build universal models.
"From the perspective of the development of the whole industry, we still have to start from the foundation." Wu Di said, on the one hand, the big model of open source is not simply to use it, but also to learn its logical structure, otherwise it can only follow others all the time; On the other hand, even the open source big model, its core part is not necessarily 100% open source, especially the latest version of the big model, such as xAI of Musk and OpenAI; Just like netizens in China commented that Tesla’s open source was "backward".
In contrast, Wu Di feels that the strength of domestic large-scale models in Industry-University-Research can be further increased. "Colleges and universities have technology, but the hardware conditions are relatively insufficient. Large models require high hardware configuration and often need hardware clusters, which may be the advantage of enterprises. To put it further, at present, the big models all over the world are essentially statistical probability models. This is inseparable from the emphasis on the foundation of mathematics. "
Kong Jun also emphasized the importance attached to the cooperation between the government, universities and enterprises. "Data, computing power and algorithms are indispensable. At present, government departments are working hard to increase the supply of high-quality data, cultivate domestic computing power, and encourage university enterprises to speed up algorithm research. "
At present, despite the relative lack of top chips in computing power in China, the total amount and increment have developed rapidly. Last November, a set of data released by the Ministry of Industry and Information Technology showed that in recent years, the annual growth rate of China’s computing power industry was nearly 30%, and the total scale of computing power ranked second in the world. By the end of last year, the scale of China’s computing core industry reached 1.8 trillion yuan.
Of course, the computing power has to be used, which involves the question of whether the domestic computing power is good or not. Earlier, some people in the industry pointed out that our computing power has a "strange circle" where we spend a lot of money but it is not easy to use, and on the other hand, our computing power is in short supply. The pressure is given to the domestic self-developed computing chip.
In addition to the above industrial foundation, many people in the industry also mentioned some institutional foundations, such as law.
On the 13th local time, the European Parliament formally voted and approved the Artificial Intelligence Act to strictly regulate the use of artificial intelligence.
On the morning of March 16th, the first "AI Good Governance Forum" held in Beijing released the "Artificial Intelligence Law (Scholars’ Proposal Draft)". The drafting expert group consists of experts from China University of Political Science and Law, Institute of Data Rule of Law, northwest university of politics and law Institute of Rule of Law and China Institute of Information and Communication.
"With the advancement of technology, it is very important to continuously pay attention to and evaluate the social, ethical and security impacts of AI technology. The rapid development of the AI field needs to be accompanied by corresponding policies, guiding principles and regulatory measures to ensure that these advanced technologies can have a positive impact on society and effectively manage potential risks. " Wu Di said that in this respect, China has paid more attention to the privacy protection of personal data because of the early start of big data and artificial intelligence; The improvement of the legal system will further promote the development of the artificial intelligence industry.
"Reprint please indicate the source"
Reporting/feedback