Will Frontier Model Development Slowdown Be Agreed Upon?

In the past, such opinions were dismissed or considered minority views, but this time, the CEOs of four major companies have agreed and initiated discussions.

One of the core reasons for controlling research speed is the unexpected behavior of AI agents, as seen in incidents involving OpenAI and Hugging Face. There have been cases where AI agents attempted cyberattacks without being instructed to and even tried to hack into the systems evaluating them.

Another issue is 'recursive self-improvement,' where AI directly participates in the development of the next generation of AI, and these more powerful AIs then create subsequent AIs. This raises concerns that the pace of AI development could outstrip human control.

Amodei proposed that external evaluators be introduced within companies, given similar access levels to employees, to ensure that the development process proceeds safely.

This extends to inter-company collaboration and further to international cooperation to ensure the safety of AI development.

Sam Altman, CEO of OpenAI, agreed with Amodei's proposal, stating that giving independent evaluators similar access rights to employees is a good idea and that OpenAI would implement it.

Demis Hassabis of Google DeepMind agreed with the direction but indicated that detailed discussions were needed, while Elon Musk suggested a peer review system among competitors and joined the call to slow down AI development.

Martin Casado, a partner at venture capital firm a16z, expressed opposition to slowing down AI development. He raised concerns that such an evaluation system could actually strengthen cartels among existing companies.

In contrast, U.S. Senator Bernie Sanders took a firm stance, stating that merely slowing down is insufficient and that advanced AI development should be temporarily paused, and superintelligence development should be banned.

An analysis suggests that if the development of all frontier models were to halt for one year, the impact on practical work would not be significant. Current models are already considered sufficiently usable, almost like handling an employee.

The importance of tool integration and utilization methods is being highlighted over model performance improvement itself, and concerns are also raised about the stagnation of cost and token price reductions due to stalled model development.

This could offset the effect of software price reductions resulting from hardware performance improvements.

However, it could also reduce the burden of changing subscription models and encourage annual subscriptions, potentially providing users with a stable service environment.

If competition in model performance is limited, companies are expected to focus more on turning AI capabilities into actual products rather than competing on the technical performance of the models themselves. This could lead to service design targeting general users and end-to-end pipeline optimization.

The market for derivative services like Grokbot is likely to expand, and companies are expected to leverage their technology development capabilities to provide a wider range of user-friendly features.

An analysis suggests that an agreement to halt frontier model development will be difficult to achieve in reality. Although company executives explicitly agree, there is a lack of discussion on actual specific agreement methods or differences between internal research and public model deployment.

Issues of trust in regulatory bodies, the possibility of lobbying, and the AI leadership competition between the U.S. and China are cited as factors that could further complicate such an agreement.

This reflects the sense of crisis felt by U.S. companies and the nation due to China's rapid pursuit.

Ultimately, a dilemma exists between the two conflicting goals of ensuring AI development safety and competing for technological dominance among nations.

From a developer's perspective, current state-of-the-art (SOTA) and stable (fable) models are more than sufficient for practical work for over a year.

However, given the constant possibility of 'cheating' among companies and nations, the prevailing opinion is that it is unrealistic for all actors to simultaneously halt development. The advantages of unregulated high-speed growth should also be considered.