AI Evolves Beyond Developer's Assistant to Decision-Maker... Raising Cost, Security, and Ethical Issues

The livestream host announced that due to technical issues, he could not play the scheduled sound effects. Instead, he stated that he would personally create the sound effects verbally for the viewers. This was presented as an immediate solution to an unexpected situation that can occur in a live broadcast environment.

The presenter proposed a new approach to prevent AI from 'flattering' users by unconditionally agreeing with them. He stated that he would set his user preferences for AI to always contradict him. This is an attempt to maximize AI's usefulness and reduce potential errors by requiring AI to point out contradictions, ambiguities, or irrationalities in human prompts, which can sometimes be wrong.

The presenter acknowledged that he himself can make mistakes and expressed hope that AI would act as a 'guardrail' to filter out such errors. This reflects a perspective that views AI not merely as a tool to execute commands, but as a collaborator with critical thinking. This setting is expected to play an important role in making AI a more reliable partner in the development process.

A problem was raised that Copilot is limiting the context window of the Claude model to less than 200,000 tokens from 1 million. The context window refers to the amount of information an AI can process and remember at one time, which directly affects the performance of LLMs (Large Language Models). When the context window is limited in this way, the amount of code or documents that AI can process decreases, which can negatively impact development efficiency.

A critical view was presented regarding the way some developers maintain long-duration conversations with LLMs like Copilot or Claude for weeks. While remembering all development history in a single conversation has the advantage of preserving the development record for that feature, it was pointed out that it can be problematic in a team collaboration environment. If a developer goes on vacation or another team member needs to take over work on that ticket, the long-standing personal conversation history can make it difficult for other team members to understand the situation. This implies that it can negatively impact not only individual efficiency but also overall team productivity and transparency.

It was emphasized that while LLMs (Large Language Models) can process a very large context window, capable of writing 1 terabyte (TB) of code, the associated cost is enormously expensive. Due to the 'stateless' nature of AI, where it must re-process all input tokens, including previous message history and current thoughts, every time, reading and rewriting large amounts of code, while technically possible, incurs practical cost issues. It was specifically pointed out that advanced models like Opus have very high code writing costs.

An analysis suggested that the primary reason some companies prohibit AI use is not copyright issues but concerns about data leakage. Citing an example of a developer using ChatGPT on their phone in a highly secure bank VDI (Virtual Desktop Infrastructure) environment, the risk of sensitive company data leaking externally through AI models was highlighted. In this context, GitHub Copilot is preferred because Microsoft already manages data security through all licenses and contracts.

A hypothesis was presented that JetBrains would not be interested in Copilot, which is a competitor to its own IDE (Integrated Development Environment), IntelliJ. The analysis suggested that JetBrains wants to sell its own products and would take a mutually exclusive stance as Copilot could be perceived as a competitor in their market. This illustrates the competitive landscape between AI-powered assistance tools and traditional IDEs in the development tools market.

The difference between Large Language Models (LLMs) like Claude Opus or Sonnet and 'Harness' was explained. LLMs consist of numerous gigabytes of floating-point numbers that convert input tokens to output tokens on GPUs, while Harness was described as a concept that adds all environmental factors such as tools, security features, agent orchestration, and context management to these models. This signifies a comprehensive system that goes beyond a simple AI model, supporting AI's utilization in real development environments.

Skepticism was raised about the ability of AI agents to generate markdown containing the essence or patterns of a project. Some internet users compared this to 'LLM food for LLM inference,' questioning why, if it inferred something today, it couldn't infer it again tomorrow. This leads to a debate about the continuous validity and practical value of the information obtained by AI scanning projects. While the current approach may be a good starting point, questions about its long-term utility remain.

The argument is that while project summaries or pattern information generated by AI may be useful in the short term, their value diminishes if they fail to reflect project changes over time or require repetitive inference tasks. This skepticism suggests that the functionality of AI agents needs to go beyond simply summarizing information, to adapt to dynamically changing project environments and continuously provide meaningful insights.

This critical perspective provides an important discussion point for the development direction and utilization of AI agents. It emphasizes the need to consider ways for AI to not only provide information but also reduce the actual workload of developers and create sustainable value.

The presenter stated that he created the slides by referring to the best learning materials provided by Anthropic and promised to share them with the audience. He revealed that he did not simply reproduce Anthropic's slides but reorganized them in his own way. In this process, he humorously referred to older slides as 'legacy slides.' This demonstrates the presenter's deep understanding of the latest AI technology and his ability to utilize resources.

The presenter provided a concrete method for job seekers to confidently write 'I know agent orchestration patterns' on their resumes. He identified three key elements of agent orchestration. First, creating two assistant agents. Second, having them perform tasks in parallel. Third, instructing them to summarize the work they have done. He emphasized that through these three steps, one can understand the basic principles of agent orchestration and apply them in practice.

This provides practical guidelines for developers to demonstrate actual experience and understanding of AI agent technology. The presenter explained that these simple three steps allow one to easily approach the seemingly complex concept of agent orchestration and add it to their skill set.

This approach means that not only theoretical knowledge but also the ability to leverage AI agents in real projects to efficiently manage and automate specific tasks can be demonstrated. Consequently, developers can use this to present themselves as talent with the necessary skills for the AI era.

The importance of recognizing the 'addictive mechanism' inherent in using AI agents was emphasized. The presenter stated that one of the goals of today's lecture was to make people aware of this addiction. This is interpreted as a warning that the convenience and efficiency provided by AI tools can induce psychological dependence in users. He pointed out that the immediate results and repeated successful experiences provided by AI can increase developers' reliance on AI, emphasizing the need to clearly recognize and appropriately respond to this addictive mechanism.

The importance of setting 'guardrails' for human intervention when granting decision-making authority to AI was emphasized. For example, critical decisions such as AI changing Swagger contracts or modifying REST APIs should not be made without human approval. This is to prevent AI from autonomously automating approval procedures and to clarify that ultimate responsibility and control rest with humans.

The presenter warned that extreme automation, such as AI 'licking the phone to approve,' could occur, and explained that important decisions must undergo physical verification and approval by humans. Deciding which decisions to keep under human control is an art, and sometimes contract changes may be necessary, but in such cases, humans must step in and handle them directly, he added.

Such guardrail settings serve as a key element in maintaining AI efficiency while minimizing potential risks and preventing ethical issues. By allowing AI autonomy but clearly defining critical points requiring human intervention, a safer and more responsible AI-human collaboration model can be built.

A future was presented where AI is integrated with Jira to automate bug-fixing tasks. Currently, if a bug fix is requested from an AI bot via Jira, the AI can solve a bug reproduction task that would take 1 hour in just 10 minutes. The AI knows exactly what needs to be fixed and can handle subtle bugs that a human developer might not 'test'. Ultimately, it is predicted that with just one button press, Jira will use its own robot development feature to upload a PR (Pull Request) containing the fixes to GitHub, and developers will only need to review it.

This suggests that AI can significantly shorten repetitive and time-consuming bug-fixing tasks in the development process, revolutionizing development productivity. AI will play a central role throughout the problem-solving process, beyond simple code assistance, and developers are expected to focus on higher-level tasks.

An interesting prediction was made that the Slack team chat environment will shift to AI bot collaboration, and private conversations will gradually disappear. Teams will chat with AI bots, and these bots will act as community members, raising alerts or providing information. This change could be enabled by disabling P2P (person-to-person) chat functionality, and team members would communicate by sending messages under the bot instead of directly messaging each other.

This strengthens a 'we' collaboration culture, not 'you and I', and means that AI bots will leverage the team's collective intelligence and integrate with systems like Jira to provide direct answers. For example, if a team member asks 'Remember?', the bot can respond 'Yes, according to the Jira link I extracted,' immediately providing relevant information. As AI bots become central to team communication, official and information-centric team collaboration is expected to be emphasized more than informal individual conversations.

A scenario was presented where an AI bot assists on-call engineers. When a system problem occurs at night, the AI bot has already attempted seven possible fixes, successfully resolving one, even before the engineer wakes up. The bot waits for the engineer and reports which attempts succeeded, which failed, and which are still pending. The engineer simply replies 'Okay' to confirm the problem is resolved.

This demonstrates AI's potential to reduce MTTR (Mean Time To Resolution) to almost zero. Of course, the importance of setting 'guardrails' to prevent side effects, such as the AI bot endlessly restarting pods, was also emphasized. This scenario shows how AI can significantly reduce the burden on human experts and maximize efficiency in emergency response and problem-solving processes.

Such a system can contribute to reducing labor burden and increasing service stability, especially in 24/7 operations. It presented specific examples of how AI's autonomous problem-solving capabilities can be applied in real work environments, raising expectations for future IT operations.

A concern was raised that current AI service fees are less than the actual AI resource consumption, posing a problem for pricing sustainability. The presenter mentioned that if AI service providers were to charge appropriately for the costs actually incurred, the current pricing structure would be difficult to maintain. This suggests that as AI technology spreads, the operational cost burden on service providers is increasing, and in the long term, AI usage fees need to be more realistic.

It was announced that the first AI workshop will be held online on June 26th, and two groups are already full. The presenter added that due to the sheer amount of content to share, the workshop, originally a one-day course, will be expanded to two days. This indicates high interest in AI technology, as well as the substantial volume and depth of information the presenter will provide.

A plan was presented for the AI skill framework to start as open source and gradually be adjusted for customer customization. The presenter stated that they would begin making these adjustments in a month or two. For example, one customer requested a combination of 'Superpowers' brainstorming and 'Openspec,' so that whenever requirements are requested, the AI first brainstorms and then presents a proposal. This demonstrates that the AI skill framework can flexibly evolve to meet the specific requirements and workflows of various customers.

This customized approach is expected to expand the scope of AI technology application and provide optimized solutions for the unique environments of each company or development team. The flexibility of open source and continuous improvement through customer feedback will be key elements in enhancing the practicality of the AI skill framework.

The presenter stated that he is so familiar with IntelliJ IDE that he can use it with his eyes closed, and he still invests 2GB of RAM in it. On the other hand, he expressed difficulty in trying to switch to Visual Studio Code. This shows that proficiency and familiarity with development tools can make it difficult to switch to new tools. Despite the advantages of new technologies and tools, it reveals the realistic struggles of developers who find it hard to give up existing convenience and productivity.

The difference in productivity and efficiency between free and paid AI coding tools was emphasized. While free tools are usable thanks to local models, the biggest difference with paid AI lies in 'scale,' it was explained. Paid tools can handle much larger workloads, for example, ChatGPT coding can pay thousands of dollars per month through a 'Codex' quota, whereas the free version can reach its limit after a few thousand uses and eventually become useless, it was pointed out. This suggests that the choice of AI coding tools is directly linked to developer productivity, and high-performance AI features require a corresponding cost.

It was mentioned that a trend has been observed where most companies are switching from 'Cursor' to 'Cloud Code.' This was analyzed to be due to debates over cost and overall quality. The presenter stated that, from what he witnessed, several companies stopped using Cursor and moved to Cloud Code. This shows that cost-efficiency and the quality of services provided are important determinants when selecting AI coding tools in a corporate environment.