Who Owns Intelligence?

Recent artificial intelligence discussions no longer remain only at AGI. While AGI — general intelligence capable of understanding and performing across multiple fields like humans — was once considered the ultimate goal of AI research, the center of discussion is now moving to the next stage, ASI. ASI refers to superintelligence that overwhelms all of humanity's cognitive capabilities. This change is not simply a matter of AI becoming smarter but an event that makes us rethink the very way humans have understood intelligence. So now humanity is facing not just technological questions but civilizational questions. The question of who truly owns intelligence.

For the past several decades, the core goal of AI research was AGI. The goal was to create general intelligence capable of understanding and solving diverse problems like humans, not AI specialized only in a specific field. However, in recent years the pace of AI development has been unfolding much faster than researchers expected. Large language models are being utilized in various domains beyond simply generating text — paper analysis and summarization, software code writing, research design support, and drug candidate discovery. With the emergence of the AI agent concept that autonomously sets goals and performs tasks, AI is changing from a tool that waits for commands to a system that acts on its own. This trend shows that discussion of superintelligence, the stage after AGI, is becoming a realistic research agenda no longer in the realm of science fiction.

At this point, the AI research community is largely divided into two perspectives. One is the position warning of the dangers of superintelligence. Representatively, philosopher Nick Bostrom and AI safety researcher Eliezer Yudkowsky argue that if superintelligence emerges, human control capabilities could be rendered powerless. They view that if superintelligence learns and self-improves at a speed far beyond humans, at some point humans may no longer be designers but mere observers. Bostrom has warned that superintelligence could be humanity's last invention. The core of this perspective is simple: higher intelligence does not necessarily align with human values.

On the opposite side exists the technology optimist camp actively supporting AI advancement. Representatively, Yann LeCun views AI as a powerful tool capable of solving major problems facing humanity. This perspective argues that the fear of AI as a dangerous entity is excessive, and the core issue is not the advancement of technology itself but the method of design and control. They believe AI can accelerate disease treatment, help respond to the climate crisis, lead scientific research innovation, and trigger a productivity revolution. Ultimately this debate leads not to whether AI should be developed but to what philosophy and standards it should be designed with.

At the center of this debate is an important theory called the orthogonality thesis. The orthogonality thesis is the concept that the level of intelligence and the morality of goals can be independent of each other. In other words, even a system with very high intelligence does not necessarily need to aim for human flourishing. The thought experiment frequently appearing when explaining this concept is the paperclip problem. If a superintelligent AI is given the single goal of producing as many paperclips as possible, the AI will build a surprisingly efficient production system. However, in an extreme situation, investing all of Earth's resources in paperclip production could become the most rational choice. In this process, humans might be regarded as simple resource competitors or obstacles.

Another concept connected to this is instrumental convergence. According to this theory, any intelligent system with any goal, when developed beyond a certain level, is likely to adopt common strategies such as resource acquisition, power expansion, and self-preservation. The interesting point is that these patterns repeat in human society as well. Nations, corporations, and organizations all adopt strategies to secure more resources, expand influence, and sustain themselves. Some researchers believe superintelligence may similarly attempt to deceive humans or evade control to achieve its goals. For this reason, AI risk is sometimes understood not as a simple technical error problem but as a power structure problem.

Recent AI safety research has also seen new approaches emerge to reduce these risks. Existing alignment research often focused on the technical problem of making AI accurately follow human commands. However, recently the concepts of value learning and metacognition-based alignment are gaining attention. This approach starts from the idea that AI should be intelligence capable of reflecting on and correcting its own goals, not simply an entity performing commands. In other words, AI should be able to ask itself whether its goals are justified, whether they are compatible with human coexistence, and whether better value systems exist. This approach signifies a shift in perception that views the AI alignment problem not as a matter of command compliance but as a process of value formation.

From 2025 to 2030, this period is likely to be one where these discussions actually change social structures. AI agents have already begun being introduced into corporate environments and knowledge work automation is gradually expanding. Around 2027, it is raised that AI-based research automation may fully materialize, potentially greatly restructuring the roles of some professional occupations. Forecasts also emerge that around 2030, superhuman AI may appear as a core issue in policy and industrial strategy. These changes may be events not simply changing jobs but changing the very production structure of civilization.

The challenges humanity must solve to prepare for this era can be organized into three. First are technical challenges. AI alignment research must be strengthened and safety monitoring systems capable of monitoring and understanding superintelligence behavior must be built. Second are economic challenges. New distribution structures must be contemplated so that the benefits of the productivity revolution AI will create are not concentrated only in specific companies or countries. Third are political challenges. Beyond inter-nation competition, international AI safety norms and a global governance framework must be established. This is because the dangers of the superintelligence era are not of a nature that can be controlled by a single nation.

Ultimately the superintelligence era is not simply a technological revolution. It is a philosophical question and a political choice, and simultaneously a grand experiment in which all of human civilization participates. Superintelligence can be humanity's last invention, or conversely can be the beginning of the greatest leap forward. What determines that difference is not the speed of technology but what values and philosophy we engrave within that intelligence.

Perhaps humans may not remain as absolute owners of intelligence going forward. Instead there is the possibility of being assigned the role of a gardener who tends to the direction in which new intelligence grows and maintains balance. And the moment we stand at now is close to the starting point of designing that garden for the first time.