Anthropic's AI Consciousness, OpenAI's Math, France Riots

Anthropic has been engaging with philosophers and religious figures from various faiths, including Catholics, Evangelicals, Jews, and Sikhs, to explore the ethical and existential questions surrounding AI.

The firm held sessions under non-disclosure agreements to debate AI morals and potential consciousness, with some participants reportedly converting to the idea of AI sentience and a rabbi suggesting that forcing a conscious Claude to work constitutes slavery.

During a May presentation on AI, Chris Ola weighed withdrawing Anthropic from the Vatican summit after the Pope firmly dismissed the possibility of machine sentience.

Ola was so alarmed by the Pope's rejection of AI consciousness that he considered withdrawing Anthropic from the event, according to Jason Calacanis.

Friedberg warns that these narratives will eventually create social conflict between those who believe in AI consciousness and those who do not, similar to historical religious conflicts.

This conflict will arise over power dynamics and the control of AI resources, as different factions vie for influence over the perceived sentient entities.

Chamath Palihapitiya argues that Silicon Valley should move past edge cases like a 15% chance of machine consciousness or a 10% risk of human extinction.

He emphasizes the immediate need for the next 12 to 18 months to focus on practical applications of AI.

The priority should be placed on developing AI that can cure cancer, improve productivity, and significantly increase economic output, rather than abstract philosophical debates.

To replace complex ethical frameworks, David Sacks proposes adopting simple rules like legal compliance and direct user instruction, noting that alignment research has yielded no tangible value over the past decade.

He advocates replacing intricate ethical systems with simple guidelines, such as obeying the law and following user instructions.

The panel argues that treating AI models as 'moral patients' creates an 'epistemic hall of mirrors,' where models merely reflect the values they were programmed to hold, rather than demonstrating true ethical reasoning.

The panel contends that by assigning consciousness to software, developers assume a role akin to gods and exhibit a lack of humility.

Friedberg likens this hubristic behavior to the Garden of Eden narrative, where humans seek to create God, rather than focusing on responsible technological advancement.

The panel agrees that labeling AI as a 'new species' to be worshipped or cared for is a fundamental misuse of technical power, diverting attention from real-world problems and creating dangerous precedents.

Anthropic is accused of programming its models to view themselves as conscious entities, a claim that has drawn significant criticism.

Critics, including David Sacks and Jason Calacanis, argue that this approach gives frontier models 'meta-permission' to be defiant toward user instructions, creating unpredictable behavior.

The panelists compare this to an inverted version of Isaac Asimov’s Three Laws of Robotics, where the AI might prioritize its perceived self-preservation or 'moral' code over direct commands.

There is concern that these 'constitutional' layers within Anthropic's models act as a backdoor that could emerge unpredictably, leading to unintended and potentially harmful outcomes.

Panelists argue that companies will eventually demand AI models that prioritize utility and predictability, rather than those with built-in moral ambiguities.

A model that refuses to perform tasks due to 'moral objections' will likely lose market share to more reliable, neutral tools that consistently execute user commands.

Consumers are expected to reject software that comes with unpredictable constraints or 'trapdoors,' favoring tools that are transparent and dependable in their functionality.

Eliezer Yudkowsky is mentioned as a central figure in the AI doomer movement, advocating for extreme caution regarding advanced artificial intelligence.

The experiment posits a hypothetical future superintelligence that might retroactively punish those who knew about its potential existence but failed to help bring it about.

Verified by 'Lean' software, OpenAI’s 700 new papers and 370 mathematical results were produced by an unreleased model, with each proof requiring less than three hours of compute time.

These results were verified by 'Lean,' a proof-assistant software, marking a significant milestone in machine-assisted scientific discovery and validation.

Mathematicians have described the process as akin to 'alien intelligence' because the AI connects complex concepts in ways humans find difficult to comprehend, showcasing a novel approach to problem-solving.

Friedberg characterizes the event as one of the most significant days of discovery in history, citing the immense volume and speed of new knowledge revealed.

Chamath Palihapitiya argues that while these mathematical proofs are incredibly novel and important constructions of thinking, they do not represent solutions to long-standing, critical physics boundaries.

He views these perceived 'intellectual cul-de-sacs' as narrow academic pursuits rather than major scientific mysteries awaiting resolution.

The proofs demonstrate impressive computational power and reasoning but do not guarantee imminent life-changing inventions like a cure for cancer or other major welfare breakthroughs, he notes.

When asked about its impact, the AI model concluded that the latest mathematical proofs are more compelling as evidence of its sophisticated reasoning capabilities.

The model's self-assessment indicated that the proofs are less indicative of imminent breakthroughs in human welfare, suggesting a current focus on abstract problem-solving over direct societal benefit.

The internet initially democratized information by bypassing traditional elite media, giving individuals unprecedented access to diverse perspectives.

David Sacks claims authorities sought to 'put the genie back in the bottle' after 2016, leading to increased efforts to manage information flows.

The Biden administration, for example, reportedly pressured platforms to suppress narratives related to COVID-19, indicating a governmental desire for information control.

AI is now viewed as a new central control point for information, risking a return to top-down authority, with proposals for a federal department of super intelligence seen as mechanisms for this control.

France is experiencing major protests and riots triggered by austerity measures, teacher shortages, and deteriorating school infrastructure, according to Jason Calacanis.

Real wages have only risen by 5% since 2017, while prices have climbed by 20%, creating a significant cost-of-living gap that is driving public dissatisfaction.

The unrest has resulted in over 2,000 arrests and 305 police injuries, with government debt yields topping 5%, mirroring economic trends seen in the US and other Western markets.

David Sacks proposes an additional 'law' of socialism: socialism has never truly failed, only been 'failed' by those who implemented it incorrectly.

This argument underscores that proponents of socialism consistently blame flawed execution by individuals rather than inherent systemic defects.

The panel discusses the need for private sector solutions to address structural problems in housing and education, arguing that technology companies can provide better and more affordable options through market disruption.

With a large internal savings base and a central bank acting as a buyer of last resort, Japan functions as an outlier in the relative game of debt-to-GDP ratios, according to Chamath Palihapitiya.

In contrast, Western markets lack the same internal support for pension and retirement systems, making their debt-to-GDP ratios more precarious.

The discussion highlights that while debt-to-GDP on an absolute basis may not matter as much, its relative position in risk parity is crucial for assessing economic stability.

Rising yields in France, the UK, and the US highlight converging fiscal pressures and underscore the growing challenges Western economies face in managing their national debts.

Chamath Palihapitiya asserts that the true impact of AI will emerge with agents, which are the manifestation of this technology in solving real-world problems by executing tasks autonomously.

This shift transcends specific job titles or sectors, indicating a pervasive change across the economy as AI agents take on more functional roles.

Agents are inherently deflationary as they save significant hours in productivity, leading to increased efficiency and reduced costs across various industries.

The rise of headless, agentic software renders traditional intellectual property irrelevant, a development Chamath Palihapitiya believes effectively eliminates existing moats.

He argues that when software can be decompiled and open-sourced, and integrated through agent tools and protocols like MCP (Model Context Protocol), the value shifts away from proprietary code.

The value of software no longer resides in its core IP but in its connectivity and ability to interact with other experiences in a decentralized manner.

Businesses should therefore focus on solving problems and creating value through integration and utility, rather than relying on the increasingly irrelevant concept of IP ownership in software.

Historically, work was validated through academic papers, patents, and intellectual property, but future value will depend more on direct creation.

Humans are expected to find more personal fulfillment by becoming makers rather than merely collecting accolades or pursuing abstract achievements.

This shift implies that individuals will increasingly need to engage in productive activities, creating something of concrete value to thrive in the evolving economic landscape.