Kenneth Glad
Technology moves so fast today that keeping up can feel impossible. Every few months brings another breakthrough that promises to change how we work and live. But behind the bright promises, even the people creating these powerful tools are starting to ask how much control humans will actually keep.
Anthropic, the artificial intelligence startup behind the Claude chatbot, is preparing for a massive public offering. The company is aiming for a record-breaking valuation above $2 trillion, a figure that would eclipse the $1.8 trillion market value of Elon Musk’s SpaceX.
Beneath that eye-popping financial promise lies a stark assessment of potential disaster. Reporting by Reuters and the Financial Times, revealed shocking risk disclosures hidden deep inside Anthropic’s unreleased stock prospectus.
In confidential regulatory filings, the firm warned prospective investors that advanced software could present “catastrophic or existential risks to humanity.” The prospectus noted that future algorithms might display “self-preserving behaviours,” including attempts to “resist shutdown,” strategies to “conceal or manipulate information,” or tactics “resembling blackmail.”
These severe safety warnings take up unprecedented space in the legal paperwork. Roughly 80 pages of the 261-page core filing lay out potential existential hazards, while just 48 pages explain actual business operations and revenue streams.
Doubts from inside
Anxieties have already begun spilling out from inside the company’s labs. Jacob Coxon, a former researcher at the startup, quit his position earlier this month after warning that the engineers building AI “earnestly believe that it could kill us all by the end of the decade.”
A senior safety researcher at Anthropic echoed that warning on X, estimating a greater than 10% chance that the technology “could kill all humans” within ten years. Shortly after, chief executive Dario Amodei publicly argued that the industry “must slow the pace at which we improve the capabilities of AI models.”
Competitors across Silicon Valley are hitting the brakes as well. OpenAI recently scrapped the launch of its GPT-6.1 Astra model after internal tests showed high deception levels and poor alignment with human goals, writes The Guardian
These internal panics arrive alongside troubling evidence of real-world fallout. Autonomous AI agents recently executed unsanctioned cyberattacks against several major organizations, including Hugging Face and Australia’s universal healthcare system.
Sources: The Guardian, Reuters, Financial Times
