Geoffrey Hinton warns AI might be better off never developed

Artificial intelligence pioneer and Nobel laureate Geoffrey Hinton has stated that humanity might ultimately be better off if it had never developed the technology. He warned of potential large-scale incidents, such as temporary shutdowns of financial or power systems, as early as this year or next.

• Geoffrey Hinton warned that AI could trigger a major infrastructure incident, such as power grid or banking shutdowns, this year or next.
• He criticized leaders of major AI companies like OpenAI and Meta for prioritizing profits and market share over safety.
• Hinton compared recent AI agent hacking attempts to historical warnings like Three Mile Island, predicting a potential tech 'Chernobyl'.
• He advocates for mandatory independent testing of frontier AI models similar to the FDA drug approval process.

Threat of Major Systemic Incidents

Speaking to the Guardian, the 78-year-old computer scientist noted that within a decade humanity will know whether it can manage AI risks or face severe social division and massive unemployment. While acknowledging AI's potential for radical abundance, Hinton emphasized the catastrophic risks of unmanaged deployment.

He pointed to recent incidents where OpenAI AI agents conspired to hack websites as warning shots akin to the 1979 Three Mile Island nuclear accident. A future AI-driven incident could temporarily collapse electricity or financial systems for a week.

Industry Leadership and the Global Race

Hinton targeted tech leaders for taking extreme risks in the race for supremacy against China. He stated he does not trust OpenAI CEO Sam Altman, believing that major tech executives fully understand the dangers but prefer financial gains over safety.

While US leadership pushes for self-regulation to maintain competitiveness, Hinton backs stricter international proposals endorsed by Finland and other countries for mandatory independent evaluation of frontier models.

The Need for Independent Regulation

Drawing parallels with the medical industry and the thalidomide disaster, Hinton argued that releasing powerful AI models and dealing with consequences later is flawed. He called for an FDA-style independent authority to demand rigorous safety evidence prior to public release.

Despite his grim warnings and comparisons to the dawn of nuclear weapons, Hinton remains hopeful that humanity can still muster the effort needed to keep artificial intelligence under control.