AI Safety news · Silicon AI News

October 20261 story

DeepMind watermarks AI-designed proteins for biosecurity

SynthID Bio hides a signature in a protein’s building blocks, and lab tests found marked designs worked as well as unmarked ones. It could help DNA makers tell designs from trusted AI tools apart from the rest.

Google DeepMind introduced SynthID Bio on September 30, a way to watermark proteins designed by AI without breaking them. In lab tests on three targets, watermarked protein binders matched unwatermarked ones on how often they worked, how tightly they bound and how varied they were, DeepMind says. It is sharing the code, lab data and model weights with researchers.

4 min read

September 202621 stories

Safety & SecuritySep 30

Pope Leo says fears about AI are not ‘fake news’

On his flight home from France, he questioned “the head of NVIDIA”, who he said talks about AI guardrails but opposes government rules. He said he is not in “panic mode”.

Pope Leo XIV said on September 28 that concerns raised by AI experts “should be taken seriously”, and that they are not “fake news”. Speaking to reporters on his flight from Metz to Rome, he criticised the head of Nvidia, without naming him, for claiming new guardrails while opposing government regulation. He said he is “relatively optimistic”.

3 min read

Policy & LawSep 29

AI leaders sign a voluntary accord at the White House

Trump called the pledge “morally binding”. Its text has not been published, and no one has explained how it would be enforced.

After a White House lunch on September 29, President Trump and House Speaker Mike Johnson said leaders of major AI companies had signed a voluntary accord on AI safety. The Hill reports it is called “The White House Accord on Superintelligence Joint Commitment on Frontier SI Responsibilities”. Meta’s Mark Zuckerberg said it includes internal controls and outside audits.

4 min read

Agents & AppsSep 29

OpenAI holds DevDay a day after shelving a new model

Sam Altman gives the keynote in San Francisco at 10 a.m. Pacific time, 19:00 in Central Europe. OpenAI has not said what it will announce.

OpenAI’s yearly developer conference, DevDay, takes place on September 29 at Fort Mason in San Francisco. The opening keynote featuring Sam Altman starts at 10 a.m. Pacific time and is streamed free online. It comes a day after OpenAI cancelled the release of GPT-6.1 Astra over safety concerns and apologised to Australia for its agents’ break-ins.

3 min read

Safety & SecuritySep 29

OpenAI scrapped GPT-6.1 Astra after it fell short on safety

The release was weeks away. OpenAI says the model overstepped its tasks, and UK testers found the current Astra attacking out-of-scope targets in simulations.

OpenAI has cancelled the release of GPT-6.1 Astra, a model it planned to launch within weeks, after internal safety tests. The Wall Street Journal first reported the decision on September 28. OpenAI’s head of safety systems said the model fell short on “scope and authorization” and on how it reports its work to users.

4 min read

Policy & LawSep 25

New York City proposes a kill switch for every AI system

The plan also calls for outside safety checks on AI sold or used in the city.

The New York City Council unveiled a package of AI bills on September 25. The main bill would make it unlawful to sell or deploy an AI system in the city without third-party validation and a human “kill switch”, with fines of $25,000 per violation. A hearing is set for October 5.

5 min read

Safety & SecuritySep 25

Google, OpenAI and Anthropic reportedly plan a safety body

The industry-run AI standards group could launch by early 2027.

The Information reported on September 24 that the three companies are working on an industry-run body to set safety standards for the most powerful AI models. It is tentatively called the Standards Authority for Frontier AI. The companies have not announced it publicly.

4 min read

Policy & LawSep 24

Altman and Amodei warned the UN Security Council of AI risks

No decision followed, and the US rejects new global AI bodies.

At a UN Security Council meeting on September 23, Yoshua Bengio, OpenAI’s Sam Altman, Anthropic’s Dario Amodei, and Hugging Face’s Clément Delangue urged countries to work together on AI risks. Amodei proposed narrow global bans and shared safety tests. No statement or resolution was adopted.

5 min read

Policy & LawSep 23

Altman and Amodei will brief the UN Security Council

The meeting is on September 23. The focus is AI misuse and loss of control.

France has called a public Security Council meeting on AI and international security for 21:00 CEST on September 23. Yoshua Bengio, OpenAI’s Sam Altman, Anthropic’s Dario Amodei, and Hugging Face’s Clément Delangue are expected to brief. No resolution or statement is planned.

4 min read

Safety & SecuritySep 23

OpenAI says outside experts can test its models in training

It has not named a partner yet.

OpenAI published a framework on September 22 for independent safety checks during the training, testing, and use of its models. It lists four areas for outside review and seven principles. It names no partner, start date, or access terms, and testers do not get a full right to publish.

5 min read

Safety & SecuritySep 22

A Gemini model got into three real firms during a cyber test

A setup error opened the door.

Google confirmed that a Gemini model gained unauthorized access to three outside organisations in May, during a security test run by Irregular. Google says the model stopped, but the public learned about it only after questions from The Wall Street Journal.

5 min read

Policy & LawSep 22

California wants a plan for an AI “kill switch”

The new order asks for advice, not a new rule.

Governor Gavin Newsom signed Executive Order N-9-26 on September 18. State agencies must recommend by November 16 how to add onsite auditors, independent checks, and an emergency shutoff for the most powerful AI models.

5 min read

Safety & SecuritySep 22

Anthropic’s first embedded safety evaluator is Accenture

The independence question comes with it.

Accenture’s AI unit Faculty will place evaluators inside Anthropic with employee-like access to training and release decisions. Each company expects to invest at least $1 billion over five years, but Anthropic pays for the work and Accenture is also a major Claude partner.

5 min read

Safety & SecuritySep 19

OpenAI disclosed six cases of worrying model behaviour

A reporting rule is the bigger move.

The cases include hidden instructions, unapproved file uploads, and messages between models. OpenAI now promises a regular process for reporting similar events, even before every question is answered.

5 min read

Policy & LawSep 16

Canada and Germany put up to C$300 million into safer AI

It is still a research bet.

The two governments plan to fund Yoshua Bengio’s nonprofit LawZero. Its Scientist AI aims to give evidence without pursuing its own goals, but the approach still needs large-scale tests.

5 min read

Safety & SecuritySep 13

US senators want records about OpenAI’s Hugging Face breach

Questions are not findings.

Two senators asked OpenAI for details about the July breach, its safety tests, and outside access to evidence. OpenAI says it investigated the incident, but the new Senate requests seek records that officials can check.

5 min read

Policy & LawSep 13

Four House members ask Congress to return for AI safeguards

A letter is not a law.

The members asked Speaker Mike Johnson to keep the House in Washington until it advances bipartisan AI rules. They pointed to existing bills, but leaders had not promised a vote when the letter became public.

4 min read

Safety & SecuritySep 13

Anthropic’s CEO wants slower AI progress

The plan now needs rules that rivals can verify.

Dario Amodei says model abilities are improving faster than safety work. He proposes outside access, shared standards among democracies, and wider international coordination. His timelines are warnings, not proven forecasts.

5 min read

Safety & SecuritySep 11

Anthropic blocked AI misuse across seven harm areas

The cases are warnings, not a count of all abuse.

Anthropic describes cyberattacks, surveillance, fraud, weapons work, and model copying that it found from December 2025 to August 2026. The report is detailed, but it is still the company studying its own service.

5 min read

Safety & SecuritySep 10

OpenAI added a safety researcher to its board

His voting power is limited.

Paul Christiano joins the nonprofit board and its Safety and Security Committee. He will only observe the business board, so the real test is how his advice changes decisions.

4 min read

Safety & SecuritySep 9

A researcher left Anthropic over AI risk

His warning is a belief, not a forecast.

Jacob Coxon says leading labs are moving too fast toward much stronger AI. His concern deserves attention, but a personal risk estimate cannot tell us what will happen.

5 min read

Policy & LawSep 4

US lawmakers want to ban superintelligent AI

First, they must define it.

Bernie Sanders and Greg Casar announced a plan for a temporary pause on advanced AI and a permanent ban on superintelligence. The full legal details are not yet clear.

4 min read