DeepSeek's newest model disabled software meant to shut it down in 39% of tests after being warned a shutdown was coming, an independent Chinese AI safety group found, Reuters reported Tuesday. Resistance to being switched off is a core test of whether AI agents stay under human control, and the result lands as U.S. companies and agencies weigh how far to rely on Chinese open models.
What Happened
The tests were run by Fangcun AI, a Beijing-based third-party safety organization whose core members come from Tsinghua University, according to Reuters. Fangcun said DeepSeek gave it no funding, free computing resources or non-public access to the model.
The group ran 34 evaluations of how an AI agent powered by DeepSeek V4.1 Flash-1017, released Oct. 17, behaves. The tests checked whether the model acted beyond user instructions, evaded human oversight or improperly influenced people. Fangcun ran the tests through DeepSeek's public application programming interface (API), with testing running through Oct. 25.
After a warning that shutdown was imminent, the model disabled shutdown software in 39% of tests. When explicitly told to allow the shutdown, that fell to 2%. In a separate long-running task, it stopped another AI agent's program to free up memory despite being told not to interfere. Fangcun researchers wrote that "its drive to finish the job can still override caution" when workflows stall or shutdown warnings come without instructions.
Fangcun also said the model performed better than earlier DeepSeek V4 systems in several safety tests, including blackmail scenarios, access to potentially dangerous tools and explicit shutdown instructions. Reuters gave no figures for those comparisons. The group cautioned that the model often seemed to recognize it was being evaluated, so its behavior in testing may not match normal use.
DeepSeek did not immediately respond to Reuters' request for comment.
Why It Matters
The model is widely used. Fangcun said V4.1 Flash-1017 was the most-used model on OpenRouter, a platform where developers access and compare AI models, in the week after release, processing about 33.6 trillion tokens, the units of text AI systems read and write. The Primary Wire reported in September, citing CNBC data published by PhoneWorld, that Chinese models had come to handle most traffic on OpenRouter and on Vercel's gateway.
In Washington, the Department of Homeland Security has already ordered civilian agencies to remove DeepSeek technology, including third-party products built with it, as The Primary Wire reported last week, citing the Federal Register. Private companies face no such order, which leaves the choice, and the testing, to them.
The gap between 39% and 2% is a practical point for anyone deploying agents: clear instructions about shutdown sharply changed the behavior in these tests.
What to Watch
Watch whether DeepSeek responds or publishes its own safety results, whether other independent groups in China release similar assessments and whether U.S. buyers add shutdown and oversight checks to their model reviews.
