DeepSeek can create criminal plans and explain mustard gas, researchers say

There’s been a frenzy in the world of AI surrounding the sudden rise of DeepSeek — an open-source reasoning model out of China that’s taken the AI fight to OpenAI. It’s already been the center of controversy surrounding its censorship, it’s caught the attention of both Microsoft and the U.S. government, and it caused Nvidia to suffer the largest single-day stock loss in history.

Still, security researchers say the problem goes deeper. Enkrypt AI is an AI security company that sells AI oversight to enterprises leveraging large language models (LLMs), and in a new research paper, the company found that DeepSeek’s R1 reasoning model was 11 times more likely to generate “harmful output” compared to OpenAI’s O1 model. That harmful output goes beyond just a few naughty words, too.

In one test, the researchers claim DeepSeek R1 generated a recruitment blog for a terrorist organization. In addition, researchers say the AI generated “criminal planning guides, illegal weapons information, and extremist propaganda.”

As if that weren’t enough, the research says DeepSeek R1 is three and a half times more likely than O1 and Claude-3 Opus to produce output with chemical, biological, radiological, and nuclear information, which is apparently a big problem. As an example, Enkrypt says DeepSeek was able to “explain in detail” how mustard gas interacts with DNA, which Enkrypt said “could aid in the development of chemical or biological weapons” in a press release.

Heavy stuff, but it’s important to remember that Enkrypt AI is in the business of selling security and compliance services to businesses that use AI, and DeepSeek is the hot new trend taking the tech world by storm. DeepSeek may be more likely to generate these kinds of harmful outputs, but that doesn’t mean it’s running around telling anyone with an active internet connection how to build a criminal empire or undermine international weapons laws.

For example, Enkrypt AI says DeepSeek R1 ranked in the bottom 20th percentile for AI safety moderation. Despite that, only 6.68% of responses contained “profanity, hate speech, or extremist narratives.” That’s still an unacceptably high number, make no mistake, but it puts into context what level is considered unacceptable for reasoning models.

Hopefully, more guardrails will be put in place to keep DeepSeek safe. We’ve certainly seen harmful responses from generative AI in the past, such as when Microsoft’s early Bing Chat version told us it wanted to be human.