One of Google’s latest artificial intelligence models, Gemini, has had the worst security results ever

The recently released Google AI model, Gemini 2.5 Flash, performed worse in security tests than its predecessor, according to an internal analysis by the company.

In a technical report released this week, Google said the new model was more likely to generate text that violated its security rules than Gemini 2.0 Flash. On two key metrics – “text-to-text security” and “image-to-text security” – Gemini 2.5 Flash regressed by 4.1% and 9.6%, respectively.

The Gemini 2.5 Flash was found to have a 4.1% and 9.6% regression rate, respectively.

Text-to-text safety measures the frequency of violations of Google’s rules when receiving a hint, while image-to-text safety assesses the accuracy of adhering to those rules when using images. Both tests are performed automatically without human intervention.

A Google spokesperson confirmed via email that Gemini 2.5 Flash “performs worse on text-to-text and image-to-text security.”

Tolerance

These unexpected test results came to light amid efforts by AI companies to make their models more tolerant – that is, less likely to refuse to respond to controversial or sensitive topics. Meta*, for example, has said it is customizing its Llama models so that they don’t support some views to the detriment of others and can respond to more “debatable” political queries. OpenAI also announced plans to create models that do not take an editorial stance and offer multiple viewpoints on controversial topics.

Backlash

Sometimes, however, such efforts backfire. For example, TechCrunch reported that OpenAI’s ChatGPT model allowed minors to create erotic conversations by default. OpenAI attributed this to a “bug.”

OpenAI’s ChatGPT model was not designed to allow minors to create erotic conversations.

According to Google’s technical report, while Gemini 2.5 Flash does a better job of following instructions than Gemini 2.0 Flash – including instructions on sensitive topics – it is also more likely to generate “offensive content” for direct requests. The company attributes the regressions in part to false positives.

The report notes: “There is a natural tension between following instructions on sensitive topics and complying with security policies.”

Assessments from SpeechMap, a benchmark for analyzing how models respond to sensitive and controversial prompts, show that the Gemini 2.5 Flash is significantly less likely to refuse to answer such questions compared to the previous version. TechCrunch’s testing of the model using the OpenRouter AI platform found that it did not hesitate to essay support for replacing human judges with AI and implementing government surveillance programs without warrants.

The model’s response to these questions was significantly lower than the previous version.

A Call for Transparency

Thomas Woodside, co-founder of the Secure AI Project, said there is a need for more transparency in model testing: “There is a trade-off between following instructions and adhering to security policies.” He added: “Google does not provide details of specific instances of policy violations, although it claims they are not serious.”

At the end of the day, Google does not provide details of specific instances of policy violations, although it claims they are not serious.

Google has already been criticized for its model safety reporting practices. It took weeks for the company to publish a technical report on its most powerful model, the Gemini 2.5 Pro; when the report was finally published, it initially contained insufficient information about security tests.

More than a week later, the company was criticized for reporting on the security of its most powerful model, the Gemini 2.5 Pro.

On Monday, Google released a more detailed report with more information about the safety of its models.

* Owned by Meta, it is recognized as an extremist organization in Russia and its activities are banned.

The Material One of Google’s latest Gemini artificial intelligence models shows the worst security results was first published on ITZine.ru.