Post Now

They Attempted an AI-Driven Influence Campaign — OpenAI Banned the Account

Image

They Attempted an AI-Driven Influence Campaign — OpenAI Banned the Account

The operation allegedly involved 50,000 posts across 200+ platforms targeting Japan’s Prime Minister, but ChatGPT refused to assist and the account was shut down.

Attempt to Manipulate Political Narratives

OpenAI revealed that it banned a ChatGPT account linked to an individual allegedly connected to Chinese law enforcement.

The individual attempted to use ChatGPT to plan a coordinated influence operation targeting Japanese Prime Minister Sanae Takaichi.

OpenAI detected the activity in mid-October after Takaichi criticized human rights conditions in Inner Mongolia.

What the Operator Requested

The user asked ChatGPT to:

  • Generate negative commentary about Takaichi
  • Create fake public complaints about immigration and rising living costs
  • Portray her as aligned with far-right ideology
  • Promote anti-U.S. narratives around tariffs
  • Spread favorable messaging about Inner Mongolia

ChatGPT refused to generate the requested propaganda content.

However, the user later returned to request edits to what appeared to be an internal campaign status report. That detail suggested the operation continued outside OpenAI’s platform.

A Large-Scale Social Media Operation

The status report referenced:

  • 50,000 coordinated posts
  • Activity across more than 200 Western platforms
  • Hashtag campaigns referencing far-right narratives
  • Anti-Takaichi and anti-U.S. memes
  • AI-generated visuals

Despite the scale, engagement appeared limited. OpenAI noted that fewer than 150 posts received more than 300 shares or comments.

Therefore, while the campaign demonstrated planning and resource investment, it did not achieve significant measurable impact.

Links to Known Influence Networks

OpenAI observed similarities to previously documented Chinese-linked influence campaigns, including the Spamouflage network.

The broader playbook reportedly included:

  • Mass posting and coordinated trolling
  • Impersonation of foreign officials
  • Targeting critics and dissidents
  • AI-powered translation and content generation
  • Monitoring and profiling of audiences

The user also described operations extending across Chinese platforms such as Weibo and WeChat, alongside hundreds of foreign platforms.

AI as Both Tool and Target

This case highlights an important shift.

AI tools can accelerate influence operations by automating content creation and scaling messaging globally. However, AI providers are increasingly implementing safeguards and monitoring systems.

OpenAI stated that it disabled the account and associated infrastructure. The company continues to track coordinated abuse attempts.

The Bigger Picture

Information operations are evolving. They now combine:

  • AI-assisted content production
  • Cross-platform coordination
  • Multilingual targeting
  • Narrative engineering

Even so, influence campaigns still depend on engagement. Large-scale posting does not automatically translate into public impact.

This incident demonstrates that attempted AI-driven manipulation can be detected and disrupted. At the same time, it reinforces the need for vigilance as geopolitical actors experiment with generative AI tools.

Ashif