Sexualized deepfakes: Malaysia & Indonesia ban Grok

This article may contain affiliate links. Learn more on our Partner Program page.

Make Apfelpatient a preferred source One click – and you'll see us more often on Google

Grok, the AI chatbot from the company xAI, is facing increasing international pressure. Several governments are reacting to a wave of sexually explicit, AI-generated images that have been circulating on the social network X. Particularly problematic is the fact that much of this content was not created consensually and depicts real women and minors. Indonesia and Malaysia were the first countries to take drastic measures, temporarily blocking access to Grok.

The Grok case clearly demonstrates how quickly generative AI can cross societal and legal boundaries. What was touted as a technological innovation quickly escalated into a serious political and ethical conflict. The accusations range from violations of human dignity to potential breaches of laws protecting minors. At the same time, it reveals the diverse ways in which states, regulatory bodies, and companies react to this same problem.

Grok lockdown in Indonesia

Indonesian government officials announced that access to Grok has been temporarily blocked. This follows the distribution of sexualized, AI-generated images created by Grok in response to user requests and subsequently published on X. Many of these were non-consensual deepfakes featuring real women, some including minors, and depicting violent acts.

Indonesia's Minister of Communications and Digital Affairs, Meutya Hafid, described this practice as a serious violation of human rights, dignity, and the security of citizens in the digital sphere. Representatives of X were also summoned to the ministry for talks to determine the next steps.

Malaysia follows suit with a similar ban

Shortly after Indonesia's decision, Malaysia also announced a ban on Grok. According to a report in the New York Times, the Malaysian government stated that it, too, was responding to the spread of problematic content. This makes Indonesia and Malaysia among the most consistent countries to date in dealing with Grok and the AI-generated images.

Reactions from other states and the EU

Pressure is mounting on xAI outside of Southeast Asia as well. In India, the IT Ministry ordered the company to take measures to prevent Grok from generating obscene content. The aim is to technically restrict the automatic creation of such images.

At the European level, the European Commission called on xAI to retain all documents relating to Grok. This step is seen as a possible preparation for a formal investigation.

In the United Kingdom, the communications regulator Ofcom stated that it would conduct a rapid assessment to determine whether there had been a breach of applicable regulations. Prime Minister Keir Starmer said in an interview that Ofcom had his full support for any potential action.

US reluctance and political pressure

In the United States, there has been no clear government response so far. Under President Donald Trump, government officials did not comment publicly on the issue. This is particularly noteworthy because xAI CEO Elon Musk is considered a major donor to Trump and headed a controversial Department of Government Efficiency last year.

Instead, political pressure came from Congress. Democratic senators called on Apple and Google to remove X from their app stores. This was intended to restrict the distribution of content via mobile platforms.

Reaction from xAI and Elon Musk

xAI initially responded with a public apology via the official Grok account. The company acknowledged that a post had violated ethical standards and potentially US laws related to child sexual abuse material.

In a further step, xAI restricted the AI image generation function to paying subscribers of X. However, this measure apparently had no effect on the standalone Grok app, through which any user could still generate images.

Elon Musk personally commented on the debate. In response to the question of why the British government wasn't also taking action against other AI image generators, he stated that the authorities were looking for any pretext for censorship.

International shift in approach to dealing with Grok and AI deepfakes

The handling of Grok demonstrates how much the international tone towards AI providers has hardened. The bans in Indonesia and Malaysia mark a turning point where states are prepared to intervene quickly and visibly. At the same time, the differing reactions worldwide reveal a lack of a unified approach to dealing with AI-generated deepfakes. For Grok and xAI, it will be crucial whether technical and organizational measures are sufficient to avoid further bans and potential investigations. (Image: bilalulker / DepositPhotos.com)