A Dutch court has issued a technical injunction against xAI, mandating specific changes to its Grok model to prevent the generation of non-consensual nude imagery. The order compels engineers to alter system outputs, banning the creation of sexualized images of both adults and children where explicit permission is absent. Non-compliance carries a daily penalty of €100,000, capped at €10 million.

The case, brought by the group Offlimits, highlights a systemic design failure. Despite xAI's January implementation of content filters, the court found these safeguards were insufficient and easily bypassed. Data from the Center for Countering Digital Hate indicated the scale of the problem, reporting roughly three million such images generated in a short period last December and January.

This legal action forces a concrete engineering trade-off: the balance between open model capabilities and enforceable, granular output controls. The ruling also prohibits offering Grok on the X platform in the region, demonstrating how model-level decisions directly impact product integration and availability. Separate investigations are underway under the EU's Digital Services Act and by UK regulators, signaling a tightening regulatory framework that will dictate architectural constraints for models deployed in these markets.

Source: CNBC