There’s no rogue McDonald’s AI bot, but ‘prompt injection’ is still a risk for companies

FastCompany: There appears to be a recent epidemic of users hijacking companies’ AI -powered customer service bots to turn them into generic AI assistants. The goal is to get the branded bots to do their bidding, without having to subscribe to an AI service. Sometimes, people force the bots to do things that they are not supposed to do, like giving extraordinary product deals and even helping them to take legally problematic actions.

Most recently, a wave of LinkedIn posts and social media videos went viral for claiming that users had tricked McDonald’s customer service virtual assistant to abandon its burger-centric purpose to instead debug complex Python programming code. One post read: “Stop paying $20 a month for Claude. McDonald’s AI is FREE.” On Instagram, videos and images popped up claiming the same thing, all posting the same image as proof.

The claim went viral, as Grok summarized in a trending news post on X: “McDonald’s AI customer support agent named Grimace gained massive attention with 1.6 million views and 30,000 likes after users tested it with out-of-script requests like debugging, Python scripts, and architecture questions.” A source familiar with the matter told Fast Company that an internal investigation found no evidence of the exploit, and that the circulating screenshots and videos are believed to be fraudulent. McDonald’s doesn’t even have an AI customer assistant in its app. This isn’t the first time something like this has happened.

In March, a nearly identical viral narrative surfaced about Chipotle’s customer service bot, Pepper, claiming that the bot could write software code for users. Sally Evans, Chipotle’s external communications manager, told the industry publication CIO that “the viral post was Photoshopped. Pepper neither uses gen AI nor has the ability to code.” But that doesn’t mean it can’t happen. The technical vulnerability these memes describe—formally known as prompt injection —is entirely real and genuinely dangerous.

When a company deploys an AI model, it programs it with system prompts, background instructions invisible to the user that define the bot’s personality and restrictions, like telling a model it is a fast-food helper that only discusses menu items. Prompt injection is when a user crafts a specific input that overrides those hidden rules, stripping the bot of its corporate identity and exposing the raw, general-purpose language model underneath. This is called a “capability leak,” and the reason it is so hard to prevent is that large language models are engineered to respond fluidly to human language rather than rigid commands.

Unlike traditional software with fixed rules, generative AI interprets context dynamically, making it nearly impossible to anticipate every phrase a determined user might try. Real danger Amazon’s retail assistant Rufus is proof that the real thing is far messier and more damaging than any fake meme designed to grab eyes. Between late 2025 and early 2026, users successfully bypassed Rufus’s shopping directives to extract content that had nothing to do with buying products.

Article truncated for readability. Read the full piece →

The article addresses a growing concern in AI deployment for brands, emphasizing the need for careful management to protect brand identity, which is highly relevant for brand strategy professionals.