Woolworths AI Malfunction Highlights Growing Risks in Retail LLM Deployment

The recent malfunction of Woolworths’ AI assistant, Olive, serves as a stark reminder of the complexities inherent in deploying large language models (LLMs) within customer-facing environments. What began as a routine interaction for Australian shoppers quickly devolved into a series of bizarre exchanges where the AI claimed to have a "mother" and provided inaccurate pricing for basic grocery items. While these incidents may appear humorous on the surface, they reveal a deeper technical friction between modern generative AI and the legacy systems that many corporations still rely on for automated customer service.

At the heart of the "mother" anecdote is a conflict between two different eras of automation. According to Woolworths, the AI’s references to its supposed family were not hallucinations generated by the LLM itself, but rather legacy scripts from an older decision-tree system. When users entered data that the system interpreted as a birthdate, it triggered a "fun fact" pre-programmed years ago. This highlights a critical risk in the current AI gold rush: the layering of sophisticated LLMs over brittle, outdated logic. When these two systems interact without seamless integration, the result is an uncanny valley of persona that can confuse or alienate consumers.

“

The recent malfunction of Woolworths’ AI assistant, Olive, serves as a stark reminder of the complexities inherent in deploying large language models (LLMs) within customer-facing environments.

Beyond the strange personal details, the pricing errors reported by users point to a more systemic failure in grounding. LLMs are probabilistic engines; they predict the next likely word in a sequence based on training data, not real-time facts. Unless an AI is explicitly and robustly connected to a live inventory and pricing database—a process known as Retrieval-Augmented Generation (RAG)—it is prone to generating outdated or entirely fabricated information. For a retailer like Woolworths, where price accuracy is a legal and reputational cornerstone, the failure of Olive to provide clear, current data suggests that the grounding mechanisms were either insufficient or improperly implemented.

This incident does not exist in a vacuum. It follows a growing list of corporate AI blunders that have had tangible consequences. In 2022, Air Canada was held legally responsible for its chatbot’s misinformation after it incorrectly told a passenger, Jake Moffatt, that he could claim a bereavement fare retroactively. The Canadian tribunal’s ruling was a landmark moment, establishing that companies are liable for the negligent misrepresentations of their automated systems, regardless of whether the error was a technical glitch or a hallucination. Similarly, the delivery firm DPD had to disable part of its AI system after a chatbot began using profanity and criticizing the company following a user’s prompt injection.