The AI safety community publishes hundreds of papers a year on alignment, interpretability and robustness. Meanwhile, a small business installs a chatbot on its website that confidently quotes the wrong price and offers a refund nobody authorised. The distance between those two worlds is the governance gap, and it is widening.
Disclosure. Singularity AI is operated by Daedalus Design, which builds websites, including AI assistants, for small businesses. This piece discusses that market. We have tried to keep it to what the evidence and the law say, not what any provider sells.
The research is not the problem
Governance research is in better shape than it is often given credit for. The NIST AI Risk Management Framework sets out how to identify, measure and manage AI risk across a system's life. The OECD AI Principles give a common baseline for trustworthy AI. The EU AI Act turns much of this into law, and the UK's AI Security Institute tests frontier models before release. Interpretability and evaluation research keeps improving our ability to understand what models do.
None of it reaches the average small business website. The typical deployment is a thin wrapper around a frontier model: a widget, a few sentences of system prompt written by whoever set it up, perhaps a PDF of the product catalogue. There is no evaluation before launch, no record of what the assistant says, and no plan for when it goes wrong. The gap is not a failure of research. It is a failure of translation.
This is already a legal problem
It is tempting to treat chatbot mistakes as embarrassing rather than consequential. The courts disagree. In Moffatt v Air Canada (2024), a Canadian tribunal held the airline liable after its website chatbot gave a customer wrong information about bereavement fares. Air Canada argued that the chatbot was responsible for its own words. The tribunal rejected that outright: a company is responsible for all the information on its website, whether it comes from a static page or a chatbot. The same year, the parcel company DPD had to disable part of its customer-service bot after a customer prompted it to swear and to write a poem criticising the company.
In Europe the rules are now explicit. Article 50 of the EU AI Act, which has applied since 2 August 2026 and was not delayed by the Digital Omnibus, requires that people are told when they are interacting with an AI system unless it is obvious. UK businesses serving EU customers are in scope. At home, UK GDPR already applies to whatever personal data a chatbot collects, and the Information Commissioner's Office has published detailed guidance on AI and data protection.
Three failures, in research terms
The safety literature gives useful names to what goes wrong.
1. Specification failure
The specification problem asks whether we can write down what we want a system to do. For a business chatbot, the specification is the system prompt, and most are badly underspecified. They say "be helpful" and "answer questions about our services" but say nothing about pricing guarantees, refunds, legal or medical questions, or when to hand over to a human. The Air Canada case is a specification failure: nothing constrained what the assistant could promise.
2. Robustness failure
Robustness research asks how systems behave under inputs they were not designed for. A customer-facing chatbot sees them constantly: slang, sarcasm, other languages, questions about competitors, and people trying to break it for fun, as DPD discovered. The model provider's safety training helps with generally harmful content. It is not designed for the specific ways a particular business can be embarrassed or exposed. There is also a subtler problem: research from the Oxford Internet Institute in 2026 found that chatbots tuned to be warmer and friendlier made more mistakes and were more inclined to tell users what they wanted to hear [Oxford Internet Institute, 2026]. The persona a business chooses can make its assistant less reliable.
3. Accountability failure
Interpretability research asks why a model produced an output. The business equivalent is simpler and more important: what did the assistant say, and to whom? Without logging, there is no answer. If a customer says the chatbot promised them something, the business cannot check. "The AI did it" is not a defence; the deploying organisation owns the outcome.
What good practice looks like
Closing the gap does not require a safety research team. It requires a short list of things done properly.
- Disclose it. The first message should make clear that the customer is talking to an automated assistant, and how to reach a person. In the EU this is now law; everywhere it is good practice.
- Define what it may commit to. Prices, refunds, delivery dates, bookings and anything resembling legal, financial or medical advice need explicit rules. "Be helpful" is not a guardrail.
- Give it a safe fallback. When it does not know, it should say so and escalate, not improvise. Ask your provider to show you what it does with a question it cannot answer.
- Enforce limits in code, not just in the prompt. Topic filters, output checks for commitments the business does not make, and rate limits add defence in depth. A system prompt alone can be talked around.
- Log and review. Keep a record of conversations, in line with your privacy notice and UK GDPR, and have someone review a sample regularly. Patterns of error only surface when someone looks.
- Collect only what you need. Minimise personal data, set a retention period and give people a way to have it deleted.
- Keep a kill switch, and test it. You should be able to turn the assistant off immediately if something goes wrong.
The path forward
Small businesses are not expected to run interpretability research on their chatbots. They are expected to know what their assistant can say, to keep a record of what it does say, and to have a plan for when it gets things wrong. These are not high bars. They are the basic operational requirements that the AI industry has largely failed to communicate to the businesses adopting its tools, and that the law increasingly assumes are in place.
The governance gap is not a technology problem. The research community knows what good governance looks like. The work that remains is making it ordinary: affordable, expected and built in by default for every business putting an AI assistant in front of its customers.
References
Moffatt v. Air Canada, 2024 BCCRT 149 (Civil Resolution Tribunal of British Columbia, 14 February 2024).
European Union. Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 50, as amended by Regulation (EU) 2026/1744. eur-lex.europa.eu.
Information Commissioner's Office. "Guidance on AI and data protection." ico.org.uk.
NIST (2023). "Artificial Intelligence Risk Management Framework (AI RMF 1.0)." NIST AI 100-1.
OECD. "OECD AI Principles." oecd.ai.
Oxford Internet Institute (2026). "Friendly AI chatbots make more mistakes and tell people what they want to hear, study finds." 29 April. oii.ox.ac.uk.