Who is responsible when AI makes mistakes? What science fiction left out

Who is responsible when AI makes mistakes? Not the model. Responsibility stays with people and organisations: the developer who built the system, the organisation that deployed it, the operator who runs it and, within limits, the user who acts on its output. Naming these roles is easy. The hard part is deciding, before launch, which of them owns which decision.

Science fiction spent eighty years describing machines that build, calculate and talk. Most of it arrived: the computer that answers questions, the assistant that writes code, the agent that runs a workflow overnight. The genre was right about capability and quiet about accountability. In 2026 capability is cheap, so accountability is the part a business owner has to design.

Asimov’s rules (1942)

In 1942, in the story “Runaround”, Isaac Asimov wrote three rules for robots. The classic stories that followed turn on a rule meeting a situation it did not foresee. The robot obeys the letter, misses the purpose, and nobody has worked out who is responsible for what happens next. The rules existed. An accountable owner did not.

HAL (1968)

HAL, the computer in 2001: A Space Odyssey (1968), talks, plans and runs a spacecraft. It has no owner who signs for it. When its judgement and the crew’s interests part, there is no audit trail anyone reads, no agreed stop condition and no escalation route. The only human oversight left is a crew member disconnecting it by hand. Capability arrived first; accountability never did.

The people who build, deploy and run the system, each for what they controlled. The developer answers for how the model and the software were made. The deploying organisation answers to its customers for putting AI into its product. The operator answers for running it. The end user answers for acting on output they could have checked.

This is not legal advice. AI liability depends on the jurisdiction, the contract and the facts, and a claim may run through contract, negligence or product liability; ask a lawyer before relying on any of it. The public frameworks point the same way: the OECD AI Principles hold AI actors accountable according to their role, the NIST AI Risk Management Framework asks organisations to make named teams and individuals responsible for AI risk, and the EU AI Act splits duties between providers and deployers.

The gaps appear where roles overlap. A company that builds its own AI tool and uses it with customers is developer, deploying organisation and operator at once, and each team assumes another one holds the risk. So who is accountable when AI fails there? Whoever was named in advance, or nobody.

The cheap part

An AI platform can produce a working prototype in an evening. We use those tools ourselves, inside our engineering and inside Vendo AI, our own sales product. The output still needs checking: Veracode’s 2026 GenAI Code Security Report found a 56% security pass rate across more than 100 models, and in the Stack Overflow Developer Survey developers’ trust in the accuracy of AI output fell to 29% from 40%. None of that makes the tools bad. It makes them tools.

The expensive part

What a business needs after the prototype is a system that keeps working: architecture that survives growth, data handled correctly, security that is checked rather than assumed, and a named person who answers when something breaks at two in the morning. For a product owner, AI governance comes down to four decisions, written down before launch:

• →A named owner: one person, not a committee, who can stop the system and answers for the result.
• →An audit trail: what the system received, what it produced and who approved it, kept for traceability and explainability.
• →Stop conditions: where AI must not act alone, such as payments, automated decisions about people or anything outside its documented scope.
• →An escalation path: who is called, how quickly, and what the user sees while a person takes over.

None of these is a clause a lawyer adds at the end. Each changes the architecture, the interface, the documentation and the running cost, so each belongs in the product logic from the start. Human in the loop means something only when that person has the authority, the information and the time to overrule the machine.

Our rule is short: no development without product logic. A product operator decides what deserves to be built, builds it to operate and answers for the result. For a new product, accountability starts before the first line of code.

The Leadership Gate is where accountability becomes a decision. It has one owner, the CEO, and it ends with reasons: proceed, reshape or decline. We decline when the product logic does not hold, when the commercial model cannot support operation, or when we could not take responsibility for the outcome. The full sequence is set out in how our process works.

We hold our own AI to the same rules. The Vendo AI case study shows an accountable AI product in production: every signal source is a separate opt-in module with its own analytics, and a conversation can pass to a human in real time. On this site, Vendo AI collects context and helps arrange a call; it does not make commercial decisions or issue offers.

Ask them of your own team, of any vendor and of any AI builder you are comparing. The answers belong in writing.

• →Who is the named owner of this system in production?
• →Which decisions may the AI take alone, and which need a person’s approval?
• →What is logged, for how long, and who reads it when something goes wrong?
• →Which conditions stop the system, and who is called when they do?
• →If you are choosing between an AI builder and a team: who answers for the system after launch?

If nobody can answer the first question, the prototype is where the project ends, however fast it was made. If an AI-built product is already live, has users and has stalled, Existing Product Review is the route where we review the code, the business logic and the market, and decide whether to invest.

Science fiction predicted the machines. The rules for the people around them were always the hard part.