The Difference Between AI Safety, AI Ethics, and AI Governance

Three terms that get used interchangeably, and that's a problem for practitioners.

The VP of Engineering sends an email with the subject line "AI safety concerns."

What exactly does that mean?

The engineering team thinks it's about hallucination rates and failure modes.

Legal thinks it's about regulatory exposure risks.

The product ethics lead is certain it's about the recent bias tracking metrics.

They all show up to the same meeting. All with different agendas. They all are wrong. Everyone was answering a different question.

This is a real dynamic, and it happens constantly. "AI safety," "AI ethics," and "AI governance" are more often than not considered synonyms. Mainstream and tech media certainly treat them as the same. Even in professional discussions, they get confused.

For practitioners, this creates genuine confusion about who owns which problems, which teams need to be involved, and what "handling it properly" actually means.

AI Safety, AI Ethics, and AI Governance are not synonyms. The difference is important.

In my experience, the hardest problems to untangle are the ones where all three questions are confused from the start.

AI Safety: What Could Go Wrong

AI safety is a technical discipline concerned with preventing AI systems from producing catastrophic, unintended, or uncontrollable outcomes. This is often the case when an AI system begins pursuing outcomes its designers didn't anticipate.

It's the medical AI that confidently recommends the wrong treatment. It's the autonomous vehicle that misidentifies a pedestrian. It's a critical infrastructure system that behaves unpredictably under edge conditions it wasn't tested on.

These are safety failures in the engineering sense. The system does something harmful. In the world of AI, harm can occur at a scale or speed that humans can't easily contain.

The people drawn to AI safety tend to be researchers, ML engineers, and systems architects. The tools are model evaluation, red teaming, robustness testing, and the kind of adversarial probing that tries to find the edges where a system breaks.

AI Ethics: What Should We Do

AI ethics is concerned with the values and principles that should guide how AI systems are designed, deployed, and used. It's asking a different question entirely. It's not what happens in extreme failure cases. The concern is whether the system does the right thing in the ordinary case, day after day, for the people it affects.

The core concerns of AI ethics are the ones this newsletter spends most of its time on: fairness, accountability, transparency, privacy, and the measurable potential for harm to those involved. An AI ethics lens asks whose interests the system serves, who gets hurt when it makes mistakes, and whether the people it affects have any real recourse. It also asks whether decisions can be explained and scrutinized.

AI ethics operates at the intersection of technical decisions and moral philosophy, which is part of why it attracts a more diverse group of practitioners than AI safety does. Engineers, designers, social scientists, legal scholars, policy researchers, and domain experts all have relevant things to say about whether a system is treating people fairly.

It also means that AI ethics often produces disagreement. Reasonable people with different values and different frameworks can look at the same system and reach different conclusions about whether it's ethical. That's uncomfortable for engineers who want clean optimization targets.

It's also unavoidable.

AI Governance: How Do We Make Sure We Do It

AI governance is concerned with the structures, processes, and rules that make ethical commitments real and enforceable. Ethics tells you what you should do. Governance is how you make sure you actually do it, and how you demonstrate that to everyone outside your team.

Governance operates at two levels. Internal governance includes the organizational mechanisms teams put in place to manage AI risk: model cards, risk assessments, review boards, audit trails, documentation requirements, and approval processes for deploying high risk systems. These are the structures that catch problems before they reach users, and that create accountability when they fail.

External governance is what happens outside your organization: regulation, standards bodies, and compliance frameworks. The EU AI Act. The NIST AI Risk Management Framework. Sector specific rules from financial regulators, healthcare authorities, and employment law. These create obligations your organization must satisfy. Increasingly, you also have to demonstrate that you do.

The people drawn to governance work tend to come from legal, compliance, policy, and risk management backgrounds. They think in terms of documentation, audit trails, liability, and accountability structures. Engineers sometimes find this frustrating. The reverse is also true.

That tension is actually important. The goal of governance isn't paperwork. It's making sure that the values articulated in an AI ethics framework translate into actual system properties, and that those properties can be verified. Governance without ethics produces compliance theater. Ethics without governance produces intentions that don't survive a production rollout.

Where They Overlap, and Where They Don't

The three disciplines are related but genuinely distinct, and the distinctions matter for knowing who to pull into a conversation.

Safety and ethics overlap when a system's failure mode produces harm. A model that systematically misidentifies faces from one demographic group is both a safety failure (it's wrong in high stakes contexts) and an ethics failure (it distributes errors unfairly). The same event. Two different failure modes. Two different sets of fixes.

Ethics and governance overlap when the question is whether ethical commitments are real. Your team may have made deliberate choices about fairness definitions and accountability mechanisms. Governance is what ensures those choices are documented, auditable, and verifiable. They need to exist in a form that survives turnover, audits, and regulatory inquiries. Documented and verified, not just in the engineering team's collective memory.

Safety and governance overlap at the question of standards and certification. For high risk applications, external governance frameworks often specify what safety testing is required and what evidence of that testing must be preserved.

What they don't share is scope. Safety is primarily asking about catastrophic failure. Ethics is primarily asking about how the system treats people in the ordinary case. Governance is primarily asking about accountability. A system can be safe, yet deeply unethical. A team can have excellent internal governance and still ship a system with serious safety problems.

Treating all of these as the same issue leads to each one being addressed poorly.

A Map Worth Keeping

Three questions. Each separate but important.

Safety asks what could go catastrophically wrong.

Ethics asks what we should do, and whose interests we're serving.

Governance asks how we make sure we actually do it, and how we prove that we did.

These are three different questions. They require three different kinds of expertise. And when something goes wrong with an AI system in production, you will almost always find that at least two of these questions weren't being asked carefully enough.

Which problem the VP is actually worried about matters enormously. The team that knows these distinctions will walk into that meeting with the right conversation. Everyone else will have three conversations at once and accomplish nothing.

Great Architecture Isn't Built. It's Conducted The Software Conductor is Lee Atchison's practical guide to surviving the transition from developer to software architect, leading complex systems and teams without losing the score. Take a Look!

​

Lee Atchison is a software architect, author, and technology thought leader. He is the author of Architecting for Scale (O'Reilly) and The Software Conductor, and was the founder and CTO of Product Genius, an AI startup. He writes about software architecture, cloud systems, and AI at Software Architecture Insights.