Scaling AI Responsibly: What the FCA and the AA Are Learning in Practice

There is a version of the AI conversation that happens at conferences.

Governance frameworks. Responsible adoption. Ethics by design. Human oversight.

All important. None particularly difficult to say.

The harder question is what happens when AI leaves the presentation deck and starts making decisions, changing workflows and affecting real people.

That is where the experiences of the UK Financial Conduct Authority and the AA become interesting. Their contexts could hardly be more different. The FCA is working with regulated financial firms to understand how AI can be deployed safely across markets. The AA is applying AI inside a 120-year-old customer operations business, where a bad decision can affect someone stranded at the roadside in the middle of the night.

Yet both point toward the same conclusion.

Responsible AI is less about writing the perfect framework and more about building the organizational capability to test, learn, intervene and remain accountable.

The FCA's Answer Is Not Another AI Rulebook

Jessica Rusu, Chief Data, Information and Intelligence Officer at the FCA, has taken a deliberately practical approach to AI governance.

The FCA has chosen not to introduce a separate set of AI-specific rules. Instead, it is relying on its existing principles-based regulatory framework and using initiatives such as AI Live Testing to understand how AI behaves in real-world financial applications. Rusu has described the approach as deliberately technology-positive and focused on helping firms innovate while protecting consumers and markets. (FCA)

That distinction matters.

A credit-scoring model, an internal productivity assistant and a customer-facing financial chatbot do not create the same risks. Treating them as though they do can produce a compliance exercise without producing better outcomes.

The FCA's AI Live Testing programme therefore focuses on the AI system, rather than simply the underlying model. That means looking at the model alongside its data, deployment environment, human oversight, governance, testing and input and output controls. (FCA)

The question becomes less:

"Is this AI model responsible?"

And more:

"Does this particular system produce the outcomes we intended, in the environment in which we are actually deploying it?"

That is a much harder question.

It is also a much more useful one.

Test It Before You Trust It

The FCA's AI Live Testing programme is essentially an attempt to move responsible AI out of theory and into controlled reality.

Participating firms work with FCA regulatory teams and technical partner Advai through three stages: discovery, framework validation and AI system testing. The testing considers both quantitative and qualitative evidence, including how systems perform in their intended context. (FCA)

The FCA's second cohort, announced in April 2026, includes firms such as Barclays, Experian, Lloyds Banking Group's Scottish Widows and UBS. The applications being tested range from agentic payments and anti-money-laundering detection to credit-score insights and targeted investment support. (FCA)

This is important because many organizations still confuse an impressive demonstration with a deployable system.

A model can perform brilliantly in a controlled test and behave very differently once customers, employees, incomplete data, unusual edge cases and operational constraints enter the picture.

Responsible scaling therefore requires a feedback loop.

Build.

Test.

Observe.

Find the failure modes.

Adjust.

Test again.

That is closer to engineering than policy writing.

The Sandbox Is More Than a Testing Environment

The FCA has also been experimenting with a broader idea: sometimes the fastest way to make innovation safer is to give organizations a place where they can experiment together.

Its Supercharged Sandbox provides firms with infrastructure, synthetic datasets and technical support so they can develop and test AI applications without having to build the entire environment themselves. The FCA has been expanding this capability in response to demand. (FCA)

That creates an interesting dynamic.

The regulator isn't simply standing outside the innovation process waiting to assess the finished product.

It is getting closer to the work.

The firms learn what responsible deployment requires. The regulator learns what firms are actually encountering. Both sides accumulate information that can improve the next iteration.

That is particularly valuable in AI because the technology is moving faster than conventional regulatory cycles.

The FCA has explicitly framed AI Live Testing as a way to avoid "POC paralysis", where promising AI systems remain trapped indefinitely in pilots because organizations cannot establish enough confidence to deploy them. (FCA)

The lesson for businesses is broader than financial services.

If experimentation is going to happen anyway, organizations need environments where experimentation can happen safely.

Then There Is the AA

The AA presents the other side of the equation.

The FCA is trying to understand how organizations should govern AI.

The AA is trying to make AI work inside a complex, operationally critical business.

Imagine the scenario.

A family breaks down on the M6 at midnight. The customer may not know what is wrong with the vehicle. The organization needs to determine what resource to send, where it is, whether the vehicle can be repaired, and what happens to the people inside it.

That is not a theoretical AI use case.

It is a chain of decisions involving a real customer in a vulnerable situation.

And that changes the definition of "responsible."

The objective isn't simply to make an AI system accurate.

It is to make the entire customer experience better without losing the human judgment required when circumstances fall outside the expected pattern.

The AA's AI Story Started Before AI

One of the most useful details in the AA story is that its AI transformation did not begin with AI.

The company had already spent years rebuilding its digital infrastructure and changing how customers reported breakdowns and interacted with the organization.

That foundation matters.

AI is often presented as though it can simply be placed on top of an existing business and instantly transform it.

In reality, AI tends to amplify the quality of the system underneath it.

Weak processes produce automated weak processes.

Fragmented information produces faster access to fragmented information.

Strong digital foundations, clear processes and connected data give AI something much more useful to work with.

The AA's experience therefore offers a less glamorous but more important lesson.

AI readiness is often built years before the AI arrives.

Adoption Cannot Be Ordered Into Existence

The AA's experience with internal AI adoption provides another useful insight.

When organizations introduce new technology, leadership often assumes adoption will follow the rollout.

It doesn't.

People have to find a reason to use it.

When the AA introduced ChatGPT Enterprise to employees, adoption spread beyond the technology function. Employees across areas of the business began experimenting with the technology and building their own applications.

That matters because successful AI transformation is rarely just a technology deployment.

It is a behavior-change problem.

The organization has to create enough safety for people to experiment, enough guidance to prevent reckless experimentation and enough autonomy for employees to discover uses leadership may never have anticipated.

The most valuable AI use cases are not always found in the strategy document.

Sometimes they are discovered by the employee who realizes on a Tuesday afternoon that a repetitive task they've performed for three years could disappear.

The Contact Centre Could Become Something Completely Different

Harj Johal's perspective on the AA's customer operations points toward an even bigger shift.

The conventional AI narrative says contact centres are vulnerable because machines can answer more questions.

The more interesting possibility is that AI changes what the human does.

Instead of moving a customer between multiple people, each responsible for one fragment of a problem, a single employee could have access to vehicle information, customer history, logistics, external services and operational data at the same time.

The human becomes the orchestrator.

In the breakdown scenario, that could mean recognizing that a vehicle cannot be repaired roadside, arranging the appropriate recovery vehicle, identifying accommodation and managing the customer's situation from beginning to end.

AI isn't necessarily replacing the person making the decision.

It is expanding the amount of information and capability that person can bring to the decision.

That distinction could define the next generation of AI-enabled work.

The Real Competitive Advantage May Be Curiosity

There is another thread connecting these stories.

Curiosity.

AI transformation requires people who are willing to ask:

What else could this do?

What happens if we try it differently?

Where could this fail?

What did we learn from the last experiment?

That mindset matters at every level, from the employee experimenting with a new workflow to the regulator trying to understand emerging risks.

Technical expertise helps.

But technological change moves too quickly for expertise alone to be enough.

Organizations need people who can continuously learn, question assumptions and adapt their mental models as the technology changes.

Responsible AI Is an Operating Capability

The FCA and the AA are operating at completely different ends of the AI spectrum.

One is responsible for protecting consumers and markets across a heavily regulated financial system.

The other is trying to improve how a major customer-service organization operates.

Yet the underlying lesson is remarkably similar.

Neither organization treats responsible AI as something that can be solved once.

The FCA is testing systems in context, learning from firms and adapting its understanding of emerging risks. The AA is building AI into workflows where employees, customers, data and operational consequences all interact.

Both approaches recognize something that gets lost in much of the AI debate.

The difficult part isn't getting AI to work once.

It's building an organization capable of knowing when it is working, when it isn't, what to do when it fails, and who remains accountable either way.

That is what responsible scaling actually looks like.