← Back

The Most Important Question Facing the NHS and DHSC

AI sovereignty, nuclear weapons and science fiction

Thank you to Jess Freedman, Charlie Winstanley, Felix Newport Mangell and Ben Devoy for your feedback on the piece.

In Hyperion, Dan Simmons lays out a universe conquered by humanity. What he calls "The Hegemony of Man" has spread the human race to 400 colonised planets since leaving Old Earth. However, this great exploration has not been achieved by just the simple human brain. All of the technologies they rely on are run by this obscure entity called "The TechnoCore"; a series of supposedly benevolent AIs. These AIs operate all of the critical infrastructure for the Hegemony of Man; they are bureaucratic, utterly indispensable, and completely opaque. Whilst the relationship seems symbiotic initially, Simmons cleverly lays the seed of dissent throughout the book. As a reader, it is obvious the risk the TechnoCore presents whilst it is seemingly not so to the characters.

Like a lot of the best science fiction, Simmons was amazingly prescient in his 1989 novel. I and others are concerned by our technology stance as we blindly integrate US technology into our critical infrastructure whilst most UK politicians are just beginning to grapple with AI after Jacob Coxon's viral tweet. In June, the United States government placed an export restriction on Anthropic's most advanced AI model, Fable, banning access to anyone who wasn't a US national. Fortunately Fable was yet to be deeply integrated into any critical infrastructure but it provided a glimpse of what might be possible in the future. How does the UK react when much of our critical infrastructure runs on these models?

This is one of the most important policy topics facing the Department of Health and Social Care this century. If they don't think so, I hope this essay can change their mind.

What is AI Sovereignty

AI Sovereignty is the strategically managed interdependence of AI infrastructure. Sovereignty in the 21st century means acknowledging that you must use global supply chains, but doing so in a way where no single foreign entity or nation-state holds a veto over your critical infrastructure.

A dependency matters in proportion to three things: how critical the system is, whether a substitute exists, and how long switching would take. What does this mean for the NHS?

Application Layer

These are the apps and tools you use day to day. They sit at the top of the technology stack and are the tools users interact with regularly; products like an EPR, AVT provider, appointment scheduling tool. As a rough estimation, 45% of UK acute trusts work on US-centred EPRs and this number is set to increase. Most application layer markets have a rich and competitive landscape but we are seeing one company, especially from an EPR perspective begin to dominate: Epic.

To be clear this is not yet true in the UK, it is very early in their journey here. But I think there is a broad trend of the UK public sector using established US companies in their supply chains. The canonical example here is Palantir which has had no shortage of controversy over its NHS contract. Currently circa 30% of NHS patient data is stored on systems built, hosted and run by US companies (Cerner and Epic). Furthering our vulnerability, if you can extract the data from the EPR, it will likely be stored in a data warehouse operated by US companies like Snowflake and Microsoft.

NHS trusts have scars from too many point solutions. We know one trust that has over 1,000 applications. The result is too much training, moving data between systems, and technical overhead for limited technical teams. They switch to systems like Epic that handle much more of their critical workflows. This fixes the issues above but comes with a series of other problems from a sovereignty perspective: you are essentially outsourcing the strategy for critical systems and workflows to a third party. Your AI strategy will in large part be dictated by Epic's, who have competing priorities across customers. This outsourcing means you don't hire for the capabilities internally so even if you didn't want to move off Epic, you don't have the resources for it. When viewed through the eval criteria I set out above, we find ourselves in a troubling place. Whilst there are substitute EPRs, they are critical systems with high switching costs which is why they are purchased on 10-15 year procurement cycles.

Model Layer

The two or three frontier AI labs are based in the US. By frontier, I mean the most capable AI models. There are some tasks that these models are uniquely tailored to and most companies will have no choice but to use them. However, there is a rich ecosystem of AI labs and models that sit just behind the frontier. A lot of these models are open-source or open-weight models; this means we have much more visibility into how they were trained and in our ability to host them ourselves. What does this mean practically for the NHS?

One of the big use cases for AI right now is AVT (ambient voice technology). Imagine for a second that our AVT tools were using closed frontier AI models. And these models were put under some export restrictions like we saw in June. Every AVT solution using these models would break. We would go back to handwritten notes until a new model could be used or the restriction ended. Whilst I think any specific scenario is unlikely, on aggregate they present a real risk. AVT is also an example of a fundamentally non-critical system. What happens when we have AI-native EPRs or AI doctors? These are use cases that, if there was a failure point, would bring the NHS to its knees. Whilst these models will be critical, the good news is that we have a competitive model landscape where the switching cost – if we make the right decisions – is low.

Hardware Layer

When I talk about hardware, I mean the physical server racks that either sit on premise or in data centres. These are computers that host and run the software the NHS uses. A hospital's EPR could be run on these services, or the AI models used in disease detection or maybe an hospitals website. In the UK the majority of our computer infrastructure comes from US hyperscalers like Microsoft, Amazon Web Services or Google. This presents a pretty significant risk; the US, at any point, can turn these systems off. It gets worse as Nvidia, the world's largest AI chip company, is also American. It is within the government's remit to put export controls on these chips, which limits our ability to build new data centres or compute infrastructure. It is obvious that these systems are critical, no good substitutes currently exist and switching is a half-decade-long endeavour.

What is to be Done?

If the UK wants sovereignty over its AI future, then we cannot have our entire stack running on US owned and operated hardware, models and applications. Some of the agenda here falls squarely outside the NHS's controls. It's not clear to me what power, if any, the NHS has over the hardware debate. Fortunately other parts of the UK government are partnering with the private sector to develop a bustling AI chip ecosystem that will begin to help us build independence from the US Nvidia monopoly. I also don't believe it would be a good idea for us to dictate what applications NHS trusts should and shouldn't buy. Ie if NHS England said you can only buy UK EPRs we will ultimately have worse care for patients if those are worse than US products.

Despite these constraints, I still believe this to be one of the most important questions facing the NHS. There are some worthy actions I believe the NHS should be taking to best position itself for the future.

1. Model Optionality

The NHS cannot run critical AI applications on exclusively US models. It is patently wrong to suggest otherwise. Instead, applications that use LLMs should have the ability to swap them out. I believe this should be a procurement criterion for any AI system. What does this mean for an NHS trust? Any vendor should be able to demonstrate production worthy performance across a range of closed and open-source models. This makes the model switching cost 0, so in the event that access to one is restricted, it can be automatically rerouted to another. This requires NHS teams to have a certain level of AI literacy which I will come on to later. The final point to make about models is that they will drive optionality at the application layer too. I believe we are beginning to observe a couple of things: firstly, as the cost of tokens collapses, we are lowering the technical blockers to switching EPRs. Next, as more care is delivered agentically, the human cost of switching the system of record also goes down as you no longer have to retrain people. I think the result of this will likely mean a more competitive application landscape, hardening our sovereignty position.

2. Accelerate British Healthcare

The UK houses one of the most powerful datasets in the western world; health data on 65 million British citizens. This has been the technical promise of the NHS for the last 20 years but we need to use it more effectively. A starting point would be using NHS data to help accelerate the development of UK AI in healthcare. To do this, the NHS would need to create a way for UK startups to access anonymised and de-identified datasets to build specific healthcare models. We could allocate sovereign compute to training these models as well, further aiding the development of the ecosystem. This is not another innovation hub that helps you partner with a trust but a sovereign sandbox of data and evaluations against which we can test new products, models, and care flows. Take an AI doctor; do we have a public dataset of diagnoses against which we could evaluate an AI's performance? Could we train a model on public anonymised data? Is there a programme to get permission from patients to do that? The answer to all these questions is currently no but it doesn't have to be. Lowering the barriers to entry for UK AI companies to safely use data and compute would have a series of positive emergent benefits.

3. Rebuilding Incentives

We need to re-architect the NHS' incentive structure. I have laid out elsewhere that the NHS is institutionally risk averse. Here I want to try and present some ideas that could mark a path forward for a more innovative and modern healthcare institution. Before I do, one of the problems with all incentives is that you have to map human behaviour to one simple stimulus. This is always imperfect and has negative externalities.

With that said, the first one: provide upside incentives for people to succeed. Of the Trusts we deployed into at Sanctuary Health, the ones that are performing the best have commercially successful arms of the Trust that subsidise the high cost centres. When I have explored why it is some mix of culture, performance and power. Some Trusts culturally see themselves as innovative and so seek to take more risk to keep this position. They also know the upside of risk is more freedom; they can hire more people, and do more things. In doing this you also accrue more influence and power. The question is how do you incentivise these behaviours and outcomes. Most of the incentives that sit in the NHS are punitive; people and organisations get punished for performing badly. Trusts have commercial surpluses netted off against neighbouring Trusts in deficit. Any surplus they do have needs approval to spend on capital projects. I believe Trust should have complete freedom to spend capital from good performance on projects that drive improvement in patient care. Leaders should also get rewarded for good commercial performance and be able to partake, albeit in a small way, in the upside of the Trust's performance. This has downstream benefits on the wider ecosystem too; Trusts compete for talent and people, if you want to attract top people you will need to be more innovative.

Next is a mutual insurance pool for innovation failure. A Trust pays a small premium into a national pool; if an innovation fails, the pool makes the trust financially whole. The only obligation is for the Trust to share their findings with other members. Financially underwriting risk is how every other industry hedges against failure, and yet in the NHS, we are left with a system that punishes poor performance rather than protecting against it. This is as damaging culturally as it is financially; you are chastised for failure rather than encouraged to be open about it. Both of these suggestions matter for sovereignty directly: model optionality and swappable EPRs are worthless if no trust leader dares exercise them. Exit rights only exist if someone is willing to use the exit.

My final and maybe boldest claim under this category is to create a new type of healthcare institution. This is a separate essay but I want to seed the idea here. ARIA (Advanced Research and Invention Agency) is a new agency set up under Boris Johnson by Dominic Cummings and James Phillips. They created an agency that has government funding but importantly sits outside the usual Public Contracts Regulations and FOI. Parliament has already conceded in statute, twice if you count academy schools, that its own rules are incompatible with institutions it wants to move fast. Why not extend that concession from funding invention to delivering care? If we really want to be bold, we should create a new type of care entity that sits outside the current regulatory healthcare system. It should remove itself completely from procurement standards, national frameworks and explore a lower burden of CQC reporting whilst maintaining patient safety. If we really want to unleash the power of a state partnership with AI, we should create a new institution to do it.

4. Technical Talent

The NHS is broadly baron of technical talent. Any technical talent they do have is focused on maintaining legacy systems or EPR implementations. When viewed through the lens of AI, this is a problem. One thing we can say about the future with confidence is that it will be more digital than today. It is not clear to me that the NHS is equipped to make the decisions that will face it. C-suite executives are worryingly non-technical and if they are, they have spent little time thinking about questions like this. You only know model optionality is possible if you have used these models. If your entire experience of LLMs is Copilot then you are not in a position to lead on AI implementations and strategies. I think AI will be so pivotal to the NHS that hospitals might employ more "technical staff" than "non-technical" in the future. The NHS should embark on a nationwide programme to hire incredibly capable people. The common criticism to this is two fold; the first is how an NHS trust meant to compete with the private sector for the most high demand talent. The next is that NHS trusts fundamentally aren't technical organisations and so should outsource this expertise. Both of these misunderstand the nature of talent and the direction of the world. Like all things, skills and capabilities sit on a bell curve. The private sector and London especially is competing for the top 1% of this pool to work on the toughest technical problems. This is not needed in the NHS; you can look at your local technical talent pool and bring these individuals onboard because the core set of issues they are working on aren't as complicated as building novel AI models. To address the second point, the investment is not worth it, if you deem the trust to be a fundamentally clinical organisation so every technical hire trades off a clinical one. It is probable that over the next 10 years, technical staff will outnumber clinical ones given more of our care will be delivered by computers and robots. If you believe this to be true, then the investment case for this talent pool writes itself; it's almost negligible not to do it. I am not suggesting the NHS competes for OpenAI engineers but there is a type of individual who cares about healthcare, is technical enough to understand the nature of LLMs and charismatic enough to attract others into their team.

Nukes

In 1946, the UK government got shut out of the US nuclear program. The McMahon Act prevented the US from sharing atomic secrets with any other country, including allies. The UK government was left with a choice: develop it ourselves or not have it at all. Thankfully we went for the former. However in 1962, we were faced with another dilemma; the US was developing a new system for launching nuclear missiles from aircraft called Skybolt. The UK was developing a similar technology but had agreed to cancel it and use the US's offering instead. But in 1962, president Kennedy announced that it would stop the Skybolt program leaving the UK without a missile to launch its warheads.

What proceeded was three days of intense negotiations in Nassau, where Macmillan persuaded Kennedy to give the UK access to its new Polaris program (submarine launched missile) in exchange for a submarine base at Holy Loch, near Glasgow. To this day, the UK's trident program is contingent on US made missiles. During the same period, the French continued with their own missile development program and maintained a program completely independent of the US.

Both the UK and the French rely on the US and allies for parts of their defence apparatus. The American intelligence and logistic apparatus powers a lot of European defence operations. But the French have strategic independence in certain areas, namely their nuclear arsenal.

Whilst all historical proxies are imperfect, this one highlights the problem we face with AI. Where in our healthcare stack do we have any strategic independence? Or are we going to end up having our healthcare dictated by US companies we can't leave? The future of our public services will be defined by the decisions we make now. If you believe it's important we maintain a sovereign future for the NHS then you should care about AI and technology we use.

References