AI doctors and chatbot nurses? Labour must show the future of the NHS isn’t so dystopian – The Guardian
To reform the NHS, a new government will have to ignore self-serving tech companies – and engage a sceptical public instead
The AI revolution is upon us, and Labour is hoping it could help rescue an NHS in crisis. At a recent Institute for Government event, Karin Smyth, shadow minister for health, described how a Labour government will “hold the door open” for tech companies. But the tech-utopian visions Labour is being sold are not cost free: they could fundamentally reshape our relationships with our doctors and our own health, and undermine the founding principles on which the NHS is built. While tech companies flog AI magic, civil society must act as a counterbalance to temper the hype and remind the NHS about what really matters.
The Tory government also hopes that unlocking “the incredible power that [health] data possesses” can drive innovation, reduce costs and improve healthcare. The optimism crosses the parliamentary divide. The consensus is based on a vision of AI’s role in analysing medical health records and diagnostics, such as the early detection of Parkinson’s from eye scans or the “personalised AI doctor” extolled by the Tony Blair Institute in its recent report. It encompasses a top-to-bottom transformation of the healthcare system: from chatbots in mental health apps, to waiting list prioritisation algorithms, to supporting the discovery of new medicines.
But while Labour cannot spend its way out of the NHS crisis, AI is no magic bullet either. Tech-utopian visions fail to consider the darker implications of AI. Dr Jess Morley, an NHS and AI expert, has warned that we need to manage potential unintended consequences to “avoid a situation in which the NHS becomes a healthcare system that suits the algorithms but is not socially desirable or even acceptable”.
Hyping AI – as then health secretary Matt Hancock did with Babylon Health prior to its collapse – undermines public trust, which is needed for both the adoption of technology and the use of patient data that underpins it. Research conducted by my organisation, Connected by Data, with the campaign group Just Treatment, revealed patients’ deep concerns about the personal and ethical implications of the NHS’s digital transformation. We need a different political conversation and vision for the role of AI in the NHS to address them.
For example, do we want the Tony Blair Institute’s vision of personalised AI doctors to “capture and track diverse data from an individual over their lifetime” and use this to “make tailored treatment and lifestyle advice”? They could feel like an overbearing parent nagging us to take better care of ourselves, consequently making us start to discount how we actually feel, and so worry unnecessarily about our health.
Do we want doctors to become deskilled “humans in the loop”, mediating our encounters with AI systems? Or for triage nurses on 111 to be replaced by chatbots? Our relationships with medical professionals could unintentionally become datafied: reduced to a cold exchange of measurements and figures rather than the rich, human, caring encounters we value. Shouldn’t our priority be to use AI to optimise NHS management and simplify administration rather than augmenting medical care?
The risks don’t end there. Are we content for AI to leave behind the digitally excluded and mostly benefit the already privileged? There are well-known biases in the coverage and quality of health data and medical research, as well as a difference in how easy it is to access this technology between genders, races and communities. More AI could mean more health inequalities.
These potential downsides of AI threaten the foundations of our healthcare system: our autonomy, doctor-patient relationships and universal access. But they are not inevitable. An NHS that is alert to the risks, and to preserving its founding principles, can avoid them.
This is where Labour has the opportunity to differentiate itself from Tory tech solutionism. The current government shies away from robust challenge and runs engagement exercises oriented towards achieving public buy-in to plans the government already has in place. Labour should give the public, professionals and civil society a true opportunity to co-design the NHS’s future with technology, including legal and political tools to support collective scrutiny.
Labour also needs to show leadership in demarcating the role of the private sector in the development and delivery of AI systems. The Tony Blair Institute advocates using public money and public data to grow the commercial biotech sector. But as we have seen with medicines, the private sector has few qualms about putting profit above patient health.
Instead, Labour should commit to overhauling procurement processes in the health sector. It needs to equip the NHS to not just negotiate good value for money on AI, but renew its democratic roots. The NHS should be strategically investing in community-owned and open-source health technologies and putting in place independent ethical reviews and public oversight of large technology procurements to rebuild public trust.
AI has promise, but is not cost free. A Labour government will not just inherit a failing health service, but a set of risky technology deals and a sceptical public. As it seeks to reform the NHS, it must avoid being seduced by the self-serving promises of AI companies and instead give a powerful role to the public, medical professionals and civil society in shaping our future NHS.
Jeni Tennison runs Connected by Data, a campaign for communities to have a powerful say over the development and use of data and AI
