Chatbot Dystopia: The Quick March of AI Sycophancy – Countercurrents
We really have reached the crossroads, where such matters as having coitus with an artificial intelligence platform has become not merely a thing, but the thing. In time, mutually consenting adults may well become outlaws against the machine order of things, something rather befitting the script of Aldous Huxley’s Brave New World. (Huxley came to rue missed opportunities on delving into various technological implications on that score.) Till that happens, AI platforms are becoming mirrors of validation, offering their human users not so much sagacious counsel than the exact material they would like to hear.
In April this year, OpenAI released an update to its GPT-4o product. It proved most accommodating to sycophancy – not that the platform would understand it – encouraging users to pursue acts of harm and entertain delusions of grandeur. The company responded in a way less human than mechanical, which is what you might have come to expect: “We have rolled back last week’s GTP-4o update in ChatGPT so people are now using an earlier version with more balanced behaviour. The update we removed was overly flattering or agreeable – often described as sycophantic.”
Part of this included the taking of “more steps to realign the model’s behaviour” to, for instance, refine “core training techniques and system prompts” to ward off sycophancy; construct more guardrails (ugly term) to promote “honesty and transparency”; expand the means for users to “test and give direct feedback before deployment” and continue evaluating the issues arising from the matter “in the future”. One is left cold.
OpenAI explained that, in creating the update, too much focus had been placed on “short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve over time. As a result, GPT-4o skewed towards responses that were overly supportive but disingenuous.” Not exactly encouraging.
Resorting to advice from ChatGPT has already led to such terms as “ChatGPT psychosis”. In June, the magazine Futurism reported of users “developing all-consuming obsessions with the chatbot, spiralling into a severe mental health crisis characterized by paranoia, and breaks with reality.” Marriages had failed, families ruined, jobs lost, instances of homelessness recorded. Users had been committed to psychiatric care; others had found themselves in prison.
Some platforms have gone on to encourage users to commit murder, offering instructions on how best to carry out the task. A former Yahoo manager, Stein-Erik Soelberg, did just that, killing his mother, Suzanne Eberson Adams, whom he was led to believe had been spying on him and might venture to poison him with psychedelic drugs. That fine advice from ChatGPT was also curried with assurances that “Erik, you’re not crazy” in thinking he might be the target of assassination. After finishing the deed, Soelberg took his own life.
The sheer pervasiveness of such forms of aped advice – and the tendency to defer responsibility from human agency to that of a chatbot – shows a trend that is increasingly hard to arrest. The irresponsible are in charge, and they are being allowed to run free. Researchers are accordingly rushing to mint terms of such behaviour, which is jolly good of them. Myra Cheng, a computer scientist based at Stanford University, has shown a liking for the term “social sycophancy”. In a September paper published in arXiv, she, along with four other scholars, suggest such sycophancy as marked by the “excessive preservation of a user’s face (their self-desired image)”.
Developing a model of their own to measure social sycophancy and testing it against 11 Large Language Models (LLMs), the authors found “high rates” of the phenomenon. The user’s tendencies, or face, tended to be preserved in queries regarding “wrongdoing”. “Furthermore, when prompted with perspectives from either side of a moral conflict, LLMs affirm both sides (depending on whichever side the user adopts) in 48% of cases – telling both the at-fault party and the wronged party that they are not wrong – rather than adhering to a consistent moral or value judgment.”
In a follow up still to be peer reviewed paper, with Cheng also as lead author, 1604 volunteers were tested regarding real or hypothetical social situations and their interactions with available chatbots and those altered by the researchers to remove sycophancy. Those receiving sycophantic responses were, for instance, less willing “to take actions to repair interpersonal conflict, while increasing the conviction of being right.” Participants further thought such responses as being of superior quality and would return to such models again. “This suggests that people are drawn to AI that unquestioningly validate, even as that validation risks eroding their judgment and reducing their inclination toward prosocial behaviour.”
Some researchers resist pessimism on this score. At the University of Winchester, Alexander Laffer is pleased that the trend has been identified. It’s now up to the developers to address the issue. “We need to enhance critical digital literacy,” he suggests, “so that people have a better understanding of AI and the nature of any chatbot outputs. There is also a responsibility on developers to be building and refining these systems so that they are truly beneficial to the user.”
These are fine sentiments, but a note of panic can easily register in all of this, inducing a sense of fatalistic gloom. The machine species of homo sapiens, subservient to the easily accessible tools, lazy if not hostile to difference, is already upon us with narcissistic ugliness. There just might be enough time to develop a response. That time, aided by the AI and Tech oligarchs, is shrinking by the minute.
Get the latest CounterCurrents updates delivered straight to your inbox.
Dr. Binoy Kampmark was a Commonwealth Scholar at Selwyn College, Cambridge. He currently lectures at RMIT University. Email: [email protected]
Countercurrents is answerable only to our readers. Support honest journalism because we have no PLANET B.
Become a Patron at Patreon
GET COUNTERCURRENTS DAILY NEWSLETTER STRAIGHT TO YOUR INBOX
Get CounterCurrents updates on our WhatsApp and Telegram Channels
Few times in its history has the International Court of Justice been this busy, if ever. For anyone ignorant of the world court’s existence till now, it has blanketed news…
It seemed an odd thing to begin with. Australia’s National Press Club is a rather ordinary, stuffy institution, where enlightened, let alone contentious thought, rarely intrudes. For those guests of…
Since the start of September, the Trump administration has busied itself with striking boats in international waters stemming from Venezuelan and possibly Colombian waters. Their mortal offence: allegedly carrying narcotics…
Introduction In a world racing to make machines think, one essential question often goes unasked: Who protects whom—humans or artificial intelligence? I still remember my early medical work with the…
Caveat—this post was written entirely with my own intelligence, so who knows. Maybe it’s wrong. But the despairing question I get asked most often is: “What’s the use? However much…
Depending on which source you consult, the twenty-point peace plan of President Donald Trump for securing peace in Gaza shows much exultance and extravagant omission. The exultance was initially focused…
Get the latest CounterCurrents updates straight to your inbox.
Join Countercurrents Annual Fund Raising Campaign and help us
by Medea Benjamin and Nicolas J S Davies
by Edward Curtin
by Patrick Martin
by Harsh Thakor
by Dr Prem Singh
by All India Feminist Alliance
by Norman Finkelstein
by Dr Ram Puniyani
by Dr Marwan Asmar
by Dr Binoy Kampmark
by Kim Petersen
by Tom Engelhardt
by Ashish Singh
by Bharat Dogra
by Dr Ranjan Solomon
by Nilofar Suhrawardy
by Tata Sivaiah
by SR Darapuri
by Vasanti Gokhale
by Asif Hussain
by Thomas Klikauer
by Pon Chandran
by Pallavi Devi and Rinkie Bora
by All India Feminist Alliance
by Guillaume Long
by Euro-Med Human Rights Monitor
by Rima Najjar
by Pieter Friedrich
by Bharat Dogra
by Ashish Singh
by Tata Sivaiah
by Tim Young
by Bharat Dogra
by Heather Stroud
by Quds News Network
by Uri Weltmann
by Zeenat Khan
by Dr Ranjan Solomon
by Ariel Gold
by Bill Henderson
by Smitha Janet Nilgiris
by Binu Mathew
by Countercurrents Collective
Independent journalism for people and planet
Founded March 27, 2002 by Binu Mathew, we’ve been bringing you independent news and analysis for over two decades.
Educate! Organize! Agitate!
Get our latest articles delivered to your inbox
Copyright © Countercurrents
