Judge rejects claim that forcing OpenAI to keep ChatGPT logs is mass surveillance – Ars Technica
OpenAI will fight order to keep all ChatGPT logs after users fail to sway court.
After a court ordered OpenAI to “indefinitely” retain all ChatGPT logs, including deleted chats, of millions of users, two panicked users tried and failed to intervene. The order sought to preserve potential evidence in a copyright infringement lawsuit raised by news organizations.
In May, Judge Ona Wang, who drafted the order, rejected the first user’s request on behalf of his company simply because the company should have hired a lawyer to draft the filing. But more recently, Wang rejected a second claim from another ChatGPT user, and that order went into greater detail, revealing how the judge is considering opposition to the order ahead of oral arguments this week, which were urgently requested by OpenAI.
The second request to intervene came from a ChatGPT user named Aidan Hunt, who said that he uses ChatGPT “from time to time,” occasionally sending OpenAI “highly sensitive personal and commercial information in the course of using the service.”
In his filing, Hunt alleged that Wang’s preservation order created a “nationwide mass surveillance program” affecting and potentially harming “all ChatGPT users,” who received no warning that their deleted and anonymous chats were suddenly being retained. He warned that the order limiting retention to just ChatGPT outputs carried the same risks as including user inputs, since outputs “inherently reveal, and often explicitly restate, the input questions or topics input.”
Hunt claimed that he only learned that ChatGPT was retaining this information—despite policies specifying they would not—by stumbling upon the news in an online forum. Feeling that his Fourth Amendment and due process rights were being infringed, Hunt sought to influence the court’s decision and proposed a motion to vacate the order that said Wang’s “order effectively requires Defendants to implement a mass surveillance program affecting all ChatGPT users.”
Hunt’s fears are not unfounded, Corynne McSherry, legal director for the digital rights group the Electronic Frontier Foundation, told Ars.
“The discovery order poses genuine risks to user privacy in itself and as a precedent for the many other lawsuits around the country,” McSherry said. “And it is emblematic of a broader problem: AI chatbots are opening another vector for corporate surveillance, especially if users don’t have meaningful control over what happens to their chat histories and records.”
According to Hunt, Wang failed to “consider exempting ‘Anonymous Chats,’ which are reasonably expected to contain the most sensitive and potentially damaging information of users, from retention and disclosure in this case,” claiming that it “constitutes an overly broad and unreasonable action.”
He urged the judge to revise the order to include this exemption, as well as exemptions for any chats “discussing medical, financial, legal, and personal topics that contain deeply private information of users and bear no relevance whatsoever” to the plaintiff news organizations’ claimed interests.
For Hunt and many other users blindsided by the order, the stakes appear high. He suggested that Wang should have allowed him to intervene “because this case involves important, novel constitutional questions about the privacy rights incident to artificial intelligence usage—a rapidly developing area of law—and the ability of a magistrate to institute a nationwide mass surveillance program by means of a discovery order in a civil case.”
But Wang disagreed with Hunt that she exceeded her authority in enforcing the order, emphasizing in a footnote that her order cannot be construed as enabling mass surveillance.
“Proposed Intervenor does not explain how a court’s document retention order that directs the preservation, segregation, and retention of certain privately held data by a private company for the limited purposes of litigation is, or could be, a ‘nationwide mass surveillance program,'” Wang wrote. “It is not. The judiciary is not a law enforcement agency.”
However, McSherry warned that “it’s only a matter of time before law enforcement and private litigants start going to OpenAI to try to get chat histories/records about users for all sorts of purposes, just as they do already for search histories, social media posts, etc.” Wang’s order could become a gateway to that future, she said.
Wang rejected Hunt’s motion primarily because “whether the temporary preservation of certain chat output log data that was routinely being deleted by OpenAI throughout the course of this litigation may infringe on purported constitutional and contractual privacy rights of individual consumers that use ChatGPT” was deemed a “collateral issue” that does not directly pertain to the central question of copyright infringement.
Finding that Hunt’s intervention would not contribute “in any way” to “the development of the underlying factual issues in this case,” Wang ruled that Hunt ultimately had no right to intervene.
“None of Proposed Intervenor’s purported ‘novel’ questions are at issue in this copyright infringement action,” Wang wrote. “Even if the Court were to entertain such questions, they would only work to unduly delay the resolution of the legal questions actually at issue.”
OpenAI will have a chance to defend panicked users on June 26, when Wang hears oral arguments over the ChatGPT maker’s concerns about the preservation order.
In his filing, Hunt explained that among his worst fears is that the order will not be blocked and that chat data will be disclosed to news plaintiffs who may be motivated to publicly disseminate the deleted chats.
That could happen if news organizations find evidence of deleted chats they say are likely to contain user attempts to generate full news articles. Ars could not immediately reach a spokesperson for the lead plaintiff in the copyright lawsuit, The New York Times, for comment on this alleged privacy risk for ChatGPT users.
Wang suggested that there is no risk at this time since no chat data has yet been disclosed to the news organizations. That could mean that ChatGPT users may have better luck intervening after chat data is shared, should OpenAI’s fight to block the order this week fail. But that’s likely no comfort to users like Hunt, who worry that OpenAI merely retaining the data—even if it’s never shared with news organizations—could cause severe and irreparable harms.
Some users appear to be questioning how hard OpenAI will fight. In particular, Hunt is worried that OpenAI may not prioritize defending users’ privacy if other concerns—like “financial costs of the case, desire for a quick resolution, and avoiding reputational damage”—are deemed more important, his filing said.
OpenAI did not immediately respond to Ars’ request to comment. The company previously provided a breakdown of affected users and vowed to fight the order.
For now, ChatGPT users must wait to see the fate of their most sensitive chat logs. Intervening ChatGPT users had tried to argue that, at minimum, OpenAI should have been required to directly notify users that their deleted and anonymous chats were being retained. Hunt suggested that it would have stopped him from inputting sensitive data sooner. McSherry told Ars that more transparency will be needed as courts continue tangling with cases impacting chatbot users.
“All AI chat apps should be taking steps not only to ensure that users can delete their records and be sure they are actually erased but also to ensure that users get timely notice of demands for their information,” McSherry said. “If they aren’t already doing so, they should also commit to regular transparency reporting about demands for user data.”
Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don’t need to know everything, only what’s important.
