Tracer unveils AI tool to shield brands from ChatGPT fraud – SecurityBrief New Zealand
Tracer AI has launched a product designed to monitor and protect brands from fraudulent activities that are perpetrated via AI chatbots, focusing on OpenAI's ChatGPT.
The introduction of Tracer Protect for ChatGPT comes as enterprise concern grows about the ways in which generative AI, particularly chatbots, are being exploited to commit brand fraud, distribute counterfeits and conduct executive impersonation scams. Tracer AI’s system is built to track mentions of specific brands, products, services, and executives within ChatGPT responses and to proactively identify and mitigate schemes targeting both businesses and consumers.
Changing search habits
The escalation in generative AI use, including ChatGPT, which reportedly has approximately 400 million weekly active users worldwide, has resulted in consumers rapidly shifting from traditional search engines such as Google to AI chatbots for information and product recommendations. Tracer AI states that this mainstream adoption and the public’s general trust in chatbot answers has opened effective new channels for bad actors to carry out brand abuse campaigns. These can manifest as phishing schemes, fraudulent product recommendations, or the promotion of unsafe counterfeit goods.
Mechanisms of attack
According to Tracer AI, generative engine optimisation (GEO) is being used to promote fraudulent content within chatbot outputs, which often remains hidden from traditional search engines and detection tools. The result is a growing incidence of social engineering, impersonation, and narrative poisoning, in which misleading stories about brands are intentionally promoted to influence consumers and even affect how AI systems respond to future queries regarding those brands.
Tracer’s approach
The system’s architecture features Flora, Tracer’s agentic AI platform, which uses experience-based learning to improve its threat detection and enforcement capabilities over time. The company claims its integration of advanced automation and human analysis allows for continuous monitoring, drastically reducing the time required to identify and respond to brand misuse, and filtering out irrelevant threats.
Collaboration and platform integration
Tracer Protect for ChatGPT is powered by the Universal AI Platform from Dataiku, aiming to deliver fast and accurate brand threat detection at scale within generative AI environments.
The product leverages proprietary Human-in-the-Loop AI (HITL) methodologies, combining algorithmic speed with review by expert analysts to ensure that enforcement decisions are legally robust and aligned with client-specific requirements.
Industry perspectives
Farnell further emphasised the heightened need for advanced digital brand protection, stating, "The urgency to get ahead of this threat cannot be overstated given the increasing ease with which fraudulent content can be generated and the unprecedented consumer shift from using search engines to now using AI chatbots. Organisations must adopt equally advanced countermeasures to protect their digital presence."
"This escalating technological arms race between malicious actors and brand defenders necessitates a proactive and nuanced approach to digital security, which is why we built Tracer Protect for ChatGPT. With our first-of-its-kind brand protection product that actively monitors and analyses ChatGPT outputs to detect and neutralise brand infringements, we're helping enterprises use AI for good and get ahead of these dangerous new threats posing grave risks to brands and their customers."
Expansion plans
Tracer Protect for ChatGPT is the first of several planned products targeting generative AI ecosystems. Tracer AI intends to extend its monitoring and enforcement solutions to other platforms, including Claude, Perplexity, and Gemini, later in the year.
The company highlights its ongoing efforts in proactive brand protection, asserting that live, targeted monitoring of large language model (LLM) outputs represents a shift from reactive to anticipatory security measures. Tracer AI’s continued development in this area is marked by its use of its HITL AI, Flora, and Tracer Graph technologies. The firm has recently been recognised within the technology industry, receiving awards for both product development and leadership.
