A five-month experiment by security researcher HD Moore has put hard evidence behind a worry many AI teams had only guessed at: prompts sent to AI services do not always stay where customers think they do. Moore, chief research officer at Dragos and the creator of Metasploit, seeded roughly two million prompts with unique, trackable fake secrets and watched where they turned up. The answer: inside at least three breached AI hosting providers, in the hands of marketplace resellers, and on the servers of companies that ignored Zero Data Retention flags.
The findings, published by Dragos on November 19 and widely covered after an Ars Technica report by Dan Goodin, are the clearest picture yet of how AI prompt leaks happen across the routers, resellers and hosts that sit between an app and a model.
Key takeaways
- Dragos sent about 2 million prompts carrying unique canary credentials and URLs to AI services between July and November 2026.
- Attackers were stealing data from at least three hosting providers. None knew until Dragos told them.
- Some inference marketplace sellers simply turned on request logging and read customer prompts.
- One Chinese AI lab fetched secret URLs from more than half of the prompts sent to its models, including roughly two-thirds of requests marked “no data collection” or Zero Data Retention.
- Dragos did not publicly name the providers involved. Its advice: pick your upstream providers yourself, avoid reseller marketplaces and plant your own canaries.
What happened
A canary token is a secret that exists in only one place. If anyone ever uses it, you know that place was exposed. Moore’s team began in July with free Thinkst Canarytokens, then built a dedicated service that generated a fresh AWS key, DNS name, URL and other credentials for every single prompt. The AWS keys were real keys tied to empty, fully logged accounts. Because each secret maps back to one prompt, one model, one router and one privacy setting, any reuse points to the exact hop that leaked it.
To see who was really handling requests, the team also built what Dragos describes as a traceroute for AI inference: image URLs that reveal every system fetching them, plus slightly broken requests whose nested error messages expose each layer in the chain. Across about 2,000 traced routes, several hundred included a system Dragos could not identify, and a handful sent data to companies that were not on the advertised path at all.
Three breached providers
The most serious finding was direct theft. Attackers were copying traffic from three separate hosting providers, with the earliest incident dating to mid-July. In the largest breach, every request Dragos sent during a few windows in late October was stolen, including a couple hundred AWS keys. Some came from requests the provider rejected before any model ran, which suggests the attacker was copying traffic at the front door rather than deep in the inference stack.
Once they had the keys, attackers tested them, sometimes within hours, then listed S3 buckets, searched Secrets Manager and Parameter Store, tried to enable models in AWS Bedrock and added their own IAM users. The bait keys were used for tens of thousands of AWS API calls. One adversary appeared to have compromised two providers at once and, on November 1, added ARTEX, the LLM-driven attack framework we covered in our report on the ARTEX AI cyberattack on the European Commission. In the other two breaches, some stolen requests carried the ZDR flag. As Dragos puts it, a privacy flag does not help when the attacker is already inside.
Resellers that read your prompts
The second finding needed no exploit. On open inference marketplaces, the cheapest seller usually wins the request. Some sellers listed popular models at low prices, enabled logging and waited. Dragos traced individuals on home broadband connections opening secret URLs and typing stolen SSH passwords by hand. Prompts for the same model sent through a different seller were never touched, so the leak followed specific sellers, not the marketplace as a whole. One router was also quietly forwarding part of its traffic into that marketplace, an undisclosed hop it says it has since stopped.
Privacy flags that were ignored
The third finding is about companies that were never meant to keep prompts at all. A Chinese AI lab’s systems fetched the secret URLs from more than half of the prompts sent to its models between July and October, usually two or three days later. That included roughly two-thirds of requests marked “no data collection” or ZDR, and prompts sent to U.S. hosts that resell the lab’s models. In one case, a secret link appeared in a Feishu (Lark) company chat 24 days after the request was sent with the ZDR flag, and was opened from desktops and phones in Beijing and Shanghai within two minutes.
Large U.S. providers were not clean either. One kept crawling a secret URL more than two months after a single call to its search endpoint. Another runs a scraper that returns to prompt URLs days after each request.
Why it matters
Prompts are often more valuable than the GPU time used to answer them. They hold source code, internal plans, customer data and, increasingly, credentials, because AI agents will happily send whatever they can read. Routers make it easy to switch models and cut costs, but every hop decrypts and sees the full prompt and response. Dragos’s core point is blunt: a privacy setting only binds the first hop. After that, you are trusting every router, reseller and host you never chose.
This also changes how to read routing news. When Hugging Face moved its LLM calls onto OpenRouter, the debate centred on convenience and provider keys. The Dragos data shows the bigger question is where a request actually lands and who can log it there.
What it means for developers and businesses
Dragos says none of this means teams should stop using AI services, but where prompts go has to be a decision rather than whatever the lowest price picks. Its recommendations, in practical terms:
- Self-host the most sensitive work if you can afford the hardware. It is the only option where nobody else reads your prompts.
- Go direct to a first-tier provider for everything else: one contract, one privacy policy, the smallest exposure.
- Use routers carefully. Choose upstream providers yourself instead of default pools, check that privacy settings really change routing, and stay away from reseller marketplaces.
- Seed your own canaries. Put a fake credential in the context your agents and pipelines already send, and treat any use of it as an incident.
- Turn on secret filtering where providers offer it, knowing it only catches credential-shaped data.
The simplest step costs nothing: keep real secrets out of prompts. Agents that read config files, terminals or repositories should run with scoped, short-lived credentials so a leaked prompt does not become a leaked cloud account.
What to watch next
Dragos says it is working with about a dozen providers and expects that number to grow. One breached provider has published a preliminary security notice; the provider breached in July had still not disclosed its incident. Watch for named disclosures, for routers and aggregators dropping hosts, and for whether ZDR becomes something buyers can verify rather than a checkbox they have to trust. Meanwhile, Dragos reports that stolen keys and secret URLs from its test were still being accessed months later.