Paper examines the limits of query monitoring against distributed model extraction

What happened on 2026-10-09

At 06:37 UTC on 2026-10-09, arXiv received v1 of "The Limits of Query Monitoring against Distributed Model Extraction" (2610.13083, by Snigdha Chandan Khilar). According to the abstract, the paper studies distillation campaigns spread across thousands of accounts that each look ordinary; when network and payment metadata cannot link the accounts, only the queries remain as evidence. The work used one million real chat turns for development and 17 later weeks for a single frozen evaluation. The core finding is that detectability depends mostly on benign coordination among ordinary users: a campaign is visible only if it is more distinctive than the most distinctive group of ordinary users that week. A calibrated monitor caught naive distributed extraction in 0.86 to 1.00 of weeks. In the author's testbed, campaigns that evaded every detector retained at most 42 percent of a detectable campaign's capability. The abstract also lists limits: slow ramp-ups poison calibration, benign evaluation pipelines look like distillation, no deployable detector was found at colluder shares of 2.5 percent or below, and campaigns relaying real users' requests may be invisible to any query-only monitor.

What it means for a defender's threat model

For a growing number of independent developers, the core logic no longer lives in the client but in server-side models, prompts and processing pipelines. This paper is a reminder that moving capability to the server solves the problem of binaries being analyzed but opens another cloning path: copying the capability through large numbers of ordinary-looking queries. Per-account rate limits tend to fail against distributed campaigns, and query monitoring has clear detection floors of its own. A pragmatic reading is that detection mainly lowers the value of what can be copied and buys time; it does not guarantee discovery. Note the caveats: the data comes from a chat-style service, the work has a single author and a single testbed, and small apps have different traffic volumes and user populations, so the conclusions cannot be transferred directly.

Checks on your own service

If your product offers AI capability through an API, review a few areas. First, extend rate limits and quotas beyond the per-account level to dimensions such as paying entity, device attestation results and organization, while keeping a whitelisting process for legitimate bulk-evaluation customers. Second, record enough aggregate metrics, rather than raw sensitive content, to analyze unusual growth after the fact, within your privacy obligations. Third, review what you return: unnecessary confidence scores, full intermediate results or detailed reasoning raise the value of distillation, so trim them where they are not needed. Fourth, state in your terms of service that outputs may not be used to train competing models, as a constraint alongside technical measures. Finally, accept that detection has a floor, and differentiate on data, update speed and service quality.

Your checklist

• Extend rate limits and quotas beyond per-account, with a path for legitimate bulk users
• Trim unnecessary confidence scores and intermediate results from API responses
• State in your terms that outputs may not train competing models