Your logs show a few thousand tightly clustered full-precision queries to a small risk regressor — did someone solve for its weights or merely sample it?
answer
- volume is the wrong axis here
- look at the step size between queries
- did they cover the domain or thread through it
- count the digits you returned
- the log bounds material, not conclusions
basics
~10 sThe query pattern separates them: a solve probes tight groups of near-identical inputs and needs the full numeric reply, while a harvest is large, broad and tolerant of rounding. Logs bound material, not conclusions.
solid answer
~50 sTwo things distinguish a parameter solve from someone building a training set. A solve walks in tiny steps: many near-identical inputs differing minutely in one feature, revisiting narrow neighbourhoods, total volume in the thousands, no interest in covering the input distribution — and it is worthless without the low-order digits of the reply. A harvest for a fitted stand-in is the opposite: large, spread broadly across plausible inputs, happy with rounded answers. So the clustered, full-precision, low-volume pattern is consistent with a solve, and that hypothesis is only live if the model is small and piecewise-linear. Then state what you do not know: the log records what they could ask and what you returned, never what they computed offline, and careful sensitivity testing by a customer looks similar. The honest finding is an exposure bound plus a hypothesis.
go deeper
Know that a small number of very similar queries can matter more than a huge number of varied ones, and that the number of digits in each reply is part of the evidence.
Be ready to contrast the two traffic footprints: broad and large for fitting a copy, tightly clustered and small for solving for parameters, with precision as the discriminator.
Show you check the served model's size, structure and reply width before believing the hypothesis, and that you write the finding as an exposure bound with the offline step explicitly unknown.
Own the answer given upward: state protection as cost and precision rather than impossibility, and name what would move that line for the models you actually ship.
## The chair you are sitting in You are the incident responder reconstructing from access logs what somebody actually walked away with. The target is a small equipment-failure-risk regressor served to industrial customers over turbine and pump telemetry: engineered sensor features in, one real-valued risk score out. One account has made a few thousand calls in a pattern that looks odd. The question your organisation wants answered is not 'was there abuse' but **what does that account now hold**. ## Two hypotheses that leave different footprints **Hypothesis A — they were building a training set** for a fitted stand-in. The footprint of that is volume and breadth: many calls, inputs spread across the range of plausible telemetry, coverage of the space that the copy needs to be good over. The attacker is indifferent to how precisely each score is reported, because rounding a supervision target barely hurts. Volume tends to be large, and often spread over more than one identity. **Hypothesis B — they were solving for parameters.** The footprint is the opposite in nearly every dimension. Total volume is modest — thousands, not millions, and unremarkable against ordinary usage. The inputs come in tight families: near-identical points that differ minutely in one feature, walking along a line, revisiting narrow neighbourhoods repeatedly and then jumping elsewhere. There is no attempt to cover the input distribution, because the attacker is not trying to learn what the model predicts on realistic data; they are trying to localise where the computed function bends. And the whole exercise is worthless without the low-order digits of each returned number. So the discriminating observations are: **step size between related queries, whether the input set covers the domain or clusters in threads, total volume, and how many digits you actually returned.** ## The precondition you check before you believe hypothesis B The solve is only on the table for a model that is small and built from piecewise-linear units, and where the reply carries many significant digits. Check those three facts about the served model before you write anything down. If the served model is large or deep, the same clustered pattern is much more likely to be something else, because the query and precision cost of a solve climbs steeply with width and depth. That check is what turns a suspicious pattern into a supportable hypothesis or dismisses it. ## Say precisely what the log can and cannot establish This is the part that separates a senior answer from a plausible one. Direction of claim matters: - The log establishes **what was asked and what you returned**. That is an upper bound on the material available to them. - It does **not** establish what they computed from it. Solving happens offline, on data you already handed over; nothing about success or failure comes back through your service. - A clean-looking log does **not** establish that nothing was taken. The volume this attack needs is small enough to be unremarkable, and it can be spread across sessions or identities. - The pattern is **not** unique to an attack. A careful customer doing sensitivity analysis — nudging one sensor feature to see how the risk score responds — produces something that can look similar. Attribution needs more than the shape of the traffic. ## What the finding should say Write the exposure, not a verdict. Something with the shape of: this account received N replies at full numeric precision against a model of this size and structure; that combination is sufficient material for a parameter-recovery attempt, which if successful yields a functionally equivalent model, identical up to permutation and rescaling of hidden units; the traffic pattern is consistent with such an attempt and also with legitimate sensitivity testing; we cannot determine from our side whether the solve was performed or succeeded. And then the forward-looking part, framed honestly: the protection for anything of this size and reply width is **cost and precision, not secrecy**. If somebody asks 'can they get our weights', the correct answer is not 'never' — it is a statement about what it would cost against this specific model, with the awareness that a smaller shipped model or a wider numeric contract moves that line. ## The trap The common failure is to reason from volume alone: a few thousand calls looks like nothing next to the millions people associate with model stealing, so the incident gets closed. Volume is the wrong axis for this attack. The axis is how much of each answer went out, and against how small a model.
- The account made only a few thousand calls. Why is that not reassuring on its own?Because a parameter solve against a small model is cheap in calls and expensive in digits. Its cost lives in the precision of each reply and in the model's size, not in volume, so it is deliberately unremarkable in any usage view. Judging this incident by call count applies the yardstick of a distillation harvest to an attack that does not need one, and closes a real finding as noise.
- A legitimate customer running sensitivity analysis produces similar-looking traffic. How does that change your finding?It stops you writing a verdict. The traffic shape supports a hypothesis, not attribution, so the finding should state the exposure — this account received this many full-precision replies from a model of this size — and list both explanations. Anything stronger needs evidence outside the log, such as account context or a pattern the customer's stated use cannot account for.
- Leadership asks whether the attacker now has your model. What do you tell them?That the log bounds the material they were given and cannot show what they derived from it, because the solve runs offline. If the model is small, piecewise-linear and answered at full precision, a functionally equivalent copy is a live possibility, identical up to permutation and rescaling of units. State it as an exposure with an explicit unknown rather than a yes or a no, and note that the same claim would be much weaker for a larger model.
saying these in an interview costs you the question
- Closes the incident because the call volume was low
- Concludes from the log that the solve succeeded
- Treats a clean log as proof nothing was extracted
- Ignores how many digits of the score were returned
- Attributes the pattern to an attacker without other evidence
- Assumes only huge harvests can steal a model