You removed the mining workloads from a metered cloud account, so why does the bill keep climbing?
answer
- the payload is the cheap part
- ask what the operator must redo
- the credential is still valid
- your scaling policy grants the capacity
- a rate, not a fixed cost
basics
~20 sBecause the miner was the product, not the access. The credential and the exposed path that placed it are untouched, so capacity is created again, and the loss accrues until that credential is rotated and the path closed.
solid answer
~50 sMining is a monetisation, not an intrusion. Whatever placed the workloads, an exposed orchestrator API, a leaked deploy credential, a token scraped from an over-permissioned service identity, is still valid, so the same identity simply creates node pools again, sometimes in regions nobody was watching. In an elastic environment your own scaling policy does the heavy lifting: demand appears, capacity is granted, and the account is billed for it. Deleting workloads is the cheapest possible move for the operator to undo. What actually stops the accrual is rotating every credential the workloads ran under and every credential reachable from them, closing the entry path, and constraining what that identity is allowed to create at all. And treat the access as sold: footholds are resold routinely, and the next buyer's objective may not be one that keeps your hosts healthy.
code
text · 7 lines# same principal, same account, one afternoon
14:02 principal=svc-deploy-ci op=CreateNodePool region=eu-west nodes=40 status=OK
14:19 principal=svc-deploy-ci op=CreateNodePool region=us-east nodes=40 status=OK
15:07 principal=svc-deploy-ci op=DeleteWorkload name=xmr-worker status=OK # workloads removed here
15:41 principal=svc-deploy-ci op=CreateNodePool region=ap-south nodes=40 status=OK
15:44 principal=svc-deploy-ci op=CreateWorkload name=sys-metrics-agent status=OK
...go deeper
Understand that removing a malicious workload does not remove whatever was able to create it, and that in a pay-per-use account the cost continues for as long as that ability lasts.
Explain the recurring entry paths, an exposed management API, a leaked deploy credential, an over-permissioned workload identity, and why elastic scaling turns the victim's own capacity policy into the adversary's growth mechanism.
Sequence the work: rotate the identity used and everything reachable from it, close the path, scope the authority down, then use quota as a bound on the rate. Say plainly that the access should be assumed resellable.
Own the framing for the account and platform owners: a valid key is in someone else's hands, and the invoice is the cheapest use they have chosen. Drive who owns identity hygiene and what standing constraints prevent the next one.
## The wrong mental model The instinct that produces this situation is to treat the mining workload as the incident. It is not; it is the visible end of a chain whose valuable part is the access. Ask what the operator would need to redo if you deleted the workloads, and the answer is: one API call. Ask what they would need to redo if you rotated the credential and closed the path they used, and the answer is: the whole intrusion. So the question to answer is not "what is running" but "what is authorised to run, and who holds that authorisation now". ## What is usually still live In a metered cloud account with a container platform, the recurring entry paths are boring and few: - **An orchestrator or management API reachable from the internet**, with weak or absent authentication, or authorisation that lets an anonymous or low-privilege caller create workloads. - **A leaked deploy credential** from a repository, a build system, an image layer or a configuration file, which is valid until somebody revokes it. - **An over-permissioned workload identity** whose token any code running in the cluster can read, and which can create compute far beyond what the application needs. - **A stale identity nobody owns**, left behind by a project that ended, still holding the ability to create infrastructure. Any one of these turns deletion of the workloads into an inconvenience measured in minutes. The audit trail of a real case looks exactly like the fragment above: the same principal creating capacity, then deleting the obvious workload, then creating capacity again, sometimes with names chosen to read like platform components. ## Why elasticity makes it worse On owned hardware, an adversary's mining farm is bounded by the machines they reached. In a metered account it is bounded by your limits, your quota and your willingness to pay, which is often much larger. Two mechanics do the damage: - **Autoscaling.** Extra demand is a request for capacity, and the platform is built to satisfy it. Your scaling policy becomes the mechanism that grows the adversary's farm. - **Regional spread.** Quota and attention are usually organised around the regions you use. Creating capacity in a region you have never deployed to costs the operator nothing and takes the workload out of the place anyone is looking. This is why the loss in a metered account behaves differently from on-premises mining: it is a rate, not a fixed cost, and it keeps integrating for as long as the access is live. ## What actually stops the accrual In rough order of effect: 1. **Rotate the credential the workloads ran under**, and every credential reachable from where they ran, on the assumption that anything the compromised identity could read has been taken. 2. **Close the entry path.** Take the management API off the public internet or put real authentication in front of it; fix the leak that published the credential. 3. **Constrain the identity.** A deploy identity that can create forty-node pools in every region has more authority than its job requires; scoping that authority is what makes a repeat expensive rather than free. 4. **Bound the blast radius with quota and budget guardrails**, understanding that these limit the invoice rather than the access. 5. **Assume resale.** Access brokerage is a real market, and mining is at the bottom of it. The presence of a low-value objective says the access has not yet found its best buyer. ## The sentence that lands with the platform owner When you explain this to whoever acts on the invoice, avoid the vocabulary of payloads. The useful framing is: *somebody else currently holds a valid key to this account, and this month's bill is only the cheapest thing they have chosen to do with it.* That is what makes credential rotation and closing the path read as urgent rather than as tidying up after a nuisance.
- Which credential do you rotate first when several could have placed the workloads?The identity the workloads actually ran under, because it is the one you can prove was used, then anything that identity could read from where it ran: mounted tokens, instance metadata credentials, secrets in the same namespace. Ordering matters less than completeness, since leaving one valid credential means the access survives the whole exercise.
- Does a hard quota on compute solve this?It bounds the invoice, which is worth having, but it leaves the access intact and can starve your own production first. Treat it as a guardrail against the rate of loss, never as the fix. The access remains valid and remains sellable to somebody whose objective is not mining.
- Why would the operator create capacity in a region you never use?Because attention and quota tend to follow deployment footprint. An unused region has spare quota, no baseline anyone knows, and nobody reviewing what appears in it. It costs the operator one parameter and buys them a longer run at your expense.
saying these in an interview costs you the question
- Treats deleting the workload as the fix
- Grades severity by the payload rather than the access
- Rotates nothing because no data appeared to be taken
- Assumes the farm is bounded by hosts already compromised
- Relies on a budget cap while the credential stays valid
- Forgets that footholds are resold to different objectives