A .NET service's memory climbs for an hour; when do you take a dotnet-gcdump, when a dotnet-dump, and how do you find what keeps objects alive?
answer
- managed growth or native growth first
- object graph versus whole memory
- one tool forces a full collection
- dumpheap -stat, then gcroot
- compare two snapshots in time
basics
~20 sdotnet-gcdump captures the managed object graph over EventPipe but forces a full gen 2 collection; dotnet-dump writes a process dump analyzed with SOS. Compare type counts between two snapshots, then run gcroot on an instance of the growing type.
solid answer
~50 sFirst confirm with counters that the managed heap measured after collections is what grows. `dotnet-gcdump collect -p <pid>` triggers a gen 2 collection, walks the heap through EventPipe events and writes a small `.gcdump` of types, counts, sizes and references; take one now and one later, compare them in Visual Studio or PerfView, or print `dotnet-gcdump report`. The induced full collection can pause a large heap for a long time. `dotnet-dump collect -p <pid>` writes a full dump by default (`--type Heap` is smaller), which can be as large as the memory the process uses and may push a memory-limited container over its limit; `dotnet-dump analyze` then runs SOS: `dumpheap -stat` for the biggest types, `dumpheap -mt` for instances, and `gcroot <address>` for the chain from a root to the object. Dumps contain whatever was in memory, secrets included.
code
bash · 11 lines# two object-graph snapshots, 30 minutes apart (each forces a gen 2 collection)
dotnet-gcdump collect -p 4127 -o before.gcdump
dotnet-gcdump collect -p 4127 -o after.gcdump
dotnet-gcdump report after.gcdump
# a process dump without mapped images, then SOS analysis
dotnet-dump collect -p 4127 --type Heap -o /dumps/pricing.dmp
dotnet-dump analyze /dumps/pricing.dmp
> dumpheap -stat
> dumpheap -mt 00007f6c20a67498
> gcroot 00007f6ad09421f8go deeper
Know that dotnet-gcdump and dotnet-dump both capture what is in the managed heap, and that gcroot shows why an object is still alive.
Explain the difference between an object-graph gcdump and a process dump, and read dumpheap -stat output.
Choose the tool by cost: the induced gen 2 pause, the dump size against the container limit, the secrets inside, and compare two snapshots before naming a leak.
Set the rules for capturing dumps in production: who may take them, where they are stored, how long they are kept, and how personal data is protected.
## Start with the right question "Memory climbs" can mean the **managed heap** grows (objects the garbage collector cannot free because something still references them) or the **process** grows while the heap does not (native allocations, thread stacks, mapped files). `dotnet-counters` separates the two: compare `dotnet.gc.last_collection.heap.size`, measured after each collection, with `dotnet.process.memory.working_set`. Both tools below are about the managed heap; if only the working set grows, a managed heap snapshot will look healthy. ## dotnet-gcdump: the managed object graph `dotnet-gcdump collect -p <pid>` asks the runtime, over the diagnostic port and EventPipe, to **trigger a garbage collection** and emit heap-snapshot events describing every live object and reference. The tool rebuilds the object graph from those events and writes a `.gcdump` file. It captures: - types with instance counts and sizes; - references between objects, so you can ask what still holds a type. It does **not** capture field values or strings' contents, and it is much smaller than a process dump. The cost is the collection itself: the docs warn that walking the heap triggers a **generation 2 (full) collection that can suspend the runtime for a long time** when the heap is large. On a latency-sensitive process with a multi-gigabyte heap, that is a visible pause. The tool gives up after `--timeout` seconds (30 by default). `.gcdump` files open in Visual Studio and PerfView on Windows, where several snapshots can be compared side by side; `dotnet-gcdump report <file>` prints heap statistics (its `HeapStat` report) to the console. ## dotnet-dump: the whole process `dotnet-dump collect -p <pid>` writes a process dump without a native debugger. `--type` chooses its scope: | Type | Contents | |---|---| | `Full` (default) | all memory, including module images | | `Heap` | modules, threads, stacks, exceptions, handles, all memory except mapped images | | `Mini` | modules, threads, exceptions and stacks | | `Triage` | like `Mini`, with personal data removed | A full or heap dump can be as large as the memory the process uses, and the docs warn that collecting one can make the OS page in a lot of memory: in a container with a memory limit, that can get the container killed. A dump holds everything in memory, so treat it like a secret. `dotnet-dump analyze <file>` opens an SOS prompt, and the leak hunt usually follows these steps: 1. `dumpheap -stat` lists types with count and total size, sorted by size. 2. `dumpheap -mt <method-table>` lists instances of the suspicious type. 3. `gcroot <address>` prints the reference chain from a GC root (a static, a stack slot, a handle) to that instance. 4. `dumpheap -min 100000 -stat` narrows the view to large objects, and `dumpheap -type MyCompany.Pricing -stat` narrows it to types whose names match your own namespace. 5. `clrstack -all` and `threadpool` show what threads were doing when the dump was taken, useful when the growth comes from work piling up rather than a cache. `dotnet-dump` is not a native debugger: native stack frames are not shown, so native memory growth needs other tools. ## Choosing between them | Need | Tool | |---|---| | which types grow, from a running process, cheaply | `dotnet-gcdump`, twice | | object field values, strings, thread stacks, exceptions | `dotnet-dump` | | a crash or hang to inspect later | `dotnet-dump` | | no full-collection pause on a huge heap | `dotnet-dump`, accepting the file size | ## Reading the result - **Compare snapshots.** One snapshot shows what is big; two, taken some time apart under the same load, show what grows. A type whose count rises between them is the lead. - **Follow the root, not the type.** `gcroot` usually ends at something long-lived: a static dictionary used as a cache, an event handler subscribed by a short-lived object, a timer callback. The fix is to remove or bound that reference. - **Big is not leaking.** A large cache of fixed size is a design choice; a cache whose count only grows is the leak. The ideas behind retained size and reachability are taught with memory management in general; here the point is which .NET tool shows them and what each costs the running process.
- In dotnet-dump, what does gcroot showing a static field at the top of the chain usually mean?A static field lives as long as its type is loaded, normally the whole process, so everything reachable from it never becomes garbage. Typical cases are a static `Dictionary` used as a cache without eviction or a static event other objects subscribe to and never leave. The fix is bounding the cache, using an eviction policy or unsubscribing, not tuning the GC.
- Why can taking a full dump of a container that is already near its memory limit make things worse?Writing a full or heap dump makes the operating system page in much of the process's virtual memory. In a container with an enforced limit, that extra usage can cross the limit and get the container terminated, losing the dump and the process. The docs suggest checking the limit first, or raising it temporarily, before collecting.
- dotnet-gcdump is free to run on any production process, whatever its heap size.
- The biggest type in dumpheap -stat is always the one leaking.
- A heap snapshot will reveal native memory growth too.
- Dumps are harmless diagnostic files that can be shared freely.
- One snapshot is enough to tell what is growing.