skip to content

In a Go repo, internal/util is imported by every package and keeps growing — how do you split it?

level: seniorimportance: should knowfreq 40%

answer

  1. count callers before choosing a home
  2. most helpers have exactly one
  3. the bottom of the import graph is expensive
  4. cache keys include everything you import
  5. a name that promises nothing cannot be reviewed

basics

~20 s

Inventory what is in it and who calls each item, then move each cluster into its own internal package named for what it does. Helpers with one caller move into that caller, unexported; dead ones are deleted.

solid answer

~50 s

I start by listing what is in `internal/util` and who calls each item, because the answer is usually three or four unrelated clusters plus a long tail of one-caller helpers and dead code. Each cluster becomes its own package — `internal/retry`, `internal/clock`, `internal/pathglob` — named for what it holds. Anything with exactly one caller moves into that caller as an unexported function; anything with none is deleted. I do it as a sequence of mechanical import-rewriting commits, one cluster at a time, so each stays reviewable. The reason to bother is concrete in Go: a package at the bottom of the import graph is a dependency of every binary under `cmd/`, and because the build and test caches key on everything a package imports, one edit there invalidates the cached results of the whole repository. To stop it re-forming I add a `go list -deps` check in CI and make the default that a helper starts unexported in its only caller.

code

text · 10 lines
text
before:
	internal/util/  retry.go clock.go glob.go stringsx.go misc.go
	                imported by 40 packages and all 5 binaries

after:
	internal/retry/     many callers, one concept
	internal/clock/     many callers, one concept
	internal/pathglob/  many callers, one concept
	internal/store/     absorbed its single-caller helpers, unexported
	                    misc.go deleted: no callers

go deeper

for a junior

Know why a package called util is a warning sign: its name does not say what is in it, so nothing can be judged as belonging there or not.

for a middle

Explain the mechanics — a directory is a package, so splitting is moving files and rewriting imports — and that a helper with one caller belongs inside that caller rather than in a shared package.

for a senior

Show the sequencing and the evidence: inventory callers first, one cluster per reviewable commit, keep the build green throughout, and be able to name the concrete costs of a package that every binary imports.

for a principal

Own the prevention rather than the cleanup — the review default for where a new helper lives, the CI check on the dependency graph, and the judgement call about how much of a quarter this refactor is worth.

## Why a grab-bag package is expensive in Go specifically Cohesion arguments apply in any language, but Go adds mechanics that make an `internal/util` imported by everything measurably costly. **It sits at the bottom of the import graph.** Every package imports it, so it is a transitive dependency of every binary under `cmd/`. Whatever `internal/util` imports, all five binaries link and initialise — including packages that only one helper needed. **It defeats caching.** Go's build cache and test cache are keyed per package, on the package's own contents plus the contents of everything it imports. A package at the bottom of the graph is an input to every other package's cache key, so a one-line edit in `internal/util` invalidates the cached build output and the cached test results of the entire repository. Teams feel this as a CI run that never gets faster. **Nobody can review it.** A package's name is the promise a reviewer checks a change against. `util` promises nothing, so no addition can ever be judged as out of place, which is exactly why it grows without limit. **Its call sites read badly.** In Go the package name is part of every identifier at the call site. `util.Retry(ctx, f)` tells the reader nothing about what layer it belongs to; `retry.Do(ctx, f)` does. ## The inventory Before moving anything, produce two lists: 1. What is in the package — each exported symbol, grouped by the concept it belongs to. Four or five clusters normally fall out immediately: time and clocks, retries and backoff, path and glob handling, string or slice conversions, and a residue of one-off wrappers. 2. Who calls each symbol, and how many distinct packages do. `go list -deps ./cmd/...` shows you what each binary actually drags in, and a repository-wide search for `util.` gives the caller counts. The caller count decides the destination more than the topic does. ## The three destinations **Many callers, one clear concept -> its own package.** `internal/retry`, `internal/clock`, `internal/pathglob`. Name it for the thing, not for the layer that happens to use it, and keep it small enough that a reviewer can hold the whole promise in their head. **Exactly one caller -> into that caller, unexported.** This is the destination people forget, and it is usually the largest bucket. A helper used only by `internal/store` is not shared code at all; moved into `internal/store` as an unexported function it stops being anyone else's dependency, stops appearing in anyone's cache key, and stops being a candidate for accidental reuse. **No callers -> delete it.** Grab-bags accumulate helpers written for a caller that was later rewritten. Deleting them is free and it is the fastest way to shrink the problem. ## Doing it without a big-bang refactor Move one cluster per commit: create the new package, move the files, rewrite the imports mechanically, run `go build ./...` and `go test ./...`, and stop. Each commit is small, reviewable and bisectable, and the repository is healthy at every step. Resist the urge to also improve the code you are moving — a move commit that also changes behaviour is unreviewable, and any incident afterwards will point at it. Leaving `internal/util` in place, shrinking, for a few weeks is fine; it is much better than one 400-file commit. The order that works is smallest cluster first: it proves the mechanics and it gets the caller list shorter, so later moves are easier to reason about. ## Keeping it from re-forming The package comes back unless something replaces the habit that created it. - **A default.** A new helper starts unexported in the package that needs it. It gets promoted to its own package on the second real caller, not on the first speculative one. - **A CI check.** `go list -deps` over the binaries, or over a package you care about, asserted against an expected set, catches both the return of a universal dependency and unwanted new edges in the graph. - **A naming rule at review.** If a proposed package name does not say what is in it — `util`, `common`, `helpers`, `misc`, `base` — the change does not merge until it does. This is cheap and it is the check that actually holds. ## What an interviewer is listening for That you inventory before you move; that caller count, not taste, decides where each item goes; that the single-caller-unexported destination exists at all; that the migration is incremental; and that you name a mechanism, not a promise, for preventing the regression.

  • How do you decide between a new package and an unexported helper in the calling package?
    By caller count. Two or more packages that genuinely need it, and a concept you can name, justify a package. One caller means it belongs inside that caller as an unexported function — it is not shared code, and making it importable only invites a second, worse dependency later.
  • The team objects that the repository will end up with too many small packages. What is your answer?
    Small packages are cheap in Go: a package is a directory, imports are explicit, and an unused one is a compile error. The cost that actually hurts is a package everything imports, because it inflates every binary's dependency set and invalidates every cached build and test result when it changes.
  • How would you prove the split made a difference?
    Compare `go list -deps ./cmd/<binary>` before and after — each binary's transitive dependency set should shrink — and watch CI: edits that used to invalidate the whole repository's build and test cache should now touch a handful of packages.
  • Is it acceptable to leave internal/util in place while the split is in progress?
    Yes, and it is usually the right call. Move one cluster per commit, keep the repository green at every step, and let the old package shrink to nothing before deleting it. A single enormous refactor commit is unreviewable and makes any later bisect useless.

saying these in an interview costs you the question

  • Rewrites the helpers while moving them, in one huge commit
  • Splits by layer or by team instead of by responsibility
  • Never considers moving a single-caller helper into its caller
  • Renames util to common or helpers and calls it fixed
  • Argues small packages are inherently bad in Go
  • Has no mechanism to stop the grab-bag re-forming