What would you pin down in a partner's file-drop convention before accepting daily CSV drops?

## Why this is a leadership question The engineering is not hard. What makes partner file drops fail is that nobody wrote down what a drop *is*, so every ambiguity is discovered in production by someone who cannot fix it. The value you add here is deciding the ambiguities in advance and making them enforceable. Keep the scope clear: this agreement governs **delivery** — where files go, how you know they are complete, what a repeat means, when they are late. The promises about columns, types and business meaning are a separate agreement layered on top; conflating them produces a document nobody owns. ## Location, naming and identity One prefix per partner per dataset, with credentials scoped to exactly that prefix — write-only if the platform supports it. Isolation is cheap now and expensive to retrofit after two partners have shared a bucket. Partition by **arrival** date so a prefix, once closed, never changes and is therefore a stable replay unit. File names should carry the dataset, the window they cover, a run or sequence id, and a part number: `orders_20260820_run03_part-0000.csv.gz`. That lets an operator recognise a re-drop, spot a gap in sequence, and attribute a bad file to a run without opening it. Opaque UUID names are collision-safe and diagnostically useless. ## Atomicity of the upload A 4 GB CSV uploaded directly to its final key is visible-in-progress under some upload paths and, at minimum, invites a loader to start reading a prefix whose contents are still changing. Require the partner to write to a staging prefix and copy to the final key on completion, or to upload under a temporary name and copy. The loader watches only the final prefix. ## Completeness Nothing is read without a completeness signal. A zero-byte marker written last is the minimum. A manifest is better: it names every object, carries per-file row counts and a batch id, and lets you verify what you loaded against what was declared and fail loudly on a short batch. The manifest is also what makes orphan files from a failed partner run inert — you load what it names and ignore the rest. ## Re-drops and corrections This is the clause most often missing and most often costly. Two files exist for 20 August: is the second a correction that replaces the first, or additional rows that supplement it? Pick one, write it down, and make the load a partition overwrite so the semantics are enforced by the code rather than by hope. If corrections are real, require them to restate the full window rather than sending deltas, because a delta with no ordering guarantee is unmergeable. ## Absence must be an event An empty prefix is indistinguishable from a broken partner. Require an explicit zero-row drop with its marker on days with no data, so "nothing happened" is a positive statement. Pair that with an expected-by deadline on your side and an alert when the marker has not appeared — a missing drop must page as loudly as a failed load, and by default it does not. ## The file dialect Since the format is CSV, the delivery agreement has to pin the dialect, because it is a property of the bytes rather than of the schema: delimiter, quote and escape characters, whether quoted fields may contain newlines, character encoding, header row presence, the null token, the decimal separator, line terminator, and the timestamp format with timezone. Configure your reader explicitly against those rather than letting it sniff. Compression and whether it is applied per file belongs here too. ## Operations, security and money Retention on the raw zone, and who may read it. Encryption expectations and whether the drop may contain personal data — the answer determines the bucket's access model, its lifecycle policy and possibly its region. An error channel: when you reject a drop, where does that land for them, and what is the resend procedure? Rate and size expectations, so a partner's one-off 400 GB backfill does not arrive unannounced at 09:00 on a Monday. And a named owner on both sides, with an escalation path. An agreement with no accountable human degrades into folklore within two quarters. ## Make it testable and make it enforced Require a test drop against a staging prefix before go-live, exercising the empty-day case, a re-drop, and a deliberately malformed file. Version the agreement itself, so "which version were you sending in July" is answerable. Then encode as much as you can in the loader: reject files whose names do not match the pattern, refuse prefixes without a marker, compare against manifest counts, quarantine rather than crash on a malformed row, and alert on missing drops. Every rule you leave to human diligence will be violated, usually during someone else's on-call. ## The judgment call How much you can demand depends on leverage. A partner who gains nothing from your pipeline will not build a manifest emitter because you asked nicely. Rank your requirements: atomic upload, a completeness signal and unambiguous re-drop semantics are non-negotiable because without them correctness is impossible; manifests, row counts and format upgrades are worth trading away for adoption. Know which line you are defending before the meeting.