File & Object Storage

Chunking plus content addressing stores identical bytes once, lets an edited file re-upload only the chunks whose hashes changed, and turns each chunk's key into an integrity check on read.

A server-side change journal records every change in order with a monotonic position. Each device stores the last position it applied, so one small request returns exactly what it missed instead of scanning and diffing the whole file tree.

Proxying gigabytes pins app servers to slow byte streams for minutes. Instead the app hands out a short-lived signed upload URL, the phone sends bytes directly to the object store, and app servers handle only small control requests.

Fixed-size blocks cut at fixed offsets, so an insert shifts every later boundary and hash. Content-defined chunking cuts where a rolling hash of nearby bytes matches a pattern, so boundaries follow the content and realign after the edit.

Each upload names the base revision it was edited from, and the server commits only if that still matches the current revision. A losing upload is kept as a conflicted copy beside the original unless the format allows a safe automatic merge.

The file is split into numbered parts uploaded independently under one session; each part is retried on its own, the client resumes by asking which parts already landed, and a final complete call assembles them into one object.

Return the folder's children in capped pages through a cursor: read a sorted index on parent ID and name, and resume after the last name returned. Never load the whole folder, use deep offsets, or read blobs to build a listing.

Storing full paths on every entry makes a folder rename rewrite all million descendant rows, a huge write that is hard to make atomic. Storing parent-id pointers makes it a one-row update, but path resolution then costs one lookup per level.

Three-way replication costs 3x raw storage and survives two lost copies, with simple reads and repairs. A Reed-Solomon (6,3) code costs 1.5x and survives any three lost fragments, but reads and repairs must touch several nodes and decode.

Splitting notification from fetch keeps the long-poll tier cheap: it holds many idle connections and only says something changed, while the device reads the actual entries from its cursor through the normal paginated, authorized change-list call.

If the client skips uploading content any user already stored, the skipped upload reveals that someone else holds that exact file, letting an attacker confirm a document exists or brute-force its secret fields.

An upload can skip a chunk the server says it already holds while the collector deletes that zero-reference chunk; the committed manifest then points at missing bytes. A deletion grace period plus a locked check at commit closes the gap.

Preserve the edit: restore the edited file, recreating its parent path, rather than dropping it, because losing work is worse than an unexpected file reappearing. Record the restore in the journal so every device sees it.

Committing first lists files whose bytes are missing; storing bytes with no record leaves untracked orphans. A pending record before upload, then a verified, conditional flip to committed afterwards, keeps listings truthful and leftovers findable.

Uploads land in a private quarantine location; the completion event queues scan jobs, and only a pass marks the record published and exposes the object for serving. Failures are rejected, and nothing is served before a verdict.

Smaller chunks find more duplicate data and shrink uploads after small edits, but multiply index entries, manifest size and requests; larger chunks do the reverse. Choose by replaying real workload data at several sizes and picking the knee.

Keep one journal per shared folder and give each device a cursor per folder it can see. Copying each change into 50,000 personal journals makes every write a huge fan-out and makes joins and permission changes expensive.

The client hashes each part and the whole file; the store verifies every part on arrival and rejects mismatches, and after assembly the backend checks the whole-file hash before committing, so corruption between disk and store is caught.

The server rejects the expired cursor with a distinct reset signal, and the device performs a full resync: list the current tree with a fresh cursor, reconcile it against local files without discarding unsynced local work, then continue incrementally.

Drive tiering from access data in the metadata database: move blobs unread for a measured period to cheaper, slower tiers, expire old versions and trash on schedule, and weigh per-object, minimum-duration and retrieval costs against the savings.