When would you store a dataset in HBase rather than as a Hive table on HDFS?

## Same storage, opposite access paths Both options ultimately put bytes on HDFS. The difference is what sits between the query and those bytes. ## What a Hive table on HDFS is A directory tree of large files — typically ORC or Parquet — described by a metastore entry. A query engine prunes partitions by directory, then reads files sequentially, exploiting columnar layout and predicate pushdown. There is no per-row index and no random write path: files are written whole, and "updating a row" classically means rewriting a partition. Hive 3 adds ACID `UPDATE`/`DELETE`/`MERGE` on managed transactional tables, but the implementation writes delta files that compaction later merges — it is designed for batch-scale corrections, not for thousands of point updates per second. Strengths: scans, aggregations, joins, SQL, cheap storage, many engines reading the same files. Weaknesses: single-row lookup means scanning; latency is seconds at best; frequent small mutations produce file explosions. ## What HBase is HBase is a distributed sorted map: `(row key, column family, column qualifier, timestamp) → value`. The design consequences matter more than the data model: - **Row keys are sorted bytes.** All rows are kept in lexicographic order, which makes point lookups and contiguous range scans on the key cheap, and makes anything *not* keyed by the row key a full scan. - **Regions and RegionServers.** The key space is cut into contiguous ranges called regions; each region is served by exactly one RegionServer, and regions split as they grow. HMaster handles assignment and balancing; ZooKeeper tracks live servers and the location of the meta table. - **Write path.** A write appends to the write-ahead log (WAL) for durability, then lands in an in-memory MemStore. When the MemStore fills it flushes to an immutable HFile on HDFS. Reads merge MemStore and HFiles, aided by block cache and Bloom filters. - **Compaction.** Minor compaction merges some HFiles; major compaction rewrites all files in a region and physically drops deleted and expired cells. This is the price of the write-optimized design and needs scheduling on busy clusters. - **Column families** are the physical grouping — each family is stored separately, so families should be few and should share access patterns. Strengths: millisecond point reads and writes, very high write throughput, sparse wide rows, versioning by timestamp, in-place updates, TTL per family. Weaknesses: no SQL and no joins natively (Apache Phoenix layers SQL on top), no secondary indexes out of the box, a full table scan is far slower than reading ORC, and everything hinges on the row key you chose on day one. ## The decision Ask what the *dominant* query is. **HBase fits** when requests arrive as "give me the record for this key" or "give me this key's events between two timestamps": user profiles, device or sensor state keyed by device id, a serving store behind an API, message or event stores accessed by entity, and workloads with heavy random updates. **A Hive table fits** when requests are "aggregate everything for last quarter grouped by region": reporting, ad-hoc analytics, feature computation over history, anything expressed naturally in SQL joins. **Very often the answer is both.** The classic Hadoop-era architecture computes in batch over Hive tables and publishes the results into HBase for low-latency serving. Choosing one to do both jobs is the mistake: HBase as an analytics store means scanning a key-value engine, and a Hive table as a serving store means seconds-long lookups. ## The row key is the whole design Because regions are contiguous key ranges, a monotonically increasing row key — a timestamp, a sequence number — sends every new write to the single region holding the highest range, so one RegionServer saturates while the rest idle. The standard remedies are salting (prefix with a hash bucket), hashing or reversing the leading component, and pre-splitting the table into regions at creation. The cost is that a scan then has to hit every bucket. This tradeoff is why "design the row key" is the first question in any real HBase design discussion, and why migrating later is painful: the key is baked into every write. ## Interview framing A weak answer says "HBase is NoSQL, Hive is SQL". The strong answer names the access path — sorted key ranges served from memory plus HFiles versus full sequential scans of columnar files — explains that both sit on HDFS, and ends with the combined architecture plus the row-key caveat.