skip to content

In Qlik Sense, what fails first as an app's in-memory model outgrows the server's RAM?

level: seniorimportance: should knowfreq 42%

answer

  1. everything lives in memory at once
  2. something needs the space twice, briefly
  3. users are still fine when the first thing breaks
  4. distinct values cost more than rows
  5. the timestamp column is usually the culprit

basics

~20 s

Reloads fail first. The engine builds the new model while the previous one is still resident and serving users, so a reload needs headroom well above the app's steady-state footprint. Chart calculation under concurrency degrades next.

solid answer

~50 s

Qlik holds the **whole** data model in memory, so RAM is the binding constraint, and it is consumed in three places: the app's resident footprint, the extra copy built during a reload, and per-session calculation and cache memory for concurrent users. The reload is the first casualty — it needs roughly the app's footprint again before it can swap the new model in — so you see failed or aborted reloads long before dashboards break. Then heavy chart calculations start to fail or queue under concurrency, particularly `Aggr()` over high-cardinality fields. The levers, in order of payoff: cut **cardinality** (drop unused columns, split timestamps into a date plus an hour rather than keeping second-level values), aggregate at load time, move to **incremental reload** backed by QVD files, and split one large app into smaller ones — with on-demand app generation if users need to drill into detail. Section access can also reduce data per user at load.

code

text · 14 lines
text
// 1. Pull only rows changed since the last successful run
Fact:
LOAD OrderID, CustomerID, OrderDate, Sales, ModifiedAt
FROM [lib://DB/orders]
WHERE ModifiedAt >= '$(vLastReload)';

// 2. Append previously stored rows that were not just reloaded
CONCATENATE (Fact)
LOAD OrderID, CustomerID, OrderDate, Sales, ModifiedAt
FROM [lib://Data/fact.qvd] (qvd)
WHERE NOT Exists(OrderID);

// 3. Persist the merged set for the next run
STORE Fact INTO [lib://Data/fact.qvd] (qvd);

go deeper

for a junior

Know the basic fact that a Qlik Sense app holds its whole data model in memory, so app size is bounded by server RAM rather than by warehouse capacity.

for a middle

Explain the mechanics: values are stored once per distinct value in a symbol table, so cardinality drives memory, and a reload temporarily needs room for both the old and the new model.

for a senior

Diagnose from symptoms. Failing reloads with healthy dashboards points at headroom and script-side fixes; degraded charts under concurrency points at expression cost. Know the levers — cardinality, load-time aggregation, incremental QVD reload, splitting apps — and which one the symptom calls for.

for a principal

Own the capacity plan: host sizing from measured footprints plus reload headroom and concurrency allowance, a staggered reload schedule, growth tracking per app, and the policy question of how much detail belongs in memory versus left in the warehouse.

## The memory model you are budgeting against Qlik's engine stores each field as a **symbol table** — one entry per *distinct value* — plus a bit-stuffed pointer array per record. Two facts follow directly, and they drive every sizing decision: - Memory scales with **distinct values**, not with row count nearly as steeply. Ten million rows of a field with 400 distinct values is cheap; ten million rows of a second-resolution timestamp is not. - A column you never use still costs its symbol table and its pointers. Loading `SELECT *` is the most common self-inflicted memory problem in Qlik. On top of the resident model sits **session memory**: each user's selection state, chart caches and in-flight calculations. The model itself is shared across sessions — users do not each get a copy — but calculation memory is per-request and can spike hard on a single expensive object. ## The failure order 1. **Reloads.** A reload builds the new model in memory while the currently published version is still resident and being served, then swaps. Peak usage during reload is therefore roughly the old footprint plus the new one, plus whatever intermediate tables the script materialises before joins and drops. So the first symptom of an app outgrowing its host is an aborted or failing reload, on a machine where the dashboard itself is still fine. 2. **Concurrency.** Next, chart calculations start to queue, time out or fail — typically the expensive ones first: nested `Aggr()` over a high-cardinality dimension, large pivot tables, expressions with no set-analysis narrowing. 3. **Everything.** Past that point you are into eviction, swapping and service restarts, and users see sessions dropped rather than slow charts. A useful diagnostic consequence: if reloads fail but interactive use is healthy, you have a *headroom* problem and script-side fixes apply. If interactive use degrades under load while reloads are fine, you have a *concurrency and expression cost* problem and the fix is in the charts. ## Levers, roughly in order of payoff **Reduce cardinality.** The largest single win is almost always a timestamp. Splitting `OrderTimestamp` into `Date(Floor(OrderTimestamp)) as OrderDate` plus `Hour(OrderTimestamp) as OrderHour` replaces millions of distinct values with a few thousand plus twenty-four. Dropping unused columns, and dropping the component fields after building a composite key, are the same move applied to width. **Aggregate at load.** If no chart ever needs order-line detail, load the daily aggregate. The temptation to keep detail "just in case" is what turns a 4 GB app into a 40 GB one. **Incremental reload with QVDs.** A QVD is Qlik's own column-store file format, and reloading from one is dramatically faster than re-querying the source. The standard pattern loads only rows changed since the last run, concatenates the previously stored rows that were not just reloaded, and stores the merged set back for next time. This attacks reload duration and source load; it does not shrink the resident model. **Split the app.** One app per subject area, sized to its audience, beats one universal app. Where users still need row-level detail, **on-demand app generation** builds a small child app for the selection the user made, so the detail is materialised only when someone asks for it. Newer releases also offer modes that push queries to the source rather than loading everything — check what your release actually supports before promising it, as this area has changed across versions. **Section access.** Qlik's row-level security mechanism can reduce data at load per user or group. It is a security feature first, but it also means each user's session works against less data. ## What does not help Adding chart-level filters does not reduce the resident model — the data is loaded regardless of what any chart shows. Neither does set analysis: it changes what an expression aggregates, not what is in memory. And renaming or restructuring fields for readability has no memory effect unless it changes cardinality. Candidates who reach for UI-side fixes to a RAM problem are signalling that they have not sized a Qlik deployment. ## Operating it Size the host from measured app footprints plus reload headroom plus a concurrency allowance, and schedule reloads so that two large apps do not reload simultaneously — serialising the reload window is often the cheapest fix available. Track app size over time rather than at a point: models grow by a column at a time, and the reload that fails is rarely the one whose change caused it.

  • Why does a reload need more memory than the app at rest?
    The engine builds the new model while the existing one is still resident and serving users, then swaps them, so peak usage is roughly both models at once plus any intermediate tables the script materialises before joins and drops. That is why headroom, not steady-state footprint, is the number you size against, and why staggering large reloads is often the cheapest fix.
  • Which single change usually shrinks a Qlik app the most?
    Cutting the cardinality of a timestamp. The engine stores one symbol-table entry per distinct value, so a second-resolution timestamp on a large fact table can dominate the app's memory. Splitting it into a date field plus an hour field replaces millions of distinct values with a few thousand. Dropping unused columns is the same principle applied to width.
  • Does incremental reload reduce the app's memory footprint?
    No. It reduces reload duration and load on the source system by fetching only changed rows and merging them with previously stored QVD data. The resident model is the same size afterwards. If memory is the constraint you still need cardinality reduction, load-time aggregation, or splitting the app — incremental reload buys a shorter reload window, not a smaller app.

A query-per-visual tool rents the warehouse's compute for each question. Qlik moves the goods into its own building first — so the ceiling you hit is floor space, and the worst moment is the day the new shipment arrives before the old one has left.

saying these in an interview costs you the question

  • Says each concurrent user gets their own copy of the data model
  • Thinks chart-level filters reduce the app's memory footprint
  • Claims incremental reload makes the resident app smaller
  • Sizes RAM from the app's steady-state footprint with no reload headroom
  • Believes row count rather than distinct values drives memory

context

resolving links…