For AI agents: the complete documentation index is at llms.txt. Every page is also available as markdown by appending .md to its URL, or by sending an Accept: text/markdown request header.

Merge-append partitions

Merge-append partitions reduce the data copied by backdated writes into large native partitions. Available since QuestDB 10.1.0 and QuestDB Enterprise 4.1.0, this opt-in layout lets a partition hold several pieces over one set of column files: a commit appends the affected pieces at the tail and leaves untouched pieces in place.

When to use it​

Use merge-append for repeated late data or corrections ingested through deduplication in narrow time ranges. It saves less when backdated rows are spread across the whole partition, and gains nothing for purely in-order ingestion.

The trade-offs are extra disk space until compaction reclaims superseded rows, and additional page-frame boundaries at piece boundaries, which can increase query overhead.

Enable or disable​

This instance-wide setting affects all partitioned WAL tables, including in-order workloads; there is no per-table opt-in. Set it in server.conf, then restart:

cairo.o3.partition.merge.append.enabled=true

The default is false, and the setting is not reloadable. To disable it, set it to false and restart. Each table's writer converts its merge-append partitions back to the standard layout when it opens; disabling is not an immediate server-wide conversion. Conversion can delay writer opening and ingestion, and may need temporary disk space for replacement files.

warning

Merge-append is supported only for WAL tables. Before converting a table to non-WAL, disable the flag, restart, run ALTER TABLE trades SQUASH PARTITIONS, then run ALTER TABLE trades SET TYPE BYPASS WAL and restart again.

Compaction​

Compaction reclaims dead space and reduces piece counts on commit. A background partition-compaction worker also sweeps WAL tables on non-read-only servers, including when merge-append is disabled:

  1. Every two minutes, look for idle partitions, skipping partitions whose event-time range is too recent, including future-dated data.
  2. Non-active split partitions become eligible for merging after all folders have been idle for 30 minutes; individual merge-append folders and Parquet dead space become eligible for compaction after one hour without writes.
  3. Build copies in the background, then let the writer check and publish them. If intervening writes make a copy stale, discard it and retry on a later sweep; stop starting new work when the sweep's I/O or time budget is spent.

The active partition is not merged by the sweep. Remote partitions and symbolically linked partitions are skipped. Most users only need the sweep interval and two idle timeouts in the partition compaction settings. Use SQUASH PARTITIONS for an explicit conversion to the standard layout and merge of split folders.

Monitor partitions​

SHOW PARTITIONS and table_partitions() expose deadRows, lastWriteTimestamp and pieceCount. For example, this projection was captured after backdated inserts into a day initially containing one million rows:

SELECT name, numRows, deadRows, lastWriteTimestamp, pieceCount
FROM table_partitions('trades');
namenumRowsdeadRowslastWriteTimestamppieceCount
2026-01-0110100011262026-10-22T17:24:07.013120Z3
2026-01-0210null1

Pieces need not imply dead rows: a cut can create pieces without copying data. lastWriteTimestamp is the layout's wall-clock update time, not the latest event timestamp; compaction can advance it without ingesting rows. It can be null when unavailable.

note

Log messages use the name "composite partition" for a merge-append partition.

Downgrade safely​

warning

QuestDB versions before 10.1.0 and QuestDB Enterprise versions before 4.1.0 refuse tables that still contain merge-append partitions. Before downgrading, stop ingestion, disable the flag and restart on the current version. Run ALTER TABLE trades SQUASH PARTITIONS for every affected table to open its writer and convert all its partitions, and wait for these operations to complete before stopping the server and installing the older binary. Do not rely on disabling the flag alone: an unopened table can retain the layout.

Materialized and live views cannot use ALTER TABLE ... SQUASH PARTITIONS. For their backing partitions, use background compaction instead:

  1. Stop ingestion and let pending WAL apply and view refresh finish. Temporarily set the following in server.conf (the live-view setting is only needed for live views):

    cairo.o3.partition.merge.append.enabled=false
    mat.view.refresh.worker.count=0
    live.view.refresh.worker.count=0
    cairo.partition.compaction.check.interval=0
    cairo.partition.compaction.idle.timeout=0
    cairo.partition.compaction.squash.idle.timeout=0

    The worker counts disable view refresh. Zero is the minimum enabled sweep interval; zero idle timeouts remove the normal waiting periods.

  2. Restart on the current version and wait for background compaction to finish. Keep WAL apply enabled so pending work and compaction commands can complete.

  3. Inspect each affected view with table_partitions('trades_1m'), replacing trades_1m with its name. Before downgrading, every native partition must report pieceCount=1, deadRows=0 and a null lastWriteTimestamp. The sweep's eligibility restrictions still apply; do not downgrade while any merge-append partitions remain.

  4. Restore the previous refresh-worker and compaction settings and restart before resuming normal operation; keep merge-append disabled when using an older binary.

QuestDB Enterprise​

Backups preserve the merge-append layout. Storage policies first fold split or merge-append partitions into the standard layout before converting to Parquet; conversion can wait for a later policy check while background compaction runs. Replicas also run compaction unless the server is read-only.