QuestDB Releases

Latest updates and improvements to QuestDB Open Source and Enterprise editions.

Showing 20 releases

September 1, 2026

QuestDB Enterprise 4.0.1 is a patch release on the 4.0.0 line. It carries two user-visible additions — OIDC sign-in via the OAuth 2.0 device flow and a retryable, deadline-aware shutdown contract that now spans Enterprise role-switch, WAL-transfer and backup teardown — plus cold-storage support for Azure Blob object metadata and a replication fix for an idle rebased table that never settled in the uploader index. The bundled OSS engine advances seven commits past 10.0.1 (the 10.0.2-SNAPSHOT line; there is no OSS 10.0.2 release), two of which are breaking changes inherited from the engine that should be reviewed before upgrading.

Breaking Changes

An overflowing INT expression now wraps modulo 2^32 in every context, including under a 64-bit cast and on the way into a 64-bit column, so an INT expression carries exactly one value at its declared type. To compute at 64 bits, widen an operand, for example to_utc(secs * 1_000_000L, tz) or i::long * j. Consequences of this rule: WHERE secs * 1_000_000 > <timestamp>, (i*j)::long and INSERT into a 64-bit column all read the wrapped value; inserting an overflowing INT expression into a designated timestamp now fails with designated timestamp before 1970-01-01 is not allowed; SELECT <overflowing constant INT arithmetic> returns INT rather than LONG, so over the PostgreSQL Wire Protocol the column descriptor moves from int8 to int4 and CREATE TABLE ... AS SELECT creates an INT column; INT arithmetic landing on -2^31 reads as NULL everywhere; window frames, WINDOW JOIN bounds, SAMPLE BY intervals and FROM/TO bounds, and DROP/DETACH/CONVERT PARTITION WHERE filters use the final wrapped value instead of inspecting arithmetic provenance; long_sequence(100_000 * 30_000) wraps negative and returns no rows; dateadd('u', 3600*1000000, ts) computes from the wrapped stride instead of failing overload resolution; and timestamp predicates and UPDATE filters accept INT expressions and sign-extend the wrapped value, with a WAL UPDATE whose WHERE clause fails intrinsic extraction now failing synchronously at sequencing time instead of suspending the table at apply. Constant reassociation is also disabled for numeric constant pairs, so a chain such as i + 1 + 4 evaluates left to right at its declared width, which changes EXPLAIN output and costs throughput on Java-evaluated constant arithmetic. Alongside the rule, this change fixes a large family of defects: HORIZON JOIN and WINDOW JOIN wrong results including a constant-false WINDOW JOIN reading an unrelated column on a Parquet master, undercounted past-only keyed windows, lazily bound owner group-by functions, spliced NULL column handling for DECIMAL, varchar and LONG256, and keyed join maps escaping cairo.query.memory.limit.bytes; LATEST ON silently dropping an indexed-symbol WHERE filter and pushing a LIMIT below the operator; NULLIF returning the value it had just excluded and INT/IPv4 casts evaluating their argument twice; json_extract answering different values at different widths and casts to SYMBOL closing a symbol table their caller still owned; IN over integer keys widening, leaking or mis-ordering elements; numerous JIT compiled-filter crashes, fail-open native backends and width divergences against the Java filter; an inclusive tolerance boundary shared by the compiled filter and Parquet row-group pruning; Parquet row-group pruning false-pruning numeric, non-finite and column-top rows; timestamp offset pushdown dropping rows for month and year offsets, wrapped shifts and hand-written and_offset calls; a NullPointerException from 'x' || concat('y'); and several factory-construction and interval-model resource leaks. Performance work amortizes the keyed WINDOW JOIN slave index (up to 2000 rebuilds down to 17), restores vectorization for mixed-width JIT filters, and makes the expression reassociation and JIT width-marking passes linear rather than quadratic.

New Features

This feature lets deployment controllers and embedders bound a single shutdown attempt without freeing objects that a worker, lifecycle component, or engine extension may still be using. ServerMain.closeBy(long deadlineNanos) takes an absolute deadline on the monotonic System.nanoTime() clock and returns true when shutdown completed and the close graph was released, or false when the deadline or an interrupt stopped the attempt, in which case every resource that may still be in use remains owned for a later retry with a fresh deadline; completed stages stay idempotent and close() keeps the existing terminal behaviour. Concurrent callers do not tear down in parallel — one caller owns shutdown while others wait up to their own deadlines, restoring any consumed interrupt status — and unexpected component failures still propagate because an explicit incomplete-shutdown signal is used rather than classifying arbitrary exceptions as timeouts. One deadline flows through boot completion, lifecycle pre-stop work, component stops, worker-pool halts, timer shards, engine readiness, and logging, with final resource release beginning only after those stages complete; components first stop admitting work and request cooperative cancellation, WorkerPool.haltBy(long) accepts the shared absolute deadline while haltWithin(long) remains for relative budgets, a worker pool counts down startup and halt latches for workers that start() never created so rollback cannot wait on impossible acknowledgements, and CairoEngine.isCloseReady(long) gives engine extensions a pre-free phase after worker pools stop. The deadline bounds only QuestDB's coordination and cooperative waits, so it cannot forcibly stop application or native code that ignores cancellation, and an incomplete attempt intentionally retains memory, file descriptors and native handles in exchange for memory safety; existing no-argument shutdown entry points and legacy worker-pool descriptors remain available, and deadline checks run only on startup rollback and shutdown paths.

Improvements

This improvement extends the deadline-aware shutdown contract to Enterprise lifecycle code, so a single monotonic absolute deadline flows through role-switch rendezvous, role-state close, lifecycle components, worker pools, WAL transfer, the engine pre-free phase, and retryable backup teardown. A successful stage releases its ownership, while an incomplete stage returns a retryable incomplete-shutdown signal and keeps every still-live Java and native owner so a later attempt can resume with a fresh deadline. During shutdown, restore and backup cancellation are signalled before waiting for long-running work, the lifecycle orchestrator coordinates with any in-flight role switch and repeatedly requests cooperative cancellation while time remains, and the final close graph is released only after the engine reports readiness, which prevents a role switch or native task from observing engine-owned state after the engine frees it. Native backup teardown closes admission first and then uses the deadline remainder to acquire its teardown write lock and drain the Rust manager, rounding the native wait down to whole milliseconds so conversion cannot extend the caller's deadline; on timeout the native handles, Tokio runtime, backup agent and scheduler state are retained for retry, and restore cancellation is mapped to a structured RestoreCancelledException so a shutdown-requested cancellation ends boot cleanly while an unrequested one preserves the original failure. A bounded timeout can therefore leave the server stopped but still holding Java and native resources until a retry or the terminal process-exit policy reclaims them, cancellation is cooperative rather than forced, and deployment termination grace must cover role.switch.timeout plus the Enterprise teardown reserve when the shutdown hook should finish in-process.

Bug Fixes

The uploader's notified-transaction watermark was overwritten by every transaction notification instead of only being raised. Because events are not delivered in transaction order — a rebased table can drain data for transactions 1 and 2 followed by a table-created event carrying transaction 0 — a trailing lower value dragged the watermark down, so the transaction log reader was never remapped and an idle rebased table retried forever instead of settling in the uploader index. This fix only raises the watermark in the drain path, allowing the uploader to tolerate any event order while leaving the rebase ordering unchanged.
A queue segment published its frozenForEnqueues flag before bumping the tail by the freeze offset, while the empty check in tryDequeue() reads the flag before the tail. In the window between the two writes a dequeuer could pair a set flag with the pre-freeze tail and subtract the freeze offset from a tail that was never bumped, making the frozen clause always true and reporting the queue empty while it still held committed items; the full fence only ordered visibility, so the race occurred on x86 as well as ARM. This fix bumps the tail before publishing the flag, so a set flag always implies a bumped tail and a clear flag with a bumped tail simply retries, adding no additional synchronization. It also repairs the sized ConcurrentQueue constructor, which validated its size argument and then ignored it, always allocating a 32-slot first segment.
This fix addresses two symbol-map correctness issues that could produce invalid symbol mappings. Previously, SymbolMapReaderImpl.updateSymbolCount() removed the wrong cache range when the visible symbol count shrank and left maxOffset at the previous high-water mark, so keyOf() could return a key outside the current symbol count; the symbol cache and maxOffset are now shrunk together with the visible symbol count. Additionally, WalTxnDetails.saveSymbols() relied on an assertion to require dense Write-Ahead Log symbol diff keys, which production builds do not enforce, allowing sparse or invalid diffs to pass through without a useful diagnostic. Symbol diff keys are now validated at runtime, and if a diff is sparse, Write-Ahead Log apply reports the relevant transaction and symbol metadata and suspends the table instead of applying a potentially inconsistent mapping, favoring containment over continuing with an invalid mapping. The validation adds a single integer comparison per new symbol diff entry on valid input and no per-row work, but a suspended table requires operator intervention.

August 25, 2026

QuestDB Enterprise 4.0.0 is a major release. It moves the bundled OSS engine from the 9.4.4 line all the way to QuestDB 10.0.1, and adds two headline Enterprise capabilities — cold storage (old partitions tiered out to an object store as parquet, still fully queryable) and live views on replicas (incrementally maintained window-function results, refreshed independently on every node). Alongside them: a dedicated SWITCH ROLE permission with a new node_role() function, automatic sys.acl_permissions compaction so replica logins stay fast, and a deep round of role-switch, replication and backup durability fixes.

Breaking Changes

This change stops a UNION or UNION ALL from permanently widening to STRING when every branch contributes a SYMBOL column; the result column is now SYMBOL. This is a result-schema change for that query shape: HTTP /exec reports a JSON column type of SYMBOL instead of STRING, and CREATE TABLE AS derives a SYMBOL column where it previously created a STRING one. The PostgreSQL Wire Protocol continues to report the VARCHAR OID for both types, so its wire type is unchanged. Mixed SYMBOL and STRING union columns still resolve to STRING, and a pending all-symbol union segment is restored before a following EXCEPT or INTERSECT so that flat and parenthesized forms stay consistent. Internally the metadata is preserved through a specialized projection that keeps text reads streaming, initializes integer-key dictionaries lazily, translates static native symbol keys per source, falls back to text for dynamic symbol sources, and releases query-tracked native state at cursor close. No new statement or permission is introduced, and existing view authorization remains the access boundary.
This fix makes a QuestDB Enterprise instance fail to start when http.user, http.password or http.password.file is present, whether supplied through server.conf or the QDB_HTTP_USER, QDB_HTTP_PASSWORD and QDB_HTTP_PASSWORD_FILE environment variables. These are the open source basic-auth settings and QuestDB Enterprise has never read them, so an instance configured with them was serving the REST API and the Web Console with no authentication at all while the operator believed it was password-protected; remove such settings and use access-control-list users instead. Either property alone is rejected, and the check runs ahead of the open source credential-pair validation so the real reason is reported on the first attempt. It also fixes identity reporting when the access control list is disabled: the allow-all and read-only security contexts previously discarded the authenticated principal and returned the hardcoded default, so current_user() reported admin on every instance and session_user() and SHOW CREATE ... OWNED BY reported admin on read-only instances. The isPrincipalIndependent() mechanism and its boot-time shared proxy were removed so a named principal always receives a per-connection context, while anonymous connections still share a pooled context per protocol when the access control list is disabled, keeping the HTTP request path allocation-free. The Influx Line Protocol access-control bypass is now keyed on the protocol interface identifier rather than an authentication-type constant, and owner grants use an atomic grantIfExisting operation so an unknown key identifier no longer fabricates an access list entry in the replicated access-control table. Applications using QuestDB Enterprise Java APIs directly must switch to the three-argument DispatchingSecurityContextFactory(primaryFactory, replicaFactory, initialRole) constructor, stop relying on the removed EntSecurityContextFactory.isPrincipalIndependent(), and implement grantIfExisting(CharSequence, ObjList<PermissionModel>) in external AccessListWriter implementations before rebuilding.
A UNION or UNION ALL whose every branch contributes a SYMBOL column now returns a SYMBOL column instead of permanently widening it to STRING. The set-operation cursor still uses STRING internally, because independent table symbol dictionaries cannot share native keys, and the compiler restores SYMBOL once at the boundary of each all-symbol UNION segment; a later STRING leg removes the affected column from the candidate set, so mixed SYMBOL/STRING columns remain STRING. A pending UNION segment is restored to SYMBOL before a following EXCEPT or INTERSECT, so flat and explicitly parenthesized forms expose identical metadata. Text consumers stream source text without building a dictionary, while integer-key consumers initialise translation state lazily, translating native keys per union leaf where a static source dictionary exists and interning text otherwise; query-scoped dictionaries and key translations are released when the cursor closes. As a result of this change, HTTP /exec now reports SYMBOL in its JSON column metadata and CREATE TABLE AS over a symbol-only union materialises a SYMBOL column rather than STRING, so scripts that inspect result metadata or depend on the resulting column type must be updated. PostgreSQL Wire Protocol continues to expose both types as the VARCHAR OID, so the wire type is unchanged there, and EXPLAIN retains the established plan shape.
This fix corrects several defects in the query engine's memory-bound data structures and tightens configuration validation. Signed comparisons on compressed heap offsets caused OrderedMap to misread live entries as empty slots once the map heap grew past roughly 16GB, so a parallel GROUP BY could silently drop or duplicate groups; all slot-emptiness checks now share a single unsigned predicate. Budgeted structures (OrderedMap, LongChain, AbstractRedBlackTree, LongTreeChain and LimitedSizeLongTreeChain) previously rejected a growth step outright when one doubling overshot the configured ceiling, raising memory-limit errors well below the ceiling; they now clamp to the limit instead, at the cost of a runaway query holding up to twice as much native memory before it is stopped. A cached ORDER BY ... LIMIT $1 factory no longer returns wrong rows when the bind value flips sign, and a window ORDER BY with more terms than the model's no longer raises an out-of-bounds error. Native memory lifecycle issues in the HTTP header parser's multipart boundary augmenter and in the HTTP connection, client and platform-specific client constructors were fixed so a failed allocation no longer leaks or over-frees, and overflow messages now name the owning feature (ASOF JOIN, RANK() window function, DENSE_RANK() window function) and the configuration key to raise. Oversized initial heap pages are now rejected before allocation, and hot-loop memory accesses in map probing, rehashing, shard merging and red-black tree insertion were reduced. As a compatibility change, cairo.sql.small.map.page.size and cairo.sql.join.metadata.page.size now reject values below 4, cairo.sql.hash.join.light.value.page.size rejects values below 12, cairo.sql.small.map.page.size and cairo.sql.sort.key.page.size reject values above 34,359,738,352, and cairo.sql.sort.light.value.page.size rejects values above 17,179,869,176; all of these were previously accepted at startup.
Scalar subqueries in predicate position previously followed different and sometimes unsafe compilation paths: timestamp-valued scalar subqueries on the designated timestamp could lose intrinsic state during recursive compilation and fall back from covering-index and interval scans to full filters, while scalar BOOLEAN subqueries were not consistently coercible. This change leases a dedicated WHERE clause parser per recursive generation depth, evaluates timestamp scalar subqueries once per execution through a shared frozen bound that both interval pruning and the residual filter read, preserves microsecond or nanosecond precision, maps empty results to NULL and enforces one-row cardinality, so a query such as SELECT ts, value FROM t WHERE series = 's1' AND ts >= (SELECT lo FROM bounds WHERE sel = 100) now uses the same covering-index and interval plan as equivalent constant bounds. One-column BOOLEAN scalar subqueries can participate directly in bare, AND, OR and NOT predicates, and a whole-predicate runtime constant now gates the outer scan so a statement-invariant false predicate performs no outer I/O. Several previously accepted shapes are now rejected at compile time and require rewriting: a literal negative LIMIT in a correlated lateral sub-query fails with negative LIMIT is not supported in a correlated lateral sub-query; a LIMIT naming an outer column over a lateral scalar count is refused and must use a constant or bind variable; materialized views whose sub-query reads an external source such as read_parquet() fail with non-deterministic function cannot be used in materialized view: sub-query; and an UPDATE against a WAL table may no longer reference another table or instantiate a cursor function, including through a FROM-less scalar sub-query such as (SELECT 3). Lateral scalar-count compensation is also narrowed to provably one-row bare count bodies, so counts wrapped in GROUP BY, UNION or a filtering layer now return NULL where they previously returned 0 — a silent value change for existing views. Stored views and materialized views carrying a rejected shape become invalid on upgrade, cascade that invalidation to dependents and must be rewritten manually; the WAL apply path is exempt, so statements sequenced by an older binary keep applying.
HTML encoding of EXPLAIN plans was originally added as a presentation workaround for the Web Console, which did not preserve leading spaces, so the server emitted   for indentation and encoded < and > as < and >. The escaping was gated on the absence of a bind-variable service, which is the case for every HTTP request to /exec and /exp, so every REST client received escaped plans while the PostgreSQL Wire Protocol received plain text. This change removes the HTML escaping path from TextPlanSink so that child indentation uses four ordinary spaces, attribute indentation uses two ordinary spaces, comparison operators remain raw text, and literal strings such as  , < and > inside a query remain exactly as written; JSON serialization still performs the escaping required by JSON itself. HTTP and CSV EXPLAIN consumers will now receive spaces instead of   and < / > instead of < / >, and clients that parsed the previous markup must be updated. The Web Console now renders result values as text and preserves whitespace itself.

New Features

This feature moves a table's old partitions out to an object store (S3, GCS, Azure or a filesystem) as Parquet while keeping them queryable, driven by a TTL-based storage policy: ALTER TABLE trades SET STORAGE POLICY(TO PARQUET 1d, TO REMOTE 7d, DROP LOCAL 30d, DROP REMOTE 1y);. Each partition is converted to Parquet, uploaded, sealed and finally evicted from local disk, after which only metadata and symbol indexes remain local and reads fetch only the column chunks a query needs from the bucket on demand. The bucket is the source of truth: a single manager instance writes a per-table manifest and a cold-table catalog protected by a _manager.lock, while every other instance mirrors them read-only and performs its own bucket I/O, so partition bytes never traverse replication. A single PARTITION_SEAL Write-Ahead Log event is the only cross-instance signal, and the manager role can be moved at runtime with SWITCH COLD STORAGE ROLE TO MANAGER|REFRESHER [FORCE] [TIMEOUT ms] (requiring SYSTEM_ADMIN) with SWITCH COLD STORAGE STATUS reporting the current state and term. Reads are pinned to a snapshot identifier captured at cold-switch time, with self-healing re-identification so cross-host restores and storage migrations remain transparent, and a buffer coordinator coalesces concurrent chunk downloads, enforces explicit backpressure and charges pinned bytes against the query's memory budget. Remote garbage collection reclaims orphaned objects and dropped tables after configurable grace periods, table_cold_partitions('trades') exposes per-partition remote state, and Prometheus metrics report chunk hits, misses, coalescing, evictions and pinned bytes. The whole feature is off by default via cold.storage.enabled=false and is configured through the cold.storage.* and storage.policy.* property families. SHOW PARTITIONS gains two trailing columns, seqTxn and isRemotelyServed, and enabling upload is a one-way upgrade since the partition flag bits are unreadable by older binaries; replicas must be upgraded first, and non-Write-Ahead Log tables are not supported.
This feature adds live views: incrementally maintained window-function results over a single WAL-backed base table that are queried like a regular table. Window functions run once per row as new base commits are refreshed in, so a query scans precomputed output rather than reprocessing the base on every read, targeting workloads where the same window aggregate is read frequently against high-rate ingestion such as rolling VWAP, cumulative volume, running ranks and day-over-day comparisons.
```
CREATE LIVE VIEW vwap_5m
FLUSH EVERY 1s
IN MEMORY 5s
START FROM NOW
AS
SELECT ts, symbol,
       avg(price) OVER (PARTITION BY symbol ORDER BY ts ROWS 300 PRECEDING) AS vwap
FROM trades;
```
DROP LIVE VIEW [IF EXISTS] and ALTER LIVE VIEW ... RESUME|SUSPEND WAL are also supported, while structural ALTER verbs stay rejected because a live view's schema is a function of its SELECT. FLUSH EVERY and START FROM are mandatory; START FROM NOW, BEGINNING or a quoted timestamp literal defines the event-time boundary a row must satisfy to belong to the view, and a view over a base with existing history performs a resumable bounded seed before becoming active. A live view is its own WAL-backed table driven by a dedicated refresh worker pool sized by live.view.refresh.worker.count, with cairo.live.view.enabled defaulting to true; refresh appends computed rows to an in-memory tier as an un-flushed lead, flush re-serialises that lead into the view's WAL on the FLUSH EVERY cadence, and a versioned checkpoint timeline backs restart and bounded out-of-order replay. Reads route between the in-memory tier and disk under a sequencer-transaction consistency fence, falling back to the applied on-disk prefix whenever the two snapshots disagree. live_views(), SHOW CREATE LIVE VIEW, EXPLAIN, pg_class, information_schema.tables, tables() and table_columns() all recognise the new table type. Catalogue reporting also changes for existing objects to align with PostgreSQL semantics: materialized views now report pg_class.relkind 'm' and table_type MATERIALIZED VIEW, views report 'v' and VIEW, and all view kinds report is_insertable_into as false, so clients filtering on relkind = 'r' or BASE TABLE will stop listing them. SecurityContext additionally gains the abstract authorizeLiveViewCreate and authorizeLiveViewDrop methods. The V1 surface rejects joins, sub-queries, CTEs, SAMPLE BY or GROUP BY ahead of window functions, live-view-as-base, non-deterministic functions, and window shapes whose out-of-order influence is unbounded.
This feature adds <, >, and = comparisons between numeric expressions and single-column scalar sub-queries, supporting DOUBLE, FLOAT, LONG, INT, SHORT, and BYTE left operands while preserving integer precision and evaluating the sub-query once per query execution. The <=, >=, !=, and <> forms, as well as the argument-swapped forms where the sub-query sits on the left, are derived automatically, for example:
```
SELECT * FROM trades
WHERE price > (
    SELECT avg(price) FROM trades WHERE timestamp IN today()
);
```
The comparison width is chosen so that the scalar value is never narrowed: integer operands and integer cursor values compare as LONG, while FLOAT and DOUBLE values compare as DOUBLE. A scalar sub-query returning zero rows produces no match, one row supplies the cached value, and more than one row raises scalar sub-query returned more than one row, which also tightens existing designated-timestamp sub-query predicates that previously used the first row silently. Strict comparisons with a null scalar produce no match, while =, <=, and >= preserve QuestDB's sentinel-null convention. Scalar cursor state is cached during function initialisation and donated to parallel worker clones, so async filters, parallel GROUP BY, window joins, and horizon joins evaluate the sub-query only once per execution. The cursor must appear on the right-hand side unless the optimiser can rewrite the expression, cursor-to-cursor comparisons are unsupported, and correlated scalar sub-queries are not covered.

Improvements

On a table with a covering posting index, every WAL commit block re-encoded the entire partition's covered sidecar, so per-block cost grew with the partition and WAL apply fell badly behind under sustained ingestion. This improvement routes blocks that are a pure append to the last native partition through the existing fast-lag append path, writing covered fragments incrementally and deferring compaction to the auto-seal, which reduces per-block covered-index cost from proportional to the partition size to proportional to the number of new rows. The block in-order flag is re-derived with a non-overlap check so a globally in-order multi-timestamp single-segment block skips the out-of-order sort, and the fast path is gated strictly on a pure append to an open, non-parquet last native partition with no lag, dedup or replace-range, so any merge or late-data block falls back to the unchanged out-of-order path. This change also fixes a pre-existing crash-safety defect in covering indexes shipped since 9.4.0, where the chain entry's cover-end footer aliased a generation directory slot and could be clobbered in place, letting a concurrent covered reader map the sidecar too small and dereference an offset past it. The footer is de-aliased to a fixed offset under a versioned covering format, and legacy entries are migrated copy-on-write on their first extend through a single funnel, flipping the head atomically so both concurrent readers and crash recovery stay safe with no reindex required; dual-format reads keep existing tables readable through the transition.
This improvement fuses window functions that share the same window specification so that a query carrying several functions over the same PARTITION BY key uses one hash map and one probe per row instead of one map, one key copy and one probe per function. Where two outputs are provably the same accumulator, such as count(x) reading the non-null counter inside sum(x), or avg(x) and sum(x) sharing the same state, they fold onto a single component. Fusion covers cumulative sums, averages and counts, row counters, the max/min families over DOUBLE, LONG, DATE, TIMESTAMP and DECIMAL, ksum, the Welford dispersion family, first_value/last_value with and without IGNORE NULLS, bounded ROWS and RANGE frames, and whole-partition two-pass functions; a PARTITION BY expression groups too, unless it is non-deterministic. Gains are concentrated at high key cardinality, where several shapes run two to three times faster with roughly halved retained native memory, and are neutral to slightly negative at low cardinality, so the new cairo.sql.window.map.fusion.enabled property (default true, reloadable through reload_config()) acts as a kill switch; because the switch is read at compile time, turning it off for an already cached plan also requires flush_query_cache(). EXPLAIN output is unchanged at either setting. Live view refresh, restart and memory use are also improved: a view resuming from its checkpoint no longer full-scans all retained state on the first seal, the WAL clean symbol band is resolved through the on-disk hash index instead of copying the base table's entire committed dictionary on every bind, the checkpoint map memoises descents by depth so incremental seals no longer degrade past roughly 2.4M keys, seals stay incremental across a sweep, ring arenas for bounded frames are reclaimed on sweep, symbol map caches move off the Java heap into tracked native memory, and a live view now inherits its base column's symbol CACHE flag. CREATE LIVE VIEW additionally accepts projections around the window, including expressions wrapping a window function, aliased base columns and scalar columns beside the window, while shapes the refresh path cannot rebuild, such as LIMIT, ORDER BY over the window, set operations and joins, are rejected with messages naming the actual cause; a SELECT that projects no designated timestamp is now rejected at creation rather than faulting to INVALID later. This also fixes several correctness problems: sum() over a bounded RANGE frame returned the average of the frame on cached window cursors, bounded RANGE frames could wrap at extreme timestamps, ORDER BY on a NOCACHE SYMBOL column under an exclusion filter returned insertion order, generate_series could overflow, live views could serve rows whose running window values continued from an abandoned refresh cycle, an anchor-resume repair wrote the window's columns into the view's columns, and a head-miss repair repeated a memoized column's stale value.
This improvement adds a MergeUnionAll physical operator that streams an ordered UNION ALL as a two-way merge instead of concatenating branches and sorting the result. The operator is selected when the union branches expose designated timestamps of the same type, share the same scan direction, and a timestamp order is requested that matches that direction; otherwise the existing concatenating UNION ALL plan is used unchanged. The optimiser now captures a window's OVER ORDER BY as model advice and pushes order-by advice down into UNION ALL branches, so branches scan uniformly and both windowed and non-windowed queries are handled the same way, for example:
```
EXPLAIN (SELECT * FROM a) UNION ALL (SELECT * FROM b) ORDER BY ts DESC;
```
Chained and nested unions produce nested merge plans, and windows over a union can stream rather than materialize through a cached window cursor. Full scans are roughly eight to nine times faster than the sort-based plan, and LIMIT-bounded queries improve by several orders of magnitude because no full materialization is needed. Branches that disagree on timestamp type or precision continue to fall back to a sort.
Since JDK 24, Thread.sleep constructs a ThreadSleepEvent before checking whether the event is enabled, so every call allocates 40 bytes whenever the JIT cannot eliminate it. Because Os.pause() wrapped Thread.sleep(0) and appears in worker back-off loops, idle workers repeatedly paid this allocation on Graal, C1, interpreted, cold and deoptimized paths; an idle GraalVM instance with 60 workers allocated roughly 59 MB/s from this source alone. This improvement changes Os.pause() to call Thread.yield(), which reaches the same underlying yield on HotSpot while avoiding the sleep event and preserving the interrupt flag, and routes Os.sleep(millis) through an exported native qdb_sleep_millis symbol via a foreign function downcall that reissues interrupted platform sleeps. The downcall deliberately avoids a critical binding so the VM can still safepoint over the sleep, and the symbol is resolved immediately after the native library is loaded so a stale library fails class initialization with a clear error rather than killing workers during their first back-off. Because Os.pause() and Os.sleep() no longer clear the caller's interrupt flag, condition-wait loops in SOCountDownLatch, SOUnboundedCountDownLatch, SimpleWaitingLock, the concurrent hash map tree-bin contention paths, TimerShards and the lifecycle orchestrator now consume an interrupt before parking, keep waiting on their condition, and restore the flag when the operation exits, preventing an already-set flag from turning these paths into CPU spin loops. Additional hardening keeps WAL purge lock cleanup progressing across malformed tracking data and individual unlock failures, ensures asynchronous log records release their queue cursor even when validation throws, and frees the first native buffer when a log alert socket fails to allocate its second one. A platform thread inside Os.sleep() now reports RUNNABLE, shows foreign-function stub frames in stack traces, and emits no jdk.ThreadSleep event.

Bug Fixes

This fix addresses a race in the uploader's shared <db_root>/.upload scratch directory. Because the path is derived from the database root, it is shared across uploader generations, so during a teardown-and-rebuild role-switch cycle such as a hot switch or a refused-demote heal, the old uploader's final reference could unwind after the new uploader had already recreated the directory, and its drop then deleted the scratch directory out from under the live uploader. The compression path only checked that the directory existed, so from that point every upload failed with tmp path does not exist in a permanent retry loop, leaving a healthy-looking primary with replication silently wedged until restart. Temporary archive creation now recreates the directory tree instead of merely checking it, which is a no-op when the directory exists, self-healing when a stale teardown removed it, and still an error when a non-directory occupies the path. In-flight archives are unaffected by a concurrent delete because anonymous temporary files are used, so the next call converges.
This fix prevents data loss on the safe demote path. When an upload audit refuses a demote, the node is meant to keep its committed-but-not-uploaded Write-Ahead Log data, park read-only in a degraded replication state, and heal back toward primary to drain the backlog. However, the refusal arm still closed the primary role state and cleared the upload-mode latch that UploadWalDirectoryPolicy.isSeqPartInUse was gated on, so every V2 sequencer part reported as free and the node's own purge job deleted the applied transaction-log parts belonging to the very backlog the audit had declined to abandon. After healing, the rebuilt uploader could no longer read the range it had to ship and retried indefinitely, so acknowledged transactions became permanently unreplicable while the local table continued answering queries normally. The policy now carries an explicit unresolved-demote-backlog pin meaning the node owes a backlog it cannot currently upload, and part markers are honoured when either the upload-mode latch is up or the pin is set. The pin is set on both demote arms before the close that clears the latch, so no purge sweep can observe a window with neither flag set, and it is released only where the node is proven to owe nothing: a clean demote drain audit, or a retry toward replica in which every residue table has been explicitly sacrificed. A node that was never primary in the current process never sets the pin, so a plain replica keeps reclaiming its downloaded parts unchanged.
Metadata hydration can construct table sequencers as soon as the engine becomes ready, so installing the upload-aware WAL directory policy later left a race in which a sequencer captured the default policy for its lifetime, never created V2 sequencer-part upload markers, and allowed the WAL purge job to remove a part before catch-up uploaded it. This fix installs a single upload-aware policy immediately after the base server constructor returns and before lifecycle startup publishes engine readiness, and the engine keeps that policy instance throughout startup and role switches. The policy also preserves segment upload obligations across inactive and demoted roles: WAL writers record segment and sequencer-part markers while the server runs with role NONE so a later primary restart can discover retained work, and a segment is reported in use if either the upload-pending or download-pending marker exists, read under a single upload-mode snapshot so a concurrent role switch cannot probe the same marker family twice. Per-part sequencer markers stay mode-gated and are ignored in download mode, so a replica or demoted primary with no live uploader does not treat them as purge locks and cannot grow the sequencer part directory without bound.
A live view's files can reach a replica before its base table's, so registration fails and is retried on a later poll; if the node restarts first, boot finds the view definition on disk with no state file, refuses to load it and registers a dead stub that occupies the view's slot forever, leaving the view permanently empty because a replica refreshes live views locally. This fix drops the stub and rebuilds the view when a registration finds a live view with no state file and only a stub registered, so the view is left unregistered and retryable rather than stuck behind a stub if the base table is still missing. The recovery no longer unlinks and re-copies the definition file, which previously left a window in which a concurrent CHECKPOINT CREATE could abort or capture a snapshot whose restore deleted the view's data directory. Genesis now stages the state file through a temporary file and renames it into place, so a failure or crash anywhere inside the first write leaves the state file absent rather than present but unreadable; a stale staged file left by a crash is removed before writing and the registration is refused and retried when that removal fails. The on-disk format is unchanged and the state file stays node-local, so mixed-version clusters and rolling upgrades are unaffected.
This fix corrects scalar sub-query predicate handling and timestamp interval pruning, covering scalar sub-query bound hoisting, lateral count compensation, constant-false pruning in the interval model, and LATEST BY external-source propagation. As part of it, the rejection of UPDATE statements with a join on WAL tables is hoisted ahead of code generation, so on a WAL table the unsupported-shape error is now reported before column-level SELECT authorization runs; the non-WAL path is unchanged and still reaches the permission check.
This fix corrects current_user() and session_user(), which always returned the hardcoded default admin regardless of which user actually authenticated, so the Web Console displayed admin for everyone on page load, including a configured http.user or a PostgreSQL Wire Protocol user. The root cause was that the allow-all and read-only security contexts reported a fixed principal instead of the authenticated one. AllowAllSecurityContext and ReadOnlySecurityContext now carry a principal field and expose forPrincipal(principal), and the security context factories pass the authenticated principal through it. Anonymous requests allocate nothing, and derived contexts are cached per principal so the HTTP path, which re-derives the security context on every request, becomes allocation-free after the first request for a given user; the cache is bounded at 256 principals, past which an unseen principal degrades to allocate-per-call rather than growing without limit. DenyAllSecurityContext and the materialized view refresh context are never re-derived, so they cannot be downgraded to a read-allowing context, and the transient principal is copied before it is retained. current_user() and session_user() are now runtime-constant functions that snapshot the principal in init() rather than reading it once per row, so a 1000-row query performs a handful of security context reads instead of roughly one per row, while a context re-derived between executions of a cached SELECT is still picked up.
This fix stops TableReader from closing a column's file descriptor immediately after mapping it for every partition except the currently growing last one. Because the file descriptor and memory-map caches can only reuse an existing mapping while its file descriptor is still open, a later, non-overlapping reader that opened the same historical partition always created its own independent mapping. Under workloads with many long-lived, concurrent readers scanning the same large table, such as dozens of parallel connections pulling a multi-terabyte table in time-ordered batches, those duplicate mappings accumulated and could exhaust the process's virtual address space with could not mmap errors even while resident memory usage stayed low. A column's file descriptor is now kept open for as long as the reader keeps the column mapped, for every partition, so concurrent or staggered readers of the same partition share a single mapping. The trade-off is higher file descriptor usage: a reader now holds one descriptor per currently mapped column for as long as cairo.inactive.reader.max.open.partitions (default 10,000) lets it keep that partition open, so deployments with many partitions, columns and concurrent readers should ensure their ulimit -n has headroom.
This fix prevents ALTER TABLE ... ALTER COLUMN ... TYPE from crashing the whole JVM with a native SIGBUS when disk space was low instead of failing the statement with a normal error. The destination column file was sized with ftruncate(), which only sets the logical file length and does not reserve real disk blocks, so a later native write into an unbacked page faulted with SIGBUS, which cannot be caught from Java or JNI. Disk space is now reserved via TableUtils.allocateDiskSpace() (posix_fallocate), matching every other writable memory-mapping call site in the codebase, so a disk space shortfall surfaces as a catchable CairoException.
This fix prevents an idle live view fallback scan from durably invalidating a healthy view while another refresh worker processes a new base table commit. The fallback scan validates restored or replicated state by invalidating a view whose processed sequencer transaction is ahead of its base table, but it sampled the base sequencer head first and the view watermark second. A notification worker could publish and refresh the next base transaction between those two reads, so the fallback worker compared an old base head with a new view watermark and persisted a false "view is ahead of base table" invalidation. The view watermark is now read and retained first, followed by the monotonic base sequencer head, with those same retained values used for both the comparison and the diagnostic log. Since a refresh publishes the watermark only after consuming the corresponding base sequencer transaction, a newer watermark can no longer be paired with an older base head, and the impossible combination is eliminated without adding a lock on the hot path. Genuine restored or replicated ahead states still take the existing invalidation path.
This fix stops squashSplitPartitions from destroying the tail of a variable-size column (BINARY, STRING, VARCHAR) in the partition it squashes into, which produced silently wrong query results, binary is outside of file boundary errors, or unsafe memory access faults. The squash appends into the target partition through the frame's own file descriptors; when that target is the partition the writer still holds open, the writer's column append memories keep their pre-squash positions and describe a shorter file than what is on disk, so the stale append memory trims the grown data vector back on close and a later memory re-extends it zero-filled, cutting the last value's length prefix in half. This state is reachable because opening the last partition is a no-op when that partition is Parquet, leaving the writer pinned to an earlier native partition, for example after ALTER TABLE ... SET FORMAT PARQUET followed by an insert into a later day. When the squash target is the writer's open partition, the stale view is now dropped by closing without truncating and the target is re-opened so the writer resumes from sizes that match disk, with both arms running under a single distress guard. The posting index seal-purge outbox is now drained before closing the active partition in the squash path, so superseded index files are published for purging instead of leaking on disk. This change also hardens posting index publishing and closing: close() no longer sizes a mapping up just before releasing it, several call sites now arm the seal transaction tag before publishing, and the reader rejects structurally impossible or all-zero generation metadata. As a user-visible consequence, the reader's error now names the affected column and the recovery step, for example posting index is corrupt, rebuild it with REINDEX TABLE <table> COLUMN <column> LOCK EXCLUSIVE, and a query over a damaged index that previously returned a silently short result now raises an error instead.
This fix addresses two independent defects that could permanently invalidate a live view over a healthy base table, even for a column the view never referenced. First, WAL symbol map folding was coupled to every symbol column carrying a diff, ignoring the projection supplied by the caller; because WAL symbol dictionaries are shared across segments and deleted as soon as a DROP COLUMN is applied, a live-view drain could fault on a dictionary belonging to a column outside its projection. Symbol map folding now skips diffs for columns the effective projection does not reach, while a rebind that widens the projection still refolds from the start of the event file so all symbol values resolve. Second, the re-derive path that rebuilds a view from the applied base table after WAL loss replayed through a cached compiled plan and was refused whenever base metadata had moved on, which is the normal state during the DDL churn that triggers it; it now recompiles once and retries the replay. A shared guard reads the base's applied metadata ahead of both replay attempts and refuses to re-derive when a referenced column was dropped, renamed or retyped, invalidating the view and naming the column rather than silently committing results computed over a new schema. A common trigger was ALTER TABLE base ALTER COLUMN sym SYMBOL CAPACITY N, which is non-structural and deliberately leaves dependent live views valid, but causes the next WAL segment roll to delete and re-link the dictionary. The re-derive is a full rebuild from the applied base rather than an incremental drain, so recovery carries a one-off cost on the refresh worker, and the projected WAL reader overload now reports 0, null and not-found for symbol columns outside the projection.
This fix prevents PostingIndexWriter.close() from trimming the .pk file below the chain region another writer instance had already published, which physically destroyed the tail of the most recently published chain entry. Because more than one writer can be open on the same .pk file, the closing instance's cached high-water mark could lag the file, and the lost bytes read back as zeros, corrupting the generation directory and silently dropping rows from POSTING index scans. The writer now never trims below the region limit published in the chain header, the authoritative live high-water written under the header seqlock. Three further defects are addressed: publishing could itself write a regressing seal transaction when an extend happened on a reopened writer, so a slot is now clamped up to its predecessor and a monotonicity check runs immediately after the slot store and before publication, throwing a diagnostic exception instead of aborting via an assertion; the reader no longer asserts gen-dir monotonicity inside a window where torn reads are legal, and instead records the anomaly and acts only once the snapshot is proven self-consistent; and the reader now closes the release half of the entry seqlock by re-reading the generation count and retrying, so an entry rewritten in place with a smaller count cannot mix two incarnations. Genuine corruption now raises a diagnostic error naming the entry offset and generation counts. Additionally, close() no longer propagates errors on a cleanup path that must not throw, logging instead and leaving the file intact for the next open when no trusted high-water is available. A .pk file already damaged by an earlier build does not self-heal; rebuild the index with REINDEX TABLE <table> COLUMN <column> LOCK EXCLUSIVE;.
This fix resolves a crash when reading metadata for tables created before QuestDB 6.1. The per-column replacing index was followed straight out of the _meta file without validation, but on migrated tables that slot holds leftover junk, because the migration grew column entries from 16 to 32 bytes and left old file contents in the bytes it did not write, with every later rewrite copying the junk forward. An index past the column count read off the end of the mapped file, and one pointing forward read off the end of the dense column list; with assertions disabled both were silent out-of-bounds reads that fed garbage into column mapping. The replacing column index now returns -1 unless the value could have come from a real replacement, meaning it is below the column carrying it and points at a deleted column, so every chain walk is in-bounds and loop free and both cycle guards were removed. Affected tables clear the junk on their next ALTER, which rewrites the column block from in-memory metadata. As a tradeoff, a genuine chain whose bytes were corrupted now degrades silently rather than erroring.
This fix eliminates an ERROR-level log line per view on every start of a database containing views, which read like corruption despite the view definition files being present and intact. View definitions are loaded from disk into the view graph and state store by the worker pool manager, but the hydration components depend only on the engine, so as soon as the engine published READY the view compiler and metadata hydration threads walked the table name registry with an empty graph, reporting missing view definitions and skipping views during write tracker hydration. As a result, compiling all views at boot was a no-op on every start, masked only by the per-view compile requests issued later. Loading of view definitions now runs in the engine startup step, immediately after the engine loads table state and before READY is published, so every engine-gated component observes a fully loaded graph. No client-visible failure occurred previously, since the shared pools serving PostgreSQL Wire Protocol and HTTP only start after the graph is populated.
This fix tightens an internal memory bounds check so that invalid writes fail safely instead of corrupting memory. The issue was found internally and no production workload is known to trigger it. There are no SQL, API, or configuration changes.
This fix restores correct covering-index reads across Parquet format conversions. Converting a partition with a COVERING posting index to Parquet and back to native rebuilt the posting index from the decoded native data without the covering sidecar files, so covering scans returned NULL for every covered column or failed with an index-out-of-bounds reduce error, while the underlying native data remained intact. The conversion back to native now hard-links the existing key, value and covering sidecar files from the Parquet partition directory into the new native directory, mirroring what the native-to-Parquet direction already does, which also removes the index rebuild work for plain bitmap symbol indexes. Converting to Parquet had the same defect in the opposite direction whenever the indexed symbol column had a column top: the rebuild that materialised the leading NULL prefix produced key and value files only, and both directions now share a single index rebuild routine that configures covering when needed. As a behavioural change, the link path gates on the key file alone, so a Parquet partition directory holding a key file with no resolvable live value file now fails the conversion with an "index files do not exist" error, suspending a WAL table, rather than silently rebuilding a non-covering index; reaching that state requires out-of-band file loss or a crash during index creation. Building covering sidecars on the column-top path also makes CONVERT PARTITION TO PARQUET do index work proportional to rows times covered columns where it previously did almost none.
```
ALTER TABLE t ALTER COLUMN sym ADD INDEX TYPE POSTING INCLUDE (price, tag);
ALTER TABLE t CONVERT PARTITION TO PARQUET LIST '2024-01-01';
ALTER TABLE t CONVERT PARTITION TO NATIVE LIST '2024-01-01';
SELECT sum(price), first(tag) FROM t WHERE sym = 'A0';
```
This fix prevents a WAL table with a POSTING index from being suspended mid-apply during a partition squash. Under parallel WAL apply, reopening a column's key file sized the memory mapping from the reported file length, which can lag the writer that just extended the file, so the chain's head entry fell outside a short mapping and was read past its end, distressing the writer and suspending the table. The reopen now reads the header's live entry-region high-water mark through a seqlock-consistent header read and extends the mapping to cover it only when the reported length genuinely lags, leaving the common case with no extra remapping, and the header is range-checked before its values are trusted so a corrupt header raises a CairoException instead of an out-of-mapping read. A second fix stops the key file from being truncated when a reopen fails before the region is republished: the failure path now releases the mapped memory without truncation, leaving the on-disk index intact for the next open and removing a latent hazard where the file could be truncated while still mapped by concurrent readers. Neither change alters how the key file is written or the on-disk format.
This fix replaces an invalid native memory access with a controlled error when a POSTING index writer is rolled back after an I/O failure closed its mapped value file. Previously, a failed remap while flushing pending data, or a failed open of the replacement value file during truncation, left the writer holding live key or pending state over a closed mapping, so a later rollback could re-enter the flush or reencode path and produce an assertion failure, a null facade access or a native memory crash. The flush and reencode paths now reject work over a closed value memory with a CairoException before any scratch allocation or index mutation, and closing mapped read-write memory now clears its append bounds so stale addresses from a released mapping can no longer be accepted without remapping. Table-level recovery is unchanged: the caller still distresses and discards the affected writer, and the on-disk chain is recovered on the next open.
This fix prevents a table from being suspended after a REPLACE_RANGE WAL commit, used by materialized view refreshes and range-replace inserts, removes the table's last partition. The commit moved the writer's maximum timestamp back to the new last partition but left the partition timestamp boundary pointing at the removed partition, since the replace path only ever advanced it and never lowered it. The usual re-sync that re-opens the new last partition is skipped when that partition is Parquet, so the next non-replace commit applied by the same writer tripped a partition-timestamp consistency check and suspended the table. The partition-removal branch now re-derives the partition timestamp boundary from the final maximum timestamp after partition sizes are updated, which runs only for replace commits that removed or trimmed partitions and leaves the native last-partition behaviour unchanged.
A query that waits, such as sleep(), wait_wal_table(), or any long scan that periodically consults the SQL circuit breaker, could keep running after its client had already disconnected, holding the connection's socket descriptor and worker resources until the query reached its own deadline. This fix replaces the peek-based connection probe with a hangup probe that detects a FIN behind buffered data on all platforms, including Windows, using poll(POLLRDHUP) on Linux, kqueue with EVFILT_READ/EV_EOF on macOS and FreeBSD, and WSAPoll on Windows. Previously a clean client close sent an application byte before the FIN, so the peek returned buffered data instead of EOF and the breaker never tripped, letting the query run until query.timeout (60 seconds by default). This fix also wires a real, file-descriptor-backed circuit breaker into the QuestDB WebSocket egress query path, covering compilation, cursor acquisition and the result-streaming loop, so egress queries now honour query.timeout and abort with STATUS_LIMIT_EXCEEDED instead of streaming past the deadline; a stream resumed after a write park or credit grant re-checks the deadline and the connection before producing the next batch. A request-boundary reset that disarmed the breaker of a parked, protocol-switched stream is corrected, and connection probes are now throttled by wall clock (100 ms by default) so the file-descriptor-backed breaker does not turn hot loops into per-iteration syscalls, at the cost of up to one throttle window of detection lag per checking site. Bulk data phases such as INSERT AS SELECT, CREATE TABLE and UPDATE remain exempt from query.timeout by design. The now-obsolete circuit.breaker.buffer.size and net.test.connection.buffer.size settings are removed and log an advisory on start-up, or fail under config.validation.strict. A double-read race in WriterPool.Entry.getTableToken() that could throw a NullPointerException when a writer_pool() scan raced a writer close is also fixed.
Restore cancellation was previously signalled only from a component's stop(), which runs inside the reverse-topological stop loop that follows the bounded boot-thread join, so a SIGTERM landing during a long point-in-time restore burned the entire 30-second join budget before the restore was asked to unwind, and a pod with a 30-second termination grace was killed mid-restore. This fix adds a pre-join cancel hook that runs after the executor drain and before the boot join, so the join budget is spent unwinding the restore; the hook contract is a pure, non-blocking cancel signal, resource teardown stays in stop(), and a throwing hook is logged rather than propagated so it cannot abort the shutdown it was installed to accelerate. Materialized view hydration now accepts an explicit target state store, allowing a caller to hydrate a private, not-yet-installed store and publish it only after hydration completes, with closing checks that abort the load promptly; the state store's close() is now idempotent across sequential closes and clears its map after freeing state, preventing a double free. A non-creating suspension-tracker lookup is added for hot read paths such as metrics scrapes, where a missing tracker simply means the table was never suspended. Finally, a broken duplicate-log assertion in the logging test helper was corrected, which exposed and fixed a real bug where every CREATE TABLE and system-table creation logged two completion records, one premature record carrying a -1 sentinel id; completion logging now belongs entirely to the execution site, so a CREATE that is compiled but never executed emits no completion record at all.
On a primary, an apply-time invalidation could publish while another refresh worker held the same view latch; the losing worker queued a retry, but the pending-marker guard consumed that retry without minting durable invalid state or arranging another wake-up, freezing the view in a stale-but-valid state. This fix introduces a single atomic pending marker with independent invalidation and full-refresh facets, carrying the invalidation reason, base table token, base transaction, force flag and a distinct full-refresh owner, merged keep-strongest so force flags OR, same-token transactions keep the maximum, and unknown or mismatched provenance collapses to conservative. An invalidator publishes the marker before attempting the view latch and either handles it itself or lets the completing latch holder queue one authoritative wake-up, so a contention episode produces bounded queue traffic. All operational latch holders, including incremental, range, interval, full, invalidation and synchronous REFRESH ... STATS paths, release through a common finalize helper that runs cleanup before publishing follow-up work. Full refresh is now snapshot-aware: it records its fixed reader snapshot and clears an invalidation only when the base table token matches and the snapshot transaction covers the invalidation transaction, while unknown provenance, a different table epoch or a newer transaction keeps the invalidation pending. An authorization refusal from the read-only writer gate, such as a replica demote racing a refresh, is now treated as a transient role condition that defers instead of trapping the worker in an unbounded requeue loop. Suspended views retain pending invalidation and full-refresh facets and republish them after a successful RESUME WAL, and invalidation now cascades to dependent views only after an invalid state is durably persisted, so a declined invalidation no longer leaves children invalid. If queue growth fails, allocation-free retry flags are recorded on the view state and redelivered by a later refresh-job tick.
On Windows the writer temporarily grew the small .pci cover-info file to the data-index append page size (1 MiB in tests, 16 MiB on a server) and truncated it back on close, so a concurrent reader could observe the larger length and map the file after it had shrunk, failing with errno 8. This fix maps .pci at a page-aligned size derived from the payload and any existing size and parks the append pointer there, making the physical file size monotonic non-decreasing for the file's life so a reader's length-then-map sequence can never map past a shrink; a larger existing file is preserved rather than shrunk. On adjacent paths, the .pci header is now rewritten only when it is absent, shorter than the payload, or its magic, count or covered-column indices disagree with the live configuration, repairing corruption instead of rewriting unconditionally on every seal. The sync order now fsyncs .pv, each .pcN and .pci before .pk, so a covering commit can no longer leave a durable chain head over cover bytes that were never flushed, and a failure to read the length of an existing .pci now throws instead of being silently treated as a new file. Under SYNC and ASYNC commit modes each covering commit issues up to one additional sync per covered column plus one, which is inert under the default NOSYNC.
On x86 systems with SQL JIT enabled, a valid query that checks a STRING column for NULL could rarely crash QuestDB with SIGSEGV when a null or empty string header occupied the final four bytes of a mapped column file, for example SELECT status, count() FROM events WHERE status IS NOT NULL GROUP BY status. This fix makes the scalar JIT read the four-byte STRING header without crossing the mapping boundary. Before upgrading to a fixed version, the crash can be avoided by setting cairo.sql.jit.mode=off and restarting QuestDB.
Decorrelation rewrites a scalar-aggregate lateral body into GROUP BY plus LEFT JOIN, which loses the empty-group row; the previous compensation only recognised bare count() projections, so unmatched rows returned NULL for count(*) + 2, CASE WHEN count() = 0 THEN 7 ELSE count() END, count_distinct(x), approx_count_distinct(x) and for sibling columns of a count, and WHERE sub.cnt = 0 filtered out the very rows it was meant to select. This fix builds a compensation template for any projection containing a zero-on-empty aggregate (count, count_distinct, approx_count_distinct), extracts aggregate leaves into hidden carrier columns and wraps only zero-on-empty leaves in coalesce(x, 0), so count(*) + sum(v) correctly stays NULL while count(*) + 2 returns 2; constants, outer references and bind variables now carry their value onto the compensated row, and WHERE terms referencing compensated columns are hoisted above the null-extension boundary so they observe 0 rather than NULL. Keyed bodies — an implicit GROUP BY key, a SAMPLE BY body, or a WINDOW JOIN / HORIZON JOIN body — are no longer treated as scalar and correctly return NULL instead of a fabricated 0, while CROSS JOIN LATERAL and INNER JOIN LATERAL over a scalar aggregate body are promoted to LEFT OUTER so they preserve every outer row. PIVOT empty cells for approx_count_distinct now default to 0 instead of NULL, and the expression parser accepts count(*) in any expression position, so coalesce(count(*), 5) and count(*) + count(*) compile, including inside OVER() window specifications where :: casts and multi-WHEN CASE expressions were broken in the same way. A negative LIMIT reaching a lateral body through a bind variable or an outer column now fails with negative LIMIT is not supported in a correlated lateral sub-query instead of silently emptying the body.
LATEST ON over an indexed SYMBOL column with a PostgreSQL Wire Protocol bind variable could return the latest row of a previously queried key, making the WHERE filter appear to be silently ignored. LatestByValueDeferredIndexedRowCursorFactory did not override RowCursorFactory.init(), which is a no-op by default, so the symbol function was never re-initialised and kept pointing at the bind variable service of the execution context that originally compiled the factory; once the shared SELECT query cache handed the cached factory to another connection, the resolved symbol key belonged to a different connection's bind variables. This fix forwards init() to the symbol function, restoring symmetry with the sibling deferred symbol index factories. Only indexed columns queried through bind variables on fresh connections were affected; literals and casts such as $1::SYMBOL resolved through other paths and were always correct.
VARCHAR date and timestamp conversions could produce different results for native and Parquet-backed partitions, potentially excluding valid rows from queries. This fix treats a negative ASCII hint as unknown and inspects the actual bytes, preserving the ASCII fast path while supporting Unicode patterns and timezones. It also makes malformed UTF-8 handling atomic across SQL conversions, Parquet and O3 processing, row copying, TableWriter, WAL and Influx Line Protocol ingestion, so invalid input becomes NULL instead of exposing or persisting a valid prefix. Before upgrading, the original parsing issue can be worked around with an explicit format, for example to_timestamp(trim(TimeCreated), 'yyyy-MM-dd HH:mm:ss.SSS').
QuestDB now accepts SHOW CREATE DATABASE as a parenthesized query source in FROM, CTE and JOIN contexts, while filters and INCLUDE (...) / EXCLUDE (...) clauses keep their existing top-level behaviour. The parser previously read clause delimiters with helpers that consumed a closing ) as the enclosing subquery terminator, so the outer parser lost either the SHOW subquery delimiter or the category-list delimiter. This fix reads optional-clause lookahead without consuming the enclosing delimiter and returns it through the existing unparse path, reading the category list's own closing delimiter locally. Valid top-level statements are unchanged; malformed input that closes a valid category list but omits the enclosing subquery delimiter now reports the more specific error ')' expected instead of ',' or ')' expected.
This fix corrects dot_product() over two DOUBLE[]/DOUBLE[][] arrays, which could silently return a wrong result when one operand was not stored in vanilla (contiguous, row-major) layout. In the recursive code path the deepest-dimension loop read the right-hand array using the left array's flat index instead of its own cursor, so operands with different strides — for example a transposed array or a slice — selected the wrong elements. For instance, transpose([[1,3],[2,5]]) combined with [[1,5],[7,2]] now correctly yields 42 instead of 40. Arrays already stored in identical vanilla layout are unaffected, and there is no performance impact since the change simply uses an index variable that was already maintained.
This fix allows SAMPLE BY w to parse correctly, where it previously failed with one letter sample by period unit expected even though every other single-letter sampling unit, such as SAMPLE BY h, SAMPLE BY d and SAMPLE BY M, worked without a leading digit and users had to write SAMPLE BY 1w as a workaround. The parser's allow-list for the bare single-letter form was missing w, rejecting the query before it reached the sampler, even though the week unit was fully supported downstream by both the micros and nanos timestamp drivers. The change only widens what the parser accepts, with SAMPLE BY w now behaving identically to SAMPLE BY 1w, and does not alter the behavior of any query that already parsed.
This fix addresses several correctness and resource-safety issues in parallel SQL execution. first_not_null() and last_not_null() now behave correctly for IPv4 columns on the non-sharded keyed-batch path, implementing the same keyed-batch semantics as other primitive types, and batched last_not_null() implementations replace stored NULL values consistently with their row-by-row and merge paths. dim_length() now returns NULL for NULL arrays, NULL dimensions, and NULL PostgreSQL Wire Protocol array binds, while non-NULL out-of-range dimensions still report an error. Parallel reducers — including parallel filters, GROUP BY, top-K, horizon joins, multi-horizon joins, and window joins — now release their worker slot from a finally block entered immediately after acquisition, so a failure during frame navigation, temporary allocation, or cleanup can no longer permanently leak a slot and leave later executions of a cached factory spinning forever. A data race on the selectivity statistics shared across threads by a thread-safe parallel filter is resolved by packing the sample count and average into a single volatile word updated with a CAS loop, guaranteeing both consistency and visibility for the late-materialization heuristic. Array access overhead is reduced by reading dimension and element data directly through record accessors rather than materializing an ArrayView per row, NULL-producing records consistently return a NULL ArrayView, and the shared NULL array constant uses a borrowed array so an accidental close cannot mutate global NULL state. A dead VARCHAR entry was also removed from the CaseCommon dispatch table, which changes no behavior.
This fix resolves a lifecycle race where CANCEL QUERY <id> could cancel a different, unrelated query after the registry recycled a pooled query entry, causing the victim query to fail with cancelled by user even though nobody cancelled it. Cancellation now only applies to the active query that owns the requested id: each pooled entry carries a volatile lifecycle word packing the owning query id and a state, so a stale canceller holding a recycled entry fails the lifecycle check and reports the query as not found instead of cancelling the new owner. query_activity() now snapshots registry entries optimistically and only returns a row if the entry still belongs to the same active query after the row has been copied, preventing torn rows and wrong-query data while entries are unregistered and recycled. The cost is one volatile lifecycle publish on query registration and one uncontended CAS when unregistering, and a query in the cancelling state may transiently not appear in query_activity() while a canceller holds the lifecycle guard.
This fix resolves cases where some queries returned nothing over the PostgreSQL Wire Protocol, leaving the client blocked in fetchall() until the idle timeout closed the connection, five minutes later by default, while the server log showed the query finishing normally followed by an idle disconnect. The server was sending a DataRow header announcing more bytes than it went on to send, which occurred with ARRAY columns in text format when the client sends per-column format codes, and with IPv4 and GEOHASH columns requested in binary. Drivers that send a single result format code, such as psycopg2 and JDBC in its default mode, were never affected. As a tradeoff, rows that genuinely exceed pg.send.buffer.size now fail with an explicit error naming the required size instead of hanging, so jobs that previously hung will now report a failure.
Selecting a DECIMAL column over the PostgreSQL Wire Protocol could fail with unsupported type: 31 when a binary response crossed the server send-buffer boundary, because decimal types were missing from the calculation used to size the unsent tail of a fragmented DataRow. This fix adds exact PostgreSQL NUMERIC binary-size calculation for DECIMAL8, DECIMAL16, DECIMAL32, DECIMAL64, DECIMAL128 and DECIMAL256, accounting for nulls, zero, sign, precision, scale and base-10000 digit grouping, reuses the execution context's decimal scratch values for the 128-bit and 256-bit widths so the response path does not allocate, and adds a text-size estimate for decimal columns. It also introduces DecimalFunctionMemoizer, registered by SqlCodeGenerator for all decimal widths, so that a projected non-deterministic decimal function such as rnd_decimal() returns a stable value per row; sizing and serialization therefore stay aligned. The memoizer applies to every projected DECIMAL column on all protocols whenever the alias is read more than once or the function reports shouldMemoize(). Decimal storage and the binary encoder are unchanged. INTERVAL and ARRAY_STRING columns still have the same gap in the size calculators.
A client speaking a non-Influx Line Protocol binary protocol on the Influx Line Protocol TCP port caused the server to log an unhandled critical error, because EllipticCurveAuthenticator.readKeyId() converted raw key-id bytes to a String eagerly and threw CairoException on invalid UTF-8, which handleAuthentication() did not catch. Since nothing on that path closed the connection, every such connection permanently leaked a file descriptor, its receive buffer and one of the 256 default connection slots, and incremented the unhandled_errors counter exported to Prometheus. This fix validates the key id with Utf8s.validateUtf8() before conversion, so an invalid key id follows the same path as an unknown key: a single log line followed by an AuthenticatorException and disconnect. Both key-id log lines now render bytes with $safe(), hex-escaping control characters that an unauthenticated peer could previously write raw into the log. A CairoException catch is added around authenticator.handleIO(), which also closes a second instance of the same bug where a signature line that is not valid base64 stranded the connection, and the catch follows the severity and errno of the exception rather than forcing them. Finally, both the authenticator call and the security context factory call catch Throwable, logging it as critical, counting it as an unhandled error and disconnecting, so nothing escaping the authenticator can strand a connection. There is no configuration or protocol change and no behaviour change for well-behaved clients.
The documentation restructure moved ingestion/* to connect/*, which broke every client, Influx Line Protocol, PostgreSQL Wire Protocol, REST API and message-broker link in the English and Simplified Chinese READMEs. This fix repairs all dead documentation links in both files, reframes the Connect to QuestDB section around QWP as the native protocol with Influx Line Protocol, PostgreSQL Wire Protocol and REST presented as compatibility protocols, adds a link to the connect overview, retitles "First-party ingestion clients" to "First-party clients" since those clients now cover both ingestion and queries over QWP, and completes the deployment list with systemd, Azure, Hetzner and ZFS compression ordered to match the documentation sidebar.

August 24, 2026

QuestDB 10.0.1 is a maintenance release that builds on 10.0.0 with improvements across SQL, QWP, Live Views, DECIMAL, storage, and the Web Console. Expanded fuzz, fault-injection, and concurrency coverage strengthens a number of less-common execution and failure paths. The release also improves high-cardinality window queries and Live View refreshes, streams eligible ordered UNION ALL queries without a full sort, reduces idle-worker allocation on JDK 24+, and bundles Web Console 2.0.3.

Breaking Changes

This fix routes scalar subqueries in predicate position through a single, safe compilation path. Timestamp-valued scalar subqueries on the designated timestamp now preserve intrinsic state during recursive compilation, keep their covering-index and interval scan plans instead of degrading to full filters, and handle NULL, precision and cardinality consistently across comparison, BETWEEN and monotonic shapes, evaluating the subquery once per execution and sharing the frozen bound with the residual filter:
```
SELECT ts, value
FROM t
WHERE series = 's1'
  AND ts >= (SELECT lo FROM bounds WHERE sel = 100)
  AND ts <= (SELECT hi FROM bounds WHERE sel = 100);
```
One-column BOOLEAN scalar subqueries can now be used directly in bare, AND, OR and NOT predicates, and a whole-predicate runtime constant is evaluated once at cursor open so a statement-invariant false predicate performs no outer I/O. The fix also corrects EXPLAIN runtime interval state, interval builder ownership and interval sorting performance. Several behaviour changes require user action after upgrade: a negative LIMIT in a correlated lateral sub-query is now rejected at compile time; the coalesce(count, 0) compensation under a LEFT lateral applies only to provably one-row bare scalar-count bodies, so counts wrapped in GROUP BY, UNION or a filtering layer now return NULL where they previously returned 0; a LIMIT referencing an outer column over a lateral scalar count is rejected; materialized views whose sub-query reads an external source such as read_parquet() are rejected as non-deterministic; and an UPDATE against a WAL table may no longer reference a second table or instantiate a cursor function, including FROM-less scalar sub-queries such as (SELECT 3). Existing views and materialized views with these shapes fail closed on recompile or refresh, are marked invalid together with their dependants, and must be rewritten manually; the WAL apply path is exempt so statements sequenced by an older binary continue to apply.

New Features

This feature adds full light/dark theme support across the Web Console, including system-theme detection, persisted user preference, Monaco editor themes, charts, grids, dialogs, menus, and shared components. It introduces semantic design tokens in place of legacy hard-coded colors and adds a theme selector with System, Light, and Dark modes. Shared buttons, controls, menus, icons, focus states, and component styling are refreshed throughout. The default Result Quick Vis experience is replaced with a configurable ECharts chart panel that adds chart type, axis, series, partitioning, zoom, and OHLC controls, while the legacy Quick Vis panel remains available behind ?useQuickVis=1. Notebook, AI assistant, schema, import, search, footer, and onboarding surfaces are also updated, and existing preferences and persisted metric colors are preserved during migration.
This feature brings auto-refresh to grid (run-mode) cells, previously a chart-only capability, and adds a notebook-level auto-refresh default that every cell showing a chart or a grid inherits and can override individually. Grids now refresh per statement: each slot swaps its own rows in place, a failing statement keeps its old rows and carries its error, and siblings refresh independently, while cells containing DDL/DML never poll so auto-refresh never executes a write. A toolbar control sets a single cadence (Auto / Off / 1s / 5s / 10s / 30s / 1m) for the whole notebook, with Auto adapting the interval to response time between 2s and 60s, and each cell's refresh dropdown gains a "Notebook default" (inherit) option, with overrides marked by a dot and clearable in one action. One click refreshes every chart and grid, superseding in-flight background polls. Drawing a chart no longer implies auto-refresh: a new chart inherits the notebook default and renders once until the user opts in, while legacy draw cells that polled through the old implicit per-view Auto receive an explicit, resettable Auto override on load so existing dashboards keep refreshing. A new set_notebook_autorefresh tool sets the default and reset_cell_overrides clears per-cell values in the same atomic call, set_cell_autorefresh accepts null to clear an override, and apply_notebook_state gains auto_refresh_default with snapshots reporting the effective default, auto_refresh_blocked for write cells, and last_refresh_error per cell. The configured interval holds across hide/reveal and tab refocus, grid frames persist with their refresh errors throttled to one write per 10s window, and with the editor hidden Cmd+Enter runs the statement of the selected tab rather than a stale result index.
This feature introduces an Editor settings modal, accessible from the editor tabs menu, that exposes two new options. "Run with selection" controls whether editor run actions execute the highlighted text or always run the statement under the cursor, while shared query links continue to run their selected fragment regardless of this setting. "Maximum column width" caps how wide result columns can auto-size, or leaves sizing fully automatic when left empty, and columns remain manually resizable by dragging. The SQL editor is no longer locked while a query runs, so text can be typed, lines can be added above a running query, or a new run can be started; the run indicator follows the query as it moves and results are still applied to the correct statement, or cleanly dropped if the statement is rewritten mid-flight. Result grid column widths are now measured from the actual rendered font, wide columns share available space more sensibly instead of being hard-capped, widths re-settle once web fonts finish loading, and long array values are truncated with an ellipsis. Both settings persist in local storage as editor.runWithSelection and grid.maxColumnWidth, and apply to both the main result grid and Notebook result cells.

Improvements

This improvement makes a UNION or UNION ALL whose every branch contributes a SYMBOL column return a SYMBOL column instead of permanently widening it to STRING. The set-operation cursor still uses STRING internally, because independent table symbol dictionaries cannot share native keys, so the compiler restores SYMBOL once at the boundary of each all-symbol UNION segment; a pending segment is also restored before a following EXCEPT or INTERSECT, so flat and explicitly parenthesized forms expose the same metadata. Columns that mix SYMBOL and STRING legs remain STRING. Text consumers stream source values through without building a dictionary, while integer-key consumers initialize translation state lazily and release query-scoped dictionaries and key translations when the cursor closes, with native allocations charged to the query memory tracker. PostgreSQL Wire Protocol continues to expose both types as the VARCHAR OID, HTTP /exec now reports SYMBOL in its JSON column metadata, and CREATE TABLE AS over a symbol-only union materializes a SYMBOL column rather than a STRING one.
Since JDK 24, Thread.sleep constructs a ThreadSleepEvent before checking whether the event is enabled, so every call allocates 40 bytes whenever the JIT cannot eliminate the disabled event. Because Os.pause() wrapped Thread.sleep(0) and appears in worker back-off loops, idle instances allocated continuously: an idle instance with 60 workers produced roughly 59 MB/s of garbage and triggered a young collection every few seconds. This improvement makes Os.pause() call Thread.yield() and routes Os.sleep(millis) through the exported native qdb_sleep_millis symbol, which reissues interrupted platform sleeps so an interrupt neither shortens the sleep nor clears the interrupt flag; the downcall deliberately avoids a critical binding so the VM can still safepoint over a sleeping thread, and the symbol is resolved immediately after the native library loads so a stale library fails fast with a clear error. Since these calls no longer clear the caller's interrupt flag, the condition-wait loops in SOCountDownLatch, SOUnboundedCountDownLatch, SimpleWaitingLock, the concurrent hash map tree-bin contention paths, TimerShards and LifecycleOrchestrator.close() now consume an interrupt explicitly and restore it when the operation exits, preventing an already-set flag from turning those waits into CPU spin loops. The change also hardens cleanup paths: WalPurgeJob contains and logs unlock failures while continuing its cleanup walk and only publishes a delete-entire-WAL encoding after a completed segment scan, AsyncLogRecord releases its queue cursor even when validation throws, and LogAlertSocket frees its first native buffer when allocation of the second one fails.
This improvement lets several window functions that share the same window specification share a single hash map and a single probe per row, instead of each function owning a private map, key copy and probe. Cumulative sums, averages and counts, row counters, the max/min families, ksum, the Welford dispersion family, first_value/last_value, bounded ROWS and RANGE frames and whole-partition two-pass functions are all covered, and PARTITION BY expressions group as well when they are deterministic. Gains concentrate at high key cardinality, where multi-function queries run roughly two to three times faster with about half the retained native memory, while low-cardinality shapes are broadly neutral. A new reloadable configuration property, cairo.sql.window.map.fusion.enabled (default true), acts as a kill switch; because the switch is read at compile time, turning it off for an already cached plan also requires flush_query_cache(). EXPLAIN output is unchanged at either setting. Live views additionally avoid a full state scan on restart, keep the WAL clean symbol band on disk instead of copying the base table's whole symbol dictionary per drain, memoize checkpoint map descents by depth, stay incremental across a sweep, reclaim ring arenas on sweep, move symbol map caches off the Java heap, and inherit the base column's symbol CACHE flag. CREATE LIVE VIEW now accepts expressions around a window function, aliased base columns and scalar columns beside the window, and rejected shapes such as LIMIT, ORDER BY over the window, set operations and joins now name the offending clause; a SELECT that projects no designated timestamp is rejected at creation time rather than faulting to INVALID later. This change also fixes several correctness problems: a live view could serve rows whose running window values continued from an abandoned refresh cycle after an apply-lag deferral, sum() over a bounded RANGE frame returned the frame average on cached window cursors, bounded RANGE frames overflowed at extreme timestamps, ORDER BY on a NOCACHE SYMBOL column under an exclusion filter returned insertion order, generate_series overflowed for LONG and TIMESTAMP arguments, symbol cache offsets are guarded against 32-bit word offset wrap, and a projected live view could write window columns into the view's own columns during an anchor-resume repair or repeat a memoized column's stale value during a head-miss repair.
This improvement adds a MergeUnionAll physical operator that performs a streaming 2-way merge of already-ordered UNION ALL branches instead of concatenating them and sorting the result. The operator is chosen when both branches carry designated timestamps of the same type, scan in the same direction, and the downstream consumer asks for that same timestamp order. To make branch scan directions consistent, the optimizer now captures a window's OVER ORDER BY as order-by advice and pushes order-by advice down into UNION ALL branches. Chained and nested unions nest 2-way merges, and windowed queries over a union stream through the merge rather than materializing a cached window. Queries that request no timestamp order, or whose branches disagree on timestamp type or precision, keep the existing concatenating UNION ALL behaviour. Benchmarks show roughly 8-9x faster full scans and orders-of-magnitude faster LIMIT queries compared with the previous sort-based plan.
```
EXPLAIN (SELECT * FROM a) UNION ALL (SELECT * FROM b) ORDER BY ts;
```
This improvement bumps @questdb/sql-parser from 0.1.16 to 0.1.17 and advances the end-to-end submodule. The parser release removes isOrdered from the grammar so it no longer appears in autocomplete; chevrotain remains pinned at 11.1.1 and no other packages moved.
This improvement drops the autoRefreshMigrated marker field and the load-time migrateImplicitChartAutoRefresh step, so an undefined auto-refresh value on a chart cell now always means "inherit the notebook default" with no version-detection logic on the read path and no dummy field persisted on every notebook record. Charts saved before the auto-refresh work that relied on the implicit per-view Auto now load as "inherit", which resolves to Off when the notebook has no default, and can be re-enabled per cell or by setting a notebook-wide default. Browsers that already ran a stamped build keep an inert autoRefreshMigrated: true property in stored records; nothing reads it and no schema change is required.
This improvement adds Web Console telemetry for notebooks and the MCP server to reveal how these features are used, with no functional changes: every emit is a fire-and-forget trackEvent wrapped around existing handlers and batched through the existing IndexedDB send pipeline. Notebook tracking covers create and duplicate, layout mode, cell add, delete, duplicate, move, rename, run and draw, resize and size reset on gesture end only, view changes and maximize, chart settings, markdown apply, the variables popover, result tab switching, and the onboarding modal funnel. MCP lifecycle tracking covers pairing prompted, accepted and declined along with the granted permission level, manual pairing, connect, disconnect and connect-failed, permission changes, setup command copy, the agent changes popper, and the status popover, with intentional teardowns excluded from mcp.disconnected so the event only counts real drops. MCP tool calls emit one event per call named per tool, such as mcp.add_cell or mcp.run_query, with an outcome of ok, denied, tool_error, dispatch_error or aborted, a denial reason code where applicable, and duration; unrecognized tool names emit mcp.unknown_tool with the name capped at 100 characters. Outcome classification lives in a dedicated utility that prefix-matches known denial message constants rather than parsing serialized tool results, and payloads are limited to bounded enums, counts and booleans so no SQL text, table or cell names, variable values, tokens or error messages leave the browser.

Bug Fixes

This fix moves engine.buildViewGraphs() into the engine lifecycle envelope, so _view definitions are read from disk into the view graph and state store immediately after the engine loads its table state and before it publishes READY. Previously the graph was populated later, when the worker pool manager started, so components gated on engine readiness — the view compiler job and metadata hydration — observed an empty graph and logged an ERROR-level cannot compile view, missing view definition line for every view on every start, together with a view does not exist message while hydrating the write tracker. Startup compilation of all views was effectively a no-op, masked by the per-view compile requests issued once the graph was finally built, and the write tracker silently skipped views. No queries were affected, because the shared pools serving PostgreSQL Wire Protocol and HTTP only start after the graph is populated; worker pool construction now remains the sole responsibility of the worker pool manager step.
Tables migrated from the pre-6.1 layout carried junk in the per-column replacing index slot of _meta, because the migration grew column entries from 16 to 32 bytes and left the old file contents in the bytes it did not write, with every later rewrite copying the junk forward. Following that index produced an assertion error or, with assertions disabled, silent out-of-bounds reads past the mapped file or the dense column list that fed garbage into column mapping. This fix validates the value before it is used: a replacing index is only accepted when it could have come from a real replacement, meaning it sits below the column carrying it and points at a deleted column, and anything else is treated as "no replacement", matching the behaviour of the versions that wrote those bytes. Every chain walk is therefore in-bounds and loop free, and affected tables clear the junk on their next ALTER, which rewrites the column block from in-memory metadata. As a tradeoff, a genuine replacement chain whose bytes were corrupted now degrades silently rather than raising an error.
Some queries returned nothing over PostgreSQL Wire Protocol, leaving the client blocked until the idle timeout closed the connection while the server log showed the query finishing normally. The server was sending a DataRow header that announced more bytes than it went on to send, which happened for ARRAY columns in text format when the client sends per-column format codes, and for IPv4 and GEOHASH columns requested in binary format; drivers that send a single result format code, such as psycopg2 and JDBC in its default mode, were never affected. As a tradeoff of this fix, rows that genuinely exceed pg.send.buffer.size now fail with an explicit error naming the required buffer size instead of hanging, so jobs that previously hung will report a failure.
A LATEST ON query over an indexed SYMBOL column with a PostgreSQL Wire Protocol bind variable could return the latest row of a previously queried key instead of the requested one, with the WHERE filter silently appearing to be ignored:
```
SELECT Enabled, AttributePath, DeletedDate
FROM AttributeList
WHERE AssetKey = $1
LATEST ON Timestamp PARTITION BY AssetKey;
```
The deferred indexed row cursor factory did not override the row cursor factory init() method, which is a no-op by default, so the bind variable function was never re-bound to the current execution context. Once the shared query cache handed the factory to another connection, the stale value belonged to a different bind variable service. This fix forwards initialisation to the symbol function, restoring symmetry with the sibling deferred symbol index factories. Queries using a literal value, an explicit ::SYMBOL cast, or an unindexed column were never affected.
VARCHAR date and timestamp conversions could produce different results for native and Parquet-backed partitions, potentially excluding valid rows from queries. This fix treats a negative ASCII hint as unknown and inspects the actual bytes, preserving the ASCII fast path while correctly supporting Unicode patterns and timezones. It also makes malformed UTF-8 handling atomic across SQL conversions, Parquet and O3 processing, row copying, TableWriter, WAL and Influx Line Protocol ingestion, so invalid input becomes NULL instead of exposing or persisting a valid prefix.
A live view could become permanently invalid over a healthy base table, for a column it never referenced, with the only remedy being DROP plus CREATE. Two independent defects caused this. First, the WAL reader folded a symbol map diff for every symbol column that carried one, ignoring the projection passed by the caller, so a live-view drain could open the WAL symbol dictionary of a column removed by ALTER TABLE ... DROP COLUMN even though the view's query never mentioned it, faulting with a missing symbol map. The symbol fold now skips any diff whose column the effective projection does not reach, while a rebind that widens the projection refolds from the start of the event file so every symbol value still resolves. Second, the re-derive recovery that rebuilds a view from the applied base table when a needed WAL segment is gone replayed through a cached compiled plan, which was refused whenever the base metadata version had moved — precisely the state produced by the DDL churn that triggers the fault. The re-derive now recompiles once and retries the replay. Because rebuilding is only safe while the view's referenced columns survive, a shared guard reads the base's applied metadata ahead of both the initial and the recompiled replay and refuses the re-derive when a referenced column was dropped, renamed or retyped, invalidating the view and naming that column instead of silently committing output computed over a new schema. A non-structural ALTER TABLE base ALTER COLUMN sym SYMBOL CAPACITY N was enough to reach the fault, since the following WAL segment roll deletes and re-links the symbol dictionary and a lagging view could bind a segment whose dictionary was gone. The re-derive is a full rebuild rather than an incremental drain, so recovery costs a one-off rebuild on the refresh worker where the view previously just stopped.
On Windows a covering-index read racing the writer could map the .pci cover-info file past its end and fail with errno 8, because the writer temporarily grew the small file to the data-index append page and truncated it back on close, letting a reader observe the larger length and map after the file had shrunk. This fix maps .pci at a page-aligned size derived from the larger of the payload and the existing file and parks the append pointer there, so the physical size is monotonic non-decreasing for the life of the file and a reader can never map across a shrink; a larger existing file is preserved rather than shrunk. On adjacent paths now owned by the same method, the .pci header is rewritten only when it is absent, shorter than the payload, or its magic, count or covered-column indices disagree with the live configuration, so corruption is repaired and later reads can resolve covers; sync() now fsyncs .pv, each .pcN and .pci before .pk, so a durable chain head can no longer sit over cover bytes that were never flushed; and a failure to read the length of an existing .pci is now reported instead of being treated as a new file, which could reopen it at size zero. Under SYNC and ASYNC commit modes each covering commit issues up to one additional msync per covered column plus one, and the change is inert under the default NOSYNC.
A client speaking a non-Influx Line Protocol binary protocol on the Influx Line Protocol TCP port — a port scanner, for instance — made the server log an unhandled critical error with a full stack trace, because the elliptic-curve authenticator converted raw key-id bytes to a string eagerly and invalid UTF-8 threw a CairoException that the authentication handler did not catch. Nothing on that path closed the connection, so every bad connection permanently leaked a file descriptor, its receive buffer and one of the 256 default connection slots, and incremented the unhandled_errors counter exported to Prometheus, which pins /status to HTTP 500 for the lifetime of the process when http.pessimistic.health.check=true. This fix validates the key id as UTF-8 before converting it, so invalid input takes the same path as an unknown key: one log line, then a disconnect. Key-id bytes are now rendered with hex-escaping in logs, so control characters supplied by an unauthenticated peer can no longer be written raw. The authentication handler also catches CairoException, which closes a second instance of the same bug where a signature line that is not valid base64 stranded the connection, while preserving the original severity and errno. Finally, both the authenticator call and the security context factory call catch Throwable, logging it as critical, counting it as an unhandled error and disconnecting, so nothing escaping a pluggable authenticator can strand a connection. Well-behaved clients are unaffected; the malformed-base64 signature path now produces a real disconnect where the connection was previously stranded, and a CairoException from the authenticator is still logged but no longer moves the unhandled-error metric. There is no configuration or protocol change.
This fix tightens an internal memory bounds check so that invalid writes fail safely instead of corrupting memory. The issue was found internally and no production workload is known to trigger it. There are no SQL, API or configuration changes.
Closing a posting index writer trimmed the .pk key file to that instance's cached high-water mark, but more than one writer instance can be open on the same file — the O3 copy path opens its own alongside the table writer's indexer. When the other instance extended the chain in between, the trim lopped off the tail of the just-published chain entry, typically its last generation-directory slot. Those bytes read back as zeros, so the generation's seal transaction regressed behind a real transaction: queries failed with an AssertionError carrying no diagnostic on assertion-enabled servers, and silently returned fewer rows from POSTING index scans otherwise, while genuinely abandoned generations were never trimmed on recovery. This fix never trims below the region limit published in the chain header, the authoritative live high-water written under the header seqlock by whichever instance extended the region last. Along the way it clamps a generation slot's seal transaction up to its predecessor so a later generation can never become visible earlier than an earlier one, validates generation-directory monotonicity before publishing and raises a diagnostic CairoException naming the writer, entry offset and generation counts instead of an assertion, re-validates the entry seqlock so an entry rewritten in place with a smaller generation count cannot mix two incarnations, moves the reader's monotonicity check out of the window where torn reads are legal so a concurrent publish retries instead of failing, and makes close non-throwing so a cleanup failure can no longer skip subsequent frees and strand the table lock. A .pk already damaged by an earlier build does not self-heal: depending on the entry it either fails reads and suspends the WAL table on the next extending commit, or reads back as an empty generation, and the remedy in both cases is to rebuild the index:
```
REINDEX TABLE <table> COLUMN <column> LOCK EXCLUSIVE;
```
This fix removes a race in LiveViewRefreshJob.scanForLaggingViews() where an idle fallback scan could durably invalidate a healthy live view while another worker was refreshing a new base-table commit. The guard that detects a view whose processed sequencer transaction is ahead of its base table sampled the base sequencer head before the volatile view watermark, so a refresh publishing the next base transaction between the two reads produced an impossible old-base/new-view pair, for example (view=1, base=0), and persisted a false view is ahead of base table invalidation. The view watermark is now read and retained first, followed by the monotonic base sequencer head, and both the comparison and the diagnostic log use the same retained values. Because a refresh publishes the watermark only after consuming the corresponding base sequencer transaction, a newer watermark can no longer be paired with an older base head, and the fix adds no hot-path lock. Genuine restored or replicated ahead states still take the existing invalidation path.
A REPLACE_RANGE WAL commit, used by materialized view refreshes and range-replace inserts, that removed the table's last partition moved the writer's maxTimestamp back to the new last partition but left partitionTimestampHi pointing at the removed partition, since the replace path only ever advanced it via Math.max and never lowered it. finishO3Commit normally re-syncs partitionTimestampHi by re-opening the new last partition, but that re-open is skipped when the new last partition is Parquet, so the next non-replace commit applied by the same TableWriter instance tripped the partition-timestamp consistency assertion in processWalCommit and suspended the table. This fix re-derives partitionTimestampHi from the final maxTimestamp in the partition-removal branch of o3ConsumePartitionUpdateSink after partition sizes are updated, covering the Parquet last-partition case that the re-open path misses. A native last partition already received the same value through the re-open, so its behaviour is unchanged, and the new logic runs only for replace commits that removed or trimmed partitions.
Decorrelation turns a scalar-aggregate lateral body into GROUP BY plus LEFT JOIN, which loses the empty-group row, and the previous compensation recognised only bare count() projections. As a result unmatched rows returned NULL for count(*) + 2, CASE WHEN count() = 0 THEN 7 ELSE count() END, count_distinct(x) and approx_count_distinct(x) bodies, and for sibling columns of a count, while WHERE sub.cnt = 0 filtered out the very rows it was meant to select. This fix builds a compensation template for any projection containing a zero-on-empty aggregate (count, count_distinct, approx_count_distinct), extracting aggregate leaves into hidden carrier columns and wrapping only zero-on-empty leaves in coalesce(x, 0), so count(*) + 2 returns 2 while count(*) + sum(v) correctly stays NULL; constants, outer references and bind variables now carry their value onto the compensated row, and WHERE terms over compensated columns are hoisted above the null-extension boundary so they see 0 rather than NULL. Keyed bodies are no longer treated as scalar, so an implicit group-by key, a SAMPLE BY body, or a WINDOW JOIN/HORIZON JOIN body correctly returns NULL instead of a fabricated 0, and CROSS JOIN LATERAL/INNER JOIN LATERAL over a scalar aggregate body are promoted to LEFT OUTER so outer rows are no longer dropped. PIVOT empty cells for approx_count_distinct now default to 0 instead of NULL, count(*) parses in any expression position so coalesce(count(*), 5) and count(*) + count(*) no longer fail with too few arguments for '*', statement rewrites now walk OVER() sub-expressions (also repairing :: casts and multi-WHEN CASE inside a window specification), and a negative LIMIT reaching a lateral body through a bind variable or outer column now fails with negative LIMIT is not supported in a correlated lateral sub-query instead of silently emptying the body. For example:
```
SELECT o.id, s.c
FROM orders o
LEFT JOIN LATERAL (
  SELECT count(*) + 2 AS c FROM trades t WHERE t.id = o.id
) s;
```
now returns 2 for unmatched rows rather than NULL. Window-function bodies such as count(*) OVER () are deliberately left uncompensated, since a window function produces no row over an empty input.
OrderedMap compresses each hash-table slot's heap offset into an unsigned 32-bit int with a +1 bias, but eight slot-emptiness scans used signed comparisons, so above roughly 16GB of map heap live entries were read as empty: the destination side of a shard merge wrote duplicate groups, while the source side silently dropped whole groups from a parallel GROUP BY. All scans now route through a single unsigned predicate. Five budgeted structures (OrderedMap, LongChain, AbstractRedBlackTree, LongTreeChain, LimitedSizeLongTreeChain) previously rejected a growth step whenever one doubling overshot the configured ceiling, producing memory-limit errors well below it, and now clamp to the ceiling instead, with the tradeoff that a runaway query can hold up to twice as much native memory before being stopped. This fix also corrects a cached ORDER BY ... LIMIT $1 returning wrong rows when the bind value flipped sign with cairo.sql.orderby.sort.enabled=false, since the cursor never re-derived its first-N early-stop flag per execution; an ArrayIndexOutOfBoundsException when a window ORDER BY had more terms than the model's, now handled by a single shared order-dismissal check; allocation-ordering and lifecycle defects in the multipart boundary augmenter, plus native memory leaks in HttpHeaderParser, HttpConnectionContext and the platform HttpClient constructors when a later allocation failed, with the same rollback applied to the Influx Line Protocol HTTP client fork; and initial heap pages that exceeded their compressed-offset ceilings and handed out truncated offsets. Budget floors were added for LongChain and SingleRecordSink, and overflow messages now name the owning feature (ASOF join, RANK() window function, DENSE_RANK() window function) and the configuration key to raise. Configuration validation is stricter: cairo.sql.small.map.page.size and cairo.sql.join.metadata.page.size reject values below 4, cairo.sql.hash.join.light.value.page.size rejects values below 12, cairo.sql.small.map.page.size and cairo.sql.sort.key.page.size reject values above 34,359,738,352, and cairo.sql.sort.light.value.page.size rejects values above 17,179,869,176. Additionally, top-K eviction now tracks the extreme node incrementally instead of re-walking the tree spine on every accepted insert, and hot-loop slot reads in OrderedMap and grandparent lookups in AbstractRedBlackTree.fixInsert() were reduced to single accesses.
This fix prevents squashSplitPartitions from destroying the tail of a variable-size column (BINARY, STRING, VARCHAR) in the partition it squashes into. The squash appends into the target partition through the frame's own file descriptors, so when that target is the partition the writer still holds open, the writer's column append memories keep their pre-squash positions and describe a shorter file than what is on disk; the stale append memory then trims the file back on close and a later memory re-extends it zero-filled, cutting the last value's length prefix in half. With assertions disabled this produced silently wrong query results, and otherwise surfaced as binary is outside of file boundary or an unsafe memory access fault. The writer can end up holding a non-last partition open because openLastPartition() no-ops when the last partition is Parquet, which pins the writer to an earlier native partition. When the squash target is the writer's open partition, the stale view is now dropped by closing without truncating and re-opening the target so the writer resumes from sizes that match disk, with partitionTimestampHi saved and restored and both arms running under a single distress guard. Both close calls in squashSplitPartitions now drain the posting-index seal-purge outbox first, so superseded .pv and .pc files are published for purge rather than leaked on disk. This fix also hardens the POSTING index: close() no longer sizes a mapping up just before releasing it (avoiding a wasted append-page allocation) and swallows a failure of its own log record so the non-throwing close contract holds; the gen-directory TXN_AT_SEAL clamp is now guarded against brand-new entries, a MAX_VALUE regression on an in-place extend throws, and six call sites that previously fell back to a 0 tag now arm the correct transaction first. The per-publish checkGenDirMonotonic scan is replaced by an assertion-only check of the slot just written, so production builds without assertions no longer detect an at-rest gen-directory regression on the writer side. On the reader side, structural checks now reject all-zero and partially zeroed generation slots that the tag sequence alone could not see, and the resulting CairoException names the column and the recovery step, for example posting index is corrupt, rebuild it with REINDEX TABLE <table> COLUMN <column> LOCK EXCLUSIVE; queries that previously returned a silently short scan over a damaged index now raise this error instead.
This fix allows SHOW CREATE DATABASE to be used as a parenthesized query source in FROM, CTE, and JOIN contexts, while filters and INCLUDE (...) / EXCLUDE (...) clauses keep their existing top-level behaviour. The parser previously read clause delimiters with helpers that, in subquery mode, consumed a closing ) while treating it as the enclosing subquery terminator, so the outer parser lost either the SHOW subquery delimiter or the category-list delimiter. Optional-clause lookahead is now performed without consuming the token, the enclosing delimiter is returned to the outer parser, and the category list's own closing delimiter is read locally. Valid top-level statements are unchanged; for malformed input that closes a valid category list but omits the enclosing subquery delimiter, the parser now reports the more specific outer error ')' expected instead of ',' or ')' expected.
This fix makes current_user() and session_user() report the principal that actually authenticated. Both functions previously returned the hardcoded default admin, because the allow-all and read-only security contexts used when per-user authorization is not enforced reported a fixed principal, so the Web Console displayed admin for every user, including a configured http.user or a PostgreSQL Wire Protocol user. AllowAllSecurityContext and ReadOnlySecurityContext now carry a principal and expose a derivation method that returns a context reporting the authenticated user, and the security context factories pass the authenticated principal through it. Anonymous requests allocate nothing, since the derivation returns the singleton unchanged for a null, empty, or default-matching principal. Because the HTTP path re-derives the security context on every request while PostgreSQL Wire Protocol and Influx Line Protocol derive once per connection, derived contexts are cached per principal in a concurrent map that is softly bounded at 256 principals; past that bound an unseen principal degrades to allocating per call rather than growing the map without limit. The derivation is final and constructs instances through an overridable factory method, with an assertion rejecting subclasses that do not override it so a restrictive context can never be silently downgraded into a permissive one; the deny-all and materialized-view refresh contexts return themselves and are never re-derived. The transient principal is copied before it is retained. current_user() and session_user() are now runtime constants that snapshot the principal in init() rather than reading the security context once per row, so a 1000-row query drops from 1004 to 5 security-context reads for current_user() and from 1000 to 1 for session_user(); results are unchanged because the security context cannot change mid-cursor and init() runs on every getCursor().
This fix prevents a POSTING index writer from dereferencing a closed value-file (.pv) mapping during rollback after an I/O failure, which previously produced an assertion failure, a null facade access, or a native-memory crash depending on the path. Two fault states are covered: when growing .pv during flushAllPending() fails to remap, the value memory is unmapped while the pending batch remains, so a subsequent rollbackValues() re-entered the flush and appended through closed memory; and when truncate() fails to open the replacement .pv, the writer retains generation metadata while the value memory stays closed, so a later rollbackValues() could reach reencodeAllGenerations() where generation zero resolved to native address zero and terminated the JVM with SIGSEGV. The sparse and dense flush paths now reject non-empty work when the value memory is closed, reencodeAllGenerations() rejects a closed value memory when generations exist, and all guards raise a CairoException before any scratch allocation or posting-index mutation. In addition, MemoryCMARWImpl.close() now clears lim and appendAddress after releasing the mapping, since the stale bounds could let checkAndExtend() accept an address without remapping and turn post-close use into an access through unmapped memory; reopening the instance reconstructs both bounds. Table-level recovery is unchanged: the caller still distresses and discards the affected writer and the on-disk chain is recovered on the next open, with the invalid memory access replaced by a controlled exception, and the close-on-remap-failure policy and the ordering of truncate()'s close and replacement open are untouched.
This fix stops ALTER TABLE ... ALTER COLUMN ... TYPE from killing the JVM with a native SIGBUS when disk space is low. The destination column file was sized with ftruncate(), which only sets the logical file length and does not reserve real disk blocks, so a later native write into an unbacked page faulted with a signal that cannot be caught from Java or JNI. The conversion now reserves space through TableUtils.allocateDiskSpace() (posix_fallocate) before the file is mapped for writing, matching every other writable memory-mapped call site in the codebase, so a disk-space shortfall surfaces as a catchable CairoException and the statement fails with a normal error instead.
This fix ensures the OIDC state parameter is verified whenever the server sets acl.oidc.state.required. Previously setupOAuth2 compared the state parameter only when a state value already sat in localStorage, so a browser that never started a login skipped the check and exchanged any ?code= sent to it. A missing or mismatched state now results in logout({ promptForLogin: true }), and OAUTH_STATE is cleared on every code path rather than only inside the old branch.
This fix renames all @questdb/mcp-bridge occurrences to the new package name @questdb/mcp-server-questdb, and replaces MCP bridge wording with MCP server. It also bumps the expected MCP version to 0.4.0.

August 6, 2026

QuestDB 10.0.0 is a major release that introduces three key features into general availability: QWP (QuestDB Wire Protocol), a compressed binary columnar protocol over WebSockets that supersedes ILP and PG Wire for both ingress and egress; Live Views for incrementally maintaining window-function result sets with millisecond update latency; and Web Console 2.0 with Jupyter-style notebooks and coding-agent integration. Beyond the headline features, this release delivers significant performance improvements—including faster sorting, smarter partition pruning, and per-query memory limits—alongside extensive bugfixes and enhanced Parquet interoperability with tools like Iceberg, Spark, DuckDB, and Trino, making it the most reliable version yet.

Breaking Changes

This change updates what existing objects report to PostgreSQL Wire Protocol catalogue clients. Materialized views now report pg_class.relkind as 'm' (previously 'r') and information_schema.tables.table_type as MATERIALIZED VIEW (previously BASE TABLE), with is_insertable_into set to false. Views now report relkind as 'v' (previously 'r') and table_type as VIEW (previously BASE TABLE), also with is_insertable_into set to false. Plain tables remain unchanged. Clients that filter on relkind = 'r' or table_type = 'BASE TABLE' will stop listing existing materialized views and views.
Every SecurityContext implementation must now supply authorizeLiveViewCreate and authorizeLiveViewDrop methods, including implementations outside the main repository.
This fix prevents compiler-internal protective double quotes from leaking into result set metadata. The SQL compiler wraps column aliases in double quotes when names contain dots (e.g., PIVOT values like FNCL 2.5 7/26) or collide with operator tokens (e.g., in, and), but these quotes were appearing verbatim in Web Console, PostgreSQL Wire Protocol clients, and CSV exports. The fix introduces SqlUtil.isQuoteProtectedAlias to recognize compiler-added quotes and SqlUtil.toColumnName to strip them wherever a projection alias becomes a metadata column name. Result set column names now display cleanly: FNCL 2.5 7/26 instead of "FNCL 2.5 7/26", t.x+1 instead of "t.x+1", and a.b instead of "a.b". Additional follow-up fixes on this branch address a stale-quote leak after alias mutation, an infinite loop on empty PIVOT values, operator-token pivot columns breaking through joins, trailing space leaks in truncated aliases, NULL data from dotted JSON UNNEST keys, and store-reuse efficiency in alias dedup loops. This is a breaking change for clients that match column names by the old quoted format.

New Features

This feature introduces live views, which are incrementally maintained window-function results over a single WAL-backed base table, queried like a regular table. Window functions run once per row as new base commits are refreshed in, and queries against the live view scan precomputed output rather than reprocessing the base on every read. The architecture targets use cases where the same window aggregate is read frequently against high-rate ingestion, such as rolling VWAP, cumulative volume, running ranks, and day-over-day comparisons. Live views are created with CREATE LIVE VIEW syntax specifying FLUSH EVERY, IN MEMORY, optional PARTITION BY, and a required START FROM clause (NOW, BEGINNING, or an explicit timestamp literal). They support anchored windows with ANCHOR expressions and DAILY sugar for cumulative aggregates that reset on a sentinel. A dedicated worker pool sized by live.view.refresh.worker.count runs the refresh workers. The in-memory tier leads disk by the un-flushed lead, serving recent rows from RAM and older applied prefix from disk via seam-ts routing with a seqTxn consistency fence. A versioned checkpoint timeline provides durable window state with copy-on-write B+ tree indexing, and out-of-order base commits derive a finite dependency interval from the window's frame for bounded repairs. Supported window functions include ranking (row_number, rank, dense_rank), cumulative aggregates (sum, avg, count, min, max, first_value, last_value, nth_value), lag, bounded-ROWS and bounded-RANGE aggregates, Welford variance/stddev/covariance/correlation, EWMA, and the full DECIMAL aggregate family. The live_views() catalogue function exposes state, lag, memory, seed metrics, and checkpoint information. SHOW CREATE LIVE VIEW, EXPLAIN, pg_class, information_schema.tables, tables(), and table_columns() all recognise the new table type. DROP LIVE VIEW and ALTER LIVE VIEW RESUME|SUSPEND WAL are supported for lifecycle management.
This feature adds opt-in memory limits that cap how much native memory a single bounded workload may allocate, throwing at the offending allocation site when the cap is crossed so a runaway is stopped at its source while unrelated workloads keep running. Three independent limits are provided: cairo.query.memory.limit.bytes for user SQL queries, cairo.mat.view.refresh.memory.limit.bytes for materialized view refresh, and cairo.wal.apply.memory.limit.bytes for WAL apply. All default to 0 (unlimited) and are dynamically reloadable via reload_config(). A MemoryTracker wraps a 16-byte native block shared with Rust, acquired per workload via QueryRegistry.register/unregister. Tracker-aware Unsafe.malloc/realloc/free overloads update both the global RSS counter and the per-query block. Wired sites include map family, sort/tree chains, hash-join chains, FastGroupByAllocator, LATEST BY rowid lists and maps, set-operation maps, encoded sort, window-join and horizon-join aggregations, SAMPLE BY fill, and parquet decode buffers. The query_activity view gains memory_used and memory_limit columns. On breach, the engine throws a CairoException with isOutOfMemory() set and a message identifying the workload type, query ID, limit, and memory tag. Coverage is best-effort; structurally bounded operators and the vectorized Rosti keyed GROUP BY hash tables remain on the global counter only.
This feature adds controls for managing hard-suspended WAL tables and a new ALTER TABLE <t> REBASE WAL statement to rebuild a table under a fresh sequencer. The cairo.wal.apply.suspended.tables configuration property is a reloadable, comma-separated list of table names that ApplyWal2TableJob skips, preventing WAL transactions from being applied. When cairo.wal.apply.suspended.write.denied is set to true, writes to a hard-suspended table are rejected instead of being queued. Non-structural ALTERs and FORCE DROP PARTITION are routed through the WAL-bypass path so they apply directly via TableWriter, while structural changes remain denied. ALTER TABLE SUSPEND WAL and RESUME WAL authorize through the same SecurityContext.authorizeResumeWal path, making them symmetric. REBASE WAL authorizes separately through authorizeRebaseWal, which defaults to authorizeSystemAdmin(). The rebase operation clones the applied table into a new directory via hard links into a .rebase/ staging directory, resets _txn and _meta with a new tableId, seeds two empty transactions, swaps the name registry, and drops the old directory. Rebasing a base table invalidates dependent materialized views, while rebasing a materialized view itself re-registers it for a full refresh. Preconditions require the table to be a WAL table, hard-suspended, and have write-deny enabled. Force-applied changes bypass the WAL and are not propagated to replicas.
This feature enables ALTER TABLE ... ALTER COLUMN ... TYPE to work on tables containing parquet partitions. Previously, the operation supported native partitions only and either failed or silently left parquet data unconverted. Conversion is lazy: parquet partitions are not rewritten by the ALTER, and decoding cost moves to the query path with a one-time decode per row group buffered for subsequent rows. O3 merges that land on a converted partition rewrite it with the new type. Conversions involving SYMBOL and chained conversions where the parquet column type no longer matches current metadata eagerly convert affected parquet partitions to native before the ALTER. Each ALTER COLUMN TYPE creates a new column index with a replacingIndex link to the previous index, and O3PartitionJob and PageFrameMemoryPool walk this chain to map current writer indexes to columns inside older parquet files. The query path carries a per-column conversion strategy and converts at accessor level using pooled sinks for zero-GC operation. Fixed-to-fixed conversions happen in the Rust decoder via post_convert(). Null sentinels, BYTE/SHORT/CHAR column tops, date/timestamp scaling, and UTF-16/UTF-8 transcoding all match the native partition behaviour.
This improvement moves covered-index column decode for aggregation and filter queries off the single dispatch thread onto the async reduce-worker pool, eliminating the dispatcher serialization that made the covered plan 3-4x slower than necessary. The covered decode now parallelizes 3-4x, and for selective predicates the covered plan is significantly faster than a full scan. count(*) WHERE sym = '<lit>' is answered from posting-index metadata with no decode at all, using O(genCount) countMatchesClamped and selectKthMatch random access into the posting index. A CoveredColumnDecoder with per-pool CoveringBuffers decodes covered columns from the sidecar and synthesizes the symbol key on both flyweight and record fast-paths, supporting all column types including variable-size (VARCHAR/STRING/BINARY/ARRAY). Concurrent-read-safe posting readers use detached per-worker cursors with a freeze before dispatch so workers never mutate shared reader state. Projection and null-pad wrappers propagate covered metadata through their column remap. Benchmarks show 2.6-3.8x speedup from parallelization on decode-heavy shapes, and at 0.1% selectivity the covered plan is 15-19x faster than a full scan for aggregations and 614x faster for count().
This feature provides a schema-only logical dump that emits round-trippable QuestDB DDL for every user object in the database, one statement per row. Since pg_dump cannot run against QuestDB due to absent pg_* catalogs and QuestDB-specific DDL (PARTITION BY, SYMBOL CAPACITY, timestamp(), DEDUP, TTL, IN VOLUME, BYPASS WAL, storage policy), this command emits native QuestDB SQL instead. Objects are emitted in dependency order: tables first, then materialized views (whose base tables are already emitted), then views (topologically sorted via the view graph so view-on-view replays succeed). The command supports INCLUDE / EXCLUDE filtering by category (TABLES, VIEWS, MATERIALIZED_VIEWS, SCHEMA, ALL). Concurrent DDL during the dump no longer fails the entire operation — objects dropped between the initial snapshot and their emit time are skipped with a logged warning rather than causing an error.
This feature enables <, >, =, <=, >=, !=, and <> comparisons between numeric columns and single-column scalar sub-queries. Supported left operand types include DOUBLE, FLOAT, LONG, INT, SHORT, and BYTE. For example: SELECT * FROM trades WHERE price > (SELECT avg(price) FROM trades WHERE timestamp IN today()). The sub-query is evaluated once per query execution and the result is cached. Integer operands compared against integer cursor values use long comparison to preserve precision beyond the exact integer range of double. Zero rows produce no match, one row supplies the cached scalar value, and more than one row raises an error. Only the <, >, and = factories are registered explicitly — FunctionFactoryCache derives the remaining operators and argument-swapped forms automatically. The implementation includes proper worker state donation for parallel execution paths including async filters, parallel GROUP BY, window joins, and horizon joins.
This feature adds statistical aggregate functions for kurtosis and skewness analysis: kurtosis() / kurtosis_samp() for sample excess kurtosis, kurtosis_pop() for population excess kurtosis, skewness() / skewness_samp() for sample skewness, and skewness_pop() for population skewness. For example: SELECT skewness(price), kurtosis(price) FROM trades WHERE symbol = 'BTC-USD' SAMPLE BY 1h. The implementation uses Pébay's one-pass online algorithm (an extension of Welford's), maintaining running mean and central moments (M2, M3, and M4 for kurtosis) plus a count per group. Partial aggregates merge with pairwise-combine formulas, enabling parallel GROUP BY execution. Sample skewness returns null for fewer than 3 values, sample kurtosis for fewer than 4, and all variants return null when every observation is equal (zero variance).
This feature introduces two new SQL functions built on a worker-continuation framework using jdk.internal.vm.Continuation. While these functions are parked, the worker carrier is released to run other queries, so the number of concurrent calls is no longer bounded by the worker pool size. wait_wal_table(table_name [, seq_txn]) blocks the current query until the WAL writer for the specified table has applied transactions up to a target seq_txn, returning true on success. When seq_txn is omitted, it captures the table's current sequencer transaction at start time. It throws if the table becomes suspended or is dropped, and respects the SQL circuit breaker for timeout and cancellation. sleep(seconds) pauses the current query for the given duration (up to 24 hours) and emits a single TIMESTAMP row containing the server's wall-clock time at wake. Both functions respect query timeout, CANCEL QUERY, and broken client connections. The continuation runtime includes WAL commit notification to fire waiters as soon as the target transaction lands, and a TimerShards daemon pool for periodic wakeups. New configuration properties include cairo.timer.shards (number of timer daemon threads, defaults to min(4, max(1, cpu_count / 4))) and griffin.query.continuation.wake.interval (millisecond interval for circuit breaker probing during park, default 1000).
This feature introduces a first-run onboarding experience for Notebooks in the Web Console. A two-step onboarding modal guides users through Notebooks capabilities (organizing queries, results, and charts into dashboards) and introduces the QuestDB MCP connection with a copy-ready setup command and an animated agent transcript. An in-notebook MCP promotion card is surfaced inside notebooks across list, grid, and empty states, and can be collapsed or dismissed. Visibility is driven by a notebook.onboarding localStorage blob with flags for MCP connection status and promotion display, and all onboarding surfaces are permanently suppressed once an MCP bridge has connected.
Agents can now edit any notebook in the background without stealing focus from the user. This feature introduces a Dexie-based controller for queued operations on unmounted buffers, a per-buffer FIFO queue to strictly order all writers, and a mount-claim protocol that handles conflicts when a user opens a buffer mid-edit. Headless cell execution runs cells on unmounted notebooks with a verification outcome machine covering scenarios such as notebook deletion, user mounting mid-run, and cell changes mid-run. Freshness gating rejects mutating tools with STATE_STALE if the user edited the notebook after the agent's last read. An activate_notebook tool brings a buffer to the foreground on user request. Footer notifications surface agent edits to non-active buffers as an unread-style indicator on the MCP pill with a transient popper and a persistent row in the pair popover.
This feature reworks the notebook cell experience with a width-aware toolbar that adapts to the cell's width across compact, standard, and expanded layouts. A table-to-chart view toggle transfers existing data between the grid and chart without re-running the query. Inline cell rename makes the cell name the single source of the chart title, with automatic migration from the old chart config name on load. Per-cell auto-refresh supports adaptive, off, or fixed intervals (1s, 5s, 10s, 30s, 1m). MCP bridge version awareness detects version mismatches — a major mismatch blocks with an upgrade command, while a minor one connects but warns, surfaced in the pairing popover, consent modal, and footer status pill. Left and right sidebars now auto-close on shrinking window sizes.
This feature introduces a cell-based notebook surface in the Web Console where users can mix SQL, markdown, and chart cells in a single notebook, each independently runnable. Query results render in a virtualized grid with support for multi-statement cells split across result tabs, row-count/timing status, truncation indicators, and one-click CSV/Parquet download. Charts support line, area, bar, stacked bar, scatter, pie, and candlestick types rendered with ECharts, with auto-inference of chart type and encodings from results. Notebooks support DECLARE-style @variables reusable across cells, drag-to-reorder cells, focus mode, resizable grid/column layout, and full-text search spanning notebook cells alongside editor tabs. Notebooks persist locally via IndexedDB and survive reloads. A new MCP bridge enables external AI coding agents (Claude Code, Codex, Cursor, OpenCode, etc.) to connect to the running Web Console over a local WebSocket and operate it. Agents must go through explicit pairing via a consent modal showing what is connecting and what it can do. Granular, revocable permissions (read/write/schema scopes) default to read-only, and SQL that cannot be safely classified is treated as a write and denied. Each bridge run generates a fresh port and pairing token held only in memory. The AI assistant provider layer supports Anthropic, OpenAI (both Responses and Chat-Completions protocols), and custom providers.

Improvements

This improvement routes WAL commit blocks that are a pure append to the last native partition through the existing fast-lag append path, writing covered fragments incrementally (O(new rows)) instead of re-encoding the entire partition's covered sidecar (O(partition)). Per-block covered-index cost drops from O(partition) to O(new rows) on the common path, making WAL drain approximately 86× faster in benchmarks. The change also fixes a pre-existing concurrency bug in covering indexes shipped since 9.4.0, where a chain entry's cover-end footer could be clobbered during append, causing concurrent covered readers to read garbage and potentially crash with an out-of-bounds read. The fix de-aliases the footer to a fixed offset under a versioned layout for all new data, with automatic COW-migration of existing format-0 heads on first extend.
This improvement extends the encoded radix sort to every column type that ORDER BY accepts and to keys of any width, across both full-sort and serial/parallel top-K paths. Previously, the encoded sort only engaged when all ORDER BY columns were fixed-width and the combined key fit 32 bytes — everything else fell back to the red-black tree sort with O(log n) comparator calls per row. VARCHAR keys use UTF-8 bytes with a terminator, STRING and non-static SYMBOL use UTF-16BE with escaping, and UUID/LONG256 use unsigned big-endian with order-preserving null remapping. Variable-width keys use a 16-byte inline prefix with overflow into a key heap; rows whose prefix already exceeds the kept boundary in top-K queries skip the full key encode entirely. Benchmarks on the ClickBench hits table show improvements from 52ms to 37ms at LIMIT 10 and from 100ms to 85ms at LIMIT 100000. The key heap is bounded by cairo.sql.sort.key.max.bytes, and the tree sort remains available via the cairo.sql.orderby.sort.enabled kill switch. This change also fixes a pre-existing bug where generateCastFunctions had no LONG128 case, which could cause UNION queries with both SYMBOL and LONG128 columns to throw an error.
Previously, wrapping the designated timestamp in any function (e.g., date_trunc('day', ts), CAST(ts AS TIMESTAMP), dateadd('h', 1, ts)) defeated partition pruning and fell back to a full table scan with a row filter. This improvement lets the optimizer push such predicates back onto the timestamp axis when the wrapping function is monotonic. A predicate is recognized, the queried value range is inverted layer by layer back to a range on the designated timestamp, and the result drives the usual interval scan and partition pruning. Functions are graded as EXACT (inverse is exact, row filter dropped), SUPERSET (inverse soundly contains the true preimage but may be wider, interval prunes but original predicate stays as residual filter), or NONE (cannot bound, predicate stays a plain row filter). Covered functions include CAST, date_trunc, timestamp_floor, timestamp_ceil, dateadd with fixed units, year, to_timezone, to_utc, and chains of these functions. Constant bounds are inverted at compile time; runtime bounds (now(), bind variables) are deferred to evaluation when the scan opens. Recognition is fail-safe: any compile or parse failure falls back to a row filter, so queries that did not previously use this path are never broken. This improvement also fixes getNextDST behavior before a zone's first transition, adds overflow hardening for fixed-unit dateadd and timestamp_ceil inverses, and introduces domain-ceiling-aware overflow checks for constant-shift inverses.
Subexpressions whose value is invariant across the rows of a single cursor but not a compile-time constant were previously re-evaluated for every row when appearing inside per-row contexts such as CASE branches or filter predicates. For example, in SELECT sum(CASE WHEN ts >= dateadd('d', -30, to_timezone(now(), 'Asia/Kolkata')) THEN amount ELSE 0 END) FROM trades, the to_timezone(...) and dateadd(...) calls were repeated on every row despite producing the same result for the whole scan. This improvement wraps such subtrees in a RuntimeConstFunction that evaluates once during init(), caches the result in a primitive field, and serves it from every getter. The fold is applied at function boundaries only, wrapping the topmost runtime-constant node without nesting. Trivial runtime-constant leaves like bind variables and now() that already cache their values are skipped to avoid unnecessary indirection. Only fixed-width scalar types are folded; variable-length types are left untouched. The wrapper delegates toPlan() to its argument, so EXPLAIN output is unchanged.
Parallel GROUP BY, TOP K, and ASOF/window joins hold per-query AsyncFilterContext instances containing one-per-worker DirectLongList buffers for matched row IDs. Under a JIT filter these lists were pre-sized to the full page frame (up to 8 MB each at the default 1M rows) and only ever grew. Previously, clear() at cursor teardown freed page-frame memory pools but left these lists at their peak size, freeing them only on query-cache eviction. This meant idle or cached factories kept 1 + workerCount peak-sized buffers allocated for their entire cached lifetime. This improvement now calls resetCapacity() on the owner and per-worker lists during clear(), returning them to the 256-long initial capacity. Idle factories no longer pin peak-sized row-id buffers. The tradeoff is that a cached factory reused for another large scan pays one reallocation to grow back. This improvement also fixes a native memory leak where resetCapacity() could re-malloc a closed list when clear() was called after close() on the horizon-join factories' failed getCursor() path, and reorders the 4 horizon-join factories to free the cursor before the frame sequence to prevent this scenario.
This improvement makes two independent optimizations to checkpoint/snapshot restore. For tables with many Parquet partitions, each partition's parquet-metadata sidecar (_pm) is now mapped and checksum-verified once instead of up to twice mapped and three times verified, and the previously serial per-partition validation now runs in parallel across the recovery pool. Workers pull parquet partitions from a shared cursor with dynamic load balancing, reusing a single set of native scratch objects across every partition they handle. The committed-size truncation check on data.parquet runs inside the worker before regeneration, removing the O(N) serial ff.length() loop. For tables with few partitions but several indexed symbol columns, the native bitmap-index rebuild now parallelizes across individual (partition, column) work items rather than whole partitions, so a non-partitioned table no longer serializes all indexed columns onto one thread. A behavioral tradeoff is that truncated capture detection now occurs inside parallel workers rather than strictly before sibling work starts, but the first failing worker trips a shared abort latch that causes all other workers to short-circuit at their next item boundary.
This improvement makes large notebooks remain responsive as they grow by virtualizing heavy cell components. Cells outside the viewport no longer keep their full editor, chart, result grid, and refresh workload active, reducing render work, memory use, and background queries. Nearby cells render normally while distant cells use lightweight placeholders. Focused, running, and recently edited cells stay active, and full content returns when the user scrolls back. Offscreen chart refreshes pause and resume when needed, so notebooks with many live charts do not continuously query and render in the background. Saved query results can be released from memory and loaded again on demand while cell sizing and run status remain available. With these optimizations in place, the notebook cell cap increases from 50 to 200.
This improvement replaces the 943 KB login-background.svg (a Figma export with every glyph outlined as vector paths) with a ~7 KB React component that renders the same binary-rain text using the theme's monospace font and a CSS gradient mask. This reduces the Web Console's distribution size by ~950 KB and the background's transfer size from ~44 KB (gzipped) to ~2 KB, while keeping the text crisp at any resolution.
This improvement extracts the result grid into a self-contained, framework-native React component built on TanStack Table (columns, pinning, ordering) and TanStack Virtual (row and column virtualization for performant rendering of wide, deep results). The grid owns grid-only concerns: cell/keyboard navigation, copy cell and column, freeze-left, move-column-to-front, reset layout, debounced column resize, scroll shadows, and markdown export. Data is supplied through a pluggable ResultGridDataSource interface with two implementations: inMemoryDataSource for bounded, fully-loaded results and usePagedDataSource for server-paged unbounded results with sparse page cache, debounced fetch windows, outlier-page purging, and a generation guard preventing stale pages from applying to current queries. A ResultGridAdapter implements the legacy IQuestDBGrid interface so existing call sites remain untouched. Column layout persistence (sizing, order, pinning) is preserved via columnLayoutStore. The new grid is enabled by default behind a useNewGrid feature flag, with the legacy grid.js path available as a fallback via useNewGrid=false or the ?useNewGrid= query parameter.

Bug Fixes

The four VWEMA window functions held args.get(3) in a field of their own and read it via getDouble, but never propagated init, initPartitionBy, toTop, or cursorClosed to it. A volume sub-expression that resolves state in init() therefore ran unbound. For example, in avg(price, 'alpha', 0.5, CASE WHEN side = 'BUY' THEN qty ELSE 0.0 END) OVER (PARTITION BY sym ORDER BY ts), the symbol comparison resolves its constant in init(), so un-initialised it reported "no such symbol" for every row, causing the CASE to weight every row 0.0 and the function to read NULL throughout instead of the volume-weighted average. All four classes now propagate all four hooks.
This fix addresses an issue where WindowContextImpl.of() converted a frame bound into the designated timestamp's units through TimestampDriver.from(long, char), which checked neither the multiply for overflow nor the int narrowing that followed. Over a nanosecond designated timestamp, RANGE 200000 DAY PRECEDING wrapped onto a positive value and was refused as a FOLLOWING frame start, RANGE 300000 DAY PRECEDING wrapped negative and silently returned rows over a 236-year frame where 821 years were asked for, and RANGE 4294967296 DAY PRECEDING narrowed to exactly 0. All three drivers now range-check the conversion and report the offending bound at its own position with a human-readable message including the unit name.
A page-frame cursor's fast skip path counted physical partition rows across row groups that parquet-level pushdown pruning had already discarded, so a resumed skip landed short and re-fed already-consumed rows. PageFrameCursor.hasActivePushdownFilter() now routes skipRows to the row-by-row path while pruning is active.
Making pushdown a third trigger for the row-by-row path exposed a defect where the walk advanced through hasNext(), which charged every row against the post-skip decode clamp. A clamp of 0, which is what LimitRecordCursor passes to size a cursor it will not read, made the walk skip nothing. This caused calculateSize() to report 0 rows for queries like WITH cte0 AS (SELECT * FROM t WHERE c2 IS NOT NULL LIMIT 16) SELECT * FROM cte0 LIMIT 40. The clamp now stays off while the walk runs and arms once the skip lands.
On rehydrate, a materialized view created but never refreshed before the demote came back valid and empty with watermark -1, and nothing kickstarted it. buildViewGraphs now enqueues the initial incremental refresh for a never-refreshed REFRESH_TYPE_IMMEDIATE view, mirroring the kickstart the persisted-state path already performs.
This fix addresses an issue where PushdownFilterExtractor pushed both null operators onto parquet row-group pruning for every column type, but a parquet null bit and a SQL NULL denote the same rows only for some types. BOOLEAN, BYTE, and SHORT carry no null sentinel, so a row group built entirely from column-top rows reported null_count == num_values and pruning discarded it for IS NOT NULL even though every one of its rows matched. CHAR, FLOAT, and DOUBLE fail the other way because their SQL NULL is broader than what the writer marks null. The extractor now refuses only the unsound direction per type.
This fix addresses an issue where isOlderThanTtl converted an hours-based TTL with an unchecked multiply. Values up to TTL 2147483647 HOURS could reach the conversion against a ceiling of 2562047 hours for nanosecond timestamps. TTL 106752 DAYS, the smallest DAYS value that trips it, wrapped past Long.MAX_VALUE into a negative span that every partition compared older than, so the next commit dropped every non-active partition silently. A TTL the timestamp type cannot express now expires nothing, guarding at the point of use rather than at the DDL to cover CREATE TABLE, ALTER SET TTL, CREATE TABLE LIKE, restored metadata, and tables written by older binaries.
The four WINDOW JOIN bound conversions called TimestampDriver.from(long, char) directly with the same unchecked multiply and int narrowing as the plain RANGE frame. On a nanosecond designated timestamp, RANGE 300000 DAY PRECEDING wrapped onto 86496 days and RANGE 4294967296 DAY PRECEDING narrowed to exactly 0. Two of the four sites fed converted bounds into intrinsicModel.mergeIntervalModel, so a wrapped bound narrowed the slave scan interval and dropped rows before evaluation. WindowContextImpl.toTimestampUnits becomes the single shared guard and all four sites call it. Queries whose bounds previously wrapped silently now fail to compile.
A client that was writing into a materialized view via QuestDB Wire Protocol and seeing no error will now receive an error. Previously, such writes were silently dropped.
A snapshot or checkpoint could capture a parquet partition whose on-disk data.parquet is a later generation than the committed parquet size recorded in _txn. The restore path trusted the partition's existing _pm sidecar in that state and left it stale, so the first read or merge after restore decoded a column chunk that lies past the committed size, resulting in a File out of specification error. On a replica that replays the WAL over the restored partition, the ApplyWal2TableJob apply fails and the table is suspended, breaking replication. This fix only trusts an existing _pm when data.parquet is exactly the committed size. When the file is longer (the snapshot captured the partition mid in-place rewrite), _pm is regenerated from the committed size. Regeneration fires only for genuinely torn partitions and is correctness-preserving, rebuilding a compact _pm for the committed footer.
This fix addresses a production out-of-memory incident on a memory-constrained instance ingesting a skewed symbol column. The WAL fast-lag commit fires an inline seal once the generation count reaches MAX_GEN_COUNT. The incremental seal branch sized its per-stride merge/trial buffers as if all 256 keys in a stride held the single hottest key's row count, inflating the allocation by up to ~256x beyond the data the stride actually merges. The incremental seal now sizes its buffers to the actual aggregate of the dirty strides, and a pre-flight defers to the full seal when even the correctly sized buffers would breach the RSS limit. The rollback path was also reworked to stream per key for every cutoff shape, bounded by the largest single key instead of the whole index, eliminating the whole-index decode that was the memory-heavy case. Additional hardening includes: poisoning the writer on post-switch failures so mutating entry points cannot re-drive publishing at a staged seal transaction, bounding native writes in posting decoders against corrupt or torn input to prevent out-of-bounds writes, and adding a live-head check to the seal-purge operator to prevent deletion of live files after a seal transaction is freed and reused.
Two malformed expressions involving CASE ... END crashed the query with an internal error instead of returning a syntax error. A binary operator with a missing right operand directly after a CASE ... END expression (e.g., SELECT SUM(CASE WHEN true THEN 1 ELSE 0 END & )) crashed with a NullPointerException because the expression parser's END keyword handler left a stale depth counter of 1, which defeated the arity guard and allowed a null left-hand side to be assigned. The fix resets the depth counter to 0 after the flush loop so the arity guard fires correctly, producing a clear error like too few arguments for '&' [found=1,expected=2]. A dot directly after CASE ... END (e.g., SELECT CASE WHEN true THEN 1 ELSE 0 END.foo) also crashed because the dot handler attempted to cast the top of the operator stack to a FloatingSequence literal, which failed with a NullPointerException or ClassCastException. The fix extends the token only when the stack top is a FloatingSequence literal and otherwise raises a syntax error. Valid queries produce the same parse as before; only the previously-crashing inputs change behavior.
This fix addresses a storage-engine corruption that could occur when a WAL REPLACE_RANGE commit in O3 mode appends brand-new partitions above the table's previous last partition. In TableWriter.processO3Block, replace mode derived partitionTimestampHi from o3TimestampMax, which is the replace-range high boundary rather than the highest timestamp actually written. For an open-ended range the boundary is Long.MAX_VALUE - 1, and getCurrentPartitionMaxTimestamp overflows on it. As a result partitionTimestampHi stayed at the stale pre-commit ceiling, finishO3Commit skipped switching the writer's active column files to the new last partition, and the next commit reused the previous partition's column descriptors, producing rows dated below the new partition's floor or suspending the table. The fix derives partitionTimestampHi from the actual highest written timestamp in replace mode. The non-replace path is unchanged, and there is no measurable performance impact.
A SAMPLE BY query that aggregates with first() / last(), filters on an indexed SYMBOL column, and uses ALIGN TO FIRST OBSERVATION could return a bucket timestamp where an aggregated value was expected. The wrong results appeared only when the designated timestamp column was left out of the SELECT list, and only for the buckets in the middle of the result. The first and last buckets were always correct, which made the problem easy to miss. There was no error and no crash, so an affected query silently produced a plausible-looking timestamp in place of the value. The root cause was that SampleByFirstLastRecordCursorFactory emitted the middle (non-boundary) rows through its data record, which special-cased the bucket-timestamp column using the base-table timestamp index rather than the projection index. The two indexes coincide only when the timestamp is projected at the same position it occupies in the base scan. When the timestamp was not projected, the base index pointed at an unrelated projected column and overwrote it with the bucket timestamp. The fix makes the data record use the same projection index as the boundary record. Adding the timestamp to the projection, switching to ALIGN TO CALENDAR, or dropping the indexed-symbol filter all avoided the problem because each routes the query away from the affected execution path.
This fix introduces three rounds of enhancements to the seeded query fuzzer and addresses 25 distinct defects surfaced by them. Cursor self-consistency checks verify that re-iterating after toTop() reproduces the result set, that preComputedStateSize() remains unchanged, and that size() and calculateSize() agree with the materialized row count. Fault injection arms filesystem, native allocation, and function faults on a fraction of queries to exercise error-path cleanup. Key engine fixes include: SAMPLE BY FILL(LINEAR) cleanup on out-of-memory no longer throws NullPointerException; covering posting index crashes on sidecar I/O errors now propagate the real failure; calculateSize() for backward interval scans now mirrors next() for correct row counts; DATE-argument window value functions (max, min, first_value, last_value, nth_value, lag, lead) no longer throw UnsupportedOperationException; rank() and dense_rank() no longer crash when a pass-through column of an unserializable type (UUID, STRING, VARCHAR, LONG256, BINARY, array, INTERVAL) is projected; lead() over high-precision DECIMAL no longer crashes; window RANGE-frame functions now use the correct designated timestamp index when the timestamp is not in the projection; sum() and avg() over Decimal256 sliding frames now evict values with scale-agnostic subtraction; TopK / ORDER BY ... LIMIT cleanup no longer crashes when a tree-chain allocation fails mid-construction; JIT filter now correctly widens nested INT products to 64 bits when a FLOAT operand is present; keyed ASOF JOIN sink-heap no longer leaks on out-of-memory during cursor open; vectorized rosti GROUP BY no longer over-frees NATIVE_ROSTI memory on wrapUp() resize; stale aggregate tasks in the shared vector-aggregate queue no longer crash later queries; GROUP BY with a trivial arithmetic expression key referenced by alias now compiles correctly; multi-key index scans no longer leak posting-index block buffers on out-of-memory; and window functions over untyped null literals now raise a clean SqlException suggesting a concrete cast instead of behaving non-deterministically across platforms. A PageFrameAddressCache.of() now reopens lists atomically so a failed reopen leaks nothing.
When a non-WAL writer ingests out-of-order data and the in-order prefix crosses a partition boundary, the partition switch sealed the earlier partition in memory but did not persist _txn. On the next lag commit, o3MoveUncommitted reclaimed the active partition's uncommitted in-order tail into the O3 buffer and reset maxTimestamp to the durable _txn value, which predated the switch. This caused the high-water mark to end up below committed data, so a rollback reloaded it and the next O3 commit reordered the partition tail. This fix introduces a new variable to track the max timestamp of the last sealed partition, updating it whenever a partition switch happens, ensuring maxTimestamp is never reset below the actual committed maximum.
This fix resolves an intermittent NullPointerException that occurred under concurrent queries when using a window function over a UNION ALL whose first branch was an IN-list scan served by a posting/covering index. The root cause was that CoveringIndexRecordCursorFactory stored a direct reference to the compiler's pooled keyValueFuncs list from IntrinsicModel. Since SqlCompiler instances are pooled and shared across threads, another thread borrowing the same compiler could clear the list between compile and execute, nulling out slots that the first thread's getCursor() then attempted to read. The fix creates an owned copy of the list in the constructor, preserving the same Function instances that the factory already owns and frees on close.
Subtracting a CHAR or STRING literal from a timestamp (e.g., SELECT * FROM t WHERE timestamp > now() - '1' LIMIT 1) failed with an internal error. The SubTimestampFunctionFactory read the right operand with getTimestamp() instead of getLong(), and calling getTimestamp() on a CharFunction threw UnsupportedOperationException. The crash also occurred on the interval-optimizer path via RuntimeIntervalModel. This fix changes the factory to read the right operand with getLong(), mirroring the symmetric AddLongToTimestampFunctionFactory which already did this correctly. As a result, - and + now produce identical, well-typed cast errors for non-numeric literals.
This fix resolves a data-ordering bug in the non-WAL out-of-order (O3) commit path where a lag commit could leave the table's maxTimestamp below the maximum timestamp actually committed to disk. When uncommitted rows of a lag commit spanned more than the active partition, o3MoveUncommitted() pulled only the active partition's rows into the sorted O3 batch. The in-order rows left on disk in the previous partition stayed committed but were not part of the sorted O3 batch, so maxTimestamp was recomputed solely from the O3 batch boundary and could end up lower than the true on-disk maximum. A later O3 commit then merged against that stale boundary and produced a single timestamp inversion. The fix reads the previous partition's actual max timestamp via readPartitionMinMaxTimestamps() and folds it into the committed maxTimestamp computation when the active partition is emptied while uncommitted rows remain in an earlier partition.
Equality and IN filters on a renamed column of a Parquet partition could return too few rows (often zero), silently dropping valid data. The defect was in Parquet row-group bloom-filter pushdown, which resolved the filtered column against the Parquet file's column names. Those names are frozen when the partition is converted to Parquet, so a later RENAME COLUMN left them stale. When another column already carried the query's current name (common after a chain of renames), the pushdown landed on the wrong Parquet column, checked that column's bloom filter, got a false negative, and skipped the entire row group. This fix makes the native-table pushdown resolve columns by stable column id, the same way the data-decode path already does. read_parquet() cursors retain name-based resolution, which is the correct semantics for arbitrary external files without QuestDB column ids. Pruning effectiveness is unchanged for the common non-renamed case, and the fix has negligible cost since getColumnIndexById is a linear scan run once per partition during filter-list preparation.
Many SQL query factories only consulted the execution context's circuit breaker inside their row-processing loop, which never runs for queries that produce no rows (empty tables, no-match filters, empty joins/aggregates), for instant single-row results, or for parallel paths that dispatch nothing when the input is empty. These queries ignored CANCEL QUERY and query timeouts entirely. This fix adds breaker consultation at the point work begins across all affected factories — at the top of hasNext() for joins and latest-by cursors, before build/aggregate loops for group-by and window functions, at cursor open for empty-cursor singletons, and before dispatch on parallel paths. A new time-throttled breaker variant always tests cancellation and timeout (both cheap, no syscall) while throttling only the heavy connection probe syscall by elapsed wall-clock time, defaulting to a 100ms window. Every query now pays for one real breaker check on its first consultation, including a single recv(MSG_PEEK) connection-probe syscall. This is a behavioral change: a query that previously ran to completion under an aggressive timeout can now abort, including catalogue and admin listings.
The incremental posting-index seal took an unguarded covered-column sidecar snapshot into native memory before the RSS pre-flight ever ran, so a skewed symbol with a covered (INCLUDE) column could still trigger an OOM and suspend the table. On a skewed symbol, a single partition's covered sidecar could be multi-GB, and the eager malloc tripped the global RSS limit during a WAL fast-lag commit. This fix adds the same headroom pre-flight before the snapshot copy, re-reading RSS memory usage per cover so it accounts for snapshots already taken in the loop. When the copy would not fit the live headroom, the incremental candidacy is cleared and execution falls through to a full seal, which rebuilds every sidecar by streaming per-key from the column files and needs no snapshot, with peak memory bounded by the largest single key rather than the entire sidecar.
When a file's length could not be read during metadata loading, MemoryCMRImpl.of() read errno after a cleanup close() call, which could overwrite the errno left by the failed length() via a native close() syscall. Since metadata reload classifies failures by errno to decide whether to retry, a clobbered errno caused transient "file does not exist" conditions — such as a _meta file briefly unreadable during a concurrent metadata change — to be misclassified as fatal. Instead of retrying until the spin-lock timeout and reporting a clean "Metadata read timeout," the lower-level "could not get length" error surfaced. This fix captures errno immediately after the failed length() call, before close() can overwrite it.
A materialized view's incremental refresh could fail transiently when the base-table reader pool was exhausted ("table busy") or when a refresh query ran out of memory. Previously both were treated as fatal — the view was invalidated immediately, and a busy failure cascaded up the dependent-view chain. This fix routes transient errors through a non-blocking, capped, timer-driven retry path. On transient failure, a per-view backoff deadline is scheduled and the view reports a new retrying status queryable via materialized_views.view_status. Each consecutive failure that does not advance the refresh watermark bumps a counter, and once it exceeds cairo.mat.view.refresh.busy.retry.limit (default 10) the view is invalidated, bounding WAL retention. The counter resets on any successful refresh. Additionally, a new cairo.mat.view.refresh.block.list configuration option accepts a comma-separated list of materialized view names that the refresh job must never refresh, serving as an escape hatch for views whose refresh crashes or destabilizes the database. Blocked views are skipped without being invalidated. The OOM path no longer calls Os.sleep between in-call step-halvings, avoiding blocking the single refresh worker that drains the entire task queue. New configuration properties: cairo.mat.view.refresh.busy.retry.timeout (default 1000ms), cairo.mat.view.refresh.busy.retry.limit (default 10), and cairo.mat.view.refresh.block.list (default empty).
The Pearson correlation denominator sqrt(sumXX * sumYY) could overflow to +Infinity for large-magnitude inputs (near +/-1e153) or underflow to 0.0 for small-magnitude inputs (near +/-1e-150), producing incorrect results of 0.0 or NaN respectively. This fix prefers the single-rounding sqrt(a * b) path when the product is finite and non-zero, and falls back to sqrt(a) * sqrt(b) when overflow or underflow is detected. The final Pearson result is clamped to [-1, 1] to absorb small rounding drift in the fallback path. The fix applies to CorrGroupByFunctionFactory, AbstractBivariateStatWindowFunctionFactory.computeCorr, and AbstractBivariateStatWindowFunctionFactory.computeCorrWelford. Normal-magnitude inputs are unaffected and retain their prior bit-exact results.
HTTP connections and contexts are reused across requests, but several parser and response objects were not fully resetting their per-request state, causing later requests to inherit stale state from earlier ones. In the worst case, a reused context could hit closed native parsing or compression buffers and crash the server. This fix reopens the header parser's quoted-value sink when pooled HTTP contexts are reused, clears per-request gzip negotiation state in HttpResponseSink, clears charset and mapped cookie state in HttpHeaderParser.clear(), fixes multipart false-boundary replay so body bytes are preserved when a boundary-like sequence is followed by non-boundary suffixes, and parses Content-Disposition parameters with quote-aware delimiter handling. Uploaded filenames containing semicolons or equals signs (e.g., a;b.csv, a=b;c.csv) now parse correctly.
A query cancellation could be silently dropped when the cancel request arrived during the narrow window between query registration and circuit breaker flag binding. NetworkSqlExecutionCircuitBreaker.cancel() always set the powerUpTime == Long.MIN_VALUE sentinel but only flipped the per-query cancelled flag when it was already attached. If a CancelRequest landed before QueryRegistry.register() bound the flag, the cancel survived only in the sentinel, which neither testTimeout() nor testCancelled() consulted. The result was that the query ran to natural completion ignoring both cancellation and timeout, pinning workers. This fix makes testCancelled() check the sentinel first so both stateful paths abort a racing cancel, corrects getState() to classify the sentinel as STATE_CANCELLED instead of STATE_TIMEOUT, and clears the sentinel in PGConnectionContext.prepareForNewQuery() to bound it to a single query.
Several per-partition code paths iterating posting columns were guarded only against absent columns (columnTop == -1) but not row-less columns (columnTop >= partition size), causing two distinct failures. In restorePostingIndexersToLastPartition, a row-less column with no .pk key file would throw index does not exist, causing WAL table suspension or direct commit failure. Conversely, blanket-skipping every row-less column was incorrect when one legitimately had a .pk (created by ADD COLUMN ... INDEX TYPE POSTING on the active partition), which would silently strand the indexer and make rows invisible to indexed predicates. This fix makes restorePostingIndexersToLastPartition discriminate by .pk file existence rather than row-less-ness, and aligns linkPartitionIndexFiles (used during parquet partition switching) with its sibling copyOrRebuildColumnIndexes by adding the full columnTop == -1 || columnTop >= partitionSize guard. The accessor was also changed from getColumnTopQuick to getColumnTop to correctly return -1 for absent columns.
The DataID class wraps a 128-bit UUID stored as two non-volatile long fields (lo/hi). The value is published by one thread (the replication WAL downloader via a JNI up-call) and read by another, but initialize() and the reader methods were not synchronized, while change() already was. This meant a reader could observe a torn half-written value or never observe the publish at all due to the lack of a happens-before edge. This fix synchronizes every accessor that touches the lo/hi pair: initialize(), isInitialized(), getLo(), getHi(), and toSink(). The fix is deliberately kept in DataID rather than making Uuid fields volatile, since Uuid is a hot general-purpose value holder used in tight loops, and the unsafe publication is specific to DataID's one-writer/other-reader pattern.
This fix addresses a scenario where a transient stat failure on a .pk file during a column rename could cause permanent posting index corruption. The root cause was that TableWriter.dropFuturePostingIndexChainEntriesBeforeLink silently treated a length() error as "no chain," which caused the hard-link step to use a stale seal transaction number for the .pv file. The only copy of the sealed values was then purged, leaving the partition with a .pk chain head referencing a nonexistent .pv file. The fix throws a CairoException.critical when length() returns a negative value on an existing .pk file, allowing the writer to distress and retry the operation. Both RENAME COLUMN and CONVERT PARTITION callers are affected. The legitimate short-file and absent-file paths retain their existing semantics.
This fix resolves a crash (AssertionError) that occurred when performing hash joins, GROUP BY, or DISTINCT operations on empty VARCHAR keys marked as non-ASCII. An empty varchar is ASCII by definition, but the Utf8Sequence contract allows producers to report isAscii() == false for empty values. In UnorderedVarcharMap, an empty non-ASCII key packed to 0, which collided with the map's empty-slot sentinel, causing silent table corruption. The fix addresses three layers: UnorderedVarcharMap.putVarchar now forces the ASCII flag for empty keys; the QWP egress request decoder now computes the ASCII flag with Utf8s.isAscii() instead of hardcoding false; and VarcharTypeDriver.appendPlainValue now forces the flag for empty values to avoid a zero-header collision with the NULL/empty sentinel. The column storage format is intentionally unchanged, as existing on-disk data remains valid.
This fix addresses two related issues in index-backed LATEST ON cursors. The first issue caused queries with an indexed SYMBOL key and a residual filter (e.g., WHERE sym = 'X' AND venue = 'Y' LATEST ON ts PARTITION BY sym) to return a row belonging to a different symbol value. The root cause was a double subtraction of partitionLo from the row id — IndexReader.getCursor() already returns row ids relative to minValue, so the extra subtraction positioned the record too early in partitions spanning multiple page frames. The fix removes the redundant - partitionLo in LatestByValueIndexedFilteredRecordCursor and LatestByValuesIndexedFilteredRecordCursor. The second issue affected queries where the same key appeared both statically and as a runtime constant (e.g., sym IN ('a', :sym) with :sym = 'a') — the duplicate inflated keyCount beyond what the found set could reach, defeating the early-exit optimization and causing unnecessary full scans of older partitions. The fix ensures deferred keys that duplicate static keys are not double-counted, and reorders the frame loop condition to short-circuit before opening the next page frame once all keys are found.
This fix resolves assertion failures that occurred when posting index covering scans (symbol IN (...)) migrated across worker threads during suspendable queries such as multi-worker HTTP parquet exports. The assertion tripwire assumed a reader is driven by one OS thread between pool acquire and release, but suspendable queries legitimately migrate connections across workers with the event loop serializing the handoff. The fix stamps the reader's operating thread on each getCursor() call, and each cursor close() path re-pools only when running on the stamping thread. Off-thread closes fall through to releaseResources(), which frees only cursor-local buffers without touching reader-shared state. The fix also extends this gate to bitmap and all-null index readers for consistency, and corrects single-key covering scan parquet exports that produced all-null covered columns by adding RecordCursorFactory.producesMaterializedPageFrames() to route metadata-only producers through the row-wise cursor-based export path instead of the zero-copy page-frame path.
Previously, CREATE MATERIALIZED VIEW queries that included a designated timestamp column but lacked a SAMPLE BY clause or GROUP BY timestamp_floor(...) column would incorrectly report "TIMESTAMP column is not present in select list," even when the timestamp was clearly in the select list. This fix distinguishes between two cases: when the timestamp column is present but no sampling interval can be inferred, the error now names the column and points to SAMPLE BY or GROUP BY timestamp_floor(...) as the two supported forms. When no designated timestamp column exists at all, the original generic message is preserved.
CANCEL QUERY <id> could cancel a different, unrelated query after the registry recycled a pooled query entry, causing the victim query to fail with "cancelled by user" even though nobody cancelled it. This fix introduces a volatile lifecycle word per pooled entry that packs the owning query ID and state. Cancellation now only applies to the active query that owns the requested ID; a stale canceller holding a recycled registry entry fails the lifecycle check and reports the query as not found. Additionally, query_activity() now snapshots registry entries optimistically and only returns a row if the entry still belongs to the same active query after the row has been copied, preventing torn rows and wrong-query data. A query in the CANCELLING state can transiently not appear in query_activity() while a canceller holds the lifecycle guard.
dot_product() over two DOUBLE[]/DOUBLE[][] arrays could return a silently incorrect result when one of the operands was not stored in vanilla (contiguous, row-major) layout. In the recursive code path, the deepest-dimension loop read the right-hand array with the left array's flat index rather than its own cursor. When both operands share an identical layout the two indices coincide, so the bug was masked. When the operands have different strides — e.g. one is transpose(...) or a slice — reading the right array with the left index selected the wrong element. This fix reads the right array with its own flat index. No behavior change occurs for arrays already stored in identical vanilla layout.
Previously, EXPLAIN plan output over HTTP and CSV was HTML-encoded with   for indentation and </> for comparison operators. This encoding was originally added as a presentation workaround for the Web Console but was applied to every HTTP request, making the response inconsistent with the PostgreSQL Wire Protocol output which returned plain text. This fix removes the HTML escaping path from TextPlanSink, so all transports now return consistent plain-text plans with ordinary spaces for indentation and raw </> characters for operators. JSON serialization still performs its own required escaping. HTTP EXPLAIN consumers will now receive spaces instead of  , and </> instead of </>.
A WAL table with a POSTING index could be suspended mid-apply during a partition squash. Under parallel WAL apply, PostingIndexWriter.of reopened a column's .pk key file and sized the mmap from the file length, which could lag the writer that just extended the file. This caused the chain's head entry to fall outside a short mapping, resulting in an AssertionError under -ea or a SIGSEGV in a -da build. This fix addresses two issues: first, the reopen mapping is now sized to cover the header's live region (regionLimit) rather than the potentially stale file length, with range checks on the header before trusting the value; second, when a reopen fails before the region is republished, the .pk file is no longer truncated back to the sentinel value, which previously could discard the live chain region on disk. On the dominant path where a cleanly-closed .pk already covers the entry region, no extra remapping occurs.
This fix replaces the previous peek-based connection probe with a hangup probe that detects a FIN behind buffered data on all platforms including Windows. The PostgreSQL Wire Protocol JDBC driver sends a Terminate message and WebSocket clients send a Close frame before the FIN, so the old recv(MSG_PEEK) probe returned that buffered byte instead of EOF and the circuit breaker never tripped. The new Net.isPeerDisconnected uses poll(POLLRDHUP) on Linux, kqueue with EVFILT_READ/EV_EOF on macOS/FreeBSD, and WSAPoll with POLLHUP|POLLERR|POLLNVAL on Windows. The QuestDB WebSocket Protocol egress path now binds a per-connection, fd-backed circuit breaker instead of the no-op breaker, so egress queries observe both query.timeout and the connection probe. Probe frequency is throttled by wall clock (default 100ms) so the fd-backed breaker does not turn hot loops into per-iteration syscalls. A stream resumed after a write park or credit grant re-checks the deadline and connection before producing the next batch. Detection is bound to the query-continuation wake interval rather than instantaneous, and a write-parked stream with a live but fully stalled reader is not aborted at query.timeout until it resumes. A double-read race in WriterPool.Entry.getTableToken() was also fixed where a writer_pool() scan racing a writer close could NPE.
This fix addresses several correctness and resource-safety issues. The IPv4 not-null aggregators now implement the same keyed-batch semantics as other primitive types, so first_not_null() and last_not_null() return correct results for IPv4 columns on the non-sharded keyed-batch path. The dim_length() function now returns NULL for NULL arrays, NULL dimensions, and NULL PostgreSQL Wire Protocol array binds, instead of failing or returning incorrect values. Every parallel reducer now releases its worker slot from a finally block entered immediately after acquisition, preventing permanent slot leaks when frame navigation, temporary allocation, or cleanup fails. A data race on selectivity statistics shared across threads by thread-safe parallel filters was fixed by packing the sample count and average into a single volatile long updated with a CAS loop. Array access overhead was reduced by reading dimension and element data directly through record accessors without materializing an ArrayView per row, and storage-delegating records forward both new methods including union, order-by materialization, window, join, and base columns under UNNEST.
This fix resolves an issue where a table with a covering posting index, converted to Parquet and then back to native, returned wrong results for every covered column. Some covering scans returned NULL for covered columns while others failed with an index-out-of-bounds error. The root cause was that convertPartitionParquetToNative rebuilt the posting index via rebuildPartitionIndexFiles, which builds only the non-covering .pv files and never the covering sidecars (.pci/.pc*). A second cause was found in the native-to-Parquet direction: when a column had a non-zero column top, copyOrRebuildColumnIndexes rebuilt the index without covering configuration, dropping covered values. The fix hard-links existing index files from the Parquet partition directory into the new native directory instead of rebuilding, since row IDs are preserved and covering sidecars are storage-independent value copies. Both conversion directions now share one rebuildColumnIndex method that calls configureCoveringIfNeeded and correctly points the sidecar build at the directory holding the native column data. This also routes plain bitmap symbol indexes through the link path on reconvert, reducing index work from a full-column scan to a few syscalls.
On x86 systems with SQL JIT enabled, a valid query checking a STRING column for NULL could rarely crash QuestDB with SIGSEGV. This occurred when a null or empty string header occupied the final four bytes of a mapped column file, causing the scalar JIT to read beyond the mapping boundary. This fix ensures the four-byte STRING header is read without crossing the mapping boundary. As a workaround for earlier versions, setting cairo.sql.jit.mode=off and restarting QuestDB avoids the crash.
This fix ensures the table details drawer correctly understands the current storage-policy grammar (TO PARQUET / TO REMOTE / DROP LOCAL / DROP REMOTE). Previously a policy containing TO REMOTE failed to parse and the drawer showed "Not configured" for tables that had one. Additionally, creating a materialized view on a policy-bearing source now generates DDL the server accepts: the policy's DROP LOCAL stage becomes a laddered materialized view TTL. Servers reject STORAGE POLICY on materialized views, so the previously generated DDL always failed. The @questdb/sql-parser dependency was bumped to 0.1.15, which adds TO REMOTE support, removes the unsupported DROP NATIVE, and registers storage_policies() for autocomplete.
This fix removes HTML entity decoding from the modern and legacy result grids, preserving result values exactly as returned by the server in rendered cells, clipboard output, and Markdown export. Previously, the Web Console applied unescapeHtml to every result value, which made ordinary query results lossy: legitimate values such as  , <, >, &, ", and ' were incorrectly transformed even though they were user data. With a paired QuestDB server change that returns EXPLAIN plans as plain text, the Web Console now treats every response value as data. Leading whitespace is also preserved so plain-text EXPLAIN plans keep their hierarchy via white-space: pre.
This fix resolves several issues with query execution in the Web Console. Previously, with all SQL selected, clicking the Cancel button on a running query opened the "Run all queries" modal instead of cancelling, because Run and Cancel shared a handler that branched on the selection. Now Cancel always cancels while a query is running, regardless of selection state. The selection is carried into the pending run so the confirmation dialog correctly reflects "Run selected queries." Additionally, triggering a multi-statement selection run while a query was already running silently ran all queries instead of the selected ones. The new-tab button, which appeared greyed out during a script run but was still clickable due to a CSS-only guard, is now gated by logic for both the button and Alt+T shortcut.

July 31, 2026

QuestDB Enterprise 3.3.4 is a feature-and-stability release that introduces column type conversions on parquet-partitioned tables, a client connect timeout with store-and-forward failover hardening, restartable backup restore, and multi-arch (arm64) Docker images. It also includes a batch of QWP ingress/egress, catalogue, and SQL correctness fixes pulled from the OSS engine, along with performance improvements such as parallel covered-index decode for aggregation and filter queries. The release builds on the QuestDB 9.4.4 engine line and advances the Enterprise build to 3.3.4.

New Features

ALTER TABLE ... ALTER COLUMN ... TYPE now works on tables that contain Parquet partitions. Previously, the operation supported native partitions only — running it against a table with any Parquet partition either failed or silently left the Parquet data unconverted. Conversion is lazy: Parquet partitions are not rewritten by the ALTER. Decoding cost moves to the query path, with a one-time decode per row group buffered for subsequent rows. O3 merges that land on a converted partition rewrite the partition with the new type. Conversions involving SYMBOL, and chained conversions where the Parquet column type no longer matches current metadata, eagerly convert the affected Parquet partitions to native before the ALTER. Null sentinels, BYTE/SHORT/CHAR column tops, date/timestamp scaling, and UTF-16/UTF-8 transcoding all match the native partition behaviour exactly.
This feature emits round-trippable QuestDB DDL for every user object in the database, one statement per row, similar to pg_dump --schema-only. It supports tables, views, and materialized views, ordered so that row-by-row replay always succeeds. Dependency ordering uses the view and materialized view graphs, with sibling dependencies name-sorted for deterministic output. The feature supports INCLUDE and EXCLUDE clauses to filter by category (TABLES, VIEWS, MATERIALIZED_VIEWS). Concurrent DDL during the dump no longer fails the entire operation — objects dropped between the initial snapshot and their emit are skipped with a logged warning rather than causing an error. Re-dumps are byte-idempotent.
This feature moves covered-index column decode for aggregation and filter queries off the single dispatch thread onto the async reduce-worker pool, eliminating dispatcher serialization. The covered decode now parallelizes 3–4× for decode-heavy query shapes such as SUM, multi-aggregation, first/last, residual filter, and GROUP BY. For selective predicates, the covered plan is significantly faster than a full scan, and COUNT(*) with a WHERE clause on a single symbol key is answered from posting-index metadata with no decode at all. The implementation includes concurrent-read-safe posting readers with detached per-worker cursors, a freeze mechanism before dispatch so workers never mutate shared reader state, and metadata-only single-key production using random access into the posting index. All column types including variable-size (VARCHAR, STRING, BINARY, ARRAY) are supported. Covered reads scale with selectivity — reading only matching rows' columns — providing 1.2–4.3× fewer bytes read from disk compared to full scans, with the advantage increasing on bandwidth-bound storage.
This change updates QuestDB Enterprise to compile against the updated open-source Parquet column type conversion API, which adds a SqlExecutionContext parameter to ParquetExportMode.determineExportMode().

Improvements

QuestDB previously fell back to a full table scan when the designated timestamp was wrapped in any function (e.g., WHERE date_trunc('day', ts) >= '2024-01-01' or WHERE CAST(ts AS timestamp) > $1). This improvement lets the optimizer push such predicates back onto the timestamp axis when the wrapping function is monotonic. A predicate g(ts) OP bound is recognized, the queried value range is inverted layer by layer back to a range on the designated timestamp, and the result drives the usual interval scan and partition pruning. Functions are graded as EXACT (inverse is exact, row filter dropped), SUPERSET (inverse soundly contains the true preimage but may be wider, interval prunes but original predicate stays as residual filter), or NONE (cannot bound, predicate stays a plain row filter). Covered functions include date_trunc, timestamp_floor, timestamp_ceil, dateadd with fixed units, to_timezone, to_utc, precision casts, and arithmetic shifts. Named-timezone functions achieve EXACT grade when the bound provably lies inside a single DST segment, falling back to SUPERSET near DST transitions. Calendar dateadd (month/year) is always SUPERSET due to day clamping. Recognition is fail-safe: any compile or parse failure falls back to a row filter. This improvement also fixes a latent bug in getNextDST for zones whose first DST transition is after 1970 (e.g., Antarctica/Troll), adds overflow hardening for fixed-unit dateadd and timestamp_ceil inverses, and introduces domain-ceiling-aware overflow checks for constant-shift inverses on microsecond columns.

Bug Fixes

On a replica, the DataID is not generated locally but learned from the object store on the first index poll by the WalDownloader, which completes after the replication component reports READY. Because the backup scheduler sits downstream in the boot dependency graph, its start could run before the DataID existed, causing a fatal boot failure. This fix adds a bounded wait (60 seconds) in the backup scheduler startup on replicas for the DataID to be established before proceeding. The fast path where the DataID is already set does not wait at all. On timeout, the boot fails with a clear, actionable message.
This fix addresses two related bugs in the index-backed LATEST ON cursors. The first bug caused queries with an indexed SYMBOL column and a residual filter (e.g., WHERE sym = 'X' AND venue = 'Y' LATEST ON ts PARTITION BY sym) to return a row belonging to a different symbol value. The root cause was that the filtered LATEST ON cursors subtracted the page frame's partitionLo from the row id a second time, even though IndexReader.getCursor() already returns row ids relative to partitionLo. This double subtraction positioned the record too early whenever the matched row fell in a page frame with partitionLo > 0, which only occurred in partitions large enough to span multiple page frames. The redundant subtraction was removed from LatestByValueIndexedFilteredRecordCursor and LatestByValuesIndexedFilteredRecordCursor. The second bug caused duplicate static/deferred keys (e.g., sym IN ('a', :sym) with :sym = 'a') to defeat the early-exit optimization, resulting in unnecessary full scans of older partitions. The deferred key was counted twice in keyCount but the found set is deduplicated, so the early-exit condition could never be satisfied. The fix ensures resolved deferred keys are only added when not already present in the static key set, and reorders the frame loop condition to short-circuit before opening the next page frame once all keys are found.
This fix resolves an AssertionError crash that occurred when a join, GROUP BY, or DISTINCT operation used a VARCHAR key that was an empty, non-ASCII varchar. The Utf8Sequence contract allows producers to report isAscii() == false for an empty value, and when this happened, UnorderedVarcharMap packed the key hash, size, and flags into a single long that equaled 0 — the map's empty-slot sentinel — silently corrupting the hash table. The fix addresses the issue at three layers: UnorderedVarcharMap.putVarchar now forces the ASCII flag for empty keys, preventing sentinel collision; the QuestDB Wire Protocol egress request decoder now computes the ASCII flag with Utf8s.isAscii() instead of hardcoding false, so empty varchar bind variables are correctly flagged; and VarcharTypeDriver.appendPlainValue now forces the ASCII flag for empty values, preventing a header collision with the NULL/empty sentinel in the plain storage format used by RecordChain and BlockFileWriter. The column storage format is intentionally unchanged, as existing on-disk data remains valid.
The SQL compiler wraps column aliases in double quotes when the bare name contains dots (e.g., PIVOT values like FNCL 2.5 7/26, auto-generated expression aliases like t.x+1) or collides with operator tokens (in, and). These protective quotes leaked verbatim into result set metadata, causing the Web Console, PostgreSQL Wire Protocol clients, and CSV exports to display column names with literal double quotes. This fix ensures protective quotes remain compiler-internal: SqlUtil.isQuoteProtectedAlias recognizes compiler-added quotes by re-testing the quoting condition, SqlUtil.toColumnName strips them wherever a projection alias becomes a metadata column name, and SqlUtil.getColumnIndexQuiet resolves compiler-side column references with a fallback retry after stripping quotes. Additional fixes address a stale-quote leak after alias mutation (truncation or dedup suffix), an empty PIVOT value that hung the compiler in an infinite loop, operator-token pivot columns breaking through joins, dotted JSON UNNEST keys returning NULL, trailing space leaking into truncated aliases, and store reuse efficiency in alias dedup loops. Result set column names now display cleanly: FNCL 2.5 7/26 instead of "FNCL 2.5 7/26", t.x+1 instead of "t.x+1", and a.b instead of "a.b".
Previously, CREATE MATERIALIZED VIEW queries that had neither a SAMPLE BY clause nor a GROUP BY timestamp_floor(...) column threw the misleading error "TIMESTAMP column is not present in select list", even when the designated timestamp column was clearly present. The actual problem was that no sampling interval could be inferred — a materialized view requires either SAMPLE BY <interval> or GROUP BY timestamp_floor(<interval>, ts) to define the fixed bucket for incremental refresh. This fix distinguishes the two cases: when the timestamp column is present but no interval is inferred, the error message now names the column and points at SAMPLE BY or GROUP BY timestamp_floor(...) as the two supported forms; when no designated timestamp column exists at all, the original generic message is preserved.
When ff.length() returned an error on an existing .pk file during column rename, the error was silently swallowed, causing the hard-link step to fall back to a stale seal transaction number. This left the partition with a .pk chain head referencing a .pv value file that no longer existed, resulting in permanent index corruption where every indexed scan of the partition would fail. This fix now throws a CairoException.critical when length() returns a negative value on an existing .pk file, instead of returning "no chain." The legitimate short-file and absent-file paths retain their existing semantics. Both callers of the affected code path (RENAME COLUMN and CONVERT PARTITION) are covered by this fix.
Suspendable queries can migrate connections and their associated TableReader across worker threads between fragments. A posting index row cursor checked out on one worker could legitimately close on another, but an assertion added in a prior release encoded the premise that a reader is driven by one OS thread between pool acquire and release. This caused assertion failures on multi-worker HTTP parquet exports of posting covering scans, leading to leaked TableReader instances and dropped PostgreSQL Wire Protocol client connections. This fix stamps the reader's operating thread on every getCursor() call, and each cursor close() path re-pools only when the close runs on the stamping thread. An off-thread close falls through to releaseResources(), which frees only cursor-local buffers and touches no reader-shared state. The same gate was extended to bitmap and all-null index readers for consistency. Additionally, this fix corrects single-key covering-index parquet exports that produced all-null covered columns by adding RecordCursorFactory.producesMaterializedPageFrames() to ensure metadata-only producers export row-wise through the record cursor instead of the zero-copy page-frame path.
The DataID class wraps a 128-bit UUID stored as two non-volatile long fields (lo/hi). The value is published by one thread (e.g., the replication WAL downloader) and read by another (e.g., a booting backup scheduler checking isInitialized()). Because the stores and loads were unsynchronized, a reader could observe a torn half-written value or never observe the publish at all due to lack of a happens-before edge. While change() was already synchronized, initialize() and the reader methods were not. This fix synchronizes all accessors (initialize(), isInitialized(), getLo(), getHi(), and toSink()) on the DataID instance. The fix is kept in DataID rather than making Uuid fields volatile, since Uuid is a hot general-purpose value holder and the unsafe publication is specific to DataID's one-writer/other-reader pattern.

July 1, 2026

QuestDB Enterprise 3.3.3 is a stability-focused patch release on the QuestDB 9.4.x engine that concentrates on reliability and correctness: hardening replication and backup around snapshot restore and role changes, and pulling in a large batch of Parquet, posting/covering-index, storage and SQL-correctness fixes from the OSS engine. Alongside the fixes it adds two Enterprise capabilities — a hot in-place primary/replica role switch and finer-grained replication control (replication.disabled.tables plus rebase-aware replication).

New Features

This feature introduces two replication capabilities. First, replication.disabled.tables provides a reloadable, comma-separated list of tables to exclude from replication — both the uploader and downloader skip listed tables. Second, replication-aware handling of ALTER TABLE REBASE WAL ensures a rebased table is re-baselined correctly on replicas instead of being rebuilt from an empty state. The uploader detects a rebased table via a permanent _rebase_new marker, skips its empty seed transaction, and uploads from seqTxn 2, recording first_txn=2 in index.msgpack. A replica refuses to build such a table from the object store when it is missing the baseline below first_txn, prompting an operator to copy the table and resume. The previously separate replication status checks are consolidated into a single getReplicationStatus(dirName) callback returning ACTIVE, SUSPENDED, or DISABLED. An additional reloadable switch, replication.primary.pause.upload.on.suspended.tables (default false), allows pausing uploads of WAL-apply-suspended tables. For permissions, ALTER TABLE … SUSPEND WAL and RESUME WAL are both authorized by the single RESUME WAL permission, while REBASE WAL requires SYSTEM ADMIN privilege due to its destructive nature.
This feature adds controls for managing hard-suspended WAL tables and introduces ALTER TABLE <t> REBASE WAL to rebuild a table under a fresh sequencer. The cairo.wal.apply.suspended.tables configuration provides a reloadable, comma-separated list of table names that ApplyWal2TableJob skips, preventing WAL transaction application. When cairo.wal.apply.suspended.write.denied is set to true, writes to a hard-suspended table are rejected instead of being queued. Non-structural ALTER statements and FORCE DROP PARTITION on suspended tables are routed through a WAL-bypass path and applied directly via TableWriter, while structural changes remain denied. REBASE WAL clones the applied table into a new directory via hard links into a .rebase/ staging directory, resets _txn and _meta with a new tableId, seeds two empty transactions, swaps the name registry, and drops the old directory. Preconditions require the table to be a WAL table, hard-suspended, and cairo.wal.apply.suspended.write.denied=true. Rebasing a base table invalidates dependent materialized views (recoverable with REFRESH MATERIALIZED VIEW <v> FULL), while rebasing a materialized view itself re-registers it for a full refresh. Pending unapplied WAL transactions are discarded by a rebase. For permissions, SUSPEND WAL and RESUME WAL share the same authorization path, while REBASE WAL requires system-admin privilege due to its destructive nature.
This feature introduces opt-in memory limits that cap how much native memory a single bounded workload may allocate, throwing at the offending allocation site when the cap is crossed so a runaway is stopped at its source while unrelated workloads keep running. Three independent, dynamically reloadable configuration properties control the limits: cairo.query.memory.limit.bytes for user SQL queries, cairo.mat.view.refresh.memory.limit.bytes for materialized view refreshes, and cairo.wal.apply.memory.limit.bytes for WAL apply. All default to 0 (unlimited). On breach, the engine throws a CairoException with a distinct message identifying the workload type, query ID, limit, and memory tag. A MemoryTracker wraps a 16-byte native block shared with Rust, acquired per workload via QueryRegistry.register/unregister. Coverage includes map family, sort/tree chains, hash-join chains, FastGroupByAllocator and function state, LATEST BY rowid lists and maps, set-operation maps, encoded sort, secondary joins, window/horizon-join aggregations, SAMPLE BY fill, and Parquet decode buffers. The vectorized (Rosti) keyed GROUP BY hash tables remain on the global RSS counter only. The query_activity view gains memory_used and memory_limit columns. Tracker-aware pooled memory classes release their native backing on cursor close and re-allocate on next use to ensure each malloc and its matching free are charged to the same tracker.
This feature introduces statistical aggregate functions for computing kurtosis and skewness. The implementation uses Pébay's one-pass online algorithm (an extension of Welford's) for numerical stability, maintaining running mean and central moments (M2, M3, M4) per group. Partial aggregates merge with pairwise-combine formulas, enabling parallel GROUP BY execution. Sample skewness returns null for fewer than 3 values; sample kurtosis for fewer than 4. The sample variants apply Fisher's bias correction. Population variants return null for empty groups. All variants return null when every observation is equal (zero variance). Example usage:
```
SELECT skewness(price), kurtosis(price)
FROM trades
WHERE symbol = 'BTC-USD'
SAMPLE BY 1h;
```

Improvements

This improvement introduces an internal continuation runtime built on jdk.internal.vm.Continuation that allows SQL functions to yield their carrier worker thread while waiting, so the number of concurrent calls is no longer bounded by the worker pool size. The wait_wal_table(table_name [, seq_txn]) function blocks the current query until the WAL writer has applied transactions up to a target sequencer transaction number, returning true on success. It respects the SQL circuit breaker for query timeout, explicit CANCEL QUERY, and broken client connections. The sleep(seconds) function pauses the current query for a given duration (up to 24 hours) and returns a single TIMESTAMP row. Both functions release their worker carrier while parked. JDK ThreadLocal is replaced with a carrier-keyed CarrierLocal along continuation-critical paths to prevent stale thread identity across yield/resume boundaries. Two new configuration properties are available: cairo.timer.shards controls the number of daemon threads driving deadline-based wakeups, and griffin.query.continuation.wake.interval sets the millisecond interval for circuit breaker probing during park.
```
INSERT INTO trades VALUES (now(), 'AAPL', 100, 150.0);
SELECT wait_wal_table('trades');
```
This improvement extends the encoded radix sort path to every column type that ORDER BY accepts and to keys of any width, across both full-sort and serial/parallel top-K paths. Previously, only fixed-width columns fitting within 32 bytes used the fast radix sort; everything else fell back to a red-black tree sort with O(log n) comparator calls per row. The SortKeyEncoder now normalizes each sort column into a byte-comparable segment: VARCHAR uses shifted UTF-8 bytes with a terminator, STRING and non-static SYMBOL use UTF-16BE with escaping, and UUID/LONG256 use unsigned big-endian with an order-preserving null remap. Variable-width keys use a 16-byte inline prefix with overflow into a key heap, and the native kernel finishes tied partitions with pdqsort. Top-K builds entries in an EncodedTopKBuffer that rejects rows up front when their leading word is beyond the kept boundary, skipping full key encoding. Benchmarks on the ClickBench hits table show improvements from 52ms to 37ms at LIMIT 10 and from 100ms to 85ms at LIMIT 100000. The key heap is bounded by cairo.sql.sort.key.max.bytes, and the tree path remains available via cairo.sql.orderby.sort.enabled=false. This change also fixes a pre-existing bug where generateCastFunctions had no LONG128 case, causing UNION queries with both a SYMBOL column requiring casting and a LONG128 column to throw UnsupportedOperationException.
This improvement addresses persistent native memory consumption from cached parallel GROUP BY, top-K, and ASOF/window join factories. Previously, AsyncFilterContext.clear() freed page-frame memory pools but left per-worker DirectLongList row-id buffers at their peak size (up to 8 MB each at the default 1M row page frame), only freeing them on query-cache eviction. Now clear() calls resetCapacity() on the owner and per-worker lists, returning them to the 256-entry initial capacity. The tradeoff is one reallocation when a cached factory is reused for another large scan. This improvement also includes a safety fix for a native memory leak: DirectLongList.resetCapacity() could re-malloc a closed list when clear() was called after close(), which occurred in the four horizon-join factories on a failed getCursor(). The fix adds a guard to skip closed lists and reorders cursor/frameSequence cleanup in horizon-join factories to match the GROUP BY/TopK factories.
This improvement eliminates redundant per-row evaluation of subexpressions whose value is invariant across all rows of a cursor but not known at compile time. A typical example is a time-window threshold like dateadd('d', -30, to_timezone(now(), 'Asia/Kolkata')) inside a CASE branch or filter predicate, where now() is already cached but the surrounding function calls were re-evaluated on every row. A new RuntimeConstFunction wrapper evaluates the subtree once during init(), caches the result in a primitive field, and serves it from every getter. FunctionParser wraps only the maximal runtime-constant subtree at function boundaries, avoiding double-wrapping. Trivial runtime-constant leaves such as bind variables and now() that already cache their values are skipped to avoid unnecessary indirection. Only fixed-width scalar types (timestamp, long, int, double, date, boolean, ipv4, geo, uuid, etc.) are folded; variable-length types are left for future work. The wrapper delegates toPlan() to its argument, so EXPLAIN output is unchanged.
This improvement reduces redundant I/O during checkpoint and snapshot restore by collapsing per-partition Parquet metadata sidecar (_pm) processing from up to two maps and three CRC verifications down to a single map and single CRC per partition, all executed in parallel. Previously, the serial validation pass dominated restore startup on cold, object, or network storage. The recovery pool now submits one worker per thread, each pulling partitions from a shared cursor with dynamic load balancing and reusing native scratch objects across partitions. The committed-size truncation check on data.parquet runs inside the worker, removing the O(N) serial ff.length() loop. Additionally, the non-Parquet bitmap index rebuild now parallelizes across individual (partition, column) work items rather than whole partitions, so a non-partitioned table with several indexed symbol columns no longer serializes all index rebuilds onto one thread. A behavioral tradeoff is that fail-fast detection of truncated captures now occurs at the parallel drain point rather than strictly before sibling work starts, though the first failing worker trips a shared abort latch that short-circuits all other workers.

Bug Fixes

This fix addresses a bug where a primary snapshot could capture a Parquet partition whose on-disk data.parquet was a later generation than the committed Parquet size in _txn. An in-place O3 rewrite (partitionMutates=false) grows the file before _txn advances. When the replica restored that torn pair and replayed the O3 merge over the partition, the stale _pm caused the merge to decode a column chunk past the committed size, resulting in a "File out of specification: Column chunk range exceeds data length" error that suspended the replica table. This fix regenerates a partition's _pm during snapshot/checkpoint restore when the restored data.parquet is longer than the committed size.
This fix addresses an out-of-memory condition in the posting index seal and rollback paths on memory-constrained instances ingesting skewed symbol columns. The sealIncremental buffers were sized at DENSE_STRIDE (256) * preAllocPerKey, resulting in up to 256x over-allocation on skewed columns. This fix right-sizes the buffers to the actual dirty-stride aggregate, adds RSS pre-flights to both the seal and rollback paths, and streams every rollback cover shape per-key instead of decoding the whole index into one buffer. The fix also closes several latent safety issues including native out-of-bounds writes in unbounded posting decoders on corrupt or torn input, and a seal-purge reuse-race that could delete a live file. The tradeoff is that the incremental-seal pre-flight is conservative and may defer to the slower full seal when the incremental path would have just fit, and the streaming rollback re-derives each surviving key rather than holding the whole index, trading per-key CPU on the rare rollback path for a peak-memory bound.
A snapshot could capture a Parquet partition whose on-disk data.parquet was a later generation than the committed Parquet size recorded in _txn. The restore path trusted the partition's existing _pm sidecar in that state and left it stale, causing the first read or merge after restore to fail with a "Column chunk range exceeds data length" error. On a replica replaying the WAL over the restored partition, this suspended the table and broke replication. The root cause was that the validation only checked whether the sidecar could resolve a footer at the committed size, but a _pm from the later generation still resolved successfully despite being stale. This fix now only trusts an existing _pm when data.parquet is exactly the committed size. When the file is longer (indicating the snapshot captured the partition mid in-place rewrite), the _pm is regenerated from the committed size, which produces a correct compact sidecar for the committed footer.
This fix addresses a production out-of-memory incident on a memory-constrained instance ingesting a skewed symbol column. The incremental seal path (sealIncremental) sized its per-stride merge and trial buffers as if all 256 keys in a stride held the single hottest key's row count, inflating allocation by up to ~256x beyond actual data. The buffers are now sized to the actual aggregate of dirty strides computed from per-key counts, and a pre-flight defers to the full seal (which streams per key) when the correctly sized buffers would still breach the RSS limit. The rollback path previously decoded the entire partition index into one buffer before filtering, which was the exact memory-heavy case that triggered the incident. Rollback now streams per key for every cutoff, bounded by the largest single key rather than the whole index. All covering index shapes (fixed-size, var-size, and addr-based) now stream their sidecar rebuilds. Error handling was hardened across seal and rollback paths: failures after the value-file switch poison the writer so all mutating entry points reject further operations, while failures before the switch clean up staged files and restore writer state. Several latent safety bugs were also fixed, including unbounded native writes in posting decoders on corrupt input (now rejected with a clean error) and a seal-purge reuse race that could delete a live file after a sealTxn was freed and republished.
This fix addresses a remaining out-of-memory path in the posting index incremental seal that was not covered by the prior streaming-fallback RSS pre-flight. The incremental seal snapshots each covered column's entire sealed sidecar into native memory before the stride-merge pre-flight runs, so a skewed symbol with a covered (INCLUDE) column could still trigger an unguarded multi-gigabyte malloc that exceeded the global RSS limit during a WAL fast-lag commit. The fix adds the same headroom pre-flight before the snapshot copy, re-reading RSS usage per cover column to account for snapshots already taken. When the copy would not fit the live headroom, the incremental path is abandoned and the full seal is used instead, which rebuilds every sidecar by streaming per-key from the column files with peak memory bounded by the largest single key rather than the entire sidecar.
This fix resolves an intermittent NullPointerException in CoveringIndexRecordCursorFactory.getCursor that occurred when multiple queries ran concurrently, such as all panels of a Grafana dashboard firing at once over PostgreSQL Wire Protocol. The factory stored a direct reference to the compiler's pooled keyValueFuncs list from IntrinsicModel. Since SQL compilers are pooled and shared across threads, a second thread borrowing the same compiler would clear that shared list (via IntrinsicModel.clear()), nulling out slots while the first thread's factory was still reading them during getCursor(). The fix creates an owned copy of the list in the constructor, preserving the same Function instances which the factory already owns and frees on close. This restores the contract already honored by FilterOnValuesRecordCursorFactory, whose key-value list parameter is consumed at construction rather than retained.
Equality and IN filters on a renamed column of a Parquet partition could return too few rows (often zero), silently dropping valid data. The defect was in Parquet row-group bloom-filter pushdown, which resolved the filtered column against the Parquet file's column names. Those names are frozen when the partition is converted to Parquet, so a later RENAME COLUMN left them stale. When another column already carried the query's current name (common after a chain of renames), the pushdown landed on the wrong Parquet column, checked that column's bloom filter, got a false negative, and skipped the entire row group. This fix makes the native-table pushdown resolve columns by stable column id, the same way the data-decode path already does. read_parquet() cursors retain name-based resolution, which is the correct semantics for arbitrary external files without QuestDB column ids. Pruning effectiveness is unchanged for the common non-renamed case, and the bloom filter format, statistics, and decode paths are untouched.
Many query factories only consulted the execution context's circuit breaker inside their row-processing loop, which never runs for queries that produce no rows (empty tables, no-match filters, empty joins/aggregates), for instant single-row results, or for parallel paths that dispatch nothing when the input is empty. This fix ensures every affected factory consults the breaker at the point its work starts: at the top of hasNext() before the empty/advance guard, before build/aggregate loops, at cursor open for shared empty-cursor singletons, and before dispatchAndAwait() on parallel paths. A new time-throttled breaker variant always tests cancellation and timeout (both cheap, no syscall) and throttles only the heavy connection probe syscall by elapsed wall-clock time, defaulting to a 100 ms window. The throttle state lives on the breaker shared across the whole query, so a CROSS JOIN that re-scans the slave once per master row issues at most one probe per window for the entire query. Every query now pays for one real breaker check on its first consultation, including a single connection-probe syscall. Queries that previously ran to completion under an aggressive timeout can now abort as expected, including catalogue and admin listings.
A SAMPLE BY query using first() or last() aggregation with an indexed SYMBOL column filter and ALIGN TO FIRST OBSERVATION could return a bucket timestamp where an aggregated value was expected. The wrong results appeared only when the designated timestamp column was omitted from the SELECT list, and only for buckets in the middle of the result set — the first and last buckets were always correct. The root cause was that SampleByFirstLastRecordCursorFactory emitted middle rows through a data record that special-cased the bucket-timestamp column using the base-table timestamp index, while the column it was handed is a projection index. The two indexes coincide only when the timestamp is projected at the same position it occupies in the base scan. When the timestamp was not projected, the base index pointed at an unrelated projected column and overwrote it with the bucket timestamp. This fix makes the data record use the same projection index as the boundary record, so when the timestamp is not projected, no column matches the special case and every aggregate reports its stored value.
Subtracting a CHAR or STRING literal from a timestamp, such as SELECT * FROM t WHERE timestamp > now() - '1' LIMIT 1, failed with an internal error. The literal was implicitly bound to the LONG operand slot of the timestamp subtraction operator, but the factory read the right operand with getTimestamp() instead of getLong(). Calling getTimestamp() on a CharFunction throws UnsupportedOperationException, which surfaced as an internal error. This fix changes SubTimestampFunctionFactory to read the right operand with getLong(), mirroring the symmetric AddLongToTimestampFunctionFactory which already did this correctly. As a result, - and + now produce identical, well-typed cast errors for non-numeric literals. For example, now() - '3 day' now reports inconvertible value: '3 day' [STRING -> LONG] instead of the previous [STRING -> TIMESTAMP].
Two malformed expressions involving CASE ... END crashed with internal errors instead of returning syntax errors. A binary operator with a missing right operand directly after CASE ... END (e.g., select sum(case when true then 1 else 0 end & )) caused a NullPointerException because the expression parser's END keyword handler left a stale depth counter that defeated the arity guard, allowing a tree node with a null left-hand side to be built. This fix resets the depth counter to 0 after the flush loop so the arity guard fires correctly, producing the error too few arguments for '&' [found=1,expected=2]. A dot directly after CASE ... END (e.g., select case when true then 1 else 0 end.foo) caused either a NullPointerException or ClassCastException because the dot handler assumed a gluable literal token on the operator stack. This fix extends the token only when the stack top is a literal; otherwise it raises the syntax error '.' is unexpected here. Valid CASE expressions and qualified table.column references are unaffected by both fixes.
This fix addresses a storage-engine corruption that occurred when a WAL REPLACE_RANGE commit in O3 mode appended brand-new partitions above the table's previous last partition. The root cause was in TableWriter.processO3Block, where replace mode derived partitionTimestampHi from o3TimestampMax (the replace-range high boundary) rather than the highest timestamp actually written. For an open-ended range, the boundary is Long.MAX_VALUE - 1, causing getCurrentPartitionMaxTimestamp to overflow. As a result, partitionTimestampHi stayed at the stale pre-commit ceiling, finishO3Commit skipped switching the writer's active column files to the new last partition, and the next commit reused the previous partition's column descriptors, producing rows dated below the new partition's floor or suspending the table. This fix derives partitionTimestampHi from the actual highest written timestamp (replaceMaxTimestamp) in replace mode, leaving the non-replace path unchanged.
This fix resolves a data-ordering bug where a non-WAL writer ingesting out-of-order data could produce out-of-order rows when the in-order prefix crossed a partition boundary. The partition switch sealed the earlier partition in memory but did not persist _txn. On the next lag commit, o3MoveUncommitted reclaimed the active partition's uncommitted in-order tail into the O3 buffer and reset maxTimestamp to the durable _txn value, which predated the switch. The high-water mark ended up below committed data, causing a rollback to reload it and the next O3 commit to reorder the partition tail. This fix ensures maxTimestamp resets to the max timestamp of the last sealed partition by introducing a new variable that is updated whenever a partition switch happens.
This fix addresses a data-ordering bug in the non-WAL out-of-order (O3) commit path where a lag commit (TableWriter.ic()) could leave the table's maxTimestamp below the maximum timestamp actually committed to disk. When uncommitted rows of a lag commit spanned more than the active partition, o3MoveUncommitted() pulled only the active partition's rows into the sorted O3 batch and emptied it. The in-order rows left on disk in the previous partition stayed committed but were not part of the sorted O3 batch, so maxTimestamp was recomputed solely from the O3 batch boundary and could end up lower than the true on-disk maximum. A later O3 commit then merged against that stale boundary and left a single timestamp inversion. This fix reads the earlier partition's actual max timestamp via readPartitionMinMaxTimestamps() and folds it into the committed maxTimestamp computation when the active partition is emptied while uncommitted rows remain in an earlier partition.
This fix resolves a non-deterministic metadata reload failure where MemoryCMRImpl.of() read errno after a cleanup close() call when a file's length could not be read. Since errno is thread-local global state, the intervening close() could overwrite the errno left by the failed length() call. The exception then carried the cleanup call's errno instead of the real one. This mattered because metadata reload classifies failures by errno: TableUtils.handleMetadataLoadException() retries the read only while CairoException.isFileCannotRead() is true. With a clobbered errno, a transient "file does not exist" condition — such as a _meta file briefly unreadable during a concurrent metadata change — was misclassified as fatal, surfacing a "could not get length" error instead of retrying. This fix captures errno immediately after the failed length() call, before close() runs.
This fix encompasses a broad set of query engine hardening changes. Key fixes include: SAMPLE BY FILL(LINEAR) cleanup no longer crashes with NullPointerException on out-of-memory; covering posting index readers now propagate sidecar I/O errors instead of swallowing them; IntervalBwdPartitionFrameCursor.calculateSize() now returns correct counts for backward interval scans with open lower bounds; DATE-argument window value functions (max, min, first_value, last_value, nth_value, lag, lead) no longer throw UnsupportedOperationException; rank() / dense_rank() no longer crash when pass-through columns of unserializable types (UUID, STRING, VARCHAR, LONG256, arrays) are projected; lead() over high-precision DECIMAL no longer crashes; window RANGE-frame functions now return correct values when the designated timestamp is not in the projection; sum() / avg() over Decimal256 now produce correct results over sliding frames by using scale-agnostic subtraction for eviction; JIT filter now correctly widens nested INT products to 64 bits when a FLOAT operand is present in the predicate; LimitedSizeLongTreeChain cleanup no longer crashes on partial construction failure; keyed ASOF JOIN sink-heap no longer leaks on out-of-memory during cursor open; vectorized rosti GROUP BY no longer over-frees NATIVE_ROSTI memory when wrapUp() resizes the map; stale aggregate tasks in the shared vector-aggregate queue no longer crash later queries after a fault aborts a cursor drain; GROUP BY keys referencing aliases of trivial arithmetic expressions now compile correctly; multi-key index scan no longer leaks posting-index block buffers on out-of-memory; and window functions over untyped null literals now raise a clean SqlException suggesting a concrete cast instead of behaving non-deterministically across platforms.
This fix addresses two distinct faults where per-partition code paths iterating posting columns were guarded only against the absent case (columnTop == -1), not the row-less case (columnTop >= partition size). In the first fault, restorePostingIndexersToLastPartition() would crash with "index does not exist" when encountering a row-less column with no .pk key file, causing WAL table suspension or INSERT failures on BYPASS WAL tables. The fix discriminates on the key file's actual presence rather than row-less-ness, correctly re-pointing indexers for row-less columns that do have a .pk (created via ADD COLUMN ... INDEX TYPE POSTING on the active partition) while skipping those without one. In the second fault, linkPartitionIndexFiles() would crash with "index files do not exist" when switching a partition to parquet and encountering a row-less column without a key file. The fix aligns the guard with its sibling copyOrRebuildColumnIndexes() to skip row-less columns on historic partitions where a .pk is never built. The row-less-without-.pk state arises from multi-partition, same-transaction out-of-order writes interleaved with ADD COLUMN. Without the first fix, a blanket row-less skip would silently strand indexers, making rows invisible to indexed predicates with no exception or suspension. The accessor was also changed from getColumnTopQuick to getColumnTop to correctly return -1 for absent columns.
This fix resolves a production bug where a query cancellation could be silently dropped, and the affected query's per-query timeout defeated along with it, when the cancel races query startup. The root cause was that NetworkSqlExecutionCircuitBreaker.cancel() always sets a powerUpTime == Long.MIN_VALUE sentinel but only flips the per-query cancelled flag when that flag is already attached. QueryRegistry.register() publishes the query and fires its listener before binding the per-query flag, so a CancelRequest landing in that window sets only the sentinel. The subsequent register() call then binds a fresh flag with value false, and neither testTimeout() nor testCancelled() consulted the sentinel — testTimeout() would compute an overflowed negative runtime that never trips, and testCancelled() only checked the flag. The query would run to natural completion ignoring both cancel and timeout, pinning workers. The fix makes testCancelled() check the sentinel first so both stateful paths abort a racing cancel, updates getState() to classify the sentinel as STATE_CANCELLED instead of mislabelling it STATE_TIMEOUT, and adds a clearCancelSentinel() call in PGConnectionContext.prepareForNewQuery() to bound the sentinel to a single query and prevent leaking across requests.
This fix addresses several issues where HTTP parser and response objects were not fully resetting their per-request state when connections and contexts were reused across requests. A reused context could hit closed native parsing or compression buffers and crash the server when handling otherwise valid requests. Specifically, the fix reopens the header parser's quoted-value sink when pooled HTTP contexts are reused, clears per-request gzip negotiation state in HttpResponseSink so clients only receive gzip responses when the current request advertises gzip support, clears charset and mapped cookie state in HttpHeaderParser.clear(), fixes multipart false-boundary replay so body bytes are preserved when a boundary-like sequence is followed by non-boundary suffixes, keeps multipart resume pointers anchored to the receive buffer during retry paths, parses Content-Disposition parameters with quote-aware delimiter handling including semicolons and equals signs inside quoted values, and resets Content-Disposition parameter state consistently after known and unknown parameters. This allows uploaded filenames such as a;b.csv and a=b;c.csv to parse correctly.
This fix addresses numerical-stability bugs in the corr() function where the Pearson denominator sqrt(sumXX * sumYY) can overflow or underflow while computing the product of two sums of squared deviations. With large-magnitude inputs (values near +/-1e153), each sum is finite (~1e306) but their product overflows to +Infinity, causing the final division to return 0.0 instead of the true correlation. With small-magnitude inputs (values near +/-1e-150), each sum is finite (~1e-300) but their product underflows to 0.0, causing the division to return NaN. The fix prefers the single-rounding sqrt(a * b) denominator when the product is finite and non-zero, preserving existing bit-exact behavior for normal inputs, and falls back to sqrt(a) * sqrt(b) when the product would overflow or underflow while both factors are non-zero. The final Pearson result is clamped to [-1, 1] to absorb small rounding drift in the fallback path. This applies to CorrGroupByFunctionFactory, AbstractBivariateStatWindowFunctionFactory.computeCorr, and AbstractBivariateStatWindowFunctionFactory.computeCorrWelford. Normal-magnitude inputs are unaffected and keep their prior bit-exact results.
This fix addresses a production issue where a materialized view's incremental refresh could be permanently invalidated by transient errors such as reader pool exhaustion (EntryUnavailableException) or out-of-memory conditions, cascading invalidation up dependent view chains. On transient "table busy" or OOM errors, the refresh now schedules a per-view backoff deadline and returns without invalidating, with MatViewTimerJob re-enqueuing the refresh once the backoff elapses. A consecutive retry counter caps retries at a configurable limit (default 10 via cairo.mat.view.refresh.busy.retry.limit), after which the view is invalidated to bound WAL retention. The materialized_views.view_status column now returns retrying while a view is in a transient-refresh backoff window. A new cairo.mat.view.refresh.block.list configuration option accepts a comma-separated list of materialized view names that the refresh job must never refresh, serving as an escape hatch for views whose refresh crashes or destabilizes the database. Listed views are skipped by all refresh paths without being invalidated. The OOM path no longer calls Os.sleep between step-halving attempts, preventing the single refresh worker from being blocked. Genuine errors such as bad SQL, type mismatches, or dropped base tables still invalidate immediately. Configuration options include cairo.mat.view.refresh.busy.retry.timeout (default 1000ms backoff between retries) and cairo.mat.view.refresh.busy.retry.limit (default 10 consecutive retries before invalidating). A blocked view grows stale and can pin base-table WAL retention until dropped or unblocked.

June 23, 2026

QuestDB Enterprise 3.3.2 is a feature release on the QuestDB 9.4.x engine. The headline addition is support for Parquet as a full table format, letting an entire table — not just individual squashed partitions — live in Parquet. The release also ships native ARM64 (aarch64) Linux binaries, adopts the simplified single-version QWP wire protocol, hardens replication and backup against transient object-store failures and in-place Parquet regeneration, and pulls in a large batch of Parquet, posting-index, SQL-correctness and performance fixes from the OSS engine.

New Features

This feature replaces the previous fixed 8-slot entry-count cache for decoded Parquet row groups with a byte-budgeted LRU cache, controlled by the new cairo.sql.parquet.cache.memory.size configuration property (default 256 MB). Each factory now declares whether it walks the base cursor monotonically or in scattered order via a ParquetDecodeHint. Scattered-access paths (sorts, hash joins, cached windows) receive the full configured budget, while monotonic paths (ASOF/LT/SPLICE joins, LATEST BY, scalar subqueries) cap at a quarter of the budget and at most 4 decoded row groups. The previous cairo.sql.parquet.frame.cache.capacity property is deprecated and no longer affects behavior. The cache uses an intrusive doubly-linked list with an IntObjHashMap index for O(1) hit lookup and LRU promotion, and performs in-place victim reuse on eviction to avoid free/munmap syscalls on the hot path. Worst-case RSS equals effectiveBudget × concurrent pools, and parallel joins open one decode pool per worker, each hinted monotonic. On a 502M-row trades table converted to Parquet, ORDER BY queries with LIMIT showed 12–31% wall-time improvements on scattered-access paths.
This feature logs an INFO line at a configurable interval with raw accounted memory, physical RSS, JVM heap usage, allocator counters, and the top 10 non-zero memory tags by absolute value. It reads Unsafe and Os memory accounting directly, so it works independently of metrics.enabled. Controlled by two hot-reloadable properties: memory.usage.log.enabled (default true) and memory.usage.log.interval (default 60s, max 24h). Example log line: memory usage [mem.accounted=821301126, mem.rss.accounted=610057798, mem.non.rss.accounted=211243328, mem.rss.limit=86442891210, rss.physical=1100009472, jvm.heap.used=164455264, jvm.heap.committed=301989888, jvm.heap.max=24024973312, malloc.count=7945, realloc.count=8, free.count=199, tags=[NATIVE_ILP_RSS=520159232, MMAP_TX_LOG=170385408, ...]].
This feature adds a SQL helper function, is_end_of_month(timestamp), which returns true when the provided timestamp falls on the final calendar day of its month. This is useful for monthly reports, billing-period logic, finance workflows, and calendar-based time-series analysis. It handles leap-year February, non-leap-year February, 30-day months, 31-day months, and nanosecond timestamp inputs.

Improvements

This improvement applies two independent optimizations to the Parquet write path shared by ALTER TABLE ... CONVERT PARTITION TO PARQUET, COPY ... TO Parquet export, and the /exp HTTP Parquet endpoint. First, symbol/dictionary statistics are now computed once per distinct dictionary key per page instead of once per row, using a reused per-column-chunk bitmap (one bit per key, up to 65,536 keys) that reduces comparisons from ~100k to ~1k per page for a 1000-symbol Zipf column with 100k-row pages. Second, the designated (monotonic) timestamp now defaults to delta_binary_packed encoding when no explicit encoding is given, significantly shrinking the timestamp column. On a 20M-row HOUR partition, conversion time dropped from 1.96s to 1.05s and Parquet file size decreased from 432 MB to 338 MB (a 47.2% reduction from the 640 MB native size). The delta_binary_packed encoding carries fixed per-block framing overhead that makes very small partitions (a handful of rows) slightly larger, but the net file-size win holds at scale. Explicit PARQUET(...) encodings still take precedence over the new default.
This improvement replaces the LimitedSizeLongTreeChain with an encoded-sort cursor for fixed-width sort keys (up to 32 bytes), eliminating random-access row-group decodes during the top-N build phase. The cursor collects encoded key and rowId pairs in a flat native buffer during the forward scan and sorts them natively. Once the buffer reaches max(2 × limit, 4096) entries (clamped to the sort memory cap), it sorts, truncates to the top limit, and rejects subsequent rows via a single native compare against the boundary entry. During the build phase, the sort cursor declares its sort-key columns to the base cursor, allowing async-filtered scans to decode only the filter and sort-key columns instead of the full projection. During the emit phase, the cursor declares the exact row set to the base cursor, enabling Parquet frames to materialize only those rows via row filtering, with a density gate falling back to full decode when declared rows cover half or more of a row group. On a 50M-row order-book table with wide array columns, the customer query at LIMIT 100000 improved from 236 ms to 185 ms warm and from 646 ms to 330 ms cold. The EXPLAIN output now prints "Encode sort light" instead of "Sort light". The legacy tree-chain cursor remains available via cairo.sql.orderby.sort.enabled=false. Two-bound NULL limit semantics changed: LIMIT null, N now returns the first N rows and LIMIT N, null returns an empty result. Rows sharing the boundary sort key are now ordered by ascending rowId instead of reverse insertion order.
This improvement enables equality predicates on a view or subquery that wraps a join to be propagated not only into the master table scan but also across equi-join keys into the slave table scans, turning full table scans into keyed lookups. Previously, filtering a view by a column of the join's master table pushed the predicate only to the master scan, while the equivalent inline query was already fully optimized. This fix covers INNER and LEFT joins, table and column aliases, and bind parameters as well as literal constants. For example, querying a view with WHERE entity_id = 'abc' now pushes the filter through events.entity_id = e.id into both slave events scans, producing index-forward scans on entity_id instead of full table scans. Constants pinned to the slave side are still not propagated to the master, and a WHERE on a left-joined slave column still runs as a post-join filter.
The parallel top-K factory (ORDER BY ... LIMIT N over page-frame tables) previously collected candidates into a per-worker tree structure driven by a compiled comparator, where every scanned row paid a tree insertion whose comparisons re-positioned the record across frames. This improvement routes encodable sort keys (fixed-width, byte-comparable keys up to 32 bytes) through flat per-worker encoded buffers of key-rowId entries instead. Each worker appends encoded entries and keeps its buffer at O(limit) through sort-and-truncate compaction; once a key threshold is known, non-qualifying rows drop with a native key compare. The owner merges the worker buffers through the same threshold filter, runs one final native sort, and emits rows by rowId. A batch-encoding fast path handles the common single fixed-width-8 column case, hoisting column base address, type dispatch, and direction transform out of the per-row loop. Per-worker encoders now share the owner's symbol rank maps instead of each independently sorting the whole symbol dictionary. Non-encodable keys (STRING/VARCHAR, non-static SYMBOL, keys over 32 bytes) keep the existing tree-chain path unchanged. In benchmarks on a 200M-row table, parquet partition queries improved from 380ms–200s (unstable) to a stable 19.76ms, and native partition queries improved from 29ms to 9.72ms. Peak native memory on the encoded parallel path is up to workerCount times the serial encoded cursor, though existing budget caps still apply.
Previously, SELECT ... FROM t LIMIT 10 on a parquet partition decoded the entire first row group before LIMIT discarded all but 10 rows, and LIMIT m, n decoded the skip-landing row group in full before jumping to the skip position. This improvement pushes a max-rows hint from the limit cursor down to the parquet decoder so the decode window covers only the rows that LIMIT will actually return. For LIMIT m, n, the window starts at the skip landing row, so the skipped prefix is not decoded either. Decoded buffers stay frame-origin-addressable by shifting published column vectors back, so records and row cursors keep using absolute frame-relative row indexes with no read-side changes. A query that random-accesses a clamped frame via recordAt re-decodes that frame to the full window, giving up the clamp for that frame in exchange for unconditional safety. In benchmarks on a table with 17 columns and 200,000 rows across two parquet row groups, SELECT * FROM t LIMIT 99990, 100010 improved from 53.04ms to 8.91ms, and SELECT * FROM t LIMIT -10, -20 improved from 26ms to 6ms. This fix also addresses a stale-pointer bug where partition frame cursors carried over a parquet metadata decoder reference from a prior call, which could crash or return wrong rows after reader eviction, and a clamp-gate bug where sorted symbol index scans could incorrectly read undecoded rows.
This improvement introduces a light cached window factory (CachedWindowLightRecordCursorFactory) that avoids materializing base columns into the record chain when the base cursor supports random access. Instead, it stores only deferred window output columns in a narrow chain plus a parallel DirectLongList of base row IDs, fetching base columns on demand via baseCursor.recordAt. An encoded sort buffer packs sort keys and row IDs into a flat DirectLongList and finalizes with Vect.sortEncodedEntries (parallelized above a configurable threshold), replacing the red-black tree sort for eligible groups. When the cursor knows the base row count, the buffer pre-sizes its backing memory to avoid geometric reallocations. The light path is controlled by the cairo.sql.window.cached.light.enabled configuration property (default true) and activates when the base cursor supports random access and every ordered group fits the encoded sort layout. Window queries with sort keys that do not fit the encoded layout (e.g., VARCHAR ORDER BY) fall back to the original tree-based factory. Additionally, lead(ts, 0) and lead(m, 0) now correctly return TIMESTAMP and DATE types respectively, matching the behavior of their lag counterparts instead of returning LONG.

Bug Fixes

This fix closes a gap in the object store client's retry coverage. While writes, deletes, copies, and streaming reads already had retry loops, whole-object reads (used by every download_and_unpack call site including restore's per-partition metadata, manifest, checkpoint history, and replication WAL index), stat calls, and lister creation were single-attempt operations. A transient S3 connection closure during a metadata download could abort a restore-on-startup and kill the server. The fix introduces a shared retry_op backing read, stat, and lister creation with jittered backoff. NotFound errors fail fast with their kind preserved so callers can map them to None, TLS certificate-trust failures also fail fast since no retry can succeed, and everything else is retried within the configured budget. Exhaustion wraps the last error with an exceeded retry attempts marker for diagnostics.
This fix addresses two latent bugs on the covering-index incremental seal path, both triggered by a pure-append out-of-order commit on a partition with a sealed posting index and more than 256 symbol keys. The first bug caused a crash after dropping an INCLUDE column: DROP COLUMN on an INCLUDE column tombstones its cover slot, but the incremental seal path did not skip tombstoned slots like the full-seal path did, resulting in an assertion error on unsupported column type -1 that distressed the writer. The second bug was a silent native memory overrun: sealIncremental sized the shared dirty-stride sidecar scratch at 8 bytes per merged value, but 16-byte types (UUID/LONG128/DECIMAL128) and 32-byte types (LONG256/DECIMAL256) wrote past the allocation through unbounded Unsafe.copyMemory. A hot symbol key with 4,001 rows and UUID+LONG256 INCLUDE columns could write 93,984 bytes past the end of a 34,048-byte allocation. The fix makes writeSidecarStrideData skip tombstoned slots and sizes the dirty-stride sidecar scratch by the widest live fixed-size cover column.
This fix addresses multiple defects where TableWriter.rollback() on a table with a covering POSTING index left the index's covering sidecars inconsistent with its row-id data. The next out-of-order commit could fail with a spurious "No space left" error citing a multi-exabyte size and distress the writer, or silently corrupt covered reads. Covered queries between the rollback and the next seal also returned NULL values. The root cause was that rollbackValues() via reencodeMonolithic() rewrote the row-id index into a fresh .pv at a bumped sealTxn but never wrote the .pc covering sidecars at that txn. The fix includes: rollback now returns early when no indexed value lies above the rollback point (the common case), avoiding unnecessary reencode work; reencodeMonolithic() rebuilds covering sidecars at the new sealTxn; seal() validates each cover's sidecar layout before snapshot copy and falls back to full seal on structural mismatch; incremental seal's clean-stride copy now bounds every offset and uses the stored stride-index sentinel as the upper bound instead of snapshot length, fixing a separate defect where post-seal gen flushes caused compounding sidecar growth; and rollback/truncate paths now sync the .pk chain publish before recording seal purge, closing a power-loss window where recovery could find a committed chain head pointing at deleted files.
This fix corrects two defects where order-sensitive aggregates first(), last(), first_not_null(), and last_not_null() could return values from the wrong row in parallel GROUP BY queries. The first defect affected per-row ordering in computeNext over DECIMAL, GEOHASH, and IPv4 columns, where the returned value came from an arbitrary row in the group rather than the row with the smallest or largest position. The second defect affected shard merge for last_not_null over every value type: when a group's value was non-null in one worker's map but null in another, last_not_null could silently drop the real value and return null. Both defects were data- and timing-dependent, surfacing only when workers reduced page frames out of row order, which is more likely under HTTP/PostgreSQL Wire Protocol worker pools or work stealing. The fix ensures computeNext decides the per-row winner by comparing row ids for the missed function families (Decimal, GeoHash, IPv4), and adds a stored-null check to every last_not_null merge path across all 14 value types (Char, Date, Decimal, Double, Float, GeoHash, IPv4, Int, Long, Str, Symbol, Timestamp, Uuid, and Varchar), mirroring the guard that first_not_null merges already carried.
Concurrent Influx Line Protocol clients sending to WAL tables over TCP could silently corrupt each other's data, with values from one connection leaking into rows written by another. The corruption affected DECIMAL columns ingested from numeric or string values, LONG256 columns, and binary-format values cast to SYMBOL columns. The server reported no error, so the problem only surfaced in query results. The root cause was that LineTcpMeasurementScheduler shared one LineWalAppender across all network IO workers, so concurrent appendToWal calls raced on its Decimal256, Long256Impl, and DirectUtf8Sink scratch buffers. This fix ensures the scheduler keeps one appender per network IO worker, indexed by worker ID, restoring the one-appender-per-thread invariant that the HTTP path and QwpWalAppender already followed. Deployments with a single network IO worker or a single connection were unaffected, as were ingestion over HTTP and QWP over WebSocket.
This fix addresses several classes of issues with Parquet file reading. Malformed or foreign-generated Parquet files could abort the entire database process through unrecoverable panics or infallible allocations sized by attacker-controlled counts in Thrift metadata, dictionary page headers, uncompressed page sizes, DELTA length streams, and array definition-level scratch buffers. All of these paths now use fallible allocation and return recoverable errors, keeping the server running. A separate correctness bug affected well-formed files including those QuestDB writes: partial range reads of multi-block DELTA-length-encoded STRING/VARCHAR/BINARY columns computed the value-bytes offset incorrectly, returning shifted or corrupted data. This occurred when an interval predicate ended inside a row group with more than 128 values. The offset calculation now uses a structural walk over the block/miniblock layout for partial reads. Additionally, IntervalBwdPartitionFrameCursor.calculateSize() had an off-by-one that under-counted rows for unbounded-low interval predicates (ts < X) in ORDER BY ts DESC queries, potentially dropping row 0 and skipping entire partitions. The non-parallel read path also previously discarded specific error messages behind a generic "corrupted" string; it now preserves and surfaces the underlying cause. Allocation failures on these paths are classified as OutOfMemory so that parquet merges under ApplyWal2TableJob back off and retry rather than suspending the table.
The WAL apply job's transaction skip calculation could walk past non-data transactions (structural metadata changes), allowing a skip decision to cross a DDL boundary. This meant two appliers of the same WAL stream (e.g. a replication primary and replica) could skip different transaction sets and reach a DDL point with different intermediate table content. When ALTER COLUMN ... TYPE SYMBOL seeds the new column's symbol map from the rows present at apply time, different intermediate content on two appliers caused different keys to be assigned to the same strings, resulting in rows silently resolving to wrong symbol values. The TRUNCATE fast path had the same exposure. This fix ensures the future-transaction scan now stops at the first non-data transaction of any kind, so a replace-range can only justify skipping transactions within the same structural-change-free window. This guarantees every applier holds identical table content at each DDL point, restoring the invariant that symbol key assignment is independent of the WAL apply method. The skip optimization becomes more conservative: a data transaction whose replacer sits beyond an interleaved DDL/SQL/TRUNCATE transaction is now applied normally instead of skipped. Materialized view refresh and replace-range ingestion without interleaved DDL keep the previous skip behavior unchanged.
The backward POSTING index reader keeps a per-key cache of generation-position entries that a full index walk resolves, and replays it on subsequent lookups of the same key. The cache-build guard keyed off an Elias-Fano mode flag, but certain early returns in the generation-load methods reset the block count without clearing this flag. When a walk passed through an Elias-Fano-encoded generation and then a lower generation that did not hold the key (reachable via split-block bloom filter false positives), the stale flag caused the guard to cache a spurious entry pointing at the lower generation but carrying the position from the EF generation. On the next lookup of the same key, replaying the spurious entry either returned row IDs belonging to a different key (wrong query results) or read a file offset far past the mapped value file and crashed the process with a SIGSEGV. This fix clears the Elias-Fano mode flag at the start of each generation-load method so a prior generation's mode cannot leak into the cache-add guard. The change does not affect the writer or the on-disk format, and indexes written before this fix are read correctly without requiring a rebuild. The forward reader was unaffected as it already reset the flag correctly.
Snapshot/checkpoint restore validates parquet partitions against _txn before generating missing _pm sidecars. The committed-size check previously ran only when _pm was absent: a restored _pm short-circuited the partition before any validation. A snapshot could pair _txn with a stale or truncated data.parquet and a matching old _pm (for example when the parquet file is regenerated in place between the moments the two files were captured), and the restore then completed silently, leaving a partition that reads garbage at query time. This fix ensures the restore agent now opens data.parquet and checks its length against the committed size from _txn for every parquet partition, regardless of whether _pm exists. A snapshot whose parquet partition is shorter than the committed size now fails the restore with a diagnostic error, and a parquet partition missing data.parquet now fails the restore even when _pm exists. Restores that previously appeared to succeed with such partitions now fail loudly at restore time rather than at query time, and the restore can be retried against an intact snapshot. Each parquet partition restore performs one extra open/length call when _pm already exists.
A query nesting a join inside an IN sub-query failed at compile time with an internal NullPointerException. The root cause was that SqlParser parsed a join's ON clause using a shared ExpressionTreeBuilder without raising an arg-stack floor, allowing the ON-clause drain loop to consume operands from the outer expression. This left the IN node with a null left operand, which the code generator dereferenced and crashed on. This fix introduces pushArgStackBottom() / popArgStackBottom() in ExpressionTreeBuilder, and parseJoin now brackets the ON-clause parse with them, raising the arg-stack floor and blocking sub-queries for the duration. A secondary fix ensures that parse errors unwinding through a raised parse-stack bottom now surface the positioned SqlException instead of an internal IllegalStateException, by clamping each restored bottom to the current stack size. Additionally, declared variables are now expanded up front in both shorthand ON branches: a variable bound to a sub-query is consistently rejected with "query is not allowed here" across all ON-clause positions, and a variable bound to a column correctly expands as a shorthand join column (e.g., ON (@c) with @c := symbol behaves like ON (symbol)).
During session ID rotation, SessionInfo.rotate() updated the coupled sessionId and rotateAt fields in sequence, while the eviction sweep read them under a different lock. An eviction interleaving between the two writes observed the new sessionId together with the stale rotateAt, computed an eviction time in the past, and dropped the old session ID immediately instead of keeping it valid through the grace period. A request still in flight on the old cookie was then rejected with HTTP 401, causing the user's Web Console session to appear to log out. This fix introduces a RotationInfo record so that rotate() performs a single atomic write of both sessionId and rotateAt, and the sweep reads that record as one consistent snapshot.
The optimiser incorrectly pushed WHERE predicates referencing only the master (left) table down into that table's sub-query for SPLICE, FULL OUTER, and RIGHT OUTER joins. These join types NULL-extend the master side for unmatched right rows, but the pushed predicate left those NULL-master rows unfiltered, producing wrong results. This fix introduces masterNullingJoinIndex() which reports the outermost downstream join that NULL-extends a given table. analyseEquals(), assignFilters(), and moveWhereInsideSubQueries() now consult it and keep such predicates as post-join filters instead of pushing them into the table's sub-query. A master-side equality spanning two master tables (WHERE t0.a = t1.b) is similarly deferred to the post-join filter when a downstream NULL-extending join is present. Three secondary fixes accompany the optimiser change: EmptySymbolMapReader.keyOf(null) now returns VALUE_IS_NULL instead of VALUE_NOT_FOUND, EmptyTableRandomRecordCursor.newSymbolTable() returns the shared immutable EmptySymbolMapReader.INSTANCE, and QueryModel.parseWhereClause() preserves each conjunct's innerPredicate origin. Additionally, a pre-existing crash where a SPLICE join feeding a subsequent join leaked a stale master alias into createJoinMetadata() (triggering an AssertionError with assertions enabled) is now resolved. The tradeoff is that master-side filters on these three join types can no longer use index or interval scans on the master table, running instead as a full master scan plus a post-join filter — the cost of correctness, as the previous plan produced wrong results.
This fix addresses a native memory leak tagged as NATIVE_INDEX_READER in the posting covering-index read path. When a row cursor was returned to the pool after its owning reader had already been closed, the cursor's blockBufferAddr (a 512-byte allocation) was re-pooled into a reader whose free-cursor list would never be drained again, causing the buffer to leak. The fix guards the re-pool operation with the reader's isOpen() state in both forward and backward readers' cursor close methods. When the owning reader is already closed, the cursor now releases its native buffers immediately rather than pooling into a dead reader, while preserving the block-buffer reuse optimization for the normal case when the reader is still open.
This fix resolves a ClassCastException that occurred when the SQL parser encountered a dotted name after an operator, such as tables()/.env. The expression parser attempted to cast an operator token to GenericLexer.FloatingSequence, but operator tokens are plain String instances, causing an unchecked cast failure. This surfaced as an internal 500-class error instead of a clean 400 bad-query response. The fix guards the fast-path concatenation in ExpressionParser so it only runs when the qualifier token is actually a FloatingSequence; otherwise it rejects cleanly with a SqlException stating '.' is unexpected here. This handles all malformed <expr> / .name patterns, not just the specific names used by vulnerability scanners.
This fix ensures that catalogue functions (tables(), all_tables(), information_schema.columns(), pg_catalog.pg_attribute) and point-lookup paths (SHOW COLUMNS, SHOW CREATE TABLE, SHOW CREATE MATERIALIZED VIEW, parquet row-group pruning) return complete results even when queried before the background metadata cache hydration finishes. Previously, the metadata cache was populated lazily on a background thread, so queries executed immediately after a restart or backup restore could observe an empty or partial table list. The fix adds hydrateAllTables() for catalogue enumerators and hydrateTableOnDemand() for point-lookup paths, which reconcile the cache against the authoritative table registry on demand. A volatile cacheComplete flag ensures these reconciliation calls short-circuit with no overhead once the cache is fully populated. A give-up budget of 8 consecutive zero-progress reconcile rounds prevents unbounded retries for genuinely unreadable tables. This is particularly important for PostgreSQL Wire Protocol clients, where drivers and BI tools automatically query column enumerators on connect — without this fix, the standard pg_class to pg_attribute introspection join would render existing tables with zero columns during the hydration window.

June 15, 2026

QuestDB 9.4.3 brings some key bug-fixes, along with parquet-native tables and new parquet-querying performance enhancements, with order-of-magnitude speedups for ORDER BY ... LIMIT queries.

New Features

This feature introduces a SQL helper function, is_end_of_month(timestamp), which returns true when the provided timestamp falls on the final calendar day of its month. It is useful for monthly reports, billing-period logic, finance workflows, and calendar-based time-series analysis. The function handles leap-year and non-leap-year February, 30-day months, 31-day months, and timestamp_ns inputs.
This feature logs an INFO line on a configurable interval with raw accounted memory, physical RSS, JVM heap usage, allocator counters, and the top 10 non-zero memory tags by absolute value. It reads Unsafe and Os memory accounting directly, so it works independently of metrics.enabled. Controlled by memory.usage.log.enabled (default true) and memory.usage.log.interval (default 60s, max 24h). Both properties support hot-reload. Example log line: memory usage [mem.accounted=821301126, mem.rss.accounted=610057798, mem.non.rss.accounted=211243328, mem.rss.limit=86442891210, rss.physical=1100009472, jvm.heap.used=164455264, ...].
This feature replaces the previous fixed 8-slot entry-count limit for decoded Parquet row groups with a byte-budgeted LRU cache, and lets factories declare their access pattern as MONOTONIC or SCATTERED. The byte budget makes the cache footprint explicit and tunable, while the access-pattern hint allows forward-mostly cursors to run on a smaller share of the budget while out-of-order cursors retain the full budget. Configured via cairo.sql.parquet.cache.memory.size (default 256 MB per cursor). The previous cairo.sql.parquet.frame.cache.capacity property is deprecated and no longer affects behavior. SCATTERED-hinted paths (sorts, hash joins, windows) receive the full budget and a much larger effective cache than the old 8 slots, while MONOTONIC-hinted paths (AsOf/Lt/Splice joins, latest by, scalar subqueries) cap at 4 decoded row groups at a quarter of the configured budget. Internal improvements include in-place victim reuse on cache miss to eliminate close()/reopen() round-trips and associated syscalls, a bounded shell pool to avoid wrapper-object churn, skipping openParquet() on cache hits, true O(1) LRU hit promotion via an intrusive doubly-linked list, and VARCHAR_SLICE decode footprint accounting.

Improvements

This improvement replaces the tree-chain comparator with a flat native buffer of encoded (key, rowId) pairs for fixed-width sort keys up to 32 bytes. During the build phase, the encoded-sort cursor collects entries and keeps the buffer at O(limit) through sort-and-truncate compaction, eliminating the O(K log K) random-access decodes per query that the tree chain required. The build phase also narrows decoded columns on async-filtered scans to only the filter and sort-key columns, and the emit phase decodes only the rows being returned via a row filter, with a density gate falling back to full decode when declared rows cover half or more of a row group. On a 50M-row Parquet partition with wide array columns, warm-cache performance improved from 236 ms to 185 ms and cold-cache from 646 ms to 330 ms. Build-phase improvements range from 3-6x across various LIMIT values, with the patch eliminating a decode-cache thrash cliff that caused master to degrade from 292 ms to 48.84 s at LIMIT 850,000. The EXPLAIN output now prints "Encode sort light" instead of "Sort light" for the encoded path. Two-bound NULL limit semantics changed: LIMIT null, N now returns the first N rows and LIMIT N, null returns an empty result, aligning with single-bound LIMIT N behavior.
This improvement routes encodable sort keys (fixed-width, byte-comparable keys up to 32 bytes) through flat per-worker EncodedTopKBuffers instead of per-worker LimitedSizeLongTreeChain structures in parallel top-K plans. Each worker appends encoded entries and keeps its buffer at O(limit) through sort-and-truncate compaction, while the owner merges worker buffers through a threshold filter and runs one final native sort. A single-column batch encoder (encodeFixed8Frame) hoists type dispatch and direction transforms out of the per-row loop for the common ORDER BY col LIMIT N shape, and per-worker encoders now share the owner's symbol rank maps instead of each sorting the full dictionary independently. On a 200M-row Parquet partition with wide array columns, the target query improved from 380 ms–200 s (unstable) on master to a stable 19.76 ms, and on native partitions from 29 ms to 9.72 ms. Non-encodable keys (STRING/VARCHAR, non-static SYMBOL, keys over 32 bytes) continue using the existing tree-chain path. Peak native memory on the encoded parallel path scales up to workerCount times the serial buffer size, bounded by cairo.sql.sort.key.max.bytes and cairo.sql.sort.light.value.max.bytes per worker.
This improvement pushes a row-count hint from LimitRecordCursor down to the Parquet decoder so the decode window covers only the rows that LIMIT will actually return. For LIMIT m, n, the window starts at the skip landing row, so the skipped prefix is not decoded either. When the row cursor is a forward 1:1 scan with no filter, the Parquet decode range is clamped via a new PageFrameMemoryPool.navigateTo overload that takes an in-frame row window. Decoded buffers remain frame-origin-addressable through column vector remapping. Queries like SELECT * FROM t LIMIT 99990, 100010 on a 200K-row Parquet partition improved from ~53ms to ~9ms, and negative limits saw similar gains. This also fixes a stale-pointer bug where FullFwdPartitionFrameCursor.next(skipTarget) and FullBwdPartitionFrameCursor.next(skipTarget) carried over a parquetMetaDecoder reference from a prior call, producing a use-after-free that could crash or return wrong rows after reader eviction. A second bug was fixed where SortedSymbolIndexRowCursorFactory could allow scattered symbol-index scans through the clamp gate, reading undecoded rows. The optimization does not apply to filtered queries (WHERE ... LIMIT N), cross joins, or native (non-Parquet) partitions.
This improvement enables equality predicates on a view or subquery that wraps a join to be pushed down not only into the master table scan but also propagated across equi-join keys into the slave scans, turning full table scans into keyed lookups. Previously, filtering a view by a column of the join's master table only reached the master scan, while the same query written inline was already optimized. For example, SELECT * FROM entity_stats WHERE entity_id = 'abc' on a view containing LEFT JOIN events ON events.entity_id = e.id now pushes entity_id = 'abc' into both events scans via index forward scans. This covers INNER and LEFT joins, table/column aliases, and bind parameters as well as literal constants. Constants pinned to the slave side are still not propagated to the master, and a WHERE on a left-joined slave column still runs as a post-join filter.
This improvement includes two changes to the Parquet write path shared by ALTER TABLE ... CONVERT PARTITION TO PARQUET, COPY ... TO Parquet export, and the /exp HTTP Parquet endpoint. First, symbol/dictionary statistics are now computed once per distinct dictionary key per page instead of once per row, using a reused per-column-chunk bitmap (one bit per key) for dictionaries up to 65,536 keys. This reduced the CPU time spent in BinaryMaxMinStats::update from 34.6% to ~0.1% of conversion CPU. Second, the designated (monotonic) timestamp now defaults to delta_binary_packed encoding when no explicit encoding is given, significantly shrinking the timestamp column. On a 20M-row partition with 17 columns, conversion time dropped from 1.96s to 1.05s and Parquet file size decreased from 432MB to 338MB (47.2% reduction vs native). Explicit PARQUET(...) encodings still take precedence over the new default. For very small partitions (a few rows), delta_binary_packed produces slightly larger files due to fixed per-block framing overhead.

Bug Fixes

This fix resolves a ClassCastException that occurred when the SQL expression parser encountered a dotted name (e.g., .env) after an operator (e.g., /). Queries like tables()/.env or select 1/.env from long_sequence(1) — commonly sent by vulnerability scanners probing endpoints like /exec?query=... — caused an unchecked cast of a plain String operator token to GenericLexer.FloatingSequence, resulting in an internal 500-class error with a stack trace instead of a clean 400 bad-query response. The fix guards the fast-path concat in ExpressionParser so it only runs when the qualifier token is actually a FloatingSequence; otherwise it rejects cleanly with a SqlException: '.' is unexpected here message. This handles every malformed <expr> / .name shape, not just the specific names used by scanners.
This fix resolves a native-memory leak of 512 bytes per occurrence (NATIVE_INDEX_READER tag) in the POSTING covering-index read path. PostingIndexBwdReader and PostingIndexFwdReader pool idle row cursors and deliberately retain each cursor's blockBufferAddr allocation for reuse. If a cursor returned to the pool after its owning reader was already closed — for example, when a reader-thread cursor outlived a concurrent reseal/reload — the cursor re-pooled into a reader whose free-cursor list would never be drained again, leaking the 512-byte block buffer. The fix guards the re-pool with the reader's isOpen() state in both readers' Cursor.close() and NullCursor.close() methods. When the owning reader is already closed, the cursor releases its native buffers immediately rather than pooling into a dead reader. The block-buffer reuse optimization is preserved for the normal case when the reader is still open.
The SQL optimizer incorrectly pushed WHERE predicates referencing only the master (left) table down into that table's sub-query for SPLICE, FULL OUTER, and RIGHT OUTER joins. These join types NULL-extend the master side for unmatched rows, so a pushed-down predicate left those NULL-master rows unfiltered, producing wrong results. The fix introduces masterNullingJoinIndex() to detect downstream NULL-extending joins and keeps such predicates as post-join filters via addPostJoinWhereClause. This applies across assignFilters(), analyseEquals(), and moveWhereInsideSubQueries(), covering single-table, multi-table, and barrier-joined branches. A master-side equality spanning two master tables is similarly deferred when a downstream NULL-extending join is present. Additionally, a pre-existing crash was fixed where a SPLICE join feeding a subsequent join leaked a stale master alias, causing an AssertionError in createJoinMetadata(). Because the predicate now runs after the join, master-side filters on these three join types can no longer use index or interval scans on the master table, which is the cost of correctness — the previous plan produced wrong results.
During session ID rotation, SessionInfo.rotate() updated the coupled sessionId and rotateAt fields in sequence, while the eviction sweep read them under a different lock. An eviction interleaving between the two writes observed the new sessionId together with the stale rotateAt, computed an eviction time in the past, and dropped the old session ID immediately instead of keeping it valid through the grace period. Requests still in flight on the old cookie were then rejected with HTTP 401, causing the user's Web Console session to appear to log out. This fix introduces a RotationInfo record so that rotate() performs a single atomic write of both sessionId and rotateAt, and the eviction sweep reads them as one consistent snapshot.
A query nesting a join inside an IN sub-query failed at compile time with an internal NullPointerException because the ON-clause parser never raised an argument-stack floor for the shared ExpressionTreeBuilder. Without the floor, the drain loop consumed operands belonging to the enclosing expression, leaving the IN node with a null left operand. The fix brackets the ON-clause parse with pushArgStackBottom() / popArgStackBottom(), isolating its operands and blocking sub-queries within the ON clause (which are unsupported and now consistently reject with "query is not allowed here"). A secondary fix clamps restored stack bottoms to the current stack size during error unwinding, so a parse error nested two or more lambda levels deep surfaces its positioned SqlException instead of a masking internal IllegalStateException. Additionally, parseJoin now expands declared variables up front in shorthand ON branches, so a variable bound to a column (e.g., @c := symbol used as ON (@c)) works as a shorthand join column, and a variable bound to a sub-query is consistently rejected across all ON-clause positions.
Snapshot restore validates Parquet partitions against _txn before generating missing _pm sidecars, but the committed-size check previously ran only when _pm was absent. A restored _pm short-circuited the partition before any validation, so a snapshot pairing _txn with a stale or truncated data.parquet and a matching old _pm completed silently, leaving a partition that reads garbage at query time. The restore agent now opens data.parquet and checks its length against the committed size from _txn for every Parquet partition regardless of whether _pm exists. Restores that previously appeared to succeed with such partitions now fail loudly with the existing "restored parquet file is shorter than committed size" diagnostic, surfacing the problem at restore time rather than at query time. Each Parquet partition restore performs one extra open/length call when _pm already exists.
The backward POSTING index reader maintains a per-key cache of (generation, position) entries resolved during a full index walk. The cache-build guard keyed off isEFMode, but early returns in loadSparseGenByPrefixSum for generations holding no values for the queried key reset isFlatMode and the block count without clearing isEFMode. When a walk passed through an Elias-Fano-encoded generation (setting isEFMode) followed by a lower generation that did not hold the key but bloom-filter false-positived it, the stale isEFMode caused the guard to cache a spurious entry pointing at the lower generation with a position from the EF generation. Replaying this entry on subsequent lookups either returned row IDs belonging to a different key (wrong query results) or read a file offset past the mapped value file, crashing the process with a SIGSEGV. This fix clears isEFMode at the start of each generation-load method (loadSparseGenByPrefixSum, loadSparseGenDirect, loadDenseGenerationCached) so a prior generation's mode cannot leak into the cache-add guard. The forward reader is unaffected as it already resets the flag. No index rebuild is required.
The WAL transaction skip optimization could cross DDL boundaries when calculating which transactions to skip, causing different appliers (e.g., replication primary and replica) to reach a DDL point with different intermediate table content. When ALTER COLUMN ... TYPE SYMBOL seeds the new column's symbol map from rows present at apply time, different intermediate content caused different key-to-string assignments, resulting in silent symbol value divergence. The future-transaction scan now stops at the first non-data transaction of any kind, ensuring every applier holds identical table content at each DDL point. The TRUNCATE early exit is now only reachable when the entire window before it consists of data transactions. This fix makes the skip optimization more conservative: a data transaction whose replacer sits beyond an interleaved DDL/SQL/TRUNCATE transaction is now applied normally instead of skipped.
This fix addresses multiple issues in the Parquet reader. Several malformed or foreign-generated Parquet inputs could abort the entire database process via unrecoverable panics or infallible allocations sized by attacker-controlled counts. Thrift metadata list reservations, RLE_DICTIONARY bit widths exceeding the target width, dictionary page num_values overrunning the buffer, unbounded decompression buffer allocations, foreign array definition-level scratch buffers, and DELTA length streams sized by attacker-controlled counts all now use fallible allocation and return recoverable errors instead of aborting. A correctness bug affecting well-formed files was also fixed: partial range reads of multi-block DELTA-length-encoded STRING/VARCHAR/BINARY columns computed the value-bytes offset incorrectly, returning shifted or corrupted values. The offset calculation now uses a structural walk over the block/miniblock layout when the length decoder was not exhausted. Additionally, IntervalBwdPartitionFrameCursor.calculateSize() had an off-by-one that under-counted rows for unbounded-low interval predicates (ts < X) in backward scans, potentially dropping whole partitions from count() and LIMIT results. The non-parallel read path now also preserves the specific guard error message instead of masking it behind a generic "corrupted" string. Allocation failures on these paths are classified as OutOfMemory so that write-path parquet merges back off and retry rather than suspending the table.
Concurrent Influx Line Protocol clients sending to WAL tables over TCP could silently corrupt each other's data, with values from one connection leaking into rows written by another. The corruption affected DECIMAL columns ingested from numeric or string values, LONG256 columns, and binary-format values cast to SYMBOL columns. The root cause was that LineTcpMeasurementScheduler shared one LineWalAppender across all network IO workers, so concurrent appendToWal calls raced on its Decimal256, Long256Impl, and DirectUtf8Sink scratch buffers. The scheduler now keeps one appender per network IO worker, indexed by worker ID. Deployments with a single network IO worker or a single connection were unaffected, as were ingestion over HTTP and QWP over WebSocket.
In a parallel GROUP BY, the order-sensitive aggregates first(), last(), first_not_null(), and last_not_null() could return a value from the wrong row. Two distinct defects were involved. First, per-row ordering over DECIMAL, GEOHASH, and IPv4 columns returned values from arbitrary rows in the group rather than the row with the smallest or largest position, because a prior change that added row-id comparison guards missed these types. Second, the last_not_null() shard merge for every value type could silently drop a real value and return null when a group's value was non-null in one worker's map but null in another with a higher row ID. Both defects were data- and timing-dependent, surfacing only when workers reduced page frames out of row order. The computeNext ordering now correctly compares row IDs for all affected types, and every last_not_null() merge now accepts the source value when the destination slot holds null, matching the guard that first_not_null() already had.
A TableWriter.rollback() on a table with a covering POSTING index left the index's covering sidecars inconsistent with its row-id data. Depending on the data that followed, the next O3 commit either failed with a spurious CairoException: No space left and a DISTRESSED writer, or silently corrupted covered reads. Covered queries issued between the rollback and the next seal also returned null values for the covered columns. The root cause was that reencodeMonolithic() rewrote the row-id index into a fresh .pv at a bumped sealTxn but never wrote the .pc covering sidecars at that transaction. This fix makes rollbackValues() return early when no indexed value lies above the rollback point, and reencodeMonolithic() now rebuilds the covering sidecars at the new sealTxn. The seal() method now validates each cover's sidecar layout before paying the snapshot copy and falls back to a full seal on structural mismatch. The incremental seal's clean-stride copy now bounds every offset it reads and uses the stored stride-index sentinel as the last stride's upper bound, fixing a secondary defect where post-seal gen flushes compounded sidecar growth on every following incremental seal. Additionally, rollback and truncate paths now sync the .pk chain publish before recording the seal purge, closing a window where a power loss could leave a committed chain head pointing at deleted files.
This fix addresses two latent bugs on the covering-index incremental seal path, both triggered by a pure-append out-of-order commit on a partition with a sealed posting index and more than 256 symbol keys, with the batch touching only some strides. The first bug caused a crash after dropping an INCLUDE column that a covering index references — the tombstoned slot routed into the fixed-stride writer, which rejected the sentinel type -1, distressing the writer. The second bug was a silent native-memory overrun when wide INCLUDE columns (UUID, LONG128, LONG256, etc.) were present — the incremental seal path sized its scratch buffer assuming 8 bytes per value, but 16- or 32-byte types could write far past the allocation through unbounded Unsafe.copyMemory. The fix ensures writeSidecarStrideData now skips tombstoned slots (matching the full-seal path behavior), and sealIncremental sizes the dirty-stride sidecar scratch by the widest live fixed-size cover column rather than assuming 8 bytes. Covered reads of a dropped INCLUDE column continue to return NULL as designed. The scratch allocation grows proportionally to the widest cover column but remains transient, freed at the end of the seal.

June 9, 2026

QuestDB Enterprise 3.3.1 is a maintenance release on the same QuestDB 9.4.x engine as 3.3.0. Driven by continued fuzz testing and stricter query-result assertions, it hardens the storage-policy and posting-index features introduced in 3.3.0 and pulls in a batch of SQL correctness and stability fixes from the OSS engine. It also adds a custom root CA option for the replication object store and an EXCLUDE column list for column-level GRANT/REVOKE.

New Features

Column-level GRANT/REVOKE now supports an EXCLUDE clause to target all columns except a specified set without listing every column explicitly. For example: GRANT SELECT ON foo(* EXCLUDE(a, b, c)) TO my_user;. The * means all columns of the table, optionally followed by EXCLUDE(...) to omit some. At statement time, the table's current columns are enumerated, excluded ones are dropped, and the permission is granted or revoked on each remaining column individually. Columns added after the statement runs are not covered automatically. Additionally, REVOKE is now strict: a REVOKE that names a table or column which does not exist will error instead of silently succeeding, catching typos that would otherwise leave the operator believing access was revoked. GRANT remains lenient about missing tables since Influx Line Protocol can auto-create tables on first ingestion. Every column named in EXCLUDE(...) must exist on the table, and duplicates or typos produce errors. Errors are positioned at the offending token for clear diagnostics.
This feature adds two optional parameters to the object store configuration string for HTTP-based services (s3, azblob, gcs): ca_cert_file=/path/to/ca.pem trusts the CA root certificate(s) in the specified PEM file in addition to the built-in Mozilla/webpki roots, and ca_builtin_roots=false drops the built-in roots to trust only the certificates from ca_cert_file. For example: s3::region=us-east-1;root=/bucket/path;endpoint=https://minio.internal;ca_cert_file=/etc/ssl/private-ca.pem;. Both keys are rejected for the fs service. The PEM is read and parsed during configuration validation, so a missing or malformed file fails fast. This is needed because the replication object store client uses rustls with the bundled webpki-roots only, ignoring the OS trust store, SSL_CERT_FILE/SSL_CERT_DIR, and AWS_CA_BUNDLE. Private or on-prem S3/Azure/GCS-compatible stores fronted by a private CA, self-signed certificate, or TLS-intercepting proxy previously failed the TLS handshake with no way to add the trusted CA. Parsing of the object store configuration string is now also stricter: a ;-separated parameter with no = is rejected with an error instead of being silently dropped.

Bug Fixes

When a partial upload advance encountered the first pending transaction whose WAL segment was still open, it consumed zero sequencer entries and incorrectly reported an error identical to a genuine catastrophic condition (missing part files). This caused the table to drop to the slow retry batch size and sleep the retry interval, even though deferring transactions in open segments is the designed behavior for partial mode. On production systems with many long-lived open WAL segments (e.g., many low-rate Influx Line Protocol writers), this resulted in hundreds of spurious errors per hour, each triggering a slow-mode flip and retry sleep, causing upload throughput to fall behind ingestion rate. This fix ensures the uploader only reports an error when zero entries were read without curtailment (the genuine unreadable-txnlog case). Zero-entry curtailment now takes the existing graceful path with no error, no slow-mode flip, and no retry sleep, and the caller resets to the fast batch size as for any successful advance.
QuestDB Enterprise 3.3.0 beta builds created sys.sp_entries with a drop_native column, but the current schema replaced it with to_remote. These are not equivalent, and since CREATE TABLE IF NOT EXISTS cannot repair an existing beta table, the writer would write into a mislabelled column and the reader would fail to compile its to_remote query. A new system migration (SysMig5) now runs on a primary before the storage-policy reader initializes. When it detects the beta schema, it drops the storage_policies system view so it can be recreated with the current definition, renames sys.sp_entries and sys.sp_links to *_v1_backup to preserve the rows, and fails startup with an actionable message. The next start creates the current schema. The migration is also safe under the replica-first upgrade procedure: StoragePolicyCheckJob catches compilation failures on the replica and retries until the primary is upgraded and the schema changes replicate. StoragePolicyReader now detects when the table ID behind sp_entries/sp_links changes, drops cached trackers, and reloads so the replica self-heals without an extra restart.
This fix addresses a set of correctness, resource, and concurrency bugs in the posting/covering index that surfaced across partition squash, the plain O3 commit path, parquet partition reseal, and the WAL fast-lag apply path. The primary issue was instability or missing rows after partition squash on tables with a POSTING index. When commitDense() rebuilt a large or squashed partition that exceeded the spill budget (cairo.posting.index.indexer.spill.bytes.max), mid-stream compaction persisted sparse generations to the .pv file, but the subsequent dense generation write orphaned them, causing failures for covering indexes or silently dropped rows for non-covering indexes. The fix consolidates through seal() when generations already exist, re-encoding every generation into a single dense gen-0 at offset 0. Additional fixes include: parquet partitions with covering posting indexes now have their .pci/.pc sidecars rebuilt during O3 reseal instead of returning NULL for covered columns; the WAL fast-lag apply path now restores covering configuration before indexing to prevent incomplete sidecars after mid-stream spills; ALTER TABLE ... ALTER COLUMN ... SYMBOL CAPACITY now preserves covering-column indices; a leaked .pv file during parquet reseal spills is now properly cleaned up via deferred seal-purges; and all seal-purge state access is now protected by a single lock for thread safety. A bare INDEX TYPE POSTING with no INCLUDE clause is now non-covering by default — the previous implicit covering behavior was unintended. Existing tables retain their persisted covering flag.
A COUNT() over a keyed GROUP BY subquery filtered on an aggregate alias (a HAVING-style predicate) reported duplicate groups that did not exist. The root cause was in the query optimizer's column propagation: propagateTopDownColumns0() had a guard that re-added grouping keys to top-down columns to prevent column pruning, but it ran before the model's own WHERE/HAVING literals had been emitted. For the COUNT() shape where the outer query contributes no key literals, the keys were left unprotected, pruning collapsed the keyed GROUP BY into a scalar aggregate, and the result was incorrect. The fix ensures retainGroupByKeysAsTopDownColumns() runs twice — once at the original early position and once after WHERE/HAVING and ORDER BY literals have been emitted. The second pass is idempotent via alias deduplication and only adds keys where the early pass missed them.
An aggregate (COUNT(), SUM(), etc.) over a UNION ALL of aliased sub-queries could fail query compilation with an AssertionError, surfacing as a 500 in the Web Console. The root cause was that the optimizer's column-propagation machinery resolved literals across UNION boundaries by name rather than by position. When the outer query selected nothing from the union (an aggregate), this pruned one branch down to a few matching-by-name columns while leaving other branches intact, causing the branches to disagree on column count. Since UNION columns are matched by position, not name, the fix removes the superseded by-name emit in favor of the indexed, by-position propagation. A rollback flag cairo.sql.legacy.union.column.propagation (default false) is available to restore the old behavior if needed.
SHOW CREATE TABLE previously resolved any object kind and rendered a CREATE TABLE statement for it. When run against a view or materialized view, it produced misleading DDL that looked like a table definition and would have created a plain table rather than the view if executed. This fix makes SHOW CREATE TABLE throw an error when the resolved token is a view or materialized view, reporting "table name expected, got view or materialized view name." The dedicated SHOW CREATE VIEW and SHOW CREATE MATERIALIZED VIEW statements should be used instead. Additionally, the sibling guards were tightened for symmetry so that SHOW CREATE VIEW <matview> now reports "got materialized view name" and SHOW CREATE MATERIALIZED VIEW <view> reports "got view name" rather than the generic "got table name."
This fix addresses numerous engine bugs surfaced by expanded query fuzzer coverage. Key fixes include: an off-by-one error when stepping past null array map keys in OrderedMapVarSizeRecord; ASOF join light path failures for STRING/VARCHAR-to-SYMBOL joins; ClassCastException in parallel keyed GROUP BY over covering index factories; memory leaks from row cursor factories not being freed on exceptions or toTop() calls; incorrect scan direction advertisement and duplicate-key handling for multi-key covering index queries; per-worker array key memory leaks in GROUP BY; incorrect group-by alias resolution for outer columns on duplicate references; unsafe LIMIT push through trivial group-by expressions when ORDER BY columns were absent from the GROUP BY; posting-index DISTINCT output named by source token instead of projection alias; JIT narrow-operand widening failures when LONG appeared only as a literal; full-fat ASOF/LT join projection failures with cross-type SYMBOL keys; SymbolConstant.valueOf() returning non-null for VALUE_IS_NULL keys; qualified column resolution failures under DISTINCT; and a missing IN function factory for IPv4 columns. The new InIPv4FunctionFactory accepts NULL, STRING, VARCHAR, SYMBOL, IPv4, and bind variables, using LongHashSet storage to safely cover the full 32-bit IPv4 range.
This fix addresses five production bugs exposed by stronger assertion coverage. Cross join skipRows() was not re-entrant: a second call while the cursor sat partway through a master row's slave scan re-skipped the already-consumed master cursor and dropped remaining slave rows, causing LIMIT and result-size calculation over cross joins to count or skip too few rows. Multi-value indexed latest-by (<indexed symbol> IN (...) LATEST ON ts) emitted result rows in index/partition discovery order without sorting across multiple partitions, yet advertised a forward scan direction, so the optimizer elided ORDER BY ts and returned unsorted rows. Parquet random access could read freed memory under some conditions: PageFrameMemoryPool.navigateTo() early-returned when the record's frame index matched the requested one, but an AsyncFilteredRecordCursor record could be bound to a reduce task's parquet buffers that were freed eagerly on collect, causing subsequent column reads to dereference freed native memory. Dense ASOF join produced stale results when its cursor was re-read because toTop() did not reset backwardScanExhausted, causing matches to be dropped on every pass after the first. Finally, COUNT(*) over a full scan threw on an empty partition missing from disk because calculateSize() opened every partition unconditionally, unlike next() which already skips zero-row partitions.

QuestDB 9.4.2 is a hardening release that builds on 9.4.1, driven by continued fuzz testing and stricter query-result assertions. It includes bugfixes for the new posting index, parquet tables, temporal joins, and a variety of other SQL queries. This release also brings a Web Console upgrade, which brings some fixes and a new utility for sharing queries with your colleagues.

New Features

This feature adds the ability to share links to queries directly from the Web Console editor. Option/Alt + L copies a link to the current query at the cursor (or the selected portion when a selection exists), while Option/Alt + Shift + L copies a link to all queries in the tab. These actions are also available from the run button dropdown as "Copy link to the query" and "Copy link to all queries" respectively, using the same mechanism as demo queries on the documentation site.
This feature adds two related capabilities for QuestDB Enterprise. The materialized view generator now correctly handles tables with STORAGE POLICY(...) by projecting each clause into the materialized view's partition unit (DAY → DAYS, MONTH → MONTHS, YEAR → YEARS), preserving the required ordering of clauses, and ladder-bumping the terminal clause so the materialized view retention outlives the source. TTL is floored at the partition unit. The table details drawer in the Web Console now surfaces storage policy information: when clauses are configured, each is rendered as a label/value card; on QuestDB Enterprise with no policy, a "Not configured" placeholder is shown. The TTL card only appears when a TTL value is actually configured.

Bug Fixes

This fix addresses a set of correctness, resource, and concurrency bugs in the posting/covering index that surfaced across partition squash, the plain O3 commit path, parquet partition reseal, and the WAL fast-lag apply path. A table with a posting index could crash the JVM (covering index) or return short/incorrect indexed counts (non-covering index) after partitions were squashed, because commitDense() assumed it wrote the first and only generation at offset 0, but a mid-stream spill flush could persist sparse generations earlier, orphaning them. The fix makes commitDense() consolidate through seal() when generations already exist. Additionally, covered reads returned NULL after an O3/squash reseal of a parquet partition because the O3 worker that rewrites a parquet partition built only the non-covering .pv and sealPostingIndexForPartition previously skipped parquet partitions entirely. On the WAL fast-lag apply path, a mid-stream spill flush during updateIndexesParallel could write an incomplete .pc covered sidecar. ALTER TABLE ... ALTER COLUMN ... SYMBOL CAPACITY did not carry over covering-column indices, so the next metadata rewrite lost the covering flag. A parquet reseal value-file leak was fixed by handing the rebuild's seal-purges to the TableWriter's deferred queue. Concurrency hardening adds a single lock for all seal-purge state accessed by parallel O3 workers.
Reading a Parquet partition could fail when an integer-family column had a DELTA_BINARY_PACKED data page containing only nulls. The integer writer short-circuited and emitted an empty values buffer with no delta header for pages with zero non-null values, causing the reader to decode block_size = 0 and reject it. This failure showed up on eager full-row-group reads such as converting a partition back to native or WAL apply re-running that conversion under replication, suspending the table on both primary and replica. The reader now treats an empty values buffer as a zero-value page and returns an iterator that yields no values, keeping already-written Parquet files readable. The writer now emits a self-describing delta header (value_count = 0) for all-null integer pages so newly written files are valid Parquet. Additionally, several reachable panics in the Parquet read path were closed — these could be triggered from the public read_parquet() SQL function or from partition conversion on malformed or foreign-produced pages. A Rust panic crosses the JNI boundary as a JVM abort with no recovery. The decoders and vendored parquet2 now reject these inputs with clean errors: DELTA_LENGTH_BYTE_ARRAY varchar pages with bit widths above 32, DELTA_BINARY_PACKED headers with invalid block sizes or zero miniblock counts, per-miniblock bit widths above 64, oversized block sizes causing multiply overflows, and ULEB128 varint values wider than 64 bits.
This fix addresses 18 engine bugs surfaced by expanded query fuzzer coverage. Key fixes include: an off-by-one in ArrayTypeDriver.getPlainValueSize() when stepping past null array map keys that caused assertion failures; ASOF Light path crashes for STRING/VARCHAR-to-SYMBOL joins due to an unimplemented SymbolJoinKeyMapping.of(RecordCursor) default; ClassCastException when parallel keyed GROUP BY ran over a CoveringIndex factory because CoveringPageFrameCursor did not extend TablePageFrameCursor; memory leaks of per-symbol PostingIndexFwdReader.Cursor instances when HeapRowCursor.of() threw mid-iteration; honest scan direction reporting and duplicate-key deduplication for multi-key CoveringIndex queries to prevent incorrect SAMPLE BY and ORDER BY behavior; per-worker array key memory leaks in GROUP BY where DirectArray backing memory was orphaned; incorrect group-by alias resolution causing wtf? errors on SELECT DISTINCT with duplicate column references; a gate on trivial-group-by LIMIT push to prevent ArrayIndexOutOfBoundsException when ORDER BY columns were absent from the GROUP BY; JIT narrow-operand widening when LONG appeared only as a literal, causing int32 wrapping divergence from the Java path; full-fat ASOF/LT join projection crashes with cross-type SYMBOL keys; SymbolConstant.valueOf() ignoring VALUE_IS_NULL keys causing stale values instead of NULL; qualified column reference resolution failures under DISTINCT; and a new InIPv4FunctionFactory for IN predicates on IPv4 columns that previously fell back to incompatible STRING comparison.
This fix addresses five production bugs exposed by strengthened query assertions. Parquet random access could read freed memory and crash the JVM: PageFrameMemoryPool.navigateTo() early-returned when the record's frame index matched the requested one, but an AsyncFilteredRecordCursor record could be bound to a reduce task's parquet buffers that were freed eagerly on collect, causing a use-after-free SIGSEGV on the next column read. The fix always re-navigates parquet records while keeping the early-return optimization for native frames. Multi-value indexed latest-by (LatestByValuesIndexedRecordCursor) emitted result rows in index/partition discovery order without sorting, yet advertised forward scan direction, so the optimizer elided ORDER BY ts and returned unsorted rows across multiple partitions. The fix sorts results into ascending designated-timestamp order. Dense ASOF join produced stale results when its cursor was re-read because toTop() did not reset backwardScanExhausted and of() reset none of the five scan fields, causing matches to be dropped on every pass after the first. Cross join skipRows() was not re-entrant — a second call while partway through a master row's slave scan re-skipped the already-consumed master cursor, causing LIMIT and result-size calculation to count too few rows. Finally, count(*) over a full scan threw on an empty partition missing from disk because calculateSize() opened every partition unconditionally unlike next() which already skips zero-row partitions.
This fix corrects a bug where count() over a keyed GROUP BY subquery filtered on the aggregate alias (a HAVING-style predicate) reported duplicate groups that did not exist. For example, SELECT count() dups FROM (SELECT ts, s, count() c FROM tab) WHERE c > 1 returned 1 instead of the expected 0 when all pairs were unique. The root cause was in propagateTopDownColumns0(), which had a guard to re-add a group-by subquery's grouping keys to its top-down columns so column pruning could not drop them, but this guard ran before the model's own WHERE/HAVING and ORDER BY literals had been emitted into the top-down list. For the count() shape, the outer query contributed no key literals, so the HAVING alias was the sole top-down contributor added only after the guard had already observed an empty list and skipped. The keys were left unprotected, pruning collapsed the keyed group-by into a scalar aggregate, and the total row count falsely passed the filter. The fix runs retainGroupByKeysAsTopDownColumns() twice: once at the original early position and once after WHERE/HAVING and ORDER BY literals have been emitted. The second pass is idempotent via addTopDownColumn() deduplication and only affects query compilation with no measurable performance change.
An aggregate function (such as count() or sum()) over a UNION ALL of aliased sub-queries crashed query compilation with an AssertionError, surfacing as a 500 error in the Web Console. The root cause was that the SQL optimizer's column-propagation logic emitted columns across UNION boundaries by name rather than by position. When aliases differed between branches, this pruned some branches incorrectly, causing a column count mismatch. This fix removes the superseded by-name emit in favor of the correct indexed, by-position propagation. A rollback flag cairo.sql.legacy.union.column.propagation (default false) is available to restore the old behavior if needed.
Previously, SHOW CREATE TABLE resolved any object kind and rendered a CREATE TABLE statement for it. When run against a view or materialized view, it produced misleading DDL that, if executed, would have created a plain table rather than the view. This fix makes SHOW CREATE TABLE reject views and materialized views with a clear error message. The sibling guards were also tightened so that SHOW CREATE VIEW run against a materialized view (and vice versa) now reports the actual object kind in the error. Users who previously relied on SHOW CREATE TABLE for views should use SHOW CREATE VIEW or SHOW CREATE MATERIALIZED VIEW instead.
When a user logged in via SSO and then logged out, the Web Console kept the client ID and automatically re-ran the OAuth flow on page refresh, effectively preventing logout. This fix introduces an explicit SSO_SESSION_ACTIVE flag that is set to false on logout to prevent auto-login on page refresh. The page now auto-refreshes on logout to clear stale information from the previous user's session, such as grid results, preventing data leakage into the next login.
This fix updates the validation debounce timeouts in the editor to 1 second while typing and 500ms when the cursor is shifted between queries.

June 3, 2026

QuestDB Enterprise 3.3.0 is a feature release built on the QuestDB 9.4.x engine. The headline additions are the new storage policy engine for tiering data between native and Parquet (local and object storage), posting index support, a new ingestion server and wire protocol with durable acknowledgements (QWP), and a move to JDK 25.

Breaking Changes

This fix reworks the storage policy SQL surface and lifecycle in QuestDB Enterprise. Previously, TO PARQUET only generated a data.parquet file alongside native columns without switching reads, and a separate DROP NATIVE step was required to flip the partition to Parquet format. This left an unused Parquet file on disk in the gap between the two operations. Now, TO PARQUET produces data.parquet and immediately removes the native columns at its TTL, eliminating the unused-file gap. DROP NATIVE is removed and replaced by TO REMOTE, which will handle uploading a partition's Parquet data to object storage once that functionality is implemented. The storage policy clauses are now TO PARQUET, TO REMOTE, DROP LOCAL, and DROP REMOTE. Example usage: ALTER TABLE t SET STORAGE POLICY(TO PARQUET 7d, TO REMOTE 30d, DROP LOCAL 90d). The drop_native column in sys.sp_entries and the storage_policies view is renamed to to_remote. TO PARQUET and TO REMOTE are not ordered relative to each other, allowing TO REMOTE < TO PARQUET for keeping both formats on local disk. The parser enforces ordering constraints such as TO PARQUET ≤ DROP LOCAL and TO REMOTE ≤ DROP LOCAL.

New Features

A storage policy defines how a table manages its cold-storage lifecycle, controlling when partitions are converted to Parquet, when native data is dropped, and when local data is removed. A policy consists of three optional settings: TO PARQUET <ttl> (convert partition to Parquet after this time), DROP NATIVE <ttl> (delete native partition, keeping only Parquet), and DROP LOCAL <ttl> (remove all local partitions). All TTL values must be positive, and later clauses must specify a TTL greater than or equal to earlier ones. Storage policies can be defined inline during table or materialized view creation (CREATE TABLE abc (...) STORAGE POLICY(TO PARQUET 3d, DROP NATIVE 10d, DROP LOCAL 1M) WAL), modified (ALTER TABLE abc SET STORAGE POLICY(...)), enabled/disabled without removal (ALTER TABLE abc DISABLE STORAGE POLICY), dropped (ALTER TABLE abc DROP STORAGE POLICY), and queried (SELECT * FROM storage_policies). The implementation includes a two-pipeline processing system: Pipeline 1 handles PARQUET_CONVERSION (reader-only) followed by PARQUET_COMMIT (writer-required), while Pipeline 2 handles direct PARQUET_COMMIT for partitions already converted but needing to switch to Parquet-only format. Staleness detection uses a squash tracker mechanism with an 8-bit counter, overflow fallback via timestamp file, and row count comparison to prevent data loss from in-place squashes. Transaction metadata uses bit 60 (parquetGenerated) and bit 61 (parquetFormat) flags per partition. Only inactive partitions can be converted, and tables must have at least 2 partitions. The feature includes four new permissions (SET_STORAGE_POLICY, REMOVE_STORAGE_POLICY, ENABLE_STORAGE_POLICY, DISABLE_STORAGE_POLICY), configurable check intervals and retry behavior, and SHOW CREATE TABLE/SHOW CREATE MATERIALIZED VIEW output includes the storage policy clause. Legacy TTL settings are deprecated in QuestDB Enterprise — SET TTL <non-zero> is rejected, but SET TTL 0 is allowed as a migration path. An ObjectStoreParquetDispatcher interface is scaffolded for future remote object-store integration, with DROP REMOTE syntax reserved but not yet supported.
This feature connects the Rust WAL uploader to the DurableAckRegistry interface so that QuestDB Wire Protocol connections receive STATUS_DURABLE_ACK frames once committed data reaches the object store. A new DurableUploadRegistry implements DurableAckRegistry using a lock-free ConcurrentHashMap<AtomicLong> to track per-table upload watermarks via monotonic CAS. It handles native UTF-8 directory name pointers from Rust without heap-allocating on the hot path, using DirectUtf8String for ASCII fast-path lookup and only materializing a heap String on first sight of a table. When a table is dropped, a MAX_VALUE sentinel is set so pending durable-ack entries unblock immediately, and late uploads after drop are absorbed by the CAS. The Rust uploader calls a new segmentUploaded JNI method after each successful segment and index upload. Rust JNI error handling was hardened: get_current_env() was removed in favor of try_get_current_env() returning Option, so log calls from non-JNI threads fall back to eprintln! instead of panicking, and call_method/call_void_method return anyhow::Error instead of panicking on missing JVM or unexpected return types.
This feature adds enterprise-side plumbing for the posting index. Backup support recognizes new posting-index file types (.pk, .pv, .pci, .pd, .pc0..pcN), including multi-segment versioned forms. DDL and backup-restore paths now track per-column index type and covering column sets through getIndexType and getCoveringColumnIndices delegates in create table, materialized view, and view operation implementations. The MetadataService.addColumn and changeColumnType methods accept a byte indexType parameter instead of boolean isIndexed. The REINDEX permission check now covers posting-indexed columns alongside legacy bitmap-indexed ones when granted at the table level without an explicit column list. The WAL transfer path removes a per-call String allocation by writing the partId-prefixed upload.pending file via Path.put. Rust uploader error handling was improved by replacing unwrap()/expect()/debug_assert_ne! with Result and ? propagation, and a divide-by-zero guard was added for corrupted V2 headers.

Improvements

This improvement introduces two related changes to QuestDB Enterprise replication. First, upload-amplification observability is added via per-table useful_* upload counters that make write amplification measurable, including replication_up_uploaded_bytes_total, replication_up_uploads_total, replication_up_useful_upload_bytes_total, and replication_up_useful_uploads_total. Amplification ratios can be computed as a single clean division (e.g., rate(uploaded_bytes_total[5m]) / rate(useful_upload_bytes_total[5m])). Second, the primary throttle window is lowered from 10s to 1s on all cloud backends (S3, Azure, GCS, FS), and the index-upload coalescer is enabled per cloud class, reducing open-segment replica lag from ~10s to ~2s. The index-upload coalescer merges all tables' index updates into at most one PUT per interval for the whole primary, decoupling index-PUT rate from both the window and the table count. For non-GCS backends, the index-upload throttle is set to 10ms, capping index PUTs at 100/s per primary regardless of fleet size. For GCS, the 1s throttle respects the per-object write cap but operates at the limit with no margin. The idle CPU usage of the coalescer wake-up was also fixed, reducing idle primary CPU from ~107% to ~7% of one core. A non-GCS deployment that explicitly set replication.primary.throttle.window.duration or replication.primary.requests.retry.interval to a sub-10ms value and left the index-upload throttle at default will now fail to boot; setting replication.primary.index.upload.throttle.interval=0 restores the previous behavior.

Bug Fixes

The SQL validation endpoint (/api/v1/sql/validate) is intended to check syntax without side effects or authorization enforcement. However, QuestDB Enterprise was executing DDL inline during compilation and enforcing permissions, so validating statements like CREATE USER, GRANT, REVOKE, BACKUP DATABASE, and storage policy changes would actually modify server state or fail on authorization. This fix guards each mutating call in EntSqlCompilerImpl with isValidationOnly(), preserving parsing and query-type reporting so validation still reports syntax errors and the correct statement type. A new EntValidationSecurityContext extends the allow-all context to bypass authorization during validation while delegating identity information (principal, session principal, auth type) to the real context. All QuestDB Enterprise security contexts implement asValidationContext() with lazy caching. OWNED BY parsing during CREATE TABLE, CREATE MATERIALIZED VIEW, and CREATE VIEW also reads identity through this validation view.
This fix enables recovery of sequencer metadata from the committed WAL log, ensuring metadata consistency after failures.

QuestDB 9.4.1 is a hardening release that builds on 9.4.0. Introduction of our new SQL fuzz-testing engine and test framework improvements has flushed out more than 60 latent bugs around query correctness and resource leaks. 9.4.1 also brings additional hardening and bugfixes for the newly released posting/covering indexes, performance enhancements for materialised views, upgrades to parquet querying, and new aggregate and window functions.

New Features

This feature introduces the array_agg() aggregate function that collects per-group double values or arrays into a DOUBLE[] result. Both array_agg(DOUBLE) and array_agg(DOUBLE[]) preserve input order across keyed and non-keyed GROUP BY and SAMPLE BY, including parallel execution. The function supports FILL(NONE), FILL(NULL), and FILL(PREV) in SAMPLE BY, while FILL(LINEAR) and FILL(VALUE) are rejected at compile time. NULL inputs to array_agg(DOUBLE) are preserved as null elements in the output array, while NULL or empty arrays passed to array_agg(DOUBLE[]) are skipped during concatenation. The output column type is DOUBLE[], suitable for downstream array functions such as array_count, array_sum, and indexing. Memory usage scales with total element count at 16 bytes per element in the build buffer, and CairoConfiguration.maxArrayElementCount caps per-group element count. As part of this change, several pre-existing SAMPLE BY ... FILL correctness bugs were fixed: fill values applied after aggregate arithmetic now return the correct fill value instead of the fill value transformed by the arithmetic; duplicate aggregates with different fill values now each receive their own fill; and multi-fill validation now correctly matches each aggregate against its corresponding fill entry using 0-based indexing.
This feature introduces regr_r2(y, x), the standard SQL coefficient-of-determination aggregate that reports how well a linear regression of Y on X fits the data on a 0–1 scale. This is useful for separating real time-series trends from noise — for example, finding sensors whose temperature is trending up with high confidence versus those drifting randomly. The function implements SQL:2003 §10.9 semantics: when Sxx = 0 it returns NULL (covering single-row and constant-X cases), when Syy = 0 with Sxx ≠ 0 it returns 1.0 (a horizontal line is a perfect fit when Y is constant), and otherwise returns Sxy² / (Sxx · Syy). The implementation introduces a shared AbstractRegressionGroupByFunction base class with a 6-slot Welford state (meanY, Syy, meanX, Sxx, Sxy, count) that provides per-row update and Chan parallel merge in one place. The existing regr_slope and regr_intercept functions were migrated onto this shared base, significantly reducing their code size. The semantics diverge from corr() at one edge: when Y is constant and X varies, regr_r2 returns 1.0 per SQL:2003, while corr() returns NULL.
This feature adds window function support for all six DECIMAL sub-types (D8, D16, D32, D64, D128, D256). The covered functions include first_value, last_value, nth_value, lag, lead, min, max, count, sum, avg, and avg(x, n). Each factory covers the same frame shapes already supported by existing primitive-typed window functions: whole partition, current row, ROWS BETWEEN, RANGE BETWEEN, partitioned and non-partitioned forms. For output types, first_value, last_value, nth_value, lag, lead, min, and max return the input decimal type; count returns LONG; sum widens to reduce overflow risk (D8/D16 widen to D64, D32/D64 widen to D128, D128/D256 widen to D256); avg(x) matches the input type; and avg(x, n) returns D256 with target scale n. NULL inputs are skipped in aggregations and propagate through value-access functions per existing window semantics. D128 and D256 ring-buffer slots are 16 and 32 bytes respectively, so memory usage for wide windows over these column types is correspondingly larger than for primitive numeric window functions.

Improvements

This improvement introduces six new configuration properties that provide opt-in byte-denominated caps on native memory growth for window function and ORDER BY operators: cairo.sql.window.cache.max.bytes, cairo.sql.window.rowid.max.bytes, cairo.sql.window.tree.max.bytes, cairo.sql.sort.key.max.bytes, cairo.sql.sort.light.value.max.bytes, and cairo.sql.sort.value.max.bytes. All caps are unset (uncapped) by default; setting any of them bounds the operator and raises a LimitOverflowException with a hint naming the specific configuration key that needs to be raised. The previous page-based configuration keys (cairo.sql.window.tree.max.pages, cairo.sql.window.rowid.max.pages, cairo.sql.sort.key.max.pages, cairo.sql.sort.light.value.max.pages, cairo.sql.sort.value.max.pages) are now deprecated but continue to be parsed — if a user has one set, the derived byte default becomes pageSize * maxPages, and an explicit new *.max.bytes value takes precedence when both are set. When sort-key materialization is engaged, the cairo.sql.sort.key.max.bytes budget is split across materialized column buffers in proportion to each column's fixed-size width, so each buffer's row capacity stays roughly balanced. Page-size configuration keys now clamp to a minimum of 1 byte at read time, preventing misconfigured zero values from propagating into downstream divisions.
This improvement adds a cost-aware clustering pass to materialized view refresh that skips unchanged buckets when an out-of-order historical write lands far behind the current commit position. Previously, the incremental refresh scanned every non-empty bucket between the O3 timestamp and the current position, including pre-existing ones that no WAL transaction touched. The optimization uses two rolling exponential moving averages on materialized view state (average commit latency and average scan latency per timestamp-unit) to drive a gap-width threshold below which adjacent cached intervals are merged into clusters. Each cluster receives its own iterator step sized to its width, so a step-group never straddles two clusters and the existing gap-skip excises gaps cheaply. Benchmarks show improvements from 160ms down to ~1.7ms (94x faster) for a 512-symbol materialized view with a 720-minute O3 lag. The materialized_views() function gains three columns (refresh_avg_commit_nanos, refresh_avg_scan_nanos_per_ts_unit, refresh_gap_threshold_ts_units) for operator visibility into the cost model, and REFRESH MATERIALIZED VIEW <name> STATS resets the EMAs when workload shape changes. A new configuration key cairo.mat.view.refresh.max.clusters (default 32) caps the number of clusters per refresh to prevent pathological many-disjoint-intervals workloads from emitting hundreds of tiny commits.
This improvement prevents materialized views from emitting bursts of no-op replace-range WAL transactions when the base table has an apply backlog. Previously, period views bypassed a guard that checked whether the base transaction watermark had actually advanced, causing each refresh iteration to commit a no-rows resetMatViewState WAL transaction even when no new data existed. Under sustained backlog, a single view could emit millions of such transactions in minutes, bloating the view's WAL and stalling WAL apply on replicas. The fix ensures insertAsSelect() only commits a watermark transaction when it actually advances (commitBaseTxn > lastRefreshBaseTxn or commitPeriodHi > lastPeriodHi). Additionally, the refresh acknowledgement logic now tracks the minimum base transaction examined across refreshed views, allowing the clean/dirty transaction handshake to converge sooner and reducing redundant re-enqueues.
This improvement reworks the parallel and serial execution of the twap() and sparkline() aggregate functions, achieving 2-3x speedups on common parallel and concurrent workloads. The parallel aggregates buffer per-group observations and sort them at merge and read time. Previously, the sort recovered run boundaries by scanning for key decreases, which conflated page frames whose key ranges ascended across a gap and forced element-wise merging. The new approach records exact per-frame batch boundaries in a lazily allocated descriptor buffer, allowing the sort to permute whole batches with single bulk copies. Consecutive frames are coalesced into a single batch, avoiding descriptor allocation entirely for groups whose frames arrive in order. Additional hot-path tuning includes insertion sort for small batch counts (n ≤ 16), pre-grown scratch lists, and consolidated memory allocation calls. The high-cardinality memory-stress scenario (1M groups, ~10 observations per group) regresses by approximately 15% due to per-group state widening from 24 to 56 bytes without enough observations to amortize the new merge step. The twap() function now validates at compile time that its timestamp argument is the table's designated timestamp, since the batch model relies on page frames being pre-sorted by that column.
This improvement enables parallel execution of the low-precision approx_percentile() function when used with LONG column arguments by replacing the heap-based HdrHistogram with an off-heap GroupByHistogram in ApproxPercentileLongGroupByFunction.
This improvement removes the non-keyed vector group-by factory (GroupByNotKeyedVectorRecordCursorFactory) and routes all non-keyed aggregation queries through AsyncGroupByNotKeyedRecordCursorFactory exclusively. To maintain parity, new batch-aware group-by functions were added: avg(int), avg(long), min(short), and max(short). Existing functions gained batch implementations on the async path: ksum(double), nsum(double), and sum(long256). Additionally, min(timestamp) and max(timestamp) now short-circuit the column scan when the argument is the designated timestamp by reading the first or last row of the page frame directly, reflected in query plans as min_designated / max_designated. A bug in SumShortGroupByFunction.getComputeBatchArgType that returned INT instead of SHORT was also corrected. Query plan output changes for non-keyed aggregations that previously surfaced as GroupByNotKeyed Vectorized and for designated timestamp min/max aggregations.
Previously, _pm sidecar files produced by Mig940, TableSnapshotRestore, and the attach-existing-parquet code path in TableWriter did not inline Parquet bloom-filter bitsets. This meant queries with equality predicates on bloom-indexed columns had to read the bitset from data.parquet at plan time on partitions that arrived through one of these paths. This improvement closes that gap by having convert_from_parquet read each chunk's bitset via parquet2::bloom_filter::read_from_slice_at_offset and inline it through RowGroupBlockBuilder::add_bloom_filter. The ParquetMetadataWriter.generate JNI bridge now always mmaps the parquet file and passes that slice, so all three Java call sites produce _pm files identical to the write path's output. MIGRATION_VERSION bumps from 428 to 429, so installs that already ran the 428 migration will overwrite their existing _pm files on the next startup. Databases already on migration 428 pay a one-off _pm regeneration cost on next startup, comparable to the original 428 migration run. The _pm file size grows by the size of any inlined bitsets for tables with bloom configuration, while tables without bloom configuration see no size change.

Bug Fixes

This fix addresses 55 distinct bugs across SQL query execution, JIT compilation, parallel aggregation, and resource management. Key corrections include: parallel reduce/filter now preserves original exception types instead of wrapping them as CairoException; SymbolFunction.getStrLen() no longer throws UnsupportedOperationException; SAMPLE BY ... FILL(value) properly rejects incompatible fill/aggregate type combinations; JIT no longer silently truncates out-of-range BYTE/SHORT literals or computes narrow arithmetic at incorrect widths; cast-to-symbol functions are now marked as thread-unsafe to prevent shared mutable state across parallel workers; parquet pushdown no longer truncates overflow-folded long constants on narrow-int columns; WhereClauseParser no longer clobbers FALSE intrinsic values set by earlier conjuncts; MinCharGroupByFunction.merge() no longer overruns the 2-byte CHAR slot; string/varchar comparison short-circuits on NULL constants now correctly handle negation; LIMIT push-down no longer propagates past GROUP BY, SAMPLE BY, WINDOW, or HORIZON JOIN; nested INT arithmetic in LONG-context predicates now correctly widens through subtrees via getLong(); and multiple resource leaks in factory construction error paths are resolved. Three behavior changes are included: INT - FLOAT now returns FLOAT instead of DOUBLE (matching the other arithmetic operators); column <= null and column >= null for STRING/VARCHAR now return matching NULL rows under QuestDB's NULL = NULL -> true convention; and nested INT arithmetic getLong() overrides now recurse through subtrees at long width.
This fix addresses nine latent production bugs across multiple components. Array function factories (DoubleArrayAddFunctionFactory, DoubleArrayDivFunctionFactory, DoubleArrayMultiplyFunctionFactory, DoubleArraySubtractFunctionFactory) now restore the declared array type before reusing the output buffer, so a null array row preceding a non-null one no longer causes failures. GenerateSeriesTimestampRecordCursorFactory and GenerateSeriesTimestampStringRecordCursorFactory now report scan direction only for constant steps, returning SCAN_DIRECTION_OTHER for bind-variable steps instead of reading from an unbound function. AsyncWindowJoinRecordCursor.calculateSize() now correctly advances the cursor past the last frame, preventing a subsequent hasNext() from wrongly returning true. AsyncFilteredRecordCursorFactory.recordCursorSupportsRandomAccess() no longer incorrectly delegates to the base factory, since the async-filtered cursor always supports random access through its own page-frame memory pool. PageFrameRecordCursorImpl.toTop() now properly releases the row cursor through Misc.free instead of nulling it without freeing, preventing a resource leak.
This fix resolves a SIGSEGV crash that occurred when an out-of-order or WAL commit deduplicated against a STRING dedup key whose per-partition data file had grown beyond 2 GiB. The native dedup merge comparer read the variable-length data offset into a 32-bit integer, but offsets are stored as 64-bit values. Once a partition's STRING .d file exceeded 2 GiB, offsets greater than or equal to 2^31 were truncated to negative values, causing a wild pointer dereference. Because this occurred during WAL apply or O3 commit, the offending transaction was replayed on every restart, causing an affected table to enter a boot-time crash loop. The fix reads the offset as int64_t. BINARY columns already used int64_t and VARCHAR uses a separate comparer, so neither is affected.
This fix resolves a crash (SIGSEGV) or stale data issue when querying only columns that were added via ALTER TABLE ADD COLUMN after a partition was converted to Parquet. When the projected column set contained no column present in the Parquet file, PageFrameMemoryPool.ParquetBuffers.decode() skipped sizing and zeroing the per-column page-address lists, leaving them with stale or uninitialized native memory. PageFrameMemoryRecord then dereferenced invalid page addresses, causing either a crash on freshly allocated buffers or wrong non-NULL values on reused buffers. The fix hoists remapColumns() out of the parquetColumns.size() > 0 guard so address lists are always sized and zeroed, and adds an early return for zero-column reads such as COUNT(*). Absent Parquet columns now correctly read as NULL, matching native-partition behavior. Queries that read at least one column present in the Parquet file are unaffected.
This fix corrects TTL validation for materialized views that use months-based TTL values. QuestDB encodes TTL as a single integer where positive values represent hours and negative values represent months. When a materialized view omitted an explicit PARTITION BY, the derived partition validation guard only checked for positive TTL values (ttlHoursOrMonths > 0), allowing months-based TTL values to bypass granularity validation entirely. This could result in invalid configurations, such as TTL 7 months being accepted on a view with a derived YEAR partition, even though 7 months is not a whole number of years. The fix widens the guard to ttlHoursOrMonths != 0, ensuring months-based TTL values go through the same PartitionBy.validateTtlGranularity() check as hours-based ones. The error message for unrecognized tokens after ALTER MATERIALIZED VIEW <name> now also lists set as a valid continuation.
This fix resolves an issue where tables with a posting-indexed symbol column (TYPE POSTING, TYPE POSTING DELTA, TYPE POSTING EF) could not be converted to Parquet via storage policy. The TableWriter.linkPartitionIndexFiles method only hard-linked the legacy bitmap layout (.k / .v files), but posting-indexed columns have no .v file — their value data lives in .pv.<colTxn>.<sealTxn> plus .pci sidecar and per-column .pc<N>.{colTxn}.{sealTxn} data files. The link against the non-existent .v returned ENOENT and the commit aborted, leaving the partition stuck partway through the switch on every retry. A new helper linkColumnIndexFiles now owns the per-column hard-link step, routing key/value file paths through IndexFactory which resolves to the correct files for both bitmap and posting index types. For posting columns, it also hard-links the .pci sidecar and per-column data files for the live generation.
A bound check was missing in the asmjit BitVectorRangeIterator::next_range against the iterator's end position. Without it, the iterator could surface a free bit past the search-region end whenever the end landed mid-BitWord. The caller then computed a range size via unsigned subtraction that underflowed, causing JitAllocator::alloc to accept an area index outside the block. The returned Span reported its requested size while the underlying memory ran past the block boundary into unmapped pages. This manifested in production as a SIGSEGV inside JitRuntime::_add's memcpy on a network worker JIT-compiling a SQL filter. This fix updates the asmjit dependency to include the corrected bound check.
The corr, stddev_pop, stddev_samp, var_pop, var_samp, covar_pop, and covar_samp functions were returning NULL on sparse-NULL data when using parallel GROUP BY. The merge algorithm called when parallel workers need to merge partial results of the aggregate was not considering empty partials. When partials were empty, a NaN value was being propagated and surfaced as a SQL NULL. This fix applies a merge guard to skip empty partials across all affected aggregates.
This fix addresses several gaps in the ALTER ADD POSTING INDEX ... INCLUDE (covering index) Parquet path. Seal never read Parquet column data, was not exercised for a WAL table's last partition, and could leak file descriptors or temp files on error paths. Large sealed blocks and high-cardinality multi-key covering scans could also exceed RSS_MEM_LIMIT, and one multi-key resume branch could loop forever returning the same frame. Key changes include: Parquet-aware seal that routes Parquet partitions through a dedicated indexParquetPartition path with batched materialisation across row groups; streaming FSST compression that splits the one-shot compress into a four-call lifecycle to keep anonymous-heap scratch in the low MiBs regardless of stride size; chunked FSST decompression that imports the symbol table once per block and decompresses 256 values per access instead of the entire block; immediate buffer freeing during cursor grow operations instead of accumulating all prior generations; a hard cap of 1,000,000 rows per PageFrame with resumable fill via parked RowCursor; and a fix to advance currentKeyIdx in the multi-key resume branch when a parked RowCursor drains, preventing infinite loops. Robustness improvements include pre-deleting leftover temp files before WAL-apply retries, tracking both temp file paths before mmapping, and using quiet removal to avoid masking real failures.
Indexed WHERE col = null could return zero rows from a Parquet partition whose SYMBOL column was entirely NULL in a row group. The Parquet decoder has a documented optimisation where it skips materialising the buffer and returns size = 0 when a column chunk's stats report null_count == num_values. Three call sites that rebuild a covering index from a decoded Parquet chunk shared the same defect: O3PartitionJob.updateParquetIndexes (after O3 insert rewrites), TableWriter.indexParquetPartition (during ALTER TABLE ALTER COLUMN ... ADD INDEX), and TableSnapshotRestore.rebuildTableFiles (during checkpoint restore). In all three sites the walk iterated zero bytes and wrote no index entries, resulting in an index that recognised no NULL rows. This fix detects the size == 0 convention and emits explicit null index entries for every row the row group covers, extracted into BitmapIndexUtils.addNullEntries so the convention is handled in one place.
The SQL validation endpoint (/api/v1/sql/validate) was not fully side-effect free: it still enforced permissions, ran some statements inline during compile, and emitted query-progress log lines plus error metrics. This fix makes validation-only compilation side-effect free. Inline execution is now skipped via fine-grained isValidationOnly() guards around individual side-effecting calls for REINDEX, TRUNCATE, VACUUM, RESUME/SUSPEND WAL, ALTER ... SET TYPE, ALTER VIEW/CREATE OR REPLACE VIEW, and COMPILE VIEW, while still running the full compile path to resolve target objects and check semantics. Authorization is bypassed through a new SecurityContext.asValidationContext() that returns a no-op allow-all view during validation. Logging and metrics are suppressed by having shouldLogSql() return false and QueryProgress.logError() return early in validation-only mode. Validating a statement against a non-existent table now correctly reports "table does not exist" instead of passing as syntactically valid. The compiled record cursor factory is freed for every non-SELECT statement and validationOnly is reset in a finally block to ensure the no-op security context view cannot outlive the request.
Adding a covering index via ALTER TABLE ... ALTER COLUMN <sym> ADD INDEX TYPE POSTING INCLUDE (...) threw a NullPointerException and suspended the table when the table's most recent partition was stored as Parquet. The indexLastPartition() method had no branch for Parquet partitions, so it attempted to index a Parquet last partition as if it had native column files, causing the covering seal() to dereference a FilesFacade that was never set for the Parquet path. This fix detects a Parquet last partition and delegates to indexParquetPartition(), the same routine already used by indexHistoricPartitions(). Non-WAL tables were unaffected since CONVERT PARTITION TO PARQUET skips the active partition, so a non-WAL table's last partition is always native.
Queries using SELECT ... LIMIT -N over a covering (posting) index with a residual filter silently returned the first N rows instead of the last N. The covering factory ignored the requested scan order when serving a page-frame cursor, always opening ascending partition frames. The parallel negative-limit machinery then collected the lowest-timestamp rows believing they were the highest. For single-key covering queries, getPageFrameCursor now honors descending order by iterating partitions latest-to-earliest and splitting row ranges into sub-frames emitted highest-first, preserving parallel filtering and enabling early termination for small N. For multi-key covering queries, which lack global timestamp ordering, the fix routes negative-limit queries to the serial FilteredRecordCursorFactory where LimitRecordCursorFactory computes last-N correctly via size plus skip. Bind-variable limits with unknown sign at compile time also take the serial path. Positive-limit and no-limit multi-key queries remain unaffected and still run in parallel.
A query filtered on a POSTING-indexed SYMBOL column could return fewer rows than a full scan of the same data, with no error reported. This occurred after an out-of-order insert that failed before its commit became durable. The write was correctly rolled back, but a background cleanup step may have already deleted index files the recovered table still needed, causing the index to fall back to an older generation whose files were gone. This fix holds back index-file cleanup until the transaction that supersedes those files has durably committed. If the transaction fails or rolls back, the cleanup is discarded with it, so the previous index generation stays on disk and recovery falls back to it correctly. Operations that copy index files to a new location, like RENAME COLUMN or Parquet conversion, also receive a matching fix: they drop an uncommitted future index generation before linking, so the column points at the generation readers actually see rather than one that may be rolled back.
This fix enables rebuilding bricked or version-mismatched WAL sequencer _meta from _meta.0 and committed metadata sidecars. The transaction log's max metadata version is treated as the committed recovery boundary, so uncommitted sidecar tails are ignored and ahead-of-log _meta state is rolled back. During rebuild, the registry table token remains authoritative for table renames. RENAME_TABLE sidecars now only advance the recovered structure version, preventing abandoned or chained rename sidecars from changing the recovered table name.
Under concurrent query load with cross-query work-stealing, twap() and sparkline() could silently return wrong results during parallel GROUP BY. Both functions maintain per-slot native buffers of (key, value) entries and run a two-pointer merge at slot-combine time. The buffers are bound to slots rather than workers, and the cursor thread work-steals tasks across queries, allowing a single slot to accumulate frames in non-monotonic order. The two-pointer merge then ran on unsorted input and silently produced wrong values. This fix introduces SortedRunsMerge, a stable bottom-up pairwise mergesort over the natural sorted runs detected in the buffer via a single scan. TwapGroupByFunction.merge() and SparklineGroupByFunction.merge() now delegate to SortedRunsMerge.compactInto, while the read paths call SortedRunsMerge.compactInPlace to handle the case where no merge phase ran but work-stealing still left the buffer in multi-run state. The merge pass is allocation-free and adds only one extra linear scan for queries that never encounter the multi-run condition.
Queries joining on SYMBOL plus UUID or DECIMAL keys could fail with an UnsupportedOperationException instead of returning results. This fix delegates the missing record accessors so these joins work correctly.

May 18, 2026

QuestDB 9.4.0 introduces a compact, high-performance posting and covering index for SYMBOL columns, a local parquet metadata sidecar that unlocks row-group pruning, parallelised SAMPLE BY FILL with new cross-column FILL(PREV) syntax, three new window functions, and sparkline() / bar() text visualisations. It also delivers meaningful GROUP BY / hash-join speed-ups and fixes a number of correctness issues across the SQL planner, the WAL apply path, and the PGWire protocol.

New Features

This feature introduces a new posting index format for symbol columns with optional covering index support, storing selected column values in sidecar files alongside the posting list. The index can be created inline, out-of-line, or added via ALTER TABLE, for example: CREATE TABLE t (ts TIMESTAMP, sym SYMBOL INDEX TYPE POSTING INCLUDE (price, qty), price DOUBLE, qty INT) TIMESTAMP(ts) PARTITION BY DAY;. Row IDs are encoded per-key using delta + Frame-of-Reference (FoR64) bitpacking with adaptive selection between delta and flat modes, with native AVX2 decode for common bitwidths and a Java scalar fallback. When a query selects only the indexed symbol and INCLUDE columns, data is read directly from sidecar files without touching column data files. All column types are supported in INCLUDE, with ALP compression for DOUBLE/FLOAT, FoR bitpacking for integers, and FSST compression for VARCHAR/STRING. Supported query shapes include WHERE sym = 'X', WHERE sym IN (...), bind variables, LATEST ON ts PARTITION BY sym, and SELECT DISTINCT sym. Hints /*+ no_covering */ and /*+ no_index */ allow disabling the covering and index paths. Configuration properties include cairo.posting.index.row.id.encoding (adaptive/ef/delta) and cairo.posting.index.auto.include.timestamp. Benchmarks show the posting index is approximately 13.6x smaller and 1.3-1.5x faster for reads than BITMAP, at a ~9% write regression. The CAPACITY clause remains valid only for BITMAP indexes.
This feature introduces a compact binary _pm sidecar file that accompanies each data.parquet partition file, storing all metadata the query engine needs: column descriptors, QuestDB column types, per-row-group column chunk byte ranges, compression codecs, encodings, and min/max statistics. It replaces the JSON metadata blob previously embedded in the parquet footer's key-value section with a purpose-built binary format and enables row group pruning via locally-stored min/max statistics and bloom filter offsets without reading the parquet file itself. The format supports lock-free concurrent read/write via MVCC footer chaining, with the parquet file size in _txn field 3 serving as the version token. This lays the groundwork for cold storage, where the _pm file stays local and provides everything the query planner needs to decide which column chunks to fetch by byte range, eliminating metadata round-trips. Migration Mig940 generates _pm files for all existing parquet partitions on engine upgrade in a non-destructive manner. If a user rolls back to an older version, modifies parquet data, and re-upgrades, the migration can be re-run by setting cairo.repeat.migration.from.version in server.conf. SHOW PARTITIONS and ParquetMetaPartitionDecoder use the _pm file to extract metadata and decode row groups without parsing the parquet footer.
This feature moves SAMPLE BY FILL queries from the sequential cursor path onto QuestDB's parallel GROUP BY fast path where the fill mode is supported. The optimizer rewrites SAMPLE BY to GROUP BY timestamp_floor_utc(...), and a unified streaming fill cursor inserts gap-filled rows above the sorted GROUP BY output. New FILL(PREV(col_ref)) syntax enables cross-column PREV, where any slot in the per-column fill list may reference the previous value of another output column instead of its own. For example, a candlestick query can carry the prior bucket's close into the next bucket's open: SELECT ts, first(price) AS open, last(price) AS close FROM trades SAMPLE BY 1h FILL(PREV(close), PREV);. The source column must exist in the SELECT list with compatible types (full-type equality for DECIMAL, GEOHASH, ARRAY, TIMESTAMP, and INTERVAL), and cannot be the designated timestamp, another PREV reference, or a SYMBOL column. This improvement also fixes several issues: infinite fill loop with ALIGN TO CALENDAR WITH OFFSET + FILL without TO, sub-day SAMPLE BY + TIME ZONE + FROM/TO grid misalignment, SAMPLE BY FILL rejection on pre-1970 timestamps, and several memory leaks in shared sort infrastructure. FILL(LINEAR) and ALIGN TO FIRST OBSERVATION remain on the cursor path. Behavior change: FILL queries with FROM now apply effectiveOffset = FROM + OFFSET, unifying with non-FILL SAMPLE BY semantics.
This feature adds three new window functions. NTILE(n) distributes rows of an ordered partition into n approximately equal buckets and returns the 1-based bucket number. CUME_DIST() returns the cumulative distribution: rows at or before the current row (including peers) divided by total rows in the partition. NTH_VALUE(expr, n) returns the n-th value (1-based) within the current window frame, or NULL when n exceeds the frame size. All three support PARTITION BY and ORDER BY, and NTH_VALUE supports ROWS and RANGE frames (bounded and unbounded). In this initial release, NTH_VALUE accepts only DOUBLE first argument; LONG and TIMESTAMP overloads follow in a separate change. NTH_VALUE requires n to be a compile-time constant and rejects IGNORE NULLS / RESPECT NULLS, FROM FIRST, and FROM LAST. NTILE and CUME_DIST run in two passes and reject explicit ROWS / RANGE / GROUPS frame clauses. NTH_VALUE RANGE ... CURRENT ROW follows the QuestDB convention of not looking ahead to peer rows, which diverges from the SQL standard / PostgreSQL on tied ORDER BY values.
This feature extends NTH_VALUE window function with LONG-returning and TIMESTAMP-returning factories, mirroring the routing of the DOUBLE variant: per-partition / whole-partition / ROWS / RANGE bounded and unbounded frames, plus the lock-in fast path for UNBOUNDED PRECEDING ... K PRECEDING and the current-row 1-row frame. The TIMESTAMP variant overrides getType() to return the argument's type so TIMESTAMP_MICROS and TIMESTAMP_NANOS subtypes propagate intact. Both factories use Numbers.LONG_NULL as the sentinel for cases where n exceeds the current frame size and for genuine NULL inputs. As a side effect of adding multiple overloads, arity validation now shifts from the per-factory body into the overload resolver, producing a "no matching function" error instead of a "wrong number of arguments" error.
This feature introduces two new functions for inline text visualization. The sparkline() aggregate function collects numeric values within a GROUP BY and renders them as a Unicode trend line using block characters (▁▂▃▄▅▆▇█), pairing naturally with SAMPLE BY to show intra-bucket trends. It supports auto-scaling, explicit min/max bounds, clamping, NULL handling, and width-based sub-sampling. The bar() scalar function renders a single numeric value as a horizontal bar (▏▎▍▌▋▊▉█) proportional to a given range, working with aggregates like sum() and window functions like min() OVER () for auto-scaling, with fractional block precision of 8 levels per character. Both functions return VARCHAR and work across all clients including psql, Web Console, JDBC, and CSV. Output size is bounded by cairo.sql.string.function.buffer.max.size.
This feature propagates PARQUET_ENCODING(...) for pass-through projected columns through the streaming Parquet export path, so COPY (SELECT ...) TO parquet and /exp preserve the configured encoding instead of silently falling back to defaults. Previously, projected query metadata dropped parquetEncodingConfig for pass-through columns, causing the Parquet writer to only see the default encoding when running through the streaming/page-frame path. The fix is intentionally scoped to pass-through columns; computed columns still derive their Parquet behavior from the projected type rather than the source column's encoding config. The writer-side encoder layout in parquet_write has also been cleaned up to mirror parquet_read more closely, with one top-level dispatch entrypoint and smaller encoding-family modules under parquet_write/encoders/.
This feature generates a starter CREATE MATERIALIZED VIEW statement from a table or an existing materialized view and inserts it into the editor for the user to tweak. From a table, the menu item is disabled for non-WAL tables and tables without a designated timestamp; the view name follows <table>_<sample>, replacing existing period suffix (e.g. my_table_5m → my_table_1h); REFRESH IMMEDIATE, SAMPLE BY, PARTITION BY, and TTL are inferred per QuestDB's default inference; column aggregates use sum/last by name pattern, with types lacking a LAST() overload (BINARY, LONG128, INTERVAL, arrays) dropped. From a materialized view (downsample), SAMPLE BY is stepped one ladder rung up; WITH BASE is re-rooted at the source mat view; aggregate args are rewritten to layer-1 aliases; COUNT() becomes SUM; COUNT(DISTINCT …) is dropped; non-decomposable aggregates fall back to LAST(); WHERE, GROUP BY, and LATEST ON are stripped; REFRESH, PERIOD, and OWNED BY are preserved; and TTL is stepped one rung up. Generation errors surface through the toast instead of being swallowed.

Improvements

This improvement introduces batched aggregate dispatch for the parallel keyed GROUP BY path. Instead of per-row virtual dispatch through GroupByFunctionsUpdater, the reducer splits each page frame into sub-batches (2048 rows by default, configurable via cairo.sql.parallel.groupby.batch.size) and processes them in two phases: a probe phase that finds or creates map entries and packs row index plus entry offset into a scratch buffer, and an update phase that calls computeKeyedBatch() once per function per sub-batch. Hot functions including count, sum, min, max, avg, bit_and, bit_or, and bit_xor override with tight Unsafe loops that skip the per-row MapValue dispatch. Benchmarks show speedups ranging from 1.16x to 4.60x on single-column count() queries, with per-function isolation tests showing 2.71x-3.03x improvements across aggregates. This also fixes a pre-existing data-correctness bug in count(uuid) where CountUuidGroupByFunction read the high 64 bits of the UUID argument twice, causing any UUID whose high half equals Long.MIN_VALUE to be silently excluded from the count.
This improvement consolidates QuestDB's non-Murmur 64-bit hash paths onto a single finalizer derived from xxHash3's 64-bit avalanche, and raises the load factor of the affected hash tables from 0.5 to 0.7. The new xxh3Avalanche64 mixer is roughly 1.7x faster than fmix64 in latency mode while matching its quality tier, allowing the removal of the weaker fastHashInt64 and fastHashLong64 FxHasher-based mixers. The six former fastHash* callers had 0.5 load factors chosen to absorb FxHasher's weak avalanche; with the stronger mixer they run denser at the project's standard 0.7. The cairo.sql.count.distinct.load.factor default is also raised. An incidental fix corrects DirectLongHashSet.rehash() arithmetic that was leaving the set operating past its configured load factor. ClickBench shows a 1.21% total time improvement across 43 queries, with single-column count() queries on Unordered4Map/Unordered8Map keys improving up to 25.7%.
This improvement extends the parallel top-K gate in SqlCodeGenerator#generateOrderBy so it also fires when a column-projection wrapper (SelectedRecordCursorFactory or VirtualRecordCursorFactory) sits between the ORDER BY ... LIMIT N and the filtered page-frame scan. Previously, any non-literal SELECT list combined with a WHERE clause forced the query onto the generic Sort light path, materializing and sorting every matching row instead of keeping a bounded heap of N. The implementation adds two default methods on RecordCursorFactory — translateOrderByColumnToBase and rewrapOverTopK — which projection wrappers override to peel themselves, allow top-K to apply to the inner factory, and re-wrap the output. An incidental fix adds explicit getColumnCrossIndex() != null guards in generateJoinAsof's slave-projection peels to prevent silently dropping the VirtualRecord layer's column translation.
Comparing a TIMESTAMP column to a runtime-constant value (for example a string bind variable sent over PostgreSQL Wire Protocol) used to re-parse that value on every row, which could dominate CPU time on large scans via repeated NumericException throws inside the implicit string-to-timestamp cast. This improvement evaluates the constant or bind-variable side once at query init, regardless of precision or which side of = it appears on. There is a small behavior change: some EXPLAIN plans will show timestamp equality arguments swapped, for example instead of filter: (ts=123) there will be filter: (123=ts). This is acceptable given EXPLAIN plans are not APIs and have no stability guarantee.
Lateral join decorrelation previously duplicated the outer query model for each correlated reference. This improvement replaces deep-cloning with lightweight QueryModelWrapper references that share the same underlying model, and introduces SharedRecordCursorFactory so the shared model executes only once at runtime. The wrapper/shared-cursor mechanism is designed as a general framework for materializing common table expressions, allowing future CTE and view support to reuse it to eliminate duplicate execution across multiple references. Shared cursor support covers all GROUP BY factory variants, including vectorized and async variants, as well as pass-through via SelectedRecordCursorFactory.
This improvement replaces JSON structured output with plain text responses plus a suggest_query tool for SQL suggestions, and adds thinking/reasoning content display for reasoning models. All JSON response format schemas, custom provider JSON parsing/repair logic, and extractPartialExplanation have been removed, along with the responseStart, explanation, and contentFragments fields from conversation messages. Full tool call and result history is now included in conversations, giving the model context of prior tool interactions across turns, and assistant responses are grouped with their tool calls in the UI on a turn basis. The max response token limit has been raised to 64,000 for Anthropic models, and summary generation for context compaction now uses streaming. Issues around running queries from the chat window when the editor is unmounted and query key mismatches between chat and editor runs have been resolved, unnecessary rerenders and layout issues have been prevented, and cancellation/abort handling has been added for queries run from the AI Chat Window.

Bug Fixes

WAL apply could store the bitmap index max row as the exclusive end when appending to an indexed SYMBOL column in a non-last partition. The next apply then treated the index as one row ahead and rolled it back before indexing new rows. For users with high-cardinality indexed symbols, this added unnecessary index scan work and could increase WAL apply latency. This fix stores the bitmap index max row as an inclusive row id and clarifies the row-id adjustment in the O3 index update path.
Malformed SQL with inner queries containing an empty LIMIT clause, such as DECLARE @pair := (SELECT symbol FROM fx_trades LIMIT ) SELECT ..., could trigger an internal error. The parser was allowing a nested expression parse to consume operands from the outer declaration expression, corrupting the assignment AST before parseDeclare() inspected it. This fix isolates ExpressionTreeBuilder operand stack frames across reentrant parseExpr() calls, so nested parses cannot consume outer operands, and rejects an empty LIMIT clause with a clear parser error: 'limit expression expected'.
This fix resolves an internal error in ASOF/LT joins when a subquery shifts the designated timestamp with dateadd() and that timestamp is later pruned from the visible projection. For example, an ASOF JOIN with a subquery selecting dateadd('s', -30, timestamp) AS timestamp could have the optimizer remove the shifted timestamp from the output while join planning still required it, causing timestamp metadata to be lost and query compilation to fail with an AssertionError. This fix preserves and restores the hidden derived timestamp needed by the join, resolves timestamp aliases case-insensitively after pruning, and validates restored timestamp expressions.
This fix tightens the entity check in generateSelectChoose so that a select-choose model with an explicit timestamp(...) clause is no longer incorrectly elided when projection columns are renamed. Previously, queries that renamed a designated timestamp column (e.g. timestamp AS ts) inside a CTE combined with SAMPLE BY could fail with SqlException: Invalid column: ts or trigger an internal assertion error. The fix now requires the child column count to match the projection count and verifies each projected column's token and alias align with the child's column name before treating the model as an entity.
This fix corrects the bucket timestamp emitted by SAMPLE BY 1d ALIGN TO CALENDAR TIME ZONE on the DST-start day when the preceding day had no data. Previously, in AbstractNoRecordSampleByCursor.nextSamplePeriod, after adjustDst promoted tzOffset to the post-transition value, the existing compensation subtracted the whole current offset rather than the delta between the current and pre-transition offsets, causing the bucket to be back-converted with the wrong side of the transition. The fix uses the offset valid at the bucket boundary and shifts localEpoch by the delta between the cursor's current tzOffset and that boundary offset, mirroring timestamp_floor_utc. Aggregation results were already correct; only the emitted bucket-start timestamp was wrong.
This fix hardens several code paths in the PostgreSQL Wire Protocol implementation that previously trusted length and dimension fields read from the wire without validation. In the pre-authentication handshake, PGCleartextPasswordAuthenticator accepted any signed int as msgLen, allowing negative values to drive pointer arithmetic outside the receive buffer; lengths are now required to sit between the protocol minimum and the receive buffer capacity. DefaultPGCircuitBreakerRegistry.cancel had an off-by-one comparison and read registry state outside the spin lock; the cancel path now runs under the lock and rejects the -1 sentinel. In the post-authentication extended query protocol, array dimension multiplication in PGNonNullBinaryArrayView and PGNonNullVarcharArrayView could wrap flatViewLength via signed integer overflow; these now use Math.multiplyExact, and PGPipelineEntry validates valueSize, dimension counts, and dimension sizes before each field read.
This fix addresses window functions nested inside arithmetic expressions (e.g. avg(x) - avg(x) OVER ()) which silently produced wrong results when combined with GROUP BY. The existing check in SqlOptimiser.rewriteSelect0 only inspected status flags on top-level window columns and missed window functions buried inside operators, function calls, or CASE branches. A new findWindowFunctionOutsideAggregatePos walker traverses each SELECT column's AST and returns the position of the first window function not nested inside an aggregate, skipping aggregate subtrees to keep shapes like max(avg(x) OVER (...)) legal while rejecting unrepresentable windows mixed with GROUP BY. An additional findInvalidAggregateOverWindowPos guard rejects wrapping an aggregate-over-window in operators or other terms with a clear error pointing users to use a sub-query. The walker also handles pure window function names used without OVER (e.g. row_number()) that previously fell through to a cryptic runtime error.
This fix resolves queries like max(avg(x) OVER (...)) GROUP BY category — a window function used as the argument of an aggregate combined with explicit GROUP BY — which previously failed with a confusing Invalid column error. The rewrite pipeline inserts an inner window model between groupByModel and translatingModel whenever an aggregate argument contains a nested window function, but the existing pass-through logic only propagated columns from the top-level window model, leaving the inner window model without the GROUP BY keys or base columns referenced from aggregate arguments. The fix adds a dedicated pass after inner window models are populated, walking groupByModel's columns and later inner window models' OVER clauses, and propagates referenced literals through the chain with an owner-index filter. Coverage includes simple literal keys, positional references, expression keys like GROUP BY upper(cat), multiple keys, multiple aggregates with distinct nested windows, and PARTITION BY on columns outside the GROUP BY set.
The WalPurgeJob could process a table token snapshot after the table had been renamed and the original name reused, resulting in a stale-token exception that crashed server-main. This fix treats the resulting exception as a retryable purge race and leaves the current registry state for the next purge pass.
The parser previously misinterpreted an AND inside a sub-expression as BETWEEN's AND operator, causing queries such as SELECT 1 BETWEEN (1 and 2) and 3 or SELECT 1 BETWEEN ARRAY[0 AND 1] AND 2 to fail. This fix tracks the scope depth when BETWEEN is encountered and uses the current scope depth to determine whether a subsequent AND belongs to BETWEEN or to the sub-expression. The same approach generalises the previous special-case handling for BETWEEN-CASE.
A JIT-compiled filter using a UUID bind variable alongside any other bind variable produced wrong results — usually zero rows, sometimes spurious matches — because the JIT addressed bind-variable slots at an 8-byte stride while the Java side wrote 16 bytes for a UUID. With a UUID slot followed by another bind, the second 8 bytes of the UUID shadowed the next bind, which then decoded garbage. The bind-variable area now uses a fixed 16-byte stride end-to-end: UUID slots are unchanged, and non-UUID slots get an 8-byte zero pad. Filters with a single UUID bind variable were already correctly addressed and continue to work.
This fix addresses several independent SQL planner issues. An ON predicate where both sides reference the slave table of a LEFT JOIN or ASOF JOIN (for example y.a = y.b) was silently discarded; the predicate is now routed to the join's outer-join expression clause. The boolean NOT optimiser skipped the UNION branch when a model contained both a union branch and a nested model, so NOT (a > b) was only rewritten on one side of a UNION ALL; it now descends into both. The SqlParser tableNamePositions and tableNames fields were promoted from static to instance fields so concurrent parser instances do not share mutable state. A potential double-free in SqlCodeGenerator.compileFilter was also removed.
Valid window queries using EMA, VWEMA, or KSUM could fail when the planner chose cached window execution. These functions already worked in streaming execution, but some query shapes require cached execution, for example when an incremental window function is used together with another window expression. In those cases, users could hit an execution failure even though the SQL was valid.
This fix corrects tables().table_write_amp_p50/p90/p99/max and tables().wal_dedup_row_count_since_start which could report values much larger than the per-commit log line for the same workload, sometimes by several orders of magnitude, when a WAL apply job interleaved data writes with non-data transactions. Two TableWriter counters (physicallyWrittenRowsSinceLastCommit and dedupRowsRemovedSinceLastCommit) were only reset on certain branches, so iterations taking non-resetting branches re-read the previous iteration's value when accumulating physical row counts and dedup counts. The fix resets both counters at the start of every processWalCommit call so each iteration's reads only see that iteration's writes, regardless of branch. As a side effect, the wal_apply_physically_written_rows Prometheus counter no longer over-counts on skip/no-op iterations, and sys.telemetry_wal.physicalRowCount is no longer attributed to skipped transactions.
After an Execute suspends a portal, its cursor is intentionally retained so the next Execute on the same portal can resume from the same row. However, when the client sent a Sync between that suspended Execute and a follow-up Bind for the same statement, the existing pre-lookup guard in msgBind no longer fired, and the subsequent lookup re-introduced the suspended entry with its cursor still alive. As a result, msgExecuteSelect skipped acquiring a fresh cursor and returned a partial result set. This fix adds a post-lookup guard in msgBind that always closes any suspended cursor on the entry being rebound before starting a fresh execution, matching PostgreSQL protocol semantics. Close, Describe, and Execute on unrelated entities continue to preserve suspended cursors.
When ARG_MAX() or ARG_MIN() was applied to a TIMESTAMP_NS column, the returned value was rendered with microsecond precision, producing dates thousands of years in the future (e.g. 54977-04-25T06:29:47.654321Z instead of 2023-01-03T12:34:56.987654321Z). The underlying long was a nanosecond value but the function reported its result type as TIMESTAMP (microseconds), causing the formatter to scale it incorrectly. This fix updates the six affected ARG_MAX/ARG_MIN variants to derive the timestamp type from the input column via ColumnType.getTimestampType(), mirroring the existing pattern used by MAX() and FIRST(). The internal storage layout is unchanged; only the reported column type and resulting display formatting differ. There is no behavioural change for TIMESTAMP (micros) inputs.
This fix resolves a rare live lock where the WAL apply thread could spin indefinitely in WalTxnDetails.readObservableTxnMeta when reading transaction metadata. The issue occurred in an edge-case commit pattern where transactions were added very quickly, causing the metadata read loop to always find new entries and never complete.
When TableSequencerImpl opens _wal_index.d and encounters ENOENT, it tries to translate the error to a dropped-table exception so callers can handle drop races gracefully. The previous translation used engine.isTableDropped(tableToken), which only returned true during the drop-to-purge window. Once WalPurgeJob swept the reverse-map entry, the original ENOENT propagated as a CRITICAL-level exception, causing concurrent inserts racing a DROP TABLE to fail unexpectedly. This fix switches the translation predicate to check whether the table token lookup by directory name returns null, covering both stages of drop in a single condition. The semantics for other callers remain unchanged.
The fast symbol-keyed path in WindowJoinFastRecordCursorFactory blindly cast the master cursor's getSymbolTable() result to StaticSymbolTable. When the master factory wrapped the symbol through a SymbolColumn (e.g. a CTE or sub-select that projected the symbol through a VirtualRecord), the cursor returned the SymbolColumn function itself, which is not a StaticSymbolTable, causing the join to fail with a ClassCastException. This fix unwraps the projected SymbolFunction down to its underlying StaticSymbolTable and tightens the plan-time gate in SqlCodeGenerator so the symbol-keyed fast path is selected only when both columns expose a static symbol table.
The JSON query endpoints emitted CHAR values as a raw char between two ASCII quotes with no escaping. When a CHAR row contained " (0x22), the response carried three literal quotes; for \ (0x5C) the closing quote was escaped away, leaving the JSON object unterminated; for any C0 control byte the raw byte was inlined verbatim. In every case strict JSON parsers failed mid-response, so any client running a SELECT over a table with such values would see a parser error instead of data. This fix ensures CHAR values are now escaped using the same rules as string values: backslash-prefix for " and \, the short \b/\f/\n/\r/\t form for the usual control chars, \uXXXX for the rest, and pass-through UTF-8 for everything from U+0020 upward.
This fix bumps @questdb/sql-parser so autocomplete now picks function categories from the cursor's grammar context instead of dumping the full function list at every identifier position, and implicit select statement suggestions are included in the statement start. It also fixes a hanging editor when there is a trailing comma in the select list. For example, typing t| now suggests trades, tables, wal_tables, etc.; SELECT * FROM trades ASOF JOIN m| now returns 3 relevant entries instead of 332 mixed results; SELECT * FROM trades WHERE price = c| now returns scalars only instead of mixing scalars and aggregates; and INSERT INTO trades VALUES (n| now returns relevant suggestions like now, now_ns, nullif instead of nothing. Manual resolutions for lodash and lodash-es have been added to resolve vulnerability scan issues.
This fix corrects a user-visible typo in the Web Console error message, changing 'An error occured, please try again' to use the correct spelling 'occurred'.

April 13, 2026

QuestDB 9.3.5 introduces lateral joins, SQL-standard UNNEST, statistical window functions, and corrects SAMPLE BY timezone handling during DST transitions. It also delivers multiple join performance improvements and important Parquet export fixes.

Breaking Changes

This fix introduces a new timestamp_floor_utc function that floors timestamps in local time and converts back to UTC internally, replacing the previous approach of wrapping the query in an extra to_utc() conversion model. For sub-day strides, it uses the standard (non-DST) timezone offset to keep bucket widths uniform in UTC space, avoiding ambiguity during fall-back transitions. DST fall-back (clocks go back) now produces two rows instead of one for the repeated local hour — previously both passes through the repeated hour were merged into a single bucket. Output timestamps can be non-monotonic in local time during fall-back, though the underlying UTC timestamps remain monotonic. Sub-day bucket boundaries are now uniform in UTC rather than in local time. These changes do not affect queries without TIME ZONE, queries with fixed-offset timezones, or super-day strides (day, week, month, year). The fix also corrects an offset-in-floor-anchor bug where DST-aware and fixed-offset paths produced different bucket assignments, fixes the materialized view refresh iterator for sub-day DST timezones with offset, and resolves a native memory leak in FillRangeRecordCursorFactory.

New Features

Users can now declare bloom filters as part of column metadata via the existing PARQUET() clause in CREATE TABLE and ALTER TABLE statements. Previously, bloom filter columns had to be specified each time a partition was converted to parquet. The PARQUET() clause accepts an optional trailing BLOOM_FILTER keyword, which can be used as the sole argument, combined with encoding, or with both encoding and compression (e.g., PARQUET(DELTA_BINARY_PACKED, ZSTD(3), BLOOM_FILTER)). The bloom filter flag is stored in bit 25 of the existing 32-bit parquetEncodingConfig field, requiring no on-disk format change. When convertPartitionNativeToParquet runs without an explicit bloom_filter_columns override, the TableWriter scans per-column metadata for the bloom filter flag automatically. An explicit bloom_filter_columns in CONVERT PARTITION WITH(...) still overrides metadata flags. SHOW CREATE TABLE renders the flag in lowercase (bloom_filter). SET PARQUET(...) replaces the entire parquet config for the column, so users must re-specify BLOOM_FILTER when changing only the encoding. FPP (false positive probability) remains a global setting (partition.encoder.parquet.bloom.filter.fpp) and is not configurable per-column. The vendored parquet-format-safe crate is patched to include bloom_filter_length on ColumnMetaData, improving compatibility with readers that rely on this field to locate bloom filter boundaries.
This feature introduces a full suite of statistical aggregate window functions. STDDEV_POP(), STDDEV_SAMP(), and STDDEV() compute standard deviation, while VAR_POP(), VAR_SAMP(), and VARIANCE() compute variance, reusing the standard deviation base via an isSqrt flag. COVAR_POP(), COVAR_SAMP(), and CORR() provide bivariate statistical analysis through a new bivariate abstract base class. All functions support all frame modes including ROWS, RANGE, PARTITION BY, and unbounded/bounded frames, as well as two-pass whole-partition variants. Non-removable frames use Welford's online algorithm for numerical stability. STDDEV is an alias for STDDEV_SAMP(), and VARIANCE is an alias for VAR_SAMP(). A fix to FunctionParser was also included to correctly resolve group-by vs window factory when an argument requires implicit cast.
This feature introduces a new server property cairo.metadata.cache.snapshot.ordered (default: false) that, when enabled, causes all table-listing functions to return rows sorted alphabetically by table name. The sort is maintained incrementally inside a new CharSequenceObjSortedHashMap data structure backed by a CharSequenceSortedList, which inserts at the correct sorted position using binary search. This means lookup remains O(1) via hash and ordered iteration is O(n), with no post-query sort step required. A new CharSequenceObjMap interface allows callers to hold either the sorted or unsorted implementation transparently. The feature applies to SHOW TABLES, all_tables(), and related table-listing functions, and correctly handles drop-and-recreate as well as rename scenarios.
Tick expressions like [2024-01, 2024-02]T09:30@America/New_York#workday;6h29m now work without requiring explicit day ranges ([2024-01-[01..31]]). The compiler detects month-level and year-level date elements with time override suffixes and expands them to individual days before applying the time. This also works with bracket ranges (2024-[01..02]T09:30), time list brackets (T[09:00,14:00]), bare expressions without brackets (2024-01T09:30), year-level dates ([2024]T09:30), and duration plus day filter combinations ([2024-01]#workday;6h29m). Month validation was added before getDaysPerMonth to prevent ArrayIndexOutOfBoundsException on invalid month values. A parseMonthLevelDate helper was extracted to deduplicate YYYY-MM parsing and validation. Heap allocation of long[] dayStarts was eliminated by compacting day starts to the front of the existing LongList and reusing positions after them for per-day parsing.
This feature introduces UNNEST as a FROM-clause operator supporting all typed array column types, multiple arrays with NULL padding, and WITH ORDINALITY. For JSON arrays, a COLUMNS(name TYPE, ...) syntax declares output columns from VARCHAR-stored JSON arrays. Supported JSON column types include DOUBLE, LONG, INT, SHORT, BOOLEAN, VARCHAR, and TIMESTAMP. UnnestRecordCursorFactory wraps the base factory and emits one output row per array or JSON element. The UnnestSource interface abstracts the difference between typed arrays (ArrayUnnestSource) and JSON arrays (JsonUnnestSource), keeping cursor and record logic shared. JSON field extraction uses simdjson via JSON Pointer queries, and a native truncated flag detects values exceeding the 4KB extraction limit, throwing an error instead of silently truncating. Example usage: SELECT u.price, u.name FROM events e, UNNEST(e.payload COLUMNS(price DOUBLE, name VARCHAR)) u. Typed array and JSON sources can be mixed in the same UNNEST call.
This feature adds lateral join support, allowing subqueries in the FROM clause to reference columns from preceding tables. Correlated lateral subqueries are decorrelated at the optimizer into standard joins, enabling set-based execution instead of per-row nested-loop evaluation. The decorrelation technique is based on the Neumann and Kemper "Unnesting Arbitrary Queries" approach. The rewriter (LateralJoinRewriter) runs three passes during SQL optimization: correlation analysis that tags literals referencing outer tables, decorrelation that builds deduplicated outer-reference subqueries and rewrites correlated references, and an elimination pass that attempts to remove outer-reference join models when all correlations resolve to equalities. Operator-specific compensation preserves semantics for GROUP BY, SAMPLE BY, window functions, DISTINCT, LATEST BY, LIMIT, and set operations. Supported syntax includes JOIN LATERAL for inner lateral joins, LEFT JOIN LATERAL for left lateral joins with NULL fill, and standalone LATERAL for implicit cross joins. Example: SELECT o.id, t.total FROM orders o JOIN LATERAL (SELECT sum(qty) AS total FROM trades WHERE order_id = o.id) t.
ALTER TABLE ADD COLUMN and ALTER TABLE DROP COLUMN now work correctly on tables whose partitions have been converted to Parquet format. Previously, schema changes on Parquet-stored partitions could produce corrupted files or query errors because the Parquet column layout was assumed to match the current table schema. Added columns read as all-NULL at read time, while dropped columns' data remains in the file until an O3 merge rewrites the affected row groups. During O3 merge, rewritten row groups reflect the current schema: added columns get all-NULL chunks, and dropped columns are omitted. Bitmap indexes are rebuilt when converting between native and Parquet formats if the index files are missing or stale. The WAL writer correctly marks new symbol columns as nullable when uncommitted rows exist at the time of ADD COLUMN.
This feature extends HORIZON JOIN to accept multiple right-hand-side (slave) tables in a single query, enabling users to aggregate columns from several time-series sources against a common master table and offset grid in one statement. Both single-threaded and parallel (page-frame-based) execution paths are supported for multi-slave HORIZON JOIN, including keyed (ON symbol) and non-keyed (timestamp-only ASOF) variants, as well as mixed keyed/non-keyed slaves within the same query. The last HORIZON JOIN in the chain carries the RANGE/LIST and AS clauses; preceding HORIZON JOIN clauses omit them. For example: SELECT avg(b.bid) AS avg_bid, avg(a.ask) AS avg_ask FROM trades AS t HORIZON JOIN bids AS b ON (t.sym = b.sym) HORIZON JOIN asks AS a ON (t.sym = a.sym) LIST (-2s, 0, 2s) AS h GROUP BY h.offset. Internal refactoring replaces LongList offsets with long[] to avoid on-heap allocations on the hot path, and replaces the always-forward scan strategy with an adaptive backward/forward approach controlled by configurable thresholds (cairo.sql.horizon.join.bwd.scan.* properties).
This feature adds RLE dictionary decoding for STRING columns in the Parquet reader by wiring the existing BaseVarDictDecoder, RleDictionarySlicer, and StringColumnSink together in decode_byte_array_dispatch. The default writer encoding for VARCHAR columns has been changed from RleDictionary to DeltaLengthByteArray, aligning it with STRING and Binary defaults.
This feature introduces the arg_max(varchar, key) aggregate function for key types: timestamp, double, long, and int. The function returns the varchar value corresponding to the row where the key column reaches its maximum. Each variant supports parallel group-by execution (Async Group By) with per-worker function cloning and pointer-based merge, using StableAwareUtf8StringHolder for efficient off-heap varchar storage that avoids copying on every new max. NULL key rows are skipped and do not affect the result, while a NULL varchar value at the max key is correctly returned as NULL.

Improvements

This improvement extends the convertSymbolJoinKeysToInt optimization, previously used only by AsOf/LT joins, to hash joins. Static SYMBOL-to-SYMBOL key pairs are now compared as integers rather than strings, with a SymbolTranslatingRecord wrapping the build-side record to translate build-side symbol IDs into the probe-side encoding. Translation runs only on the build side, so the probe side stays on a straight getInt() path with zero added overhead. All six hash join variants (inner/left/right/full outer × light/full-fat) are covered.
Queries on wide tables that reference only a subset of columns now open partitions faster. TableReader tracks which columns the current query needs via a BitSet and skips memory-mapping inactive columns when opening partitions, reducing mmap/munmap syscall overhead proportionally to the number of unreferenced columns. For a query touching 1 column out of 100, roughly 99% of mmap calls are avoided per partition open. The active column set is deduplicated and falls back to mapping all columns when the list is null, empty, or covers every column. When the active set broadens, already-open partitions map any newly-needed columns. On pool return, goPassive() clears the active column state so subsequent opens map all columns. When no active columns are set (Parquet export, direct callers), the reader maps all columns as before.
This improvement eliminates eager opening of all partitions at query start for ASOF JOIN, HORIZON JOIN, and WINDOW JOIN queries on large slave tables. Time frame cursors now pre-compute exact page frame boundaries from table metadata (column tops, row counts, partition formats) without opening partitions. Actual partition opening happens lazily on first access via ensurePartitionOpened(), so queries targeting a narrow time range skip I/O for untouched partitions. Both single-threaded (TimeFrameCursorImpl) and concurrent (ConcurrentTimeFrameState + ConcurrentTimeFrameCursorImpl) cursors follow a two-phase approach: an upfront phase that pre-computes page frame boundaries from partition metadata stored as UninitializedPageFrame entries, and a lazy phase that opens partitions and patches zero-address entries with real mmap addresses. Fallback paths that bypass lazy opening include partitions already open in the table reader, cursors with interval filters, and Parquet partitions. The concurrent path uses double-checked locking with AtomicIntegerArray for safe partition opening across worker cursors. Queries accessing a subset of partitions avoid opening irrelevant partitions entirely, with zero-GC on the hot path.
Previously, calculateInsertTransactionBlock() excluded the last loaded INSERT from the transaction block, forcing it to be processed alone. This improvement includes it in the block, allowing consecutive INSERT operations to be committed together via processWalCommitBlock() instead of one-by-one via processWalCommit(). The previous behavior caused unnecessary LAG usage on empty tables (e.g., post-TRUNCATE with few rows), creating artificial 0-row partitions that could race with backup compression. The root cause was a backward loop that assigned FORCE_FULL_COMMIT to the last loaded INSERT, and the block calculation loop broke before incrementing blockSize, excluding the transaction. The fix increments blockSize before the break, which is safe because the backward loop propagates FORCE_FULL_COMMIT from structural changes (ALTER, TRUNCATE) to the INSERT before them, so the block loop never reaches a non-data transaction.
ASOF JOIN and LT JOIN factories with multi-key symbol columns now compare join keys as integers instead of converting symbol IDs to strings. SymbolTranslatingRecord translates master symbol IDs to slave symbol IDs via a cached IntIntHashMap, enabling integer-based map lookups and memeq() comparisons instead of variable-length string hashing and comparison. This optimization applies to AsOfJoinFastRecordCursorFactory, AsOfJoinDenseRecordCursorFactory, AsOfJoinLightRecordCursorFactory, FilteredAsOfJoinFastRecordCursorFactory, and LtJoinLightRecordCursorFactory. Single-symbol joins retain their existing specialized paths. SymbolTranslatingRecord gains a hadNonExistentKey() flag set during getInt() calls, eliminating a separate hasNonExistentKey() pre-scan. Benchmarks on a 100M master / 50M slave row dataset show multi-key ASOF JOIN improving from 8.11s to 5.2s.
This improvement speeds up constant-index element access on 1D and 2D double array columns (e.g. arr[1], arr[3], arr[2,1]) by bypassing full ArrayView construction. The hot path now reads directly from AUX/data pages via Unsafe, skipping IntList operations, stride computation, and BorrowedFlatArrayView setup that BorrowedArray.of() performs per row. A new Record.getArrayDouble1d2d() method with an optimized override in PageFrameMemoryRecord takes two zero-based indices and dispatches internally between the 1D and 2D paths with a single Unsafe.getDouble return. DoubleArrayAccessFunctionFactory detects constant positive indices on 1D/2D column functions at compile time and routes through the fast path. Negative indices and 3D+ arrays fall through to the existing getArray path. Benchmarks on a 10M-row table show approximately 2.4x speedup for queries like SELECT sum(arr[1]) FROM t.
This improvement adds a table selector to the table details drawer header when there is no table details history, reusing the same selector as the metrics table selector. It also includes styling updates for the metrics header (table name, title, actions) and layout updates to the metrics dashboard.

Bug Fixes

After DROP COLUMN, a column's reader index (dense position in the live column list) diverges from its writer index (permanent ID stored as field_id in parquet files). Several code paths confused the two, producing type mismatch errors, corrupt bitmap indexes, or suspended WAL tables. This fix addresses five distinct issues: bitmap index rebuild using the wrong parquet column during checkpoint/backup recovery; a doubled parquet file path that silently skipped bitmap index rebuilds; a race condition where parallel parquet bitmap rebuild tasks read stale metadata when recovering multiple tables; COPY TO parquet export flagging the wrong column as designated timestamp after DROP COLUMN; and ALTER TABLE ALTER COLUMN TYPE crashing on tables with parquet partitions by converting parquet partitions back to native before the column type conversion starts.
This fix allows signed duration segments when parsing and compiling tick expressions (e.g., timestamp in '2026-01;-3d'). Intervals are now normalized when durations move backwards so that static and compiled paths produce matching results. The compiled path in emitSingleVar previously produced inverted intervals for non-day-level variables like $now;-1h. Additionally, addYears day-of-month clamping was corrected in Micros, Nanos, and Dates so that Feb 29 + 1y now correctly produces Feb 28 instead of Mar 1. Calendar drift in repeating month/year intervals is fixed by computing each interval from the base timestamp instead of iterating. Non-positive counts in repeating interval syntax are now rejected with a clear error message.
The projection self-reference check in doReplaceLiteral0() only inspected the first join model, missing columns from other join models. This caused "Invalid column" errors when a function wraps a column from a joined table (e.g., coalesce(c, 0) where c comes from a secondary join model) and table-prefixed columns make the translating model non-redundant. This fix replaces the single-model check with columnNotExistsInJoinModels() that scans all join models.
This fix addresses three related issues. First, CopyExportRequestTask incorrectly treated var-size columns as fully NULL when their data was entirely inlined into the aux vector, because the colTop heuristic checked the data page address (which is legitimately 0 for inlined varchars). It now checks the aux page address for var-size columns. Second, the streaming Parquet reader now supports multiple dictionary pages per column chunk by allocating a fresh buffer per varchar-slice dict page, preventing aux entries decoded against earlier dict pages from being corrupted when later ones are decompressed. Third, the ASCII flag is now normalized when serializing VARCHAR keys in SingleRecordSink and OrderedMap, so two equal varchars whose ASCII-flag provenance differs hash and compare as the same key. Without this normalization, GROUP BY, DISTINCT, and hash-join operations could treat visually identical values as distinct groups.
This fix corrects a bug in convertSymbolJoinKeysToInt() where writeSymbolAsString bits were unset while iterating over join key columns. When master and slave symbol columns had crossing indices (e.g., master.symbol1=col2, slave.symbol2=col2), the first pair's unset operation cleared the bits needed by the second pair, leaving keyTypes as STRING while the RecordSink generated getInt() code. This type mismatch produced garbled map keys and missed matches.
When DirectLongLongHashMap.restoreInitialCapacity() or rehash() encountered an allocation failure (OOM or RSS memory limit exceeded), the map was left in an inconsistent state where capacity > 0 but ptr = 0. A subsequent FastGroupByAllocator._close() call would iterate over the capacity and dereference the null pointer, causing a SIGSEGV. This fix moves capacity, mask, and ptr updates to after the allocation succeeds, so a failed malloc leaves the map in its prior consistent state.
The ALTER TABLE ALTER COLUMN TYPE statement now supports converting between DECIMAL and VARCHAR/STRING types. The type conversion validation matrix in SqlCompilerImpl was updated to include these entries. Four new conversion methods handle the data transformation: DECIMAL to VARCHAR/STRING reads each value through a loader, sets the scale on a thread-local Decimal256, and writes the string representation; VARCHAR/STRING to DECIMAL parses each string through Decimal256.ofString() and stores the result. NULL values round-trip correctly in both directions. All six DECIMAL storage sizes (DECIMAL8 through DECIMAL256) are supported for both conversion directions. The fix also corrects VARCHAR aux-vector sizing in the shared fixed-to-VARCHAR conversion path.
This fix resolves an issue where clients using the PostgreSQL extended query protocol (such as postgres.js) to iterate cursors in batches via unnamed portals would receive a "spurious execute message" error on the second Execute. The root cause was that after a Flush, the suspended cursor was freed for unnamed portals, the factory was moved into the cache, and the pipeline entry was released to the object pool, leaving pipelineCurrentEntry null when the next Execute arrived. The fix retains suspended unnamed portal pipeline entries across Flush/Sync so the next Execute can resume iteration, adds a stateSuspended flag to track portal suspension state accurately, and frees suspended cursors on abandonment (new Parse/Bind) or explicit Close. Named portals (used by JDBC's setFetchSize) are unaffected as they already survived via the namedPortals map.
Go's JSON marshaller omits the fractional part entirely when a time value has zero sub-second precision, producing values like 2026-03-31T09:02:28Z instead of 2026-03-31T09:02:28.000000000Z. This broke QuestDB timestamp patterns such as .U+ and .N+, which previously required a dot and at least one digit. This fix normalizes the fractional component into optional fraction opcodes in the micros and nanos timestamp compilers so .U+ and .N+ accept either fractional digits or no fractional part at all, while still rejecting a bare dot. Both the generic and ASM parser paths share the relaxed behavior.
This fix resolves an issue where copyOrRebuildColumnIndexes() skipped columns only when colTop == -1, but rebuildPartitionIndexFiles() also skips when colTop >= partitionRowCount (column has no data). When a symbol column was added after a Parquet partition existed, O3 merge set colTop == partitionRowCount. The Parquet-to-native conversion correctly skipped building index files, but the subsequent native-to-Parquet conversion tried to hard-link the non-existent .k/.v files, suspending the table. The skip condition in copyOrRebuildColumnIndexes() was aligned with rebuildPartitionIndexFiles() by also skipping when colTop >= partitionRowCount.
When a backup checkpoint was in progress, the backup read partition data and column files from the live database directory. If partition directories or column version files were deleted while the checkpoint was snapshotting metadata, the backup failed because it tried to read files that no longer existed. This fix adds checkpoint guards to all synchronous and asynchronous file removal paths to prevent deletion while a checkpoint is active. TableWriter.processPartitionRemoveCandidates0 now checks isInProgress() and defers old partition directory removal to async purge when a checkpoint is active, while rollback orphans bypass the guard since they are not in any committed snapshot. TableWriter.finishColumnPurge forces async-only column version purge during checkpoint to prevent synchronous deletion of .d/.i files. The same checkpoint guard is applied for column purge triggered by ALTER COLUMN CONVERT and UPDATE operations. The async purge job (O3PartitionPurgeJob and ColumnPurgeOperator) also skips file deletion during checkpoint, closing the race where the purge job runs before the checkpoint pins a table's scoreboard.
This fix resolves an issue where CopyExportFactory.getCursor() enqueues export work onto a ring queue that a worker thread picks up, but BaseParquetExporter.of() constructed the worker's SqlExecutionContext with a null BindVariableService. This caused a "bind variable service is not provided" error when bind variables were used in the COPY subquery (e.g. COPY (SELECT * FROM t WHERE ts <= $1) TO ...). The fix threads the caller's BindVariableService through CopyExportRequestTask so the worker's execution context receives it. Bind variable values are deep-copied via BindVariableServiceImpl.snapshot() before enqueueing the task, since the PostgreSQL Wire Protocol thread may clear or repopulate the BindVariableService for the next query before the worker starts. The HTTP export path keeps the direct reference since it runs synchronously on the same thread.
This fix resolves a NullPointerException that occurred when WalPurgeJob attempted to fetch metadata for a table that was concurrently dropped. A concurrent drop could close the metadata pool tenant's txFile during refresh, causing a NullPointerException rather than a CairoException. The fix updates WalPurgeJob.fetchSequencerPairs() to catch Throwable (not just CairoException) when getTableMetadata() fails, and similarly updates AbstractMultiTenantPool.get0() in the newTenant path so the pool slot is always released on failure.
This fix corrects an off-by-one error in O3PartitionJob.mergeRowGroup() that could produce row groups exceeding the 1.5x maxRowGroupSize limit by 1 row. The numChunks formula used 3 * maxRowGroupSize as the divisor, representing 2 * (1.5 * maxRowGroupSize) in real arithmetic. When maxRowGroupSize is odd, integer truncation causes a mismatch: 3 * 551 = 1653 vs 2 * (551 + 551/2) = 2 * 826 = 1652. This difference meant that for certain merge totals, the formula computed fewer chunks than needed, and the even-distribution logic gave the first chunk more rows than the 1.5x limit allowed. The fix computes numChunks from the integer-exact maxChunkTarget (maxRowGroupSize + maxRowGroupSize / 2), ensuring no chunk exceeds the 1.5x bound after remainder distribution.
The validateWindowJoins function only recursed into nestedModel and unionModel, missing WINDOW JOIN definitions inside join model subqueries. This left WindowJoinContext uninitialized (lo/hi not computed), causing aggregates to return error results when a WINDOW JOIN appeared under another join, such as inside a subquery used as a JOIN source.
The propagateTopDownColumns0() method emits join-condition columns to each join model, but the expression nodes carried the outer query's table aliases. When the join model was a view, those aliases could not be resolved within the view's own scope, so addTopDownColumn silently dropped them. The optimizer then pruned the column from the view's output, causing an InvalidColumnException at code-generation time when the join tried to look up the missing column. This fix switches to using plain column names without table-alias prefixes, which can be resolved in the join model's alias map.
This fix replaces header click-to-insert with per-column copy buttons and tracks grid selection state to disable "move column to front" when nothing is selected. It resolves Monaco context menu being clipped by overflow containers, consolidates DDL queries to strip redundant blank lines, and adds truncation for long DDL in the table details drawer. Format shortcuts are rebound to Alt+Shift+F / Alt+F, toolbar and result bar spacing is tightened, long materialized view DDLs are truncated, and the LiteEditor icon now switches based on diff vs regular mode.

March 26, 2026

This release introduces new SQL capabilities including the COPY PERMISSIONS statement for cloning access profiles between principals, per-column Parquet encoding and compression controls, and dynamic window ranges in WINDOW JOIN queries. Performance improvements span faster ORDER BY on TCA queries, expanded vectorized GROUP BY coverage, and improved Parquet I/O throughput. The release also addresses several critical bug fixes including data corruption in DECIMAL columns, JVM crashes during backup coordination, and WAL replication race conditions.

New Features

This feature introduces three new permission constants: CONVERT PARTITION TO PARQUET, CONVERT PARTITION TO NATIVE, and SET PARQUET ENCODING. Authorization checks are wired into all enterprise SecurityContext implementations, including EntSecurityContextBase, AdminSecurityContext, and AbstractReplicaSecurityContext. The PermissionParser was updated to handle multi-word permission names containing SQL keywords (such as TO in CONVERT PARTITION TO PARQUET) by adding isPermissionPrefix() to disambiguate keywords from permission name continuations. All three operations are denied on replica security contexts. Previously, these ALTER TABLE operations piggybacked on authorizeAlterTableAlterColumnType and could not be granted or revoked independently.

Bug Fixes

The WAL uploader could enter an infinite retry loop when it encountered a segment whose upload.pending marker file was never created. This occurred when WalWriter.openNewSegment() advanced minSegmentLocked even when createSegmentDir() threw due to a transient disk error. When the distressed WalWriter was later closed, a segmentClosed event fired for a segment that never had its upload.pending file created, causing the Rust uploader to retry deleting the non-existent file indefinitely. This fix makes clear_segment_pending_file in the Rust uploader treat ENOENT as success, since the desired end state — upload.pending absent — is already achieved. This is safe with respect to the WAL cleaner because phantom segments are harmless: object stores return success when deleting non-existent keys, and the cleaner's compaction watermark has gap protection that prevents it from skipping real segments.
In QuestDB Enterprise, exporting to Parquet via COPY could produce spurious "cancelled by user" errors. The SecurityCheckFactory was inserted as a RecordCursorFactory wrapper for all SELECT queries, even though it only mattered for UPDATE queries where it revalidated column-level permissions. This broke an assumption that the outermost factory for a SELECT is always QueryProgress, causing the code to fail to unwrap the factory chain correctly. QueryProgress then interacted badly with the COPY job, triggering the spurious errors. This fix moves authorization revalidation directly into UpdateOperation, eliminating the need for the SecurityCheckFactory wrapper.
After a server restart, collect_missed_segments walks WAL directories to find segments with upload.pending files left over from a previous run. Because the walk is processed asynchronously, it could encounter WAL directories created after the bounce and queue them as "closed" with a stale last_txn set to the sequencer's startup transaction. This caused the upload.pending clearing loop to remove the file prematurely, after which WalPurgeJob deleted the unprotected segment. The uploader, still needing the segment, entered an infinite retry loop and the table's replication became permanently stuck. This fix passes current_wal_id into collect_missed_segments and skips any segment where wal_id > current_wal_id. Since current_wal_id is read from _wal_index.d before ingestion starts, it reliably represents the last pre-bounce WAL ID. Post-bounce segments are handled through the normal SegmentClosed event flow and do not need recovery via collect_missed_segments.
Without this fix, revoking a user's ALTER or UPDATE permission did not take effect for already-cached prepared statements in the PostgreSQL Wire Protocol, allowing the user to continue executing those operations until the connection was closed. Previously, ALTER statements were not cached before the PostgreSQL Wire Protocol layer rewrite, so authorizing them at compile time only was sufficient. Since the PostgreSQL Wire Protocol layer can now cache these commands, this fix ensures that authorization happens at execution time as well.

QuestDB 9.3.4 delivers dynamic windows in WINDOW JOIN, Parquet row group pruning with bloom filters, new array functions, and significant performance improvements across ORDER BY, joins, and Parquet I/O.

Breaking Changes

This change improves query execution performance by folding constant sub-expressions at compile time rather than re-evaluating them on every row. For example, x < '2026-01-01'::timestamp previously kept the cast as an unevaluated function that executed per row, but now evaluates it once at compile time. Additionally, constant reassociation regroups constants of the same associative operator into a single subtree (e.g., col + 1 + 4 becomes col + 5). As a breaking change, float and double Infinity and -Infinity values are now treated as NULL at compile time. Previously, expressions like cast('Infinity' as float) remained unevaluated at runtime and returned Float.POSITIVE_INFINITY, but with constant folding these are now evaluated through FloatConstant.newInstance() / DoubleConstant.newInstance(), which collapses Infinity, -Infinity, and NaN to NULL per QuestDB's convention. CASE expressions can no longer branch on Infinity or -Infinity as distinct float/double values.
This change aligns STRING and SYMBOL ordering with the SQL standard byte-order collation and enables range filter pushdown for string columns in Parquet row group pruning. Any code relying on the previous UTF-16 code unit comparison order for string comparisons may observe different ordering results.
This change improves varchar decoding performance from parquet files by introducing a new internal column type: varchar slice. VarcharSlice aux entries store (length, pointer) pairs pointing directly into mmapped Parquet pages or per-page decompression buffers, eliminating byte copies on the read path. The default encoding for Varchar has been changed from Delta Length Byte Array to RLE Dictionary. Benchmarks show decode-only performance improving from ~4.25ms to ~0.75ms for short strings (8 bytes) at 500K rows with cardinality 256.

New Features

This feature adds minTimestamp and maxTimestamp TIMESTAMP columns to sys.telemetry_wal to capture the data timestamp range per WAL transaction event. WAL telemetry is now enabled by default regardless of the main telemetry setting, reducing the dependency on logs to investigate data writing shape. WalWriter commit log messages have been downgraded from info to debug level unless the commit has a replace range. Schema migration support was also added: when the column count mismatches the expected schema, the table is dropped and recreated, which is safe given the 1-week TTL.
This feature introduces array_sort(DOUBLE[]) and array_reverse(DOUBLE[]) scalar functions that operate on double arrays of any dimensionality. array_sort sorts each innermost-dimension slice independently, preserving the array's shape, and accepts optional boolean arguments for descending order and nulls-first placement. array_reverse reverses each innermost-dimension slice. Both functions handle NULL arrays, empty arrays, NaN values, and multidimensional inputs, and support both contiguous unit-stride and non-vanilla array layouts via separate code paths. The internal sort buffer grows on demand and stays at peak size for the cursor's lifetime to avoid allocation churn on the hot path.
This feature introduces four new SQL functions that operate element-wise across DOUBLE[] arrays. Each function works in two modes: variadic (two or more array arguments, per-row) and aggregate (single array column, GROUP BY / SAMPLE BY). The functions support full N-dimensional arrays with automatic shape broadcasting, where the output shape is the per-dimension maximum of all inputs. NULL arrays and NaN elements are skipped, and positions that receive no finite values yield null. array_elem_sum() and array_elem_avg() use Kahan compensated summation for floating-point accuracy. The GROUP BY average variant uses a uniform/variable dual-mode count tracker to avoid per-element count allocation in the common case. Parallel GROUP BY is supported via merge().
Queries on Parquet partitions can now skip entire row groups that contain no matching rows. Row group pruning uses three strategies: min/max statistics (row groups whose per-column value range does not overlap the filter values are skipped), bloom filters (row groups whose bloom filter reports no match are skipped, opt-in via the bloom_filter_columns option), and null count statistics (for IS NULL and IS NOT NULL filters). Pruning applies to all Parquet read paths including forward scan, backward scan, read_parquet(), and parallel page frame execution. Supported filter operations include equality, IN list, comparison operators (<, <=, >, >=), BETWEEN, IS NULL, IS NOT NULL, and OR-connected equalities on the same column. Supported column types include BYTE, SHORT, CHAR, INT, LONG, FLOAT, DOUBLE, TIMESTAMP, DATE, IPv4, UUID, LONG128, STRING, SYMBOL, VARCHAR, and all DECIMAL widths. Bloom filter columns and false positive probability (FPP) can be specified via ALTER TABLE ... CONVERT PARTITION TO PARQUET WITH (bloom_filter_columns = 'col1,col2', fpp = 0.01), COPY TO export, or the /exp HTTP endpoint. New configuration properties include cairo.sql.parquet.row.group.pruning.enabled (default true), cairo.partition.encoder.parquet.bloom.filter.fpp (default 0.01), cairo.parquet.export.bloom.filter.fpp (default 0.01), and cairo.parquet.export.statistics.enabled (default true).
This feature adds arg_max() and arg_min() function variants that accept a CHAR type for the value argument. Supported signatures include arg_max(char, timestamp), arg_max(char, long), arg_max(char, double), and the corresponding arg_min() variants. Null keys are ignored during aggregation, while null values are returned when the corresponding key is the max/min. All variants support parallel execution. Example usage: SELECT arg_max(status, created_at), arg_min(status, created_at) FROM events;
This feature allows WINDOW JOIN to accept column references and expressions as RANGE BETWEEN boundaries, in addition to static constants, enabling each left-hand-side row to define its own window size based on its data. For example: SELECT t.ts, sum(d.val) AS agg FROM fx_trades t WINDOW JOIN market_data d RANGE BETWEEN t.lookback minutes PRECEDING AND t.lookahead minutes FOLLOWING. Either or both of the lo/hi boundaries can be dynamic while the other remains a static constant. Boundary expressions must evaluate to an integer and must only reference left-hand-side table columns. Negative values are clamped to zero (equivalent to CURRENT ROW), and NULL values produce NULL aggregates where the row is skipped. Each boundary can optionally include a time unit suffix (seconds, minutes, hours, etc.), and when present the value is scaled to the table's timestamp resolution at runtime. Dynamic windows disable the fast symbol-keyed and vectorized execution paths; queries with an ON key equality clause fall back to the general path with a join filter instead. This feature also makes parallel HORIZON JOIN and WINDOW JOIN queries more responsive to query cancellation by adding circuit breaker checks at the top of every master-row iteration loop.
O3 commits into Parquet partitions previously replaced row groups in-place, leaving orphaned bytes in the file. Repeated merges caused file sizes to grow to 2-3x their useful data. This improvement adds a rewrite mode that periodically writes all data to a fresh file, eliminating dead space. An unused_bytes counter tracked in Parquet metadata drives the rewrite decision. Rewrite triggers when the file has a single row group, when unused_bytes / file_size exceeds a configurable ratio (default 0.5), or when absolute unused bytes exceeds a threshold (default 1 GB). In rewrite mode, untouched row groups are raw-copied with adjusted thrift offsets without decode/re-encode. A new O3ParquetMergeStrategy class computes merge/copy/split actions up-front using min/max timestamp overlap detection, replacing the old iterative merge loop. Row groups that exceed the configured size get split into multiple output groups. Small row groups (< 4096 rows) adjacent to a gap absorb the gap's O3 data to avoid proliferation of tiny row groups. WAL tables can now convert their last (active) partition to Parquet, with the TableWriter routing all WAL data through the O3 merge path. New configuration properties: cairo.partition.encoder.parquet.o3.rewrite.unused.ratio (default 0.5) and cairo.partition.encoder.parquet.o3.rewrite.unused.max.bytes (default 1g).
Users can now specify Parquet encoding and compression on a per-column basis using CREATE TABLE and ALTER TABLE SQL syntax. The syntax is PARQUET(encoding [, compression[(level)]]), where both encoding and compression are optional — use default for the encoding when specifying compression only. When omitted entirely, the column uses the global defaults. For example: CREATE TABLE sensors (ts TIMESTAMP, temperature DOUBLE PARQUET(rle_dictionary, zstd(3)), device_id VARCHAR PARQUET(default, lz4_raw)) TIMESTAMP(ts) PARTITION BY DAY;. Existing tables can be modified with ALTER TABLE sensors ALTER COLUMN temperature SET PARQUET(rle_dictionary, zstd(3)); or reset with ALTER TABLE sensors ALTER COLUMN temperature DROP PARQUET;. Per-column config appears in SHOW CREATE TABLE output. Supported encodings include plain, rle_dictionary, delta_length_byte_array, and delta_binary_packed, with type-specific restrictions. Supported compression codecs include uncompressed, snappy, gzip (0-9), brotli (0-11), zstd (1-22), and lz4_raw. RLE dictionary encoding is now supported for all column types except Boolean and Array. A new cairo.partition.encoder.parquet.min.compression.ratio configuration property (default 1.2) controls whether compressed pages are worth keeping — when a compressed column chunk fails to meet the ratio threshold, the encoder discards the compressed output and stores it uncompressed instead. Varchar dictionary encoding performance was significantly improved by switching from the default hasher to RapidHashMap and storing indices directly in a Vec<u32>, achieving ~87 Melem/s throughput compared to the original ~13 Melem/s.

Improvements

WalWriter previously hardcoded POSIX_MADV_RANDOM for memory-mapped column files, which hurts most workloads. This improvement makes the madvise hint configurable via the cairo.wal.writer.madvise.mode property with valid values: none (default, no hint), sequential, and random. The random mode is beneficial when ingesting into many tables with many columns, as it prevents the OS from speculatively reading adjacent pages under memory pressure.
This improvement replaces linear scanning with binary search for initial frame positioning in ASOF JOIN, LT JOIN, and WINDOW JOIN operations. The openSlaveFrame() method in AbstractAsOfJoinFastRecordCursor and the findRowLo()/findRowLoWithPrevailing() methods in WindowJoinTimeFrameHelper now call seekEstimate() on the first lookup to binary-search directly to the target partition instead of linearly scanning all preceding frames. This reduces the initial positioning cost from O(N) in the number of frames to O(log P) where P is the number of partitions, mirroring the optimization already present in HORIZON JOIN.
This improvement brings significant performance gains to Parquet decoding, with a median improvement of approximately 87% across column type and encoding combinations. Key optimizations include removing unnecessary allocation and zeroing in the decompression path, skipping definition decoding when null_count=0, improved Plain, DeltaBinaryPacked, Rle, and RleDictionary decoder implementations, a lookup table for boolean unpacking, specialized UUID byte swapping, batch decoding for DeltaBinaryPacked, batched nullable bitmap processing, and bulk copying for PlainPrimitiveDecoder. This improvement also fixes a bug with incorrect null sentinel values.
Vectorized execution is now applied to more non-keyed GROUP BY queries, such as SELECT first(price), last(price) FROM trades. Internally, the GroupByFunction#computeBatch() API is used for all queries without a filter. This API was previously used only in WINDOW JOIN.
This improvement introduces an encoded sort path for ORDER BY queries. SortKeyEncoder encodes column values into fixed-width, order-preserving binary keys (8/16/24/32 bytes), enabling comparisons via native uint64 operations instead of per-column type dispatch. The native sort is a three-layer hybrid: vergesort for detecting natural runs (O(n) for pre-sorted time-series data), MSD radix sort (American Flag Sort) with parallel bucket sorting across available cores via atomic work-stealing for large arrays, and pdqsort for small partitions with heapsort fallback guaranteeing O(n log n) worst-case. The entire native sort path is allocation-free with all data structures stack-allocated. It replaces the R-B tree (RecordTreeChain) with flat array collect plus radix sort for the non-random-access path, and guarantees stable sort by falling back to rowId ordering when all sort key columns are equal. Key size is capped at 32 bytes; ORDER BY clauses exceeding this fall back to the existing R-B tree sort. Benchmarks show a 5-column ORDER BY over 3M rows improving from 6.98s to 1.26s, and a 2-column ORDER BY over 1B rows improving from over 15 minutes to 63.6s with parallel sorting.
This improvement replaces string comparison (getSym() + Chars.compare) with rank-based comparison (getInt() + rank map lookup) when sorting SYMBOL columns that have static symbol tables. At cursor open, a rank map maps each symbol key to its alphabetical rank, and the comparator then compares ranks instead of strings. It also eliminates the temporary DirectIntList used when building symbol rank maps by replacing it with in-place permutation inversion after quicksort, reducing extra space from O(N) to O(1) where N is the symbol count, mainly benefiting high-cardinality symbol columns. This optimization applies to all comparator-based sort paths: tree-based sort cursors, top-K cursors, window function tree comparators, and window function internal comparators (rank(), percent_rank()). Non-static symbol columns (e.g. CAST(str AS SYMBOL)) fall back to the original string comparison path.
This improvement introduces an adaptive scan strategy for HORIZON JOIN's keyed ASOF lookup. The lookup now starts in backward-only mode, which is cheap for low-cardinality key spaces, and adaptively switches to forward scan mode within a frame when backward scan cost becomes excessive — for example, with high-cardinality symbols or rare/infrequent keys that cause deep backward scans. The switch uses two criteria: a relative threshold where backward scan cost at a position must exceed 8x the gap to trigger (with a minimum gap of 1,024 rows), and an absolute threshold of 131,072 rows to handle cross-partition boundaries where the relative check cannot trigger. Once switched, the frame stays in forward mode for its remainder. This also re-enables the ASOF row cache in HorizonJoinTimeFrameHelper and resets bookmarks on toTop() to eliminate jitter from out-of-order frame processing. Performance improvements range from 13-24% for dense and equity scenarios to 96% for sparse data distributions.

Bug Fixes

A LATEST BY ALL query on a large table could fill an OrderedMap until keyCapacity reached 2^30. When rehash() doubled to newKeyCapacity = 1L << 31, the overflow guard allowed it through because MAX_SAFE_INT_POW_2 was incorrectly set to 1L << 31. The subsequent (int) cast produced Integer.MIN_VALUE, and clear() computed ~18.4 EB for native memset, causing a SIGSEGV. This fix corrects Numbers.MAX_SAFE_INT_POW_2 from 1L << 31 to 1L << 30, so the guard rejects the overflow and throws a clean CairoException("map capacity overflow") instead of crashing the JVM. The constant was also deduplicated from Unordered4Map, Unordered8Map, and UnorderedVarcharMap, each of which had a private copy with the same bug.
The cairo.partition.encoder.parquet.statistics.enabled configuration allows users to disable Parquet statistics, but the read path (ParquetTimestampFinder, TableWriter) and the O3 merge path (O3PartitionJob.processParquetPartition) hard-depended on timestamp column statistics. When statistics were absent, getMinValueLong would hit an assertion crash with -ea or read garbage memory causing silent data corruption without -ea. This fix adds rowGroupMinTimestamp and rowGroupMaxTimestamp methods to PartitionDecoder that try Parquet column statistics first at zero cost, then fall back to decoding the first/last row from actual data pages when statistics are absent. findRowGroupByTimestamp also falls back to decoding instead of reading garbage memory. O3PartitionJob, ParquetTimestampFinder, and TableWriter have been migrated to use the new methods.
This fix resolves a crash that occurred when using SAMPLE BY with FILL(NULL), FILL(value), or FILL(PREV) in queries containing array column aggregates such as last(arr). The fill record types did not implement getArray(), and the null/value fill factories did not handle array column types when constructing fill constants. The fix adds getArray() overrides to SampleByFillRecord and FillRangeRecordCursorFactory.FillRangeRecord, and updates SampleByFillNullRecordCursorFactory and SampleByFillValueRecordCursorFactory to handle array types by yielding NullConstant.NULL, since arrays cannot be filled with scalar values.
When a JOIN query had a WHERE clause containing both a column-referencing condition and a non-column constant expression (e.g., NOW() = NOW()), QuestDB crashed with an internal AssertionError. The SQL optimizer splits the WHERE clause into separate buckets — one for column-referencing conditions and one for constant expressions. The code generator then applied each bucket as a separate FilteredRecordCursorFactory wrapper, but nesting these factories violated an internal assertion. This fix detects when the base factory is already a FilteredRecordCursorFactory, extracts the existing filter, and combines both filters with AND into a single factory instead of nesting them.
ALTER TABLE ... ALTER COLUMN and ALTER MATERIALIZED VIEW ... ALTER COLUMN failed when the column name was quoted (e.g., "MY_COL"). The token from the lexer was used as-is for the metadata lookup, so the quoted form was looked up instead of the unquoted name, resulting in a "column does not exist" error. This fix wraps the token with unquote() before the column name lookup in both compileAlterTable and compileAlterMatView.
The non-vectorized fast cursor (WindowJoinWithPrevailingFastRecordCursor) compared the first window match against the window end instead of the window start, causing the prevailing row to effectively never be included when the window already contained matches. The vectorized variant already used the correct comparison. This fix aligns the non-vectorized path to compare against the window start timestamp. The bug only affected queries using non-vectorizable aggregates (e.g., max(concat(...))) with symbol-keyed WINDOW JOIN and INCLUDE PREVAILING.
The read_parquet() function crashed with SIGSEGV when reading parquet files containing SYMBOL columns encoded by QuestDB's PartitionEncoder. The canProjectMetadata() method passed the actual column type (SYMBOL) to the Rust decoder instead of the expected type (VARCHAR), causing the Rust decoder to write INT32 symbol keys that Java then read as VARCHAR pointers. This fix passes the expected type (VARCHAR) for SYMBOL-to-VARCHAR conversions so the Rust decoder resolves dictionary entries to UTF-8 strings.
This fix addresses two bugs in WINDOW JOIN with INCLUDE PREVAILING. The first bug affected four sync cursor variants that call findRowLo(lo, hi, true): WindowJoinWithPrevailingAndJoinFilterRecordCursor, WindowJoinWithPrevailingAndJoinFilterFastRecordCursor, WindowJoinWithPrevailingFastRecordCursor, and WindowJoinFastVectRecordCursor. When the bookmarked-frame optimization kicked in and the prevailing candidate sat in a prior partition, the bookmarked path skipped initialization of prevailingFrameIndex/prevailingRowIndex, leaving them at -1/Long.MIN_VALUE. Downstream backward scans saw -1 and returned immediately, silently omitting the prevailing row from results. The fix seeds prevailingFrameIndex/prevailingRowIndex when entering via the bookmark path. The second bug was a comparison target error in WindowJoinWithPrevailingFastRecordCursor where the condition that triggers prevailing inclusion checked whether the first row's timestamp exceeded the window high boundary (masterTimestampHi) instead of the window low boundary (slaveTimestampLo).
This fix addresses several correctness, resource leak, and crash issues in the SQL engine. A missing break after the DECIMAL inner switch in generateCastFunctions caused fall-through to the BINARY case, adding a spurious BinColumn to the cast function list and shifting subsequent column indices in multi-column UNION queries, resulting in UnsupportedOperationException at runtime. The moveClauses method never incremented its position counter, so when swapJoinOrder0 needed to move multiple join clauses, only the first was moved — the rest stayed in the wrong join context, creating circular dependencies that broke topological sort on 3+ table cross joins with multi-column WHERE conditions. The hasGroupByFunc/hasOrderedGroupByFunc methods skipped children of non-aggregate functions, making nested aggregates like abs(sum(x)) invisible and causing PIVOT to reject valid aggregate expressions. Resource leaks were fixed in generateJoins where const filters were leaked on error paths, and in generateSampleBy where timezoneNameFunc, offsetFunc, sampleFromFunc, and sampleToFunc were never freed on error paths. A NullPointerException in CREATE MATERIALIZED VIEW was fixed — when WITH BASE specifies a nonexistent table not referenced in the query, getTableTokenIfExists() returned null and .isView() threw NPE instead of a descriptive SqlException. Additional latent bugs were fixed in generateFill (always reading the first fill value instead of iterating) and in GeoHash-to-VARCHAR cast functions (using wrong column type parameter).
This fix corrects a data corruption issue where set_memory_vanilla_vec() delegated to run_vec_bulk, which used TVec::size() as the loop increment. For Vec8uq this returned 8 (the number of uint64_t lanes), but one 512-bit store only covers 4 long_128bit elements or 2 long_256bit elements. The mismatch caused the bulk path to advance past unwritten elements, leaving holes filled with stale data. The fix replaces the broken run_vec_bulk call with a dedicated loop that computes the correct per-store element count from sizeof(TVec) / sizeof(T), guarded by a static_assert.
This fix defaults cairo.mat.view.parallel.sql.enabled to false on machines with fewer than 4 available processors. All other parallel SQL features (filter, group by, top-k, horizon join, window join, Parquet read) remain enabled regardless of core count. Users can still explicitly enable materialized view parallel SQL via configuration on low-core machines.
This fix resolves an issue where read_parquet failed with "encoding not supported" errors when reading Parquet files whose embedded QuestDB metadata schema had a different number of columns than the actual Parquet data. This happens when an external tool (DuckDB, Spark, PyArrow) rewrites a QuestDB-exported Parquet file — for example, dropping partition columns for hive-style directory layouts — but preserves the original key-value metadata. The decoder now compares the QuestDB metadata schema length against the Parquet column count and discards the metadata when they differ, falling back to physical type inference.

March 3, 2026

This release introduces automatic object store WAL cleanup on the primary node, TLS certificate expiration metrics for Prometheus alerting, and new array_sort() and array_reverse() functions. Performance improvements include faster ASOF and WINDOW JOINs through binary search–based frame positioning, along with a configurable WAL writer madvise mode. Several critical bug fixes address crashes in LATEST BY ALL queries, Parquet reads with missing statistics, and backup restore edge cases, while the ACL permission system has been expanded to support up to 256 permissions.

New Features

This feature introduces a WAL cleaner that runs on the primary node and automatically deletes replicated WAL data from object storage once it is no longer needed by any replica or backup. It determines what is safe to delete by consulting two sources of cleanup history — enterprise backup manifests and checkpoint history records — and always retains enough data to support the most recent N backups or checkpoints. The cleaner is conservative by default: it won't delete anything until sufficient history exists, and it picks the most conservative boundary when multiple sources or cluster nodes are involved. Key components include a checkpoint history tracker that records per-table transaction state to the shared replication object store on each CHECKPOINT RELEASE, a backup instance name registry for coordinating cleanup boundaries across multiple nodes, rate limiting and throttling for object store delete operations with auto-tuned defaults per cloud provider (S3, GCS, Azure Blob, R2, etc.), and crash recovery with periodic progress persistence so cleanup resumes where it left off after a restart. Dropped tables are cleaned up after a cooloff period (default 1h) to guard against clock skew. Key configuration properties include replication.primary.cleaner.enabled (default true), replication.primary.cleaner.interval (default 10m), replication.primary.cleaner.backup.window.count (default 5), replication.primary.cleaner.delete.concurrency (auto-tuned 4–12), replication.primary.cleaner.max.requests.per.second (service-dependent), and checkpoint.history.enabled (default true when replication is enabled).
This feature adds Prometheus gauge metrics for TLS certificate time-to-live (TTL) across all four TLS-enabled endpoints: questdb_tls_cert_ttl_seconds_http, questdb_tls_cert_ttl_seconds_http_min, questdb_tls_cert_ttl_seconds_line, and questdb_tls_cert_ttl_seconds_pg. Each gauge reports seconds until the active certificate expires. Values greater than 0 indicate seconds remaining, 0 means expired, and -1 means the certificate has not been loaded or could not be parsed. Gauges are only registered for endpoints where TLS is enabled. The TTL is computed from the certificate's notAfter field, which is extracted via a JNI call into a minimal DER/X.509 parser on the Rust side. The expiration epoch is cached and updated on reload_tls(), so the metric always reflects the active in-memory certificate, not the one on disk.
This feature adds array_sort(DOUBLE[]) and array_reverse(DOUBLE[]) scalar functions that operate on double arrays of any dimensionality. array_sort() sorts each innermost-dimension slice independently, preserving the array's shape, and accepts optional boolean arguments for descending order and nulls-first placement. array_reverse() reverses each innermost-dimension slice. Both functions handle NULL arrays, empty arrays, NaN values, and multidimensional inputs. They support both contiguous unit-stride and non-vanilla array layouts via separate code paths. The sort buffer grows on demand and stays at peak size for the cursor's lifetime to avoid allocation churn on the hot path.
This feature adds minTimestamp and maxTimestamp TIMESTAMP columns to sys.telemetry_wal to capture the data timestamp range per WAL transaction event. WAL telemetry is now enabled by default regardless of the main telemetry setting, reducing the dependency on logs to investigate data writing shape. The WalWriter commit log message has been downgraded from info to debug level unless the commit has a replace range. Schema migration support has been added so that when the column count mismatches the expected schema, the table is dropped and recreated, which is safe given the 1-week TTL.

Improvements

This improvement mirrors the optimization already present in HORIZON JOIN. Without it, the first lookup linearly scans through all slave time frames preceding the master's first timestamp, which is O(N) in the number of frames. With the seekEstimate() optimization, the initial positioning is O(log P) where P is the number of partitions. Specifically, AbstractAsOfJoinFastRecordCursor.openSlaveFrame() now calls seekEstimate() on the first slave frame lookup to binary-search directly to the target partition instead of linearly scanning all preceding frames, benefiting all ASOF JOIN and LT JOIN fast-path factories. WindowJoinTimeFrameHelper.findRowLo() and findRowLoWithPrevailing() also now call seekEstimate() on the first lookup with the same partition-skipping behavior, benefiting both sync and async WINDOW JOIN factories.
WalWriter previously hardcoded POSIX_MADV_RANDOM for memory-mapped column files, which hurts most workloads. This improvement makes the madvise hint opt-in via a new configuration property cairo.wal.writer.madvise.mode with valid values: none (default, no hint), sequential, and random. The random mode is beneficial when ingesting into many tables with many columns, as it prevents the OS from speculatively reading adjacent pages under memory pressure.

Example configuration:
```
cairo.wal.writer.madvise.mode=random
```
This improvement refactors the ACL permission system to support more than 64 permissions by migrating from 64-bit bitmasks to an exponent-based representation with 256-bit aggregate masks. Permission constants changed from long bitmasks to int exponents, and a new PermissionMask class provides 256-bit storage (4 longs) for aggregate permission sets. The permission column type in the database schema changed from long to short (storing exponents instead of bitmasks), reducing storage overhead while supporting up to 256 distinct permissions. PermissionMask.ZERO is now immutable and throws on mutation attempts, and the sentinel value handling for ALL_PERMISSIONS is properly supported across has, set, and clear operations.

Bug Fixes

This fix corrects Numbers.MAX_SAFE_INT_POW_2 from 1L << 31 to 1L << 30. The old value (2^31) does not fit in a signed 32-bit int, so the rehash overflow guard let exactly 2^31 through. The subsequent (int) cast produced Integer.MIN_VALUE, and clear() fed approximately 18 EB to native memset, causing a SIGSEGV. The crash chain occurred when a LATEST BY ALL query on a large table filled an OrderedMap until keyCapacity reached 2^30, then rehash() doubled to newKeyCapacity = 1L << 31, which truncated to a negative value and passed an enormous size to native memset. The fix makes the guard reject newKeyCapacity = 2^31, throwing a clean CairoException("map capacity overflow") instead of crashing the JVM. The constant was also deduplicated from Unordered4Map, Unordered8Map, and UnorderedVarcharMap, each of which had a private copy with the same bug.
The cairo.partition.encoder.parquet.statistics.enabled configuration allows users to disable Parquet statistics, but the read path (ParquetTimestampFinder, TableWriter) and the O3 merge path (O3PartitionJob.processParquetPartition) hard-depended on timestamp column statistics. When statistics were absent, getMinValueLong would hit an assertion crash with -ea or read garbage memory causing silent data corruption without -ea. This fix removes that hard dependency by adding rowGroupMinTimestamp and rowGroupMaxTimestamp methods to PartitionDecoder that try Parquet column statistics first at zero cost, then fall back to decoding the first/last row from actual data pages when statistics are absent. The findRowGroupByTimestamp method also falls back to decoding instead of reading garbage memory, and O3PartitionJob, ParquetTimestampFinder, and TableWriter have been migrated to use the new methods.
During backup, partitions with row_count=0 may not produce meta.msgpack. The restore process previously always attempted to download this file when hash verification was enabled, causing restores to fail with "no partition metadata found" for empty partitions. This fix skips downloading meta.msgpack for empty partitions during restore and skips hash verification in that case, while still requiring metadata for non-empty partitions.
A primary instance restored from a backup could encounter a race condition between the WalPurgeJob and dropped table request processing, where the uploader could not open the _txnlog because it had already been deleted. This fix ensures that the WalPurgeJob does not delete state that the uploader still needs. The bug was unique to backups since backups do not restore .pending files used to control this workflow. At start-up, the adjusted WalUploader's replication logic now ensures that the appropriate .pending file is recreated when missing, which also patches instances that were already restored with older versions of QuestDB Enterprise.
This fix introduces three new ACL permissions: ALTER SYMBOL CAPACITY (column-level), SET REFRESH LIMIT (table-level), and SET REFRESH TYPE (table-level). ALTER SYMBOL CAPACITY replaces the incorrect reuse of ALTER COLUMN TYPE for symbol capacity changes. A startup migration automatically grants ALTER SYMBOL CAPACITY to every entity that previously had ALTER COLUMN TYPE, at the same scope (database, table, or column level), preserving grant options. SET REFRESH LIMIT and SET REFRESH TYPE gate the previously unprotected ALTER MATERIALIZED VIEW ... SET REFRESH LIMIT/IMMEDIATE/MANUAL/EVERY/PERIOD operations. This fix also wires in the previously commented-out authorization check for ALTER TABLE SET PARAM and adds a retry loop with recompile in access list reloading to handle table reference out-of-date exceptions during ACL reload.

February 25, 2026

QuestDB 9.3.3 is a feature-rich release introducing HORIZON JOIN for markout analysis, a new twap() aggregate, SQL-standard WINDOW definitions, JIT compilation on ARM64, and file-based secrets for Kubernetes deployments. It also brings significant performance improvements across Parquet I/O, parallel GROUP BY, UNION queries, and ORDER BY on computed expressions.

New Features

Sensitive configuration options can now be loaded from files using the _FILE suffix convention, enabling seamless integration with Kubernetes Secrets, Docker Secrets, HashiCorp Vault, and other file-based secrets management systems. For example, setting QDB_PG_PASSWORD_FILE=/run/secrets/pg_password as an environment variable or pg.password.file=/run/secrets/pg_password in server.conf will load the password from the specified file. This feature works with all sensitive properties (pg.password, pg.readonly.password, http.password). File contents are automatically trimmed of whitespace. SHOW PARAMETERS displays value_source = 'file' for secrets loaded from files, and secrets are reloaded when file contents change via SELECT reload_config().
This feature allows underscores as number separators in TICK date expressions (e.g., $now-10_000T) and duration suffixes (e.g., ;1_500T), consistent with Numbers.parseInt() which already supports this. Error position reporting in date expression evaluation has also been improved to point at the specific offending location instead of the start of the expression. Invalid underscore placement (leading, trailing, consecutive) is properly detected and reported.
This feature enables QuestDB's JIT code generation to run natively on ARM64 systems in addition to x86. The changes introduce ARM64-specific implementations for register handling, caches, and code generation, while maintaining full compatibility with the existing x86 logic. Conditional compilation is used throughout to select the appropriate architecture at build time. ARM64-specific versions of Function and CountOnlyFunction structs were implemented, including methods for code generation, register setup, and loop logic that mirror the x86 implementations but use ARM64 instructions and register types.
This feature introduces the WINDOW clause, allowing users to define named window specifications that can be reused across multiple window functions within a single query. Window inheritance following the SQL standard is also supported, where a named window can reference another named window as its base, inheriting PARTITION BY, ORDER BY, and frame clauses. Chained inheritance is supported (e.g., w3 references w2, which references w1). Merge rules follow the SQL standard: PARTITION BY is always inherited from the base (child cannot specify its own), ORDER BY in the child takes precedence if specified, and frame clauses in the child take precedence if non-default. Validation ensures base windows must be defined earlier in the WINDOW clause, PARTITION BY in a child window referencing a base is rejected, and circular and self-references are prevented.
This feature introduces the twap(price, timestamp) aggregate function that computes the time-weighted average price using step-function integration, where each price is held constant until the next observation and the TWAP is the area under this step function divided by the total time span. The function supports parallel GROUP BY execution via per-worker native buffers that are merge-sorted during the merge phase, since workers process non-adjacent page frames via work-stealing and an incremental weighted-sum approach would incorrectly bridge gaps between frames. It falls back to a simple arithmetic mean when all observations share the same timestamp. SAMPLE BY with FILL modes is also supported.
This feature adds support for all six decimal types (DECIMAL8, DECIMAL16, DECIMAL32, DECIMAL64, DECIMAL128, and DECIMAL256) in the Parquet format. Write support stores decimals as fixed-length byte arrays in big-endian format per the Parquet specification. Read support includes a WordSwapDecimalColumnSink for DECIMAL128 and DECIMAL256 that correctly converts from Parquet's big-endian format by reversing each 8-byte word independently. The supported Parquet physical types are INT32 for small decimals (DECIMAL8/16/32), INT64 for DECIMAL64, and FixedLenByteArray for all decimal types.
This feature introduces array_build(nDims, size, filler1, ...) that creates DOUBLE[] or DOUBLE[][] arrays with controlled shape and fill values. The size parameter accepts a scalar integer or a DOUBLE[] (using its cardinality). Each filler can be a scalar (repeated for all elements) or a DOUBLE[] (copied element-by-element, NaN-padded if shorter, truncated if longer).
This feature introduces HORIZON JOIN, a specialized time-series join designed for markout analysis — a common financial analytics pattern where you need to analyze how prices or metrics evolve at specific time offsets relative to events such as trades or orders. For each row in the left-hand table and each offset in the horizon, the join computes left_timestamp + offset and performs an ASOF match against the right-hand table. When join keys are provided via ON, only right-hand rows matching the keys are considered. The horizon is defined using either a RANGE FROM <from> TO <to> STEP <step> clause for uniform offsets, or a LIST (<offset>, ...) clause for explicit non-uniform offsets, both aliased with AS. The pseudo-table exposes .offset (LONG) and .timestamp (TIMESTAMP) columns. Both RANGE and LIST use the same interval expression syntax as SAMPLE BY with supported units: U (microseconds), T (milliseconds), s (seconds), m (minutes), h (hours), d (days). Use cases include markout P&L analysis, event impact studies, and time-series correlation at different lags. Current limitations include no combination with other joins at the same query level, no right-hand side WHERE filters, both tables requiring a designated timestamp, positive STEP with FROM ≤ TO for RANGE, and monotonically increasing offsets for LIST.
```
SELECT h.offset / 1000000 AS horizon_sec, t.sym, avg(m.mid) AS avg_mid
FROM trades AS t
HORIZON JOIN mid_prices AS m ON (t.sym = m.sym)
RANGE FROM 1s TO 60s STEP 1s AS h
ORDER BY t.sym, horizon_sec
```
This feature adds a Table Details Drawer to the Web Console for tracking the health and details of a table or materialized view. It simplifies the table schema explanation flow by removing flow-specific schema and adding streaming support, along with a new flow for table health issues. An adaptive polling mechanism checks request latency proactively and sets the interval dynamically. WAL tables polling has been removed except for the suspension dialog. The feature also includes monitoring support in the AI Assistant docs, updated tooltip behavior and styling throughout the application, query truncation support for LiteEditor, history and navigation among right-hand-side bar drawers, and a mechanism to copy table and column names using Ctrl/Cmd+C.
This feature adds import and export support for editor tabs, makes tabs scrollable with an increased minimum width, adds a rename button that appears on tab hover, and updates the editor tab styling.

Improvements

This improvement introduces three optimizations for Parquet partition reads. Late materialization decodes only filter columns first to identify matching rows, then decodes remaining columns only for those rows, significantly reducing decoding overhead for low-selectivity queries. Zero-copy mmap page reading uses a new SlicePageReader that reads Parquet pages directly from the memory-mapped byte slice, bypassing the previous approach of copying page data into an intermediate buffer. Raw array encoding is now enabled by default for partition-to-Parquet conversion, avoiding the overhead of Parquet's nested LIST decoding. In benchmarks, an OHLC aggregation query on an 8-day Parquet partition improved from 600ms to 250ms.
This improvement speeds up Parquet export through two optimizations. SIMD-accelerated encoding uses portable SIMD intrinsics for nullable INT/LONG columns, processing 64 values per iteration for vectorized definition level encoding, yielding up to 16.5% faster throughput and 20% less CPU usage with LZ4RAW compression. A new streaming mode for TableReader applies MADV_SEQUENTIAL on mmap for aggressive read-ahead prefetching and MADV_DONTNEED before munmap for immediate page cache release. Streaming mode bypasses the mmap cache so each partition mapping is independent and fully releasable. Parquet exporters automatically enable streaming mode. Under memory pressure, the combination of both madvise hints recovers 94% of baseline performance (387 MB/s vs 82 MB/s without hints).
This improvement pushes designated-timestamp filters from outside a UNION / UNION ALL / EXCEPT / INTERSECT subquery into each branch of the set operation, enabling per-branch partition pruning on time-series tables. The optimizer only pushes filters that exclusively reference the designated timestamp column, deliberately excluding non-timestamp filters to avoid type mismatches and column resolution issues across branches with different schemas. The outer filter always stays on the wrapper model, so correctness is unchanged. For composite filters like WHERE ts IN '2025-12-01T01;2h' AND x > 5, the optimizer splits the top-level AND into separate conjuncts and processes each independently: it pushes the timestamp conjunct into branches, while x > 5 stays at the parent level only. Safety mechanisms include per-branch semantic guards that skip branches with LATEST BY, LIMIT, or non-pushable SAMPLE BY, column existence checks, and deep cloning to prevent cross-branch mutation.
This improvement adds an unordered mode to PageFrameSequence that uses SOUnboundedCountDownLatch instead of ordered collection, eliminating head-of-line blocking for reducers that don't need ordered results. Keyed GROUP BY, non-keyed GROUP BY, and Top K factories now use this unordered mode, replacing boilerplate ordered collect loops with a single dispatchAllAndAwait() call. Error field copying in storeError() was also fixed to use a dedicated CairoException instance, preventing thread-local exception recycling from corrupting stored error data. At concurrency 8, per-iteration spread (a direct proxy for head-of-line blocking) was reduced by 53–75% across query types, tail latency (p99) improved by 30–44%, and latency predictability (p99/p50 ratio) improved from 2.5–3.6x down to 1.4–1.8x. Throughput at concurrency 8 improved by 6–9% for most query types, and throughput now scales monotonically from concurrency 1 to 2 instead of regressing as on the previous implementation.
CASE expressions on symbol columns with static symbol tables now resolve string constants to integer symbol keys at initialization time and compare by integer at runtime, avoiding per-row string comparisons. Three picker specializations are used based on branch count: single-branch (one equality check), dual-branch (two equality comparisons), and multi-branch (integer-object hash map lookup). When a WHEN value is not found in the symbol table, the branch is silently skipped. Non-static symbol tables (e.g., from casts) fall back to the existing character sequence-keyed comparison.
This improvement replaces the Dragon4 double-to-string algorithm with Ryu (Ulf Adams, 2018), which produces minimal-length, correctly-rounded decimal representations with better performance. A direct double-to-decimal conversion path (Numbers.doubleToDecimal) bypasses string formatting entirely, removing the double→string→decimal roundtrip in CastDoubleToDecimalFunctionFactory. The dependency on jdk.internal.math.FDBigInteger has been eliminated, removing the --add-exports java.base/jdk.internal.math compiler flag.
When sorting by computed expressions (e.g., ORDER BY a + b * c), QuestDB's SortedLightRecordCursor previously re-evaluated the expression on every comparison during tree insertion and output, meaning sort key functions were called O(N log N) times during sorting plus O(N) during output. This improvement adds a sort key materialization layer that pre-computes expensive sort key values into off-heap buffers before sorting, reducing function evaluations from O(N log N) to O(N). Each function gets a complexity score that propagates through the expression tree. If any sort key column exceeds the configured complexity threshold (default 3, configurable via cairo.sql.sort.key.materialization.threshold), that column's values are pre-computed and stored in per-column memory buffers. A materializing record intercepts reads for materialized columns from the buffer and delegates to the base record for everything else. The feature supports two-pass operation where subsequent passes after toTop() look up existing ordinals without re-computing values. This optimization applies to fixed-size types only (no string/varchar/binary materialization) and operates on the general sort path only (no Top-K modification).
This improvement replaces the digit-by-digit loop in doubleToDecimal() with a single wide multiply via a new ofDigitsAndPower(long digits, int power) method on the Decimal interface. The old code extracted up to 17 individual digits from the Ryu significand via long division, then performed 17 wide additions (128-bit or 256-bit). The new code multiplies the significand by a single power of 10. Benchmarks show Decimal64 is 1.8x faster, Decimal128 is 2.2x faster, and Decimal256 is 2.8x faster. The three types converge to roughly the same cost since the single multiply dominates and the word-width difference matters less for one operation than for 17.
This improvement introduces a hybrid export mode for Parquet export (COPY ... TO and HTTP /exp) that passes through raw page-frame-backed columns zero-copy and materializes only computed columns into native buffers, row by row per frame. Queries with no page-frame backing (e.g., cross joins) now use a cursor-based export that materializes all columns row by row, also avoiding the temporary table detour. Only queries requiring re-partitioning (PARTITION BY override) or containing a computed BINARY column still use the temporary table path. The export mode is determined by inspecting the compiled RecordCursorFactory before constructing any temporary table: DIRECT_PAGE_FRAME for factories supporting PageFrameCursor directly (zero-copy), PAGE_FRAME_BACKED for virtual record cursor factories whose base supports page frames with no computed BINARY columns, CURSOR_BASED for no page-frame backing or descending ORDER BY with computed columns, and TEMP_TABLE for PARTITION BY overrides or computed BINARY columns. The HybridColumnMaterializer handles both PAGE_FRAME_BACKED and CURSOR_BASED modes, converting computed SYMBOL columns to STRING in adjusted metadata while dispatching via source type for correct record accessor usage. A buffer pool recycles computed-column native buffers rather than freeing them after each row group flush. This improvement also fixes several resource lifecycle issues on error paths including page-frame cursor leaks, temp directory leaks on moveFile() failure, and use-after-free on error path cleanup. The exporter hierarchy was refactored so that BaseParquetExporter holds shared state while HTTPSerialParquetExporter and SQLSerialParquetExporter extend it independently, removing the prior dependency where the SQL exporter extended the HTTP exporter.

Bug Fixes

When QuestDB fails to start due to configuration errors, a race condition between the JVM shutdown hook and the startup script could produce confusing cat and rm error messages for the hello.txt file. This fix redirects stderr to /dev/null for those commands and adds proper quoting around file paths for robustness with paths containing spaces. The actual startup failure details remain available in the stdout-*.txt log files.
Concurrent Parquet exports could produce corrupted or truncated files when multiple clients exported simultaneously through the same HTTP worker. The issue occurred because ExportQueryProcessor stored per-connection state in per-processor fields. When a worker parked one connection via PeerIsSlowToReadException and began serving another, these fields were overwritten, causing the onWrite() callback to write Parquet chunks to the wrong response on resume. This fix moves the affected fields to per-connection ExportQueryProcessorState.
This fix corrects greedy nanosecond parsing (single N followed by non-digit) to be consistent with milliseconds and microseconds. Previously, .SSSUUUN with input .1234567 produced .123456007Z (7ns) instead of the correct .123456700Z (700ns).
This fix improves the reliability of memory-mapped file handling for columns on Windows platforms. A fallback mechanism for mapping files on Windows has been introduced, which gracefully handles transient permission errors ("Access Denied" even after successful file open) by falling back to an anonymous memory map populated by reading the file contents directly. On non-Windows platforms, the original mapping logic is retained.
This fix addresses an issue where DynamicPropServerConfiguration.reload() accumulated stale entries in changedKeys across reload cycles when only a secret file changed while the properties file remained unchanged. This caused watchers to be spuriously notified about properties that did not actually change. The root cause was that changedKeys.clear() only lived inside updateSupportedProperties(), which is skipped when the properties file hasn't changed. The fix moves changedKeys.clear() to the top of reload(), inside the synchronized block, so every reload cycle starts fresh.
This fix resolves an issue where the ALTER TABLE t ADD COLUMN IF NOT EXISTS col <type> path did not fully resolve parameterized or compound column types before comparing with existing metadata. For DECIMAL and GEOHASH, the path used ColumnType.typeOf() which returns the base type constant, but metadata stores the fully encoded type (with precision/scale/bits). For array types (e.g., DOUBLE[]), the path did not parse array dimensionality brackets, causing type mismatch errors and unconsumed tokens. Unsupported array element types (e.g., INT[]) now correctly report "unsupported array element type" instead of falling through to a misleading error, and unmatched ] brackets after a type name are now detected with a helpful error pointing at the bracket.
This fix adds post-compilation validation in the SQL compiler to reject unconsumed tokens after a valid statement. Previously, trailing content was silently ignored, which could mask user errors. PostgreSQL Wire Protocol compatibility no-op handlers (RESET, CLOSE, UNLISTEN, DISCARD) now require their expected arguments instead of silently accepting bare keywords. BEGIN, COMMIT, and ROLLBACK now properly consume the optional TRANSACTION keyword. The SET statement syntax is now validated to conform to SET [SESSION | LOCAL] name { = | TO } value [, value]*, rejecting malformed forms such as missing name, invalid operator, missing value, and dangling commas.
This fix resolves two related bugs in SqlOptimiser.propagateTopDownColumns0() that caused AssertionError in SqlCodeGenerator.checkIfSetCastIsRequired when UNION sibling models ended up with different topDownColumns counts. The first bug occurred when WHERE clause and timestamp column literals were emitted to union model branches by name resolution; since UNION matches columns by position rather than name, the same alias could resolve to different column indices in different branches, causing one branch to receive extra top-down columns. The fix removes the name-based emission loops in favor of the existing index-based propagation. The second bug occurred when a GROUP BY or SAMPLE BY model as the first UNION member added non-aggregate key columns to its own topDownColumns without propagating them to union siblings. The fix ensures these columns are propagated to all union siblings whose topDownColumns are already populated.
This fix addresses performance degradation when exporting tables with 1M+ distinct symbols to Parquet. Previously, the large default batch size of 1M rows from the general CREATE TABLE AS SELECT setting prevented frequent batch commits, causing symbol index re-scaling to be deferred and degrading performance as the symbol table grew without capacity adjustments. A new configuration property cairo.parquet.export.batch.size with a default of 100K rows is now used specifically for Parquet exports.
When CREATE TABLE ... AS (SELECT ...) fails during data population with a non-Cairo exception, the partially created table and its name lock were not cleaned up. This also affects COPY parquet export, which uses temp tables internally. This fix adds fallback cleanup in COPY parquet export to match the existing HTTP export pattern.
This fix reads the len byte in DecimalBinaryFormatParser as unsigned (& 0xFF) instead of signed. A malformed Influx Line Protocol message with a high len byte (e.g. 0x80) was interpreted as negative, which skipped the VALUES parsing state and left unscaledValues empty. The subsequent load() call then hit an ArrayIndexOutOfBoundsException accessing index 0 of the empty list.
When a view name is quoted (single or double quotes), SqlParser passed the quoted token directly to getTableTokenIfExists(), which failed to find the view. This affected three code paths: SELECT ... FROM 'my_view', SELECT ... JOIN 'my_view' ON ..., and COMPILE VIEW 'my_view'. This fix adds unquote() to all three getTableTokenIfExists() calls so the view is recognized regardless of quoting.
When a CASE/WHEN expression contains a window function (e.g., lag() OVER (...)), column references outside the window function (in THEN/ELSE branches) were not emitted to the translating model. This caused "invalid column" errors when using the original column name alongside a SELECT alias (e.g., price AS p then THEN price). This fix introduces replaceWindowFunctionOrLiteral() that chains window function replacement with literal emission in a single tree traversal pass, so all column references are propagated through the model chain.
This fix introduces CompiledTickExpression, which pre-parses tick expressions containing date variables ($now, $today, $yesterday, $tomorrow) into a single long[] intermediate representation at compile time. Runtime evaluation performs only long arithmetic with no string parsing or allocations. A new DateVariableExpr encodes and evaluates $variable ± offset expressions supporting all time units plus business days. Expressions containing $ variables now produce a dynamic CompiledTickExpression instead of re-parsing the string on every query execution. The evaluation algorithm walks elements to emit intervals, applies day filter bitmasks in local time, handles timezone conversion (numeric subtraction or DST-aware toUTC), applies exchange schedule filtering with optional duration, and sorts and merges overlapping intervals. Additionally, Character.isWhitespace and Character.isDigit calls were replaced with Chars.isAsciiWhitespace and Chars.isAsciiDigit throughout IntervalUtils to avoid locale-dependent behavior.
This fix moves the circuit breaker check in DatabaseCheckpointAgent.checkpointCreate() from after ff.sync() to before it. The POSIX sync() system call flushes all dirty filesystem buffers system-wide, not just QuestDB's files. On busy hosts with heavy I/O from other processes, sync() can block for well over the 60-second query timeout. Previously, the circuit breaker was only checked after sync() returned, so a completed checkpoint was discarded when the timeout had been exceeded during the blocking call. With this fix, if the timeout is already exceeded from the table loop or lock acquisition, the operation fails fast without entering a potentially long-blocking sync(). If the timeout has not been exceeded, sync() runs to completion and the checkpoint succeeds regardless of how long sync() takes.
FILL(LINEAR) was disregarding FROM, meaning it would return calendar-aligned results instead of FROM-aligned results. This fix ensures FROM anchors the initial timestamp and TO applies as a bound. There is no intent for this to support fill expansion (i.e., filling before and after the dataset); it is unclear at this stage how pre- and post-filling should operate with linear fills.
Tick expressions containing date variables ($now, $today, etc.) used as filter predicates — e.g., WHERE now() IN '$now-100s..$now' — resolved $now once at compile time and cached the resulting interval as static timestamps. On repeated execution of a cached SQL statement, the interval never updated, causing the filter to return zero rows once now() moved past the originally computed range. The interval model path was already fixed to compile such expressions into IR via CompiledTickExpression, but the filter function path in InTimestampTimestampFunctionFactory was not updated and still used the static function. This fix detects date variables in constant tick expression strings and routes them through CompiledTickExpression, which re-evaluates the interval from IR on each execution via init().
This fix addresses a vulnerability where ChunkedContentParser.parseChunkLength() had no overflow guard on the hex chunk-size parsing loop. A crafted HTTP request with 16+ hex digits in the chunk size would overflow the long chunkSize to a negative value, corrupting the internal buffer pointer. On the next loop iteration, isEol() would dereference the corrupted pointer, causing the JVM to crash with SIGSEGV. This was reachable over the network with an unauthenticated POST request using Transfer-Encoding: chunked. The fix adds an overflow guard before each chunkSize * 16 accumulation step: if chunkSize > Long.MAX_VALUE >>> 4, the next multiply would overflow a positive long, so the input is rejected as a protocol violation. The server logs the violation and disconnects the client cleanly.
This fix resolves a crash where WINDOW JOIN and HORIZON JOIN would throw a NullPointerException when the right-hand side (slave) is a subquery that applies a timestamp interval filter or reorders columns. The root cause was that SelectedRecordCursorFactory and ExtraNullColumnCursorFactory did not implement newTimeFrameCursor(), which is required by the concurrent time frame cursor infrastructure used in these joins. Additionally, SelectedPageFrameCursor did not implement TablePageFrameCursor, causing a ClassCastException. After fixing those, a subtler double column remapping bug remained: both SelectedConcurrentTimeFrameCursor and SelectedPageFrameCursor were independently remapping column indices through columnCrossIndex, causing symbol table lookups to hit wrong columns and the timestamp column to be read from the wrong position in the address cache. This fix extracts ConcurrentTimeFrameCursor into an interface, adds newTimeFrameCursor() to the affected factories, and simplifies SelectedConcurrentTimeFrameCursor to delegate column remapping entirely to SelectedPageFrameCursor, avoiding double remapping.

February 4, 2026

This release introduces Kubernetes secrets file support for enhanced security in containerized deployments, adds TICK exchange calendar support for financial market applications, and delivers significant Parquet read performance improvements through advanced optimization techniques. The update also addresses several critical bug fixes including a rare WAL file error on Windows replicas, backup log message accumulation issues, and Parquet export corruption under concurrent connections.

New Features

This feature enables reading sensitive configuration values from files using the _FILE suffix convention, allowing native Kubernetes secret file mounts without requiring shell scripts or init containers. It works with both environment variables (QDB_ACL_ADMIN_PASSWORD_FILE) and properties (acl.admin.password.file). All properties marked as sensitive=true are supported, including acl.admin.password, acl.oidc.tls.keystore.password, replication.object.store, cold.storage.object.store, backup.object.store*, and authentication passwords. The _FILE variant takes precedence over direct values, with fallback to direct values when no _FILE variant exists.
This feature adds enterprise exchange calendar support, enabling TICK expressions to filter time intervals by stock exchange trading schedules using syntax like 2025-01-24#XNYS for NYSE trading hours. The implementation includes an EntExchangeCalendarService that loads trading schedules for 67 global exchanges from a bundled Parquet file, supports custom schedule overrides via a _exchange_calendars_custom table, and provides lazy-loading with concurrent access protection. It includes a reload_exchange_calendars() function for managing custom schedules and an exchange_calendars() table function for querying effective calendar data. The feature handles multi-session trading days, lunch breaks, and case-insensitive exchange codes.
This feature implements late materialization optimization for Parquet partitions in parallel query execution with selective filters. Instead of decoding all columns upfront, it first decodes only filter columns to identify matching rows, then decodes other columns only for rows that passed the filter, significantly reducing decoding overhead for low-selectivity queries. The feature introduces SlicePageReader for zero-copy memory-mapped page reading, bypassing intermediate Vec<u8> copies. Additionally, it defaults partitionEncoderParquetRawArrayEncoding to true for better performance by using raw array encoding instead of nested LIST decoding.

Bug Fixes

This fix resolves several race condition issues related to replication. Previously, WAL downloader tasks could leak when cancelled, causing conflicts with WalPurgeJob over file locks. The fix replaces JoinSet::shutdown with a CancellationToken to properly notify and wait for all tasks to complete, preventing file handle leaks on Windows. Additionally, this fix replaces the file-based lock mechanism with a centralized semaphore-based system using WALSegmentLockManager to coordinate access between WalPurgeJob, WalWriter, and the WAL downloader.
This fix addresses a race condition where QuestDB startup failures (such as invalid configuration) would cause confusing cat and rm error messages when the hello.txt file is removed by deleteOnExit() before print-hello.sh can process it. The fix redirects stderr to /dev/null for these commands and adds quotes around $HELLO_FILE for robustness with paths containing spaces. The actual startup failure details remain available in stdout-*.txt logs.
This fix resolves an issue where concurrent Parquet exports could produce corrupted or truncated files when multiple clients exported simultaneously through the same HTTP worker. The problem occurred because the ExportQueryProcessor stored per-connection state in per-processor fields, causing connection contexts to be overwritten when workers switched between connections. This resulted in Parquet chunks being written to the wrong response, producing invalid data that failed to open in downstream tools.

QuestDB Releases

QuestDB Enterprise 4.0.1Based on QuestDB OSS 10.0.1

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB Enterprise 4.0.0Based on QuestDB OSS 10.0.1

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB 10.0.1

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB 10.0.0

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.3.4

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.3.3Based on QuestDB OSS 9.4.3

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.3.2Based on QuestDB OSS 9.4.3

New Features

Improvements

Bug Fixes

QuestDB 9.4.3

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.3.1Based on QuestDB OSS 9.4.2

New Features

Bug Fixes

QuestDB 9.4.2

New Features

Bug Fixes

QuestDB Enterprise 3.3.0Based on QuestDB OSS 9.4.1

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB 9.4.1

New Features

Improvements

Bug Fixes

QuestDB 9.4.0

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.2.5

New Features

Bug Fixes

QuestDB 9.3.5

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.2.4Based on QuestDB OSS 9.3.4

New Features

Bug Fixes

QuestDB 9.3.4

Breaking Changes

New Features

Improvements

Bug Fixes

QuestDB Enterprise 3.2.3Based on QuestDB OSS 9.3.3

New Features

Improvements

Bug Fixes

QuestDB 9.3.3

New Features

Improvements

Bug Fixes

QuestDB Enterprise 4.0.1
Based on QuestDB OSS 10.0.1

QuestDB Enterprise 4.0.0
Based on QuestDB OSS 10.0.1

QuestDB Enterprise 3.3.3
Based on QuestDB OSS 9.4.3

QuestDB Enterprise 3.3.2
Based on QuestDB OSS 9.4.3

QuestDB Enterprise 3.3.1
Based on QuestDB OSS 9.4.2

QuestDB Enterprise 3.3.0
Based on QuestDB OSS 9.4.1

QuestDB Enterprise 3.2.4
Based on QuestDB OSS 9.3.4

QuestDB Enterprise 3.2.3
Based on QuestDB OSS 9.3.3

QuestDB Enterprise 3.2.2
Based on QuestDB OSS 9.3.2