QuestDB Releases

An overflowing INT expression now wraps modulo 2^32 in every context, including under a 64-bit cast and on the way into a 64-bit column, so an INT expression carries exactly one value at its declared type. To compute at 64 bits, widen an operand, for example to_utc(secs * 1_000_000L, tz) or i::long * j. Consequences of this rule: WHERE secs * 1_000_000 > <timestamp>, (i*j)::long and INSERT into a 64-bit column all read the wrapped value; inserting an overflowing INT expression into a designated timestamp now fails with designated timestamp before 1970-01-01 is not allowed; SELECT <overflowing constant INT arithmetic> returns INT rather than LONG, so over the PostgreSQL Wire Protocol the column descriptor moves from int8 to int4 and CREATE TABLE ... AS SELECT creates an INT column; INT arithmetic landing on -2^31 reads as NULL everywhere; window frames, WINDOW JOIN bounds, SAMPLE BY intervals and FROM/TO bounds, and DROP/DETACH/CONVERT PARTITION WHERE filters use the final wrapped value instead of inspecting arithmetic provenance; long_sequence(100_000 * 30_000) wraps negative and returns no rows; dateadd('u', 3600*1000000, ts) computes from the wrapped stride instead of failing overload resolution; and timestamp predicates and UPDATE filters accept INT expressions and sign-extend the wrapped value, with a WAL UPDATE whose WHERE clause fails intrinsic extraction now failing synchronously at sequencing time instead of suspending the table at apply. Constant reassociation is also disabled for numeric constant pairs, so a chain such as i + 1 + 4 evaluates left to right at its declared width, which changes EXPLAIN output and costs throughput on Java-evaluated constant arithmetic. Alongside the rule, this change fixes a large family of defects: HORIZON JOIN and WINDOW JOIN wrong results including a constant-false WINDOW JOIN reading an unrelated column on a Parquet master, undercounted past-only keyed windows, lazily bound owner group-by functions, spliced NULL column handling for DECIMAL, varchar and LONG256, and keyed join maps escaping cairo.query.memory.limit.bytes; LATEST ON silently dropping an indexed-symbol WHERE filter and pushing a LIMIT below the operator; NULLIF returning the value it had just excluded and INT/IPv4 casts evaluating their argument twice; json_extract answering different values at different widths and casts to SYMBOL closing a symbol table their caller still owned; IN over integer keys widening, leaking or mis-ordering elements; numerous JIT compiled-filter crashes, fail-open native backends and width divergences against the Java filter; an inclusive tolerance boundary shared by the compiled filter and Parquet row-group pruning; Parquet row-group pruning false-pruning numeric, non-finite and column-top rows; timestamp offset pushdown dropping rows for month and year offsets, wrapped shifts and hand-written and_offset calls; a NullPointerException from 'x' || concat('y'); and several factory-construction and interval-model resource leaks. Performance work amortizes the keyed WINDOW JOIN slave index (up to 2000 rebuilds down to 17), restores vectorization for mixed-width JIT filters, and makes the expression reassociation and JIT width-marking passes linear rather than quadratic.

This feature lets deployment controllers and embedders bound a single shutdown attempt without freeing objects that a worker, lifecycle component, or engine extension may still be using. ServerMain.closeBy(long deadlineNanos) takes an absolute deadline on the monotonic System.nanoTime() clock and returns true when shutdown completed and the close graph was released, or false when the deadline or an interrupt stopped the attempt, in which case every resource that may still be in use remains owned for a later retry with a fresh deadline; completed stages stay idempotent and close() keeps the existing terminal behaviour. Concurrent callers do not tear down in parallel — one caller owns shutdown while others wait up to their own deadlines, restoring any consumed interrupt status — and unexpected component failures still propagate because an explicit incomplete-shutdown signal is used rather than classifying arbitrary exceptions as timeouts. One deadline flows through boot completion, lifecycle pre-stop work, component stops, worker-pool halts, timer shards, engine readiness, and logging, with final resource release beginning only after those stages complete; components first stop admitting work and request cooperative cancellation, WorkerPool.haltBy(long) accepts the shared absolute deadline while haltWithin(long) remains for relative budgets, a worker pool counts down startup and halt latches for workers that start() never created so rollback cannot wait on impossible acknowledgements, and CairoEngine.isCloseReady(long) gives engine extensions a pre-free phase after worker pools stop. The deadline bounds only QuestDB's coordination and cooperative waits, so it cannot forcibly stop application or native code that ignores cancellation, and an incomplete attempt intentionally retains memory, file descriptors and native handles in exchange for memory safety; existing no-argument shutdown entry points and legacy worker-pool descriptors remain available, and deadline checks run only on startup rollback and shutdown paths.

This improvement extends the deadline-aware shutdown contract to Enterprise lifecycle code, so a single monotonic absolute deadline flows through role-switch rendezvous, role-state close, lifecycle components, worker pools, WAL transfer, the engine pre-free phase, and retryable backup teardown. A successful stage releases its ownership, while an incomplete stage returns a retryable incomplete-shutdown signal and keeps every still-live Java and native owner so a later attempt can resume with a fresh deadline. During shutdown, restore and backup cancellation are signalled before waiting for long-running work, the lifecycle orchestrator coordinates with any in-flight role switch and repeatedly requests cooperative cancellation while time remains, and the final close graph is released only after the engine reports readiness, which prevents a role switch or native task from observing engine-owned state after the engine frees it. Native backup teardown closes admission first and then uses the deadline remainder to acquire its teardown write lock and drain the Rust manager, rounding the native wait down to whole milliseconds so conversion cannot extend the caller's deadline; on timeout the native handles, Tokio runtime, backup agent and scheduler state are retained for retry, and restore cancellation is mapped to a structured RestoreCancelledException so a shutdown-requested cancellation ends boot cleanly while an unrequested one preserves the original failure. A bounded timeout can therefore leave the server stopped but still holding Java and native resources until a retry or the terminal process-exit policy reclaims them, cancellation is cooperative rather than forced, and deployment termination grace must cover role.switch.timeout plus the Enterprise teardown reserve when the shutdown hook should finish in-process.

The uploader's notified-transaction watermark was overwritten by every transaction notification instead of only being raised. Because events are not delivered in transaction order — a rebased table can drain data for transactions 1 and 2 followed by a table-created event carrying transaction 0 — a trailing lower value dragged the watermark down, so the transaction log reader was never remapped and an idle rebased table retried forever instead of settling in the uploader index. This fix only raises the watermark in the drain path, allowing the uploader to tolerate any event order while leaving the rebase ordering unchanged.

A queue segment published its frozenForEnqueues flag before bumping the tail by the freeze offset, while the empty check in tryDequeue() reads the flag before the tail. In the window between the two writes a dequeuer could pair a set flag with the pre-freeze tail and subtract the freeze offset from a tail that was never bumped, making the frozen clause always true and reporting the queue empty while it still held committed items; the full fence only ordered visibility, so the race occurred on x86 as well as ARM. This fix bumps the tail before publishing the flag, so a set flag always implies a bumped tail and a clear flag with a bumped tail simply retries, adding no additional synchronization. It also repairs the sized ConcurrentQueue constructor, which validated its size argument and then ignored it, always allocating a 32-slot first segment.

This fix addresses two symbol-map correctness issues that could produce invalid symbol mappings. Previously, SymbolMapReaderImpl.updateSymbolCount() removed the wrong cache range when the visible symbol count shrank and left maxOffset at the previous high-water mark, so keyOf() could return a key outside the current symbol count; the symbol cache and maxOffset are now shrunk together with the visible symbol count. Additionally, WalTxnDetails.saveSymbols() relied on an assertion to require dense Write-Ahead Log symbol diff keys, which production builds do not enforce, allowing sparse or invalid diffs to pass through without a useful diagnostic. Symbol diff keys are now validated at runtime, and if a diff is sparse, Write-Ahead Log apply reports the relevant transaction and symbol metadata and suspends the table instead of applying a potentially inconsistent mapping, favoring containment over continuing with an invalid mapping. The validation adds a single integer comparison per new symbol diff entry on valid input and no per-row work, but a suspended table requires operator intervention.

This change stops a UNION or UNION ALL from permanently widening to STRING when every branch contributes a SYMBOL column; the result column is now SYMBOL. This is a result-schema change for that query shape: HTTP /exec reports a JSON column type of SYMBOL instead of STRING, and CREATE TABLE AS derives a SYMBOL column where it previously created a STRING one. The PostgreSQL Wire Protocol continues to report the VARCHAR OID for both types, so its wire type is unchanged. Mixed SYMBOL and STRING union columns still resolve to STRING, and a pending all-symbol union segment is restored before a following EXCEPT or INTERSECT so that flat and parenthesized forms stay consistent. Internally the metadata is preserved through a specialized projection that keeps text reads streaming, initializes integer-key dictionaries lazily, translates static native symbol keys per source, falls back to text for dynamic symbol sources, and releases query-tracked native state at cursor close. No new statement or permission is introduced, and existing view authorization remains the access boundary.

This fix makes a QuestDB Enterprise instance fail to start when http.user, http.password or http.password.file is present, whether supplied through server.conf or the QDB_HTTP_USER, QDB_HTTP_PASSWORD and QDB_HTTP_PASSWORD_FILE environment variables. These are the open source basic-auth settings and QuestDB Enterprise has never read them, so an instance configured with them was serving the REST API and the Web Console with no authentication at all while the operator believed it was password-protected; remove such settings and use access-control-list users instead. Either property alone is rejected, and the check runs ahead of the open source credential-pair validation so the real reason is reported on the first attempt. It also fixes identity reporting when the access control list is disabled: the allow-all and read-only security contexts previously discarded the authenticated principal and returned the hardcoded default, so current_user() reported admin on every instance and session_user() and SHOW CREATE ... OWNED BY reported admin on read-only instances. The isPrincipalIndependent() mechanism and its boot-time shared proxy were removed so a named principal always receives a per-connection context, while anonymous connections still share a pooled context per protocol when the access control list is disabled, keeping the HTTP request path allocation-free. The Influx Line Protocol access-control bypass is now keyed on the protocol interface identifier rather than an authentication-type constant, and owner grants use an atomic grantIfExisting operation so an unknown key identifier no longer fabricates an access list entry in the replicated access-control table. Applications using QuestDB Enterprise Java APIs directly must switch to the three-argument DispatchingSecurityContextFactory(primaryFactory, replicaFactory, initialRole) constructor, stop relying on the removed EntSecurityContextFactory.isPrincipalIndependent(), and implement grantIfExisting(CharSequence, ObjList<PermissionModel>) in external AccessListWriter implementations before rebuilding.

A UNION or UNION ALL whose every branch contributes a SYMBOL column now returns a SYMBOL column instead of permanently widening it to STRING. The set-operation cursor still uses STRING internally, because independent table symbol dictionaries cannot share native keys, and the compiler restores SYMBOL once at the boundary of each all-symbol UNION segment; a later STRING leg removes the affected column from the candidate set, so mixed SYMBOL/STRING columns remain STRING. A pending UNION segment is restored to SYMBOL before a following EXCEPT or INTERSECT, so flat and explicitly parenthesized forms expose identical metadata. Text consumers stream source text without building a dictionary, while integer-key consumers initialise translation state lazily, translating native keys per union leaf where a static source dictionary exists and interning text otherwise; query-scoped dictionaries and key translations are released when the cursor closes. As a result of this change, HTTP /exec now reports SYMBOL in its JSON column metadata and CREATE TABLE AS over a symbol-only union materialises a SYMBOL column rather than STRING, so scripts that inspect result metadata or depend on the resulting column type must be updated. PostgreSQL Wire Protocol continues to expose both types as the VARCHAR OID, so the wire type is unchanged there, and EXPLAIN retains the established plan shape.

This fix corrects several defects in the query engine's memory-bound data structures and tightens configuration validation. Signed comparisons on compressed heap offsets caused OrderedMap to misread live entries as empty slots once the map heap grew past roughly 16GB, so a parallel GROUP BY could silently drop or duplicate groups; all slot-emptiness checks now share a single unsigned predicate. Budgeted structures (OrderedMap, LongChain, AbstractRedBlackTree, LongTreeChain and LimitedSizeLongTreeChain) previously rejected a growth step outright when one doubling overshot the configured ceiling, raising memory-limit errors well below the ceiling; they now clamp to the limit instead, at the cost of a runaway query holding up to twice as much native memory before it is stopped. A cached ORDER BY ... LIMIT $1 factory no longer returns wrong rows when the bind value flips sign, and a window ORDER BY with more terms than the model's no longer raises an out-of-bounds error. Native memory lifecycle issues in the HTTP header parser's multipart boundary augmenter and in the HTTP connection, client and platform-specific client constructors were fixed so a failed allocation no longer leaks or over-frees, and overflow messages now name the owning feature (ASOF JOIN, RANK() window function, DENSE_RANK() window function) and the configuration key to raise. Oversized initial heap pages are now rejected before allocation, and hot-loop memory accesses in map probing, rehashing, shard merging and red-black tree insertion were reduced. As a compatibility change, cairo.sql.small.map.page.size and cairo.sql.join.metadata.page.size now reject values below 4, cairo.sql.hash.join.light.value.page.size rejects values below 12, cairo.sql.small.map.page.size and cairo.sql.sort.key.page.size reject values above 34,359,738,352, and cairo.sql.sort.light.value.page.size rejects values above 17,179,869,176; all of these were previously accepted at startup.

Scalar subqueries in predicate position previously followed different and sometimes unsafe compilation paths: timestamp-valued scalar subqueries on the designated timestamp could lose intrinsic state during recursive compilation and fall back from covering-index and interval scans to full filters, while scalar BOOLEAN subqueries were not consistently coercible. This change leases a dedicated WHERE clause parser per recursive generation depth, evaluates timestamp scalar subqueries once per execution through a shared frozen bound that both interval pruning and the residual filter read, preserves microsecond or nanosecond precision, maps empty results to NULL and enforces one-row cardinality, so a query such as SELECT ts, value FROM t WHERE series = 's1' AND ts >= (SELECT lo FROM bounds WHERE sel = 100) now uses the same covering-index and interval plan as equivalent constant bounds. One-column BOOLEAN scalar subqueries can participate directly in bare, AND, OR and NOT predicates, and a whole-predicate runtime constant now gates the outer scan so a statement-invariant false predicate performs no outer I/O. Several previously accepted shapes are now rejected at compile time and require rewriting: a literal negative LIMIT in a correlated lateral sub-query fails with negative LIMIT is not supported in a correlated lateral sub-query; a LIMIT naming an outer column over a lateral scalar count is refused and must use a constant or bind variable; materialized views whose sub-query reads an external source such as read_parquet() fail with non-deterministic function cannot be used in materialized view: sub-query; and an UPDATE against a WAL table may no longer reference another table or instantiate a cursor function, including through a FROM-less scalar sub-query such as (SELECT 3). Lateral scalar-count compensation is also narrowed to provably one-row bare count bodies, so counts wrapped in GROUP BY, UNION or a filtering layer now return NULL where they previously returned 0 — a silent value change for existing views. Stored views and materialized views carrying a rejected shape become invalid on upgrade, cascade that invalidation to dependents and must be rewritten manually; the WAL apply path is exempt, so statements sequenced by an older binary keep applying.

HTML encoding of EXPLAIN plans was originally added as a presentation workaround for the Web Console, which did not preserve leading spaces, so the server emitted   for indentation and encoded < and > as < and >. The escaping was gated on the absence of a bind-variable service, which is the case for every HTTP request to /exec and /exp, so every REST client received escaped plans while the PostgreSQL Wire Protocol received plain text. This change removes the HTML escaping path from TextPlanSink so that child indentation uses four ordinary spaces, attribute indentation uses two ordinary spaces, comparison operators remain raw text, and literal strings such as  , < and > inside a query remain exactly as written; JSON serialization still performs the escaping required by JSON itself. HTTP and CSV EXPLAIN consumers will now receive spaces instead of   and < / > instead of < / >, and clients that parsed the previous markup must be updated. The Web Console now renders result values as text and preserves whitespace itself.

This feature moves a table's old partitions out to an object store (S3, GCS, Azure or a filesystem) as Parquet while keeping them queryable, driven by a TTL-based storage policy: ALTER TABLE trades SET STORAGE POLICY(TO PARQUET 1d, TO REMOTE 7d, DROP LOCAL 30d, DROP REMOTE 1y);. Each partition is converted to Parquet, uploaded, sealed and finally evicted from local disk, after which only metadata and symbol indexes remain local and reads fetch only the column chunks a query needs from the bucket on demand. The bucket is the source of truth: a single manager instance writes a per-table manifest and a cold-table catalog protected by a _manager.lock, while every other instance mirrors them read-only and performs its own bucket I/O, so partition bytes never traverse replication. A single PARTITION_SEAL Write-Ahead Log event is the only cross-instance signal, and the manager role can be moved at runtime with SWITCH COLD STORAGE ROLE TO MANAGER|REFRESHER [FORCE] [TIMEOUT ms] (requiring SYSTEM_ADMIN) with SWITCH COLD STORAGE STATUS reporting the current state and term. Reads are pinned to a snapshot identifier captured at cold-switch time, with self-healing re-identification so cross-host restores and storage migrations remain transparent, and a buffer coordinator coalesces concurrent chunk downloads, enforces explicit backpressure and charges pinned bytes against the query's memory budget. Remote garbage collection reclaims orphaned objects and dropped tables after configurable grace periods, table_cold_partitions('trades') exposes per-partition remote state, and Prometheus metrics report chunk hits, misses, coalescing, evictions and pinned bytes. The whole feature is off by default via cold.storage.enabled=false and is configured through the cold.storage.* and storage.policy.* property families. SHOW PARTITIONS gains two trailing columns, seqTxn and isRemotelyServed, and enabling upload is a one-way upgrade since the partition flag bits are unreadable by older binaries; replicas must be upgraded first, and non-Write-Ahead Log tables are not supported.

This feature adds live views: incrementally maintained window-function results over a single WAL-backed base table that are queried like a regular table. Window functions run once per row as new base commits are refreshed in, so a query scans precomputed output rather than reprocessing the base on every read, targeting workloads where the same window aggregate is read frequently against high-rate ingestion such as rolling VWAP, cumulative volume, running ranks and day-over-day comparisons.

CREATE LIVE VIEW vwap_5m
FLUSH EVERY 1s
IN MEMORY 5s
START FROM NOW
AS
SELECT ts, symbol,
       avg(price) OVER (PARTITION BY symbol ORDER BY ts ROWS 300 PRECEDING) AS vwap
FROM trades;

DROP LIVE VIEW [IF EXISTS] and ALTER LIVE VIEW ... RESUME|SUSPEND WAL are also supported, while structural ALTER verbs stay rejected because a live view's schema is a function of its SELECT. FLUSH EVERY and START FROM are mandatory; START FROM NOW, BEGINNING or a quoted timestamp literal defines the event-time boundary a row must satisfy to belong to the view, and a view over a base with existing history performs a resumable bounded seed before becoming active. A live view is its own WAL-backed table driven by a dedicated refresh worker pool sized by live.view.refresh.worker.count, with cairo.live.view.enabled defaulting to true; refresh appends computed rows to an in-memory tier as an un-flushed lead, flush re-serialises that lead into the view's WAL on the FLUSH EVERY cadence, and a versioned checkpoint timeline backs restart and bounded out-of-order replay. Reads route between the in-memory tier and disk under a sequencer-transaction consistency fence, falling back to the applied on-disk prefix whenever the two snapshots disagree. live_views(), SHOW CREATE LIVE VIEW, EXPLAIN, pg_class, information_schema.tables, tables() and table_columns() all recognise the new table type. Catalogue reporting also changes for existing objects to align with PostgreSQL semantics: materialized views now report pg_class.relkind 'm' and table_type MATERIALIZED VIEW, views report 'v' and VIEW, and all view kinds report is_insertable_into as false, so clients filtering on relkind = 'r' or BASE TABLE will stop listing them. SecurityContext additionally gains the abstract authorizeLiveViewCreate and authorizeLiveViewDrop methods. The V1 surface rejects joins, sub-queries, CTEs, SAMPLE BY or GROUP BY ahead of window functions, live-view-as-base, non-deterministic functions, and window shapes whose out-of-order influence is unbounded.

This feature adds <, >, and = comparisons between numeric expressions and single-column scalar sub-queries, supporting DOUBLE, FLOAT, LONG, INT, SHORT, and BYTE left operands while preserving integer precision and evaluating the sub-query once per query execution. The <=, >=, !=, and <> forms, as well as the argument-swapped forms where the sub-query sits on the left, are derived automatically, for example:

SELECT * FROM trades
WHERE price > (
    SELECT avg(price) FROM trades WHERE timestamp IN today()
);

The comparison width is chosen so that the scalar value is never narrowed: integer operands and integer cursor values compare as LONG, while FLOAT and DOUBLE values compare as DOUBLE. A scalar sub-query returning zero rows produces no match, one row supplies the cached value, and more than one row raises scalar sub-query returned more than one row, which also tightens existing designated-timestamp sub-query predicates that previously used the first row silently. Strict comparisons with a null scalar produce no match, while =, <=, and >= preserve QuestDB's sentinel-null convention. Scalar cursor state is cached during function initialisation and donated to parallel worker clones, so async filters, parallel GROUP BY, window joins, and horizon joins evaluate the sub-query only once per execution. The cursor must appear on the right-hand side unless the optimiser can rewrite the expression, cursor-to-cursor comparisons are unsupported, and correlated scalar sub-queries are not covered.

On a table with a covering posting index, every WAL commit block re-encoded the entire partition's covered sidecar, so per-block cost grew with the partition and WAL apply fell badly behind under sustained ingestion. This improvement routes blocks that are a pure append to the last native partition through the existing fast-lag append path, writing covered fragments incrementally and deferring compaction to the auto-seal, which reduces per-block covered-index cost from proportional to the partition size to proportional to the number of new rows. The block in-order flag is re-derived with a non-overlap check so a globally in-order multi-timestamp single-segment block skips the out-of-order sort, and the fast path is gated strictly on a pure append to an open, non-parquet last native partition with no lag, dedup or replace-range, so any merge or late-data block falls back to the unchanged out-of-order path. This change also fixes a pre-existing crash-safety defect in covering indexes shipped since 9.4.0, where the chain entry's cover-end footer aliased a generation directory slot and could be clobbered in place, letting a concurrent covered reader map the sidecar too small and dereference an offset past it. The footer is de-aliased to a fixed offset under a versioned covering format, and legacy entries are migrated copy-on-write on their first extend through a single funnel, flipping the head atomically so both concurrent readers and crash recovery stay safe with no reindex required; dual-format reads keep existing tables readable through the transition.

This improvement fuses window functions that share the same window specification so that a query carrying several functions over the same PARTITION BY key uses one hash map and one probe per row instead of one map, one key copy and one probe per function. Where two outputs are provably the same accumulator, such as count(x) reading the non-null counter inside sum(x), or avg(x) and sum(x) sharing the same state, they fold onto a single component. Fusion covers cumulative sums, averages and counts, row counters, the max/min families over DOUBLE, LONG, DATE, TIMESTAMP and DECIMAL, ksum, the Welford dispersion family, first_value/last_value with and without IGNORE NULLS, bounded ROWS and RANGE frames, and whole-partition two-pass functions; a PARTITION BY expression groups too, unless it is non-deterministic. Gains are concentrated at high key cardinality, where several shapes run two to three times faster with roughly halved retained native memory, and are neutral to slightly negative at low cardinality, so the new cairo.sql.window.map.fusion.enabled property (default true, reloadable through reload_config()) acts as a kill switch; because the switch is read at compile time, turning it off for an already cached plan also requires flush_query_cache(). EXPLAIN output is unchanged at either setting. Live view refresh, restart and memory use are also improved: a view resuming from its checkpoint no longer full-scans all retained state on the first seal, the WAL clean symbol band is resolved through the on-disk hash index instead of copying the base table's entire committed dictionary on every bind, the checkpoint map memoises descents by depth so incremental seals no longer degrade past roughly 2.4M keys, seals stay incremental across a sweep, ring arenas for bounded frames are reclaimed on sweep, symbol map caches move off the Java heap into tracked native memory, and a live view now inherits its base column's symbol CACHE flag. CREATE LIVE VIEW additionally accepts projections around the window, including expressions wrapping a window function, aliased base columns and scalar columns beside the window, while shapes the refresh path cannot rebuild, such as LIMIT, ORDER BY over the window, set operations and joins, are rejected with messages naming the actual cause; a SELECT that projects no designated timestamp is now rejected at creation rather than faulting to INVALID later. This also fixes several correctness problems: sum() over a bounded RANGE frame returned the average of the frame on cached window cursors, bounded RANGE frames could wrap at extreme timestamps, ORDER BY on a NOCACHE SYMBOL column under an exclusion filter returned insertion order, generate_series could overflow, live views could serve rows whose running window values continued from an abandoned refresh cycle, an anchor-resume repair wrote the window's columns into the view's columns, and a head-miss repair repeated a memoized column's stale value.

This improvement adds a MergeUnionAll physical operator that streams an ordered UNION ALL as a two-way merge instead of concatenating branches and sorting the result. The operator is selected when the union branches expose designated timestamps of the same type, share the same scan direction, and a timestamp order is requested that matches that direction; otherwise the existing concatenating UNION ALL plan is used unchanged. The optimiser now captures a window's OVER ORDER BY as model advice and pushes order-by advice down into UNION ALL branches, so branches scan uniformly and both windowed and non-windowed queries are handled the same way, for example:

EXPLAIN (SELECT * FROM a) UNION ALL (SELECT * FROM b) ORDER BY ts DESC;

Chained and nested unions produce nested merge plans, and windows over a union can stream rather than materialize through a cached window cursor. Full scans are roughly eight to nine times faster than the sort-based plan, and LIMIT-bounded queries improve by several orders of magnitude because no full materialization is needed. Branches that disagree on timestamp type or precision continue to fall back to a sort.

Since JDK 24, Thread.sleep constructs a ThreadSleepEvent before checking whether the event is enabled, so every call allocates 40 bytes whenever the JIT cannot eliminate it. Because Os.pause() wrapped Thread.sleep(0) and appears in worker back-off loops, idle workers repeatedly paid this allocation on Graal, C1, interpreted, cold and deoptimized paths; an idle GraalVM instance with 60 workers allocated roughly 59 MB/s from this source alone. This improvement changes Os.pause() to call Thread.yield(), which reaches the same underlying yield on HotSpot while avoiding the sleep event and preserving the interrupt flag, and routes Os.sleep(millis) through an exported native qdb_sleep_millis symbol via a foreign function downcall that reissues interrupted platform sleeps. The downcall deliberately avoids a critical binding so the VM can still safepoint over the sleep, and the symbol is resolved immediately after the native library is loaded so a stale library fails class initialization with a clear error rather than killing workers during their first back-off. Because Os.pause() and Os.sleep() no longer clear the caller's interrupt flag, condition-wait loops in SOCountDownLatch, SOUnboundedCountDownLatch, SimpleWaitingLock, the concurrent hash map tree-bin contention paths, TimerShards and the lifecycle orchestrator now consume an interrupt before parking, keep waiting on their condition, and restore the flag when the operation exits, preventing an already-set flag from turning these paths into CPU spin loops. Additional hardening keeps WAL purge lock cleanup progressing across malformed tracking data and individual unlock failures, ensures asynchronous log records release their queue cursor even when validation throws, and frees the first native buffer when a log alert socket fails to allocate its second one. A platform thread inside Os.sleep() now reports RUNNABLE, shows foreign-function stub frames in stack traces, and emits no jdk.ThreadSleep event.

This fix addresses a race in the uploader's shared <db_root>/.upload scratch directory. Because the path is derived from the database root, it is shared across uploader generations, so during a teardown-and-rebuild role-switch cycle such as a hot switch or a refused-demote heal, the old uploader's final reference could unwind after the new uploader had already recreated the directory, and its drop then deleted the scratch directory out from under the live uploader. The compression path only checked that the directory existed, so from that point every upload failed with tmp path does not exist in a permanent retry loop, leaving a healthy-looking primary with replication silently wedged until restart. Temporary archive creation now recreates the directory tree instead of merely checking it, which is a no-op when the directory exists, self-healing when a stale teardown removed it, and still an error when a non-directory occupies the path. In-flight archives are unaffected by a concurrent delete because anonymous temporary files are used, so the next call converges.

This fix prevents data loss on the safe demote path. When an upload audit refuses a demote, the node is meant to keep its committed-but-not-uploaded Write-Ahead Log data, park read-only in a degraded replication state, and heal back toward primary to drain the backlog. However, the refusal arm still closed the primary role state and cleared the upload-mode latch that UploadWalDirectoryPolicy.isSeqPartInUse was gated on, so every V2 sequencer part reported as free and the node's own purge job deleted the applied transaction-log parts belonging to the very backlog the audit had declined to abandon. After healing, the rebuilt uploader could no longer read the range it had to ship and retried indefinitely, so acknowledged transactions became permanently unreplicable while the local table continued answering queries normally. The policy now carries an explicit unresolved-demote-backlog pin meaning the node owes a backlog it cannot currently upload, and part markers are honoured when either the upload-mode latch is up or the pin is set. The pin is set on both demote arms before the close that clears the latch, so no purge sweep can observe a window with neither flag set, and it is released only where the node is proven to owe nothing: a clean demote drain audit, or a retry toward replica in which every residue table has been explicitly sacrificed. A node that was never primary in the current process never sets the pin, so a plain replica keeps reclaiming its downloaded parts unchanged.

Metadata hydration can construct table sequencers as soon as the engine becomes ready, so installing the upload-aware WAL directory policy later left a race in which a sequencer captured the default policy for its lifetime, never created V2 sequencer-part upload markers, and allowed the WAL purge job to remove a part before catch-up uploaded it. This fix installs a single upload-aware policy immediately after the base server constructor returns and before lifecycle startup publishes engine readiness, and the engine keeps that policy instance throughout startup and role switches. The policy also preserves segment upload obligations across inactive and demoted roles: WAL writers record segment and sequencer-part markers while the server runs with role NONE so a later primary restart can discover retained work, and a segment is reported in use if either the upload-pending or download-pending marker exists, read under a single upload-mode snapshot so a concurrent role switch cannot probe the same marker family twice. Per-part sequencer markers stay mode-gated and are ignored in download mode, so a replica or demoted primary with no live uploader does not treat them as purge locks and cannot grow the sequencer part directory without bound.

A live view's files can reach a replica before its base table's, so registration fails and is retried on a later poll; if the node restarts first, boot finds the view definition on disk with no state file, refuses to load it and registers a dead stub that occupies the view's slot forever, leaving the view permanently empty because a replica refreshes live views locally. This fix drops the stub and rebuilds the view when a registration finds a live view with no state file and only a stub registered, so the view is left unregistered and retryable rather than stuck behind a stub if the base table is still missing. The recovery no longer unlinks and re-copies the definition file, which previously left a window in which a concurrent CHECKPOINT CREATE could abort or capture a snapshot whose restore deleted the view's data directory. Genesis now stages the state file through a temporary file and renames it into place, so a failure or crash anywhere inside the first write leaves the state file absent rather than present but unreadable; a stale staged file left by a crash is removed before writing and the registration is refused and retried when that removal fails. The on-disk format is unchanged and the state file stays node-local, so mixed-version clusters and rolling upgrades are unaffected.

This fix corrects scalar sub-query predicate handling and timestamp interval pruning, covering scalar sub-query bound hoisting, lateral count compensation, constant-false pruning in the interval model, and LATEST BY external-source propagation. As part of it, the rejection of UPDATE statements with a join on WAL tables is hoisted ahead of code generation, so on a WAL table the unsupported-shape error is now reported before column-level SELECT authorization runs; the non-WAL path is unchanged and still reaches the permission check.

This fix corrects current_user() and session_user(), which always returned the hardcoded default admin regardless of which user actually authenticated, so the Web Console displayed admin for everyone on page load, including a configured http.user or a PostgreSQL Wire Protocol user. The root cause was that the allow-all and read-only security contexts reported a fixed principal instead of the authenticated one. AllowAllSecurityContext and ReadOnlySecurityContext now carry a principal field and expose forPrincipal(principal), and the security context factories pass the authenticated principal through it. Anonymous requests allocate nothing, and derived contexts are cached per principal so the HTTP path, which re-derives the security context on every request, becomes allocation-free after the first request for a given user; the cache is bounded at 256 principals, past which an unseen principal degrades to allocate-per-call rather than growing without limit. DenyAllSecurityContext and the materialized view refresh context are never re-derived, so they cannot be downgraded to a read-allowing context, and the transient principal is copied before it is retained. current_user() and session_user() are now runtime-constant functions that snapshot the principal in init() rather than reading it once per row, so a 1000-row query performs a handful of security context reads instead of roughly one per row, while a context re-derived between executions of a cached SELECT is still picked up.

This fix stops TableReader from closing a column's file descriptor immediately after mapping it for every partition except the currently growing last one. Because the file descriptor and memory-map caches can only reuse an existing mapping while its file descriptor is still open, a later, non-overlapping reader that opened the same historical partition always created its own independent mapping. Under workloads with many long-lived, concurrent readers scanning the same large table, such as dozens of parallel connections pulling a multi-terabyte table in time-ordered batches, those duplicate mappings accumulated and could exhaust the process's virtual address space with could not mmap errors even while resident memory usage stayed low. A column's file descriptor is now kept open for as long as the reader keeps the column mapped, for every partition, so concurrent or staggered readers of the same partition share a single mapping. The trade-off is higher file descriptor usage: a reader now holds one descriptor per currently mapped column for as long as cairo.inactive.reader.max.open.partitions (default 10,000) lets it keep that partition open, so deployments with many partitions, columns and concurrent readers should ensure their ulimit -n has headroom.

This fix prevents ALTER TABLE ... ALTER COLUMN ... TYPE from crashing the whole JVM with a native SIGBUS when disk space was low instead of failing the statement with a normal error. The destination column file was sized with ftruncate(), which only sets the logical file length and does not reserve real disk blocks, so a later native write into an unbacked page faulted with SIGBUS, which cannot be caught from Java or JNI. Disk space is now reserved via TableUtils.allocateDiskSpace() (posix_fallocate), matching every other writable memory-mapping call site in the codebase, so a disk space shortfall surfaces as a catchable CairoException.

This fix prevents an idle live view fallback scan from durably invalidating a healthy view while another refresh worker processes a new base table commit. The fallback scan validates restored or replicated state by invalidating a view whose processed sequencer transaction is ahead of its base table, but it sampled the base sequencer head first and the view watermark second. A notification worker could publish and refresh the next base transaction between those two reads, so the fallback worker compared an old base head with a new view watermark and persisted a false "view is ahead of base table" invalidation. The view watermark is now read and retained first, followed by the monotonic base sequencer head, with those same retained values used for both the comparison and the diagnostic log. Since a refresh publishes the watermark only after consuming the corresponding base sequencer transaction, a newer watermark can no longer be paired with an older base head, and the impossible combination is eliminated without adding a lock on the hot path. Genuine restored or replicated ahead states still take the existing invalidation path.

This fix stops squashSplitPartitions from destroying the tail of a variable-size column (BINARY, STRING, VARCHAR) in the partition it squashes into, which produced silently wrong query results, binary is outside of file boundary errors, or unsafe memory access faults. The squash appends into the target partition through the frame's own file descriptors; when that target is the partition the writer still holds open, the writer's column append memories keep their pre-squash positions and describe a shorter file than what is on disk, so the stale append memory trims the grown data vector back on close and a later memory re-extends it zero-filled, cutting the last value's length prefix in half. This state is reachable because opening the last partition is a no-op when that partition is Parquet, leaving the writer pinned to an earlier native partition, for example after ALTER TABLE ... SET FORMAT PARQUET followed by an insert into a later day. When the squash target is the writer's open partition, the stale view is now dropped by closing without truncating and the target is re-opened so the writer resumes from sizes that match disk, with both arms running under a single distress guard. The posting index seal-purge outbox is now drained before closing the active partition in the squash path, so superseded index files are published for purging instead of leaking on disk. This change also hardens posting index publishing and closing: close() no longer sizes a mapping up just before releasing it, several call sites now arm the seal transaction tag before publishing, and the reader rejects structurally impossible or all-zero generation metadata. As a user-visible consequence, the reader's error now names the affected column and the recovery step, for example posting index is corrupt, rebuild it with REINDEX TABLE <table> COLUMN <column> LOCK EXCLUSIVE, and a query over a damaged index that previously returned a silently short result now raises an error instead.

This fix addresses two independent defects that could permanently invalidate a live view over a healthy base table, even for a column the view never referenced. First, WAL symbol map folding was coupled to every symbol column carrying a diff, ignoring the projection supplied by the caller; because WAL symbol dictionaries are shared across segments and deleted as soon as a DROP COLUMN is applied, a live-view drain could fault on a dictionary belonging to a column outside its projection. Symbol map folding now skips diffs for columns the effective projection does not reach, while a rebind that widens the projection still refolds from the start of the event file so all symbol values resolve. Second, the re-derive path that rebuilds a view from the applied base table after WAL loss replayed through a cached compiled plan and was refused whenever base metadata had moved on, which is the normal state during the DDL churn that triggers it; it now recompiles once and retries the replay. A shared guard reads the base's applied metadata ahead of both replay attempts and refuses to re-derive when a referenced column was dropped, renamed or retyped, invalidating the view and naming the column rather than silently committing results computed over a new schema. A common trigger was ALTER TABLE base ALTER COLUMN sym SYMBOL CAPACITY N, which is non-structural and deliberately leaves dependent live views valid, but causes the next WAL segment roll to delete and re-link the dictionary. The re-derive is a full rebuild from the applied base rather than an incremental drain, so recovery carries a one-off cost on the refresh worker, and the projected WAL reader overload now reports 0, null and not-found for symbol columns outside the projection.

This fix prevents PostingIndexWriter.close() from trimming the .pk file below the chain region another writer instance had already published, which physically destroyed the tail of the most recently published chain entry. Because more than one writer can be open on the same .pk file, the closing instance's cached high-water mark could lag the file, and the lost bytes read back as zeros, corrupting the generation directory and silently dropping rows from POSTING index scans. The writer now never trims below the region limit published in the chain header, the authoritative live high-water written under the header seqlock. Three further defects are addressed: publishing could itself write a regressing seal transaction when an extend happened on a reopened writer, so a slot is now clamped up to its predecessor and a monotonicity check runs immediately after the slot store and before publication, throwing a diagnostic exception instead of aborting via an assertion; the reader no longer asserts gen-dir monotonicity inside a window where torn reads are legal, and instead records the anomaly and acts only once the snapshot is proven self-consistent; and the reader now closes the release half of the entry seqlock by re-reading the generation count and retrying, so an entry rewritten in place with a smaller count cannot mix two incarnations. Genuine corruption now raises a diagnostic error naming the entry offset and generation counts. Additionally, close() no longer propagates errors on a cleanup path that must not throw, logging instead and leaving the file intact for the next open when no trusted high-water is available. A .pk file already damaged by an earlier build does not self-heal; rebuild the index with REINDEX TABLE <table> COLUMN <column> LOCK EXCLUSIVE;.

This fix resolves a crash when reading metadata for tables created before QuestDB 6.1. The per-column replacing index was followed straight out of the _meta file without validation, but on migrated tables that slot holds leftover junk, because the migration grew column entries from 16 to 32 bytes and left old file contents in the bytes it did not write, with every later rewrite copying the junk forward. An index past the column count read off the end of the mapped file, and one pointing forward read off the end of the dense column list; with assertions disabled both were silent out-of-bounds reads that fed garbage into column mapping. The replacing column index now returns -1 unless the value could have come from a real replacement, meaning it is below the column carrying it and points at a deleted column, so every chain walk is in-bounds and loop free and both cycle guards were removed. Affected tables clear the junk on their next ALTER, which rewrites the column block from in-memory metadata. As a tradeoff, a genuine chain whose bytes were corrupted now degrades silently rather than erroring.

This fix eliminates an ERROR-level log line per view on every start of a database containing views, which read like corruption despite the view definition files being present and intact. View definitions are loaded from disk into the view graph and state store by the worker pool manager, but the hydration components depend only on the engine, so as soon as the engine published READY the view compiler and metadata hydration threads walked the table name registry with an empty graph, reporting missing view definitions and skipping views during write tracker hydration. As a result, compiling all views at boot was a no-op on every start, masked only by the per-view compile requests issued later. Loading of view definitions now runs in the engine startup step, immediately after the engine loads table state and before READY is published, so every engine-gated component observes a fully loaded graph. No client-visible failure occurred previously, since the shared pools serving PostgreSQL Wire Protocol and HTTP only start after the graph is populated.

This fix tightens an internal memory bounds check so that invalid writes fail safely instead of corrupting memory. The issue was found internally and no production workload is known to trigger it. There are no SQL, API, or configuration changes.

This fix restores correct covering-index reads across Parquet format conversions. Converting a partition with a COVERING posting index to Parquet and back to native rebuilt the posting index from the decoded native data without the covering sidecar files, so covering scans returned NULL for every covered column or failed with an index-out-of-bounds reduce error, while the underlying native data remained intact. The conversion back to native now hard-links the existing key, value and covering sidecar files from the Parquet partition directory into the new native directory, mirroring what the native-to-Parquet direction already does, which also removes the index rebuild work for plain bitmap symbol indexes. Converting to Parquet had the same defect in the opposite direction whenever the indexed symbol column had a column top: the rebuild that materialised the leading NULL prefix produced key and value files only, and both directions now share a single index rebuild routine that configures covering when needed. As a behavioural change, the link path gates on the key file alone, so a Parquet partition directory holding a key file with no resolvable live value file now fails the conversion with an "index files do not exist" error, suspending a WAL table, rather than silently rebuilding a non-covering index; reaching that state requires out-of-band file loss or a crash during index creation. Building covering sidecars on the column-top path also makes CONVERT PARTITION TO PARQUET do index work proportional to rows times covered columns where it previously did almost none.

ALTER TABLE t ALTER COLUMN sym ADD INDEX TYPE POSTING INCLUDE (price, tag);
ALTER TABLE t CONVERT PARTITION TO PARQUET LIST '2024-01-01';
ALTER TABLE t CONVERT PARTITION TO NATIVE LIST '2024-01-01';
SELECT sum(price), first(tag) FROM t WHERE sym = 'A0';

This fix prevents a WAL table with a POSTING index from being suspended mid-apply during a partition squash. Under parallel WAL apply, reopening a column's key file sized the memory mapping from the reported file length, which can lag the writer that just extended the file, so the chain's head entry fell outside a short mapping and was read past its end, distressing the writer and suspending the table. The reopen now reads the header's live entry-region high-water mark through a seqlock-consistent header read and extends the mapping to cover it only when the reported length genuinely lags, leaving the common case with no extra remapping, and the header is range-checked before its values are trusted so a corrupt header raises a CairoException instead of an out-of-mapping read. A second fix stops the key file from being truncated when a reopen fails before the region is republished: the failure path now releases the mapped memory without truncation, leaving the on-disk index intact for the next open and removing a latent hazard where the file could be truncated while still mapped by concurrent readers. Neither change alters how the key file is written or the on-disk format.

This fix replaces an invalid native memory access with a controlled error when a POSTING index writer is rolled back after an I/O failure closed its mapped value file. Previously, a failed remap while flushing pending data, or a failed open of the replacement value file during truncation, left the writer holding live key or pending state over a closed mapping, so a later rollback could re-enter the flush or reencode path and produce an assertion failure, a null facade access or a native memory crash. The flush and reencode paths now reject work over a closed value memory with a CairoException before any scratch allocation or index mutation, and closing mapped read-write memory now clears its append bounds so stale addresses from a released mapping can no longer be accepted without remapping. Table-level recovery is unchanged: the caller still distresses and discards the affected writer, and the on-disk chain is recovered on the next open.

This fix prevents a table from being suspended after a REPLACE_RANGE WAL commit, used by materialized view refreshes and range-replace inserts, removes the table's last partition. The commit moved the writer's maximum timestamp back to the new last partition but left the partition timestamp boundary pointing at the removed partition, since the replace path only ever advanced it and never lowered it. The usual re-sync that re-opens the new last partition is skipped when that partition is Parquet, so the next non-replace commit applied by the same writer tripped a partition-timestamp consistency check and suspended the table. The partition-removal branch now re-derives the partition timestamp boundary from the final maximum timestamp after partition sizes are updated, which runs only for replace commits that removed or trimmed partitions and leaves the native last-partition behaviour unchanged.

A query that waits, such as sleep(), wait_wal_table(), or any long scan that periodically consults the SQL circuit breaker, could keep running after its client had already disconnected, holding the connection's socket descriptor and worker resources until the query reached its own deadline. This fix replaces the peek-based connection probe with a hangup probe that detects a FIN behind buffered data on all platforms, including Windows, using poll(POLLRDHUP) on Linux, kqueue with EVFILT_READ/EV_EOF on macOS and FreeBSD, and WSAPoll on Windows. Previously a clean client close sent an application byte before the FIN, so the peek returned buffered data instead of EOF and the breaker never tripped, letting the query run until query.timeout (60 seconds by default). This fix also wires a real, file-descriptor-backed circuit breaker into the QuestDB WebSocket egress query path, covering compilation, cursor acquisition and the result-streaming loop, so egress queries now honour query.timeout and abort with STATUS_LIMIT_EXCEEDED instead of streaming past the deadline; a stream resumed after a write park or credit grant re-checks the deadline and the connection before producing the next batch. A request-boundary reset that disarmed the breaker of a parked, protocol-switched stream is corrected, and connection probes are now throttled by wall clock (100 ms by default) so the file-descriptor-backed breaker does not turn hot loops into per-iteration syscalls, at the cost of up to one throttle window of detection lag per checking site. Bulk data phases such as INSERT AS SELECT, CREATE TABLE and UPDATE remain exempt from query.timeout by design. The now-obsolete circuit.breaker.buffer.size and net.test.connection.buffer.size settings are removed and log an advisory on start-up, or fail under config.validation.strict. A double-read race in WriterPool.Entry.getTableToken() that could throw a NullPointerException when a writer_pool() scan raced a writer close is also fixed.

Restore cancellation was previously signalled only from a component's stop(), which runs inside the reverse-topological stop loop that follows the bounded boot-thread join, so a SIGTERM landing during a long point-in-time restore burned the entire 30-second join budget before the restore was asked to unwind, and a pod with a 30-second termination grace was killed mid-restore. This fix adds a pre-join cancel hook that runs after the executor drain and before the boot join, so the join budget is spent unwinding the restore; the hook contract is a pure, non-blocking cancel signal, resource teardown stays in stop(), and a throwing hook is logged rather than propagated so it cannot abort the shutdown it was installed to accelerate. Materialized view hydration now accepts an explicit target state store, allowing a caller to hydrate a private, not-yet-installed store and publish it only after hydration completes, with closing checks that abort the load promptly; the state store's close() is now idempotent across sequential closes and clears its map after freeing state, preventing a double free. A non-creating suspension-tracker lookup is added for hot read paths such as metrics scrapes, where a missing tracker simply means the table was never suspended. Finally, a broken duplicate-log assertion in the logging test helper was corrected, which exposed and fixed a real bug where every CREATE TABLE and system-table creation logged two completion records, one premature record carrying a -1 sentinel id; completion logging now belongs entirely to the execution site, so a CREATE that is compiled but never executed emits no completion record at all.

On a primary, an apply-time invalidation could publish while another refresh worker held the same view latch; the losing worker queued a retry, but the pending-marker guard consumed that retry without minting durable invalid state or arranging another wake-up, freezing the view in a stale-but-valid state. This fix introduces a single atomic pending marker with independent invalidation and full-refresh facets, carrying the invalidation reason, base table token, base transaction, force flag and a distinct full-refresh owner, merged keep-strongest so force flags OR, same-token transactions keep the maximum, and unknown or mismatched provenance collapses to conservative. An invalidator publishes the marker before attempting the view latch and either handles it itself or lets the completing latch holder queue one authoritative wake-up, so a contention episode produces bounded queue traffic. All operational latch holders, including incremental, range, interval, full, invalidation and synchronous REFRESH ... STATS paths, release through a common finalize helper that runs cleanup before publishing follow-up work. Full refresh is now snapshot-aware: it records its fixed reader snapshot and clears an invalidation only when the base table token matches and the snapshot transaction covers the invalidation transaction, while unknown provenance, a different table epoch or a newer transaction keeps the invalidation pending. An authorization refusal from the read-only writer gate, such as a replica demote racing a refresh, is now treated as a transient role condition that defers instead of trapping the worker in an unbounded requeue loop. Suspended views retain pending invalidation and full-refresh facets and republish them after a successful RESUME WAL, and invalidation now cascades to dependent views only after an invalid state is durably persisted, so a declined invalidation no longer leaves children invalid. If queue growth fails, allocation-free retry flags are recorded on the view state and redelivered by a later refresh-job tick.

On Windows the writer temporarily grew the small .pci cover-info file to the data-index append page size (1 MiB in tests, 16 MiB on a server) and truncated it back on close, so a concurrent reader could observe the larger length and map the file after it had shrunk, failing with errno 8. This fix maps .pci at a page-aligned size derived from the payload and any existing size and parks the append pointer there, making the physical file size monotonic non-decreasing for the file's life so a reader's length-then-map sequence can never map past a shrink; a larger existing file is preserved rather than shrunk. On adjacent paths, the .pci header is now rewritten only when it is absent, shorter than the payload, or its magic, count or covered-column indices disagree with the live configuration, repairing corruption instead of rewriting unconditionally on every seal. The sync order now fsyncs .pv, each .pcN and .pci before .pk, so a covering commit can no longer leave a durable chain head over cover bytes that were never flushed, and a failure to read the length of an existing .pci now throws instead of being silently treated as a new file. Under SYNC and ASYNC commit modes each covering commit issues up to one additional sync per covered column plus one, which is inert under the default NOSYNC.

On x86 systems with SQL JIT enabled, a valid query that checks a STRING column for NULL could rarely crash QuestDB with SIGSEGV when a null or empty string header occupied the final four bytes of a mapped column file, for example SELECT status, count() FROM events WHERE status IS NOT NULL GROUP BY status. This fix makes the scalar JIT read the four-byte STRING header without crossing the mapping boundary. Before upgrading to a fixed version, the crash can be avoided by setting cairo.sql.jit.mode=off and restarting QuestDB.

Decorrelation rewrites a scalar-aggregate lateral body into GROUP BY plus LEFT JOIN, which loses the empty-group row; the previous compensation only recognised bare count() projections, so unmatched rows returned NULL for count(*) + 2, CASE WHEN count() = 0 THEN 7 ELSE count() END, count_distinct(x), approx_count_distinct(x) and for sibling columns of a count, and WHERE sub.cnt = 0 filtered out the very rows it was meant to select. This fix builds a compensation template for any projection containing a zero-on-empty aggregate (count, count_distinct, approx_count_distinct), extracts aggregate leaves into hidden carrier columns and wraps only zero-on-empty leaves in coalesce(x, 0), so count(*) + sum(v) correctly stays NULL while count(*) + 2 returns 2; constants, outer references and bind variables now carry their value onto the compensated row, and WHERE terms referencing compensated columns are hoisted above the null-extension boundary so they observe 0 rather than NULL. Keyed bodies — an implicit GROUP BY key, a SAMPLE BY body, or a WINDOW JOIN / HORIZON JOIN body — are no longer treated as scalar and correctly return NULL instead of a fabricated 0, while CROSS JOIN LATERAL and INNER JOIN LATERAL over a scalar aggregate body are promoted to LEFT OUTER so they preserve every outer row. PIVOT empty cells for approx_count_distinct now default to 0 instead of NULL, and the expression parser accepts count(*) in any expression position, so coalesce(count(*), 5) and count(*) + count(*) compile, including inside OVER() window specifications where :: casts and multi-WHEN CASE expressions were broken in the same way. A negative LIMIT reaching a lateral body through a bind variable or an outer column now fails with negative LIMIT is not supported in a correlated lateral sub-query instead of silently emptying the body.

LATEST ON over an indexed SYMBOL column with a PostgreSQL Wire Protocol bind variable could return the latest row of a previously queried key, making the WHERE filter appear to be silently ignored. LatestByValueDeferredIndexedRowCursorFactory did not override RowCursorFactory.init(), which is a no-op by default, so the symbol function was never re-initialised and kept pointing at the bind variable service of the execution context that originally compiled the factory; once the shared SELECT query cache handed the cached factory to another connection, the resolved symbol key belonged to a different connection's bind variables. This fix forwards init() to the symbol function, restoring symmetry with the sibling deferred symbol index factories. Only indexed columns queried through bind variables on fresh connections were affected; literals and casts such as $1::SYMBOL resolved through other paths and were always correct.

VARCHAR date and timestamp conversions could produce different results for native and Parquet-backed partitions, potentially excluding valid rows from queries. This fix treats a negative ASCII hint as unknown and inspects the actual bytes, preserving the ASCII fast path while supporting Unicode patterns and timezones. It also makes malformed UTF-8 handling atomic across SQL conversions, Parquet and O3 processing, row copying, TableWriter, WAL and Influx Line Protocol ingestion, so invalid input becomes NULL instead of exposing or persisting a valid prefix. Before upgrading, the original parsing issue can be worked around with an explicit format, for example to_timestamp(trim(TimeCreated), 'yyyy-MM-dd HH:mm:ss.SSS').

QuestDB now accepts SHOW CREATE DATABASE as a parenthesized query source in FROM, CTE and JOIN contexts, while filters and INCLUDE (...) / EXCLUDE (...) clauses keep their existing top-level behaviour. The parser previously read clause delimiters with helpers that consumed a closing ) as the enclosing subquery terminator, so the outer parser lost either the SHOW subquery delimiter or the category-list delimiter. This fix reads optional-clause lookahead without consuming the enclosing delimiter and returns it through the existing unparse path, reading the category list's own closing delimiter locally. Valid top-level statements are unchanged; malformed input that closes a valid category list but omits the enclosing subquery delimiter now reports the more specific error ')' expected instead of ',' or ')' expected.

This fix corrects dot_product() over two DOUBLE[]/DOUBLE[][] arrays, which could silently return a wrong result when one operand was not stored in vanilla (contiguous, row-major) layout. In the recursive code path the deepest-dimension loop read the right-hand array using the left array's flat index instead of its own cursor, so operands with different strides — for example a transposed array or a slice — selected the wrong elements. For instance, transpose([[1,3],[2,5]]) combined with [[1,5],[7,2]] now correctly yields 42 instead of 40. Arrays already stored in identical vanilla layout are unaffected, and there is no performance impact since the change simply uses an index variable that was already maintained.

This fix allows SAMPLE BY w to parse correctly, where it previously failed with one letter sample by period unit expected even though every other single-letter sampling unit, such as SAMPLE BY h, SAMPLE BY d and SAMPLE BY M, worked without a leading digit and users had to write SAMPLE BY 1w as a workaround. The parser's allow-list for the bare single-letter form was missing w, rejecting the query before it reached the sampler, even though the week unit was fully supported downstream by both the micros and nanos timestamp drivers. The change only widens what the parser accepts, with SAMPLE BY w now behaving identically to SAMPLE BY 1w, and does not alter the behavior of any query that already parsed.

This fix addresses several correctness and resource-safety issues in parallel SQL execution. first_not_null() and last_not_null() now behave correctly for IPv4 columns on the non-sharded keyed-batch path, implementing the same keyed-batch semantics as other primitive types, and batched last_not_null() implementations replace stored NULL values consistently with their row-by-row and merge paths. dim_length() now returns NULL for NULL arrays, NULL dimensions, and NULL PostgreSQL Wire Protocol array binds, while non-NULL out-of-range dimensions still report an error. Parallel reducers — including parallel filters, GROUP BY, top-K, horizon joins, multi-horizon joins, and window joins — now release their worker slot from a finally block entered immediately after acquisition, so a failure during frame navigation, temporary allocation, or cleanup can no longer permanently leak a slot and leave later executions of a cached factory spinning forever. A data race on the selectivity statistics shared across threads by a thread-safe parallel filter is resolved by packing the sample count and average into a single volatile word updated with a CAS loop, guaranteeing both consistency and visibility for the late-materialization heuristic. Array access overhead is reduced by reading dimension and element data directly through record accessors rather than materializing an ArrayView per row, NULL-producing records consistently return a NULL ArrayView, and the shared NULL array constant uses a borrowed array so an accidental close cannot mutate global NULL state. A dead VARCHAR entry was also removed from the CaseCommon dispatch table, which changes no behavior.

This fix resolves a lifecycle race where CANCEL QUERY <id> could cancel a different, unrelated query after the registry recycled a pooled query entry, causing the victim query to fail with cancelled by user even though nobody cancelled it. Cancellation now only applies to the active query that owns the requested id: each pooled entry carries a volatile lifecycle word packing the owning query id and a state, so a stale canceller holding a recycled entry fails the lifecycle check and reports the query as not found instead of cancelling the new owner. query_activity() now snapshots registry entries optimistically and only returns a row if the entry still belongs to the same active query after the row has been copied, preventing torn rows and wrong-query data while entries are unregistered and recycled. The cost is one volatile lifecycle publish on query registration and one uncontended CAS when unregistering, and a query in the cancelling state may transiently not appear in query_activity() while a canceller holds the lifecycle guard.

This fix resolves cases where some queries returned nothing over the PostgreSQL Wire Protocol, leaving the client blocked in fetchall() until the idle timeout closed the connection, five minutes later by default, while the server log showed the query finishing normally followed by an idle disconnect. The server was sending a DataRow header announcing more bytes than it went on to send, which occurred with ARRAY columns in text format when the client sends per-column format codes, and with IPv4 and GEOHASH columns requested in binary. Drivers that send a single result format code, such as psycopg2 and JDBC in its default mode, were never affected. As a tradeoff, rows that genuinely exceed pg.send.buffer.size now fail with an explicit error naming the required size instead of hanging, so jobs that previously hung will now report a failure.

Selecting a DECIMAL column over the PostgreSQL Wire Protocol could fail with unsupported type: 31 when a binary response crossed the server send-buffer boundary, because decimal types were missing from the calculation used to size the unsent tail of a fragmented DataRow. This fix adds exact PostgreSQL NUMERIC binary-size calculation for DECIMAL8, DECIMAL16, DECIMAL32, DECIMAL64, DECIMAL128 and DECIMAL256, accounting for nulls, zero, sign, precision, scale and base-10000 digit grouping, reuses the execution context's decimal scratch values for the 128-bit and 256-bit widths so the response path does not allocate, and adds a text-size estimate for decimal columns. It also introduces DecimalFunctionMemoizer, registered by SqlCodeGenerator for all decimal widths, so that a projected non-deterministic decimal function such as rnd_decimal() returns a stable value per row; sizing and serialization therefore stay aligned. The memoizer applies to every projected DECIMAL column on all protocols whenever the alias is read more than once or the function reports shouldMemoize(). Decimal storage and the binary encoder are unchanged. INTERVAL and ARRAY_STRING columns still have the same gap in the size calculators.

A client speaking a non-Influx Line Protocol binary protocol on the Influx Line Protocol TCP port caused the server to log an unhandled critical error, because EllipticCurveAuthenticator.readKeyId() converted raw key-id bytes to a String eagerly and threw CairoException on invalid UTF-8, which handleAuthentication() did not catch. Since nothing on that path closed the connection, every such connection permanently leaked a file descriptor, its receive buffer and one of the 256 default connection slots, and incremented the unhandled_errors counter exported to Prometheus. This fix validates the key id with Utf8s.validateUtf8() before conversion, so an invalid key id follows the same path as an unknown key: a single log line followed by an AuthenticatorException and disconnect. Both key-id log lines now render bytes with $safe(), hex-escaping control characters that an unauthenticated peer could previously write raw into the log. A CairoException catch is added around authenticator.handleIO(), which also closes a second instance of the same bug where a signature line that is not valid base64 stranded the connection, and the catch follows the severity and errno of the exception rather than forcing them. Finally, both the authenticator call and the security context factory call catch Throwable, logging it as critical, counting it as an unhandled error and disconnecting, so nothing escaping the authenticator can strand a connection. There is no configuration or protocol change and no behaviour change for well-behaved clients.

The documentation restructure moved ingestion/* to connect/*, which broke every client, Influx Line Protocol, PostgreSQL Wire Protocol, REST API and message-broker link in the English and Simplified Chinese READMEs. This fix repairs all dead documentation links in both files, reframes the Connect to QuestDB section around QWP as the native protocol with Influx Line Protocol, PostgreSQL Wire Protocol and REST presented as compatibility protocols, adds a link to the connect overview, retitles "First-party ingestion clients" to "First-party clients" since those clients now cover both ingestion and queries over QWP, and completes the deployment list with systemd, Azure, Hetzner and ZFS compression ordered to match the documentation sidebar.

ALTER TABLE ... ALTER COLUMN ... TYPE now works on tables that contain Parquet partitions. Previously, the operation supported native partitions only — running it against a table with any Parquet partition either failed or silently left the Parquet data unconverted. Conversion is lazy: Parquet partitions are not rewritten by the ALTER. Decoding cost moves to the query path, with a one-time decode per row group buffered for subsequent rows. O3 merges that land on a converted partition rewrite the partition with the new type. Conversions involving SYMBOL, and chained conversions where the Parquet column type no longer matches current metadata, eagerly convert the affected Parquet partitions to native before the ALTER. Null sentinels, BYTE/SHORT/CHAR column tops, date/timestamp scaling, and UTF-16/UTF-8 transcoding all match the native partition behaviour exactly.

This feature emits round-trippable QuestDB DDL for every user object in the database, one statement per row, similar to pg_dump --schema-only. It supports tables, views, and materialized views, ordered so that row-by-row replay always succeeds. Dependency ordering uses the view and materialized view graphs, with sibling dependencies name-sorted for deterministic output. The feature supports INCLUDE and EXCLUDE clauses to filter by category (TABLES, VIEWS, MATERIALIZED_VIEWS). Concurrent DDL during the dump no longer fails the entire operation — objects dropped between the initial snapshot and their emit are skipped with a logged warning rather than causing an error. Re-dumps are byte-idempotent.

This feature moves covered-index column decode for aggregation and filter queries off the single dispatch thread onto the async reduce-worker pool, eliminating dispatcher serialization. The covered decode now parallelizes 3–4× for decode-heavy query shapes such as SUM, multi-aggregation, first/last, residual filter, and GROUP BY. For selective predicates, the covered plan is significantly faster than a full scan, and COUNT(*) with a WHERE clause on a single symbol key is answered from posting-index metadata with no decode at all. The implementation includes concurrent-read-safe posting readers with detached per-worker cursors, a freeze mechanism before dispatch so workers never mutate shared reader state, and metadata-only single-key production using random access into the posting index. All column types including variable-size (VARCHAR, STRING, BINARY, ARRAY) are supported. Covered reads scale with selectivity — reading only matching rows' columns — providing 1.2–4.3× fewer bytes read from disk compared to full scans, with the advantage increasing on bandwidth-bound storage.

This change updates QuestDB Enterprise to compile against the updated open-source Parquet column type conversion API, which adds a SqlExecutionContext parameter to ParquetExportMode.determineExportMode().

QuestDB previously fell back to a full table scan when the designated timestamp was wrapped in any function (e.g., WHERE date_trunc('day', ts) >= '2024-01-01' or WHERE CAST(ts AS timestamp) > $1). This improvement lets the optimizer push such predicates back onto the timestamp axis when the wrapping function is monotonic. A predicate g(ts) OP bound is recognized, the queried value range is inverted layer by layer back to a range on the designated timestamp, and the result drives the usual interval scan and partition pruning. Functions are graded as EXACT (inverse is exact, row filter dropped), SUPERSET (inverse soundly contains the true preimage but may be wider, interval prunes but original predicate stays as residual filter), or NONE (cannot bound, predicate stays a plain row filter). Covered functions include date_trunc, timestamp_floor, timestamp_ceil, dateadd with fixed units, to_timezone, to_utc, precision casts, and arithmetic shifts. Named-timezone functions achieve EXACT grade when the bound provably lies inside a single DST segment, falling back to SUPERSET near DST transitions. Calendar dateadd (month/year) is always SUPERSET due to day clamping. Recognition is fail-safe: any compile or parse failure falls back to a row filter. This improvement also fixes a latent bug in getNextDST for zones whose first DST transition is after 1970 (e.g., Antarctica/Troll), adds overflow hardening for fixed-unit dateadd and timestamp_ceil inverses, and introduces domain-ceiling-aware overflow checks for constant-shift inverses on microsecond columns.

On a replica, the DataID is not generated locally but learned from the object store on the first index poll by the WalDownloader, which completes after the replication component reports READY. Because the backup scheduler sits downstream in the boot dependency graph, its start could run before the DataID existed, causing a fatal boot failure. This fix adds a bounded wait (60 seconds) in the backup scheduler startup on replicas for the DataID to be established before proceeding. The fast path where the DataID is already set does not wait at all. On timeout, the boot fails with a clear, actionable message.

This fix addresses two related bugs in the index-backed LATEST ON cursors. The first bug caused queries with an indexed SYMBOL column and a residual filter (e.g., WHERE sym = 'X' AND venue = 'Y' LATEST ON ts PARTITION BY sym) to return a row belonging to a different symbol value. The root cause was that the filtered LATEST ON cursors subtracted the page frame's partitionLo from the row id a second time, even though IndexReader.getCursor() already returns row ids relative to partitionLo. This double subtraction positioned the record too early whenever the matched row fell in a page frame with partitionLo > 0, which only occurred in partitions large enough to span multiple page frames. The redundant subtraction was removed from LatestByValueIndexedFilteredRecordCursor and LatestByValuesIndexedFilteredRecordCursor. The second bug caused duplicate static/deferred keys (e.g., sym IN ('a', :sym) with :sym = 'a') to defeat the early-exit optimization, resulting in unnecessary full scans of older partitions. The deferred key was counted twice in keyCount but the found set is deduplicated, so the early-exit condition could never be satisfied. The fix ensures resolved deferred keys are only added when not already present in the static key set, and reorders the frame loop condition to short-circuit before opening the next page frame once all keys are found.

This fix resolves an AssertionError crash that occurred when a join, GROUP BY, or DISTINCT operation used a VARCHAR key that was an empty, non-ASCII varchar. The Utf8Sequence contract allows producers to report isAscii() == false for an empty value, and when this happened, UnorderedVarcharMap packed the key hash, size, and flags into a single long that equaled 0 — the map's empty-slot sentinel — silently corrupting the hash table. The fix addresses the issue at three layers: UnorderedVarcharMap.putVarchar now forces the ASCII flag for empty keys, preventing sentinel collision; the QuestDB Wire Protocol egress request decoder now computes the ASCII flag with Utf8s.isAscii() instead of hardcoding false, so empty varchar bind variables are correctly flagged; and VarcharTypeDriver.appendPlainValue now forces the ASCII flag for empty values, preventing a header collision with the NULL/empty sentinel in the plain storage format used by RecordChain and BlockFileWriter. The column storage format is intentionally unchanged, as existing on-disk data remains valid.

The SQL compiler wraps column aliases in double quotes when the bare name contains dots (e.g., PIVOT values like FNCL 2.5 7/26, auto-generated expression aliases like t.x+1) or collides with operator tokens (in, and). These protective quotes leaked verbatim into result set metadata, causing the Web Console, PostgreSQL Wire Protocol clients, and CSV exports to display column names with literal double quotes. This fix ensures protective quotes remain compiler-internal: SqlUtil.isQuoteProtectedAlias recognizes compiler-added quotes by re-testing the quoting condition, SqlUtil.toColumnName strips them wherever a projection alias becomes a metadata column name, and SqlUtil.getColumnIndexQuiet resolves compiler-side column references with a fallback retry after stripping quotes. Additional fixes address a stale-quote leak after alias mutation (truncation or dedup suffix), an empty PIVOT value that hung the compiler in an infinite loop, operator-token pivot columns breaking through joins, dotted JSON UNNEST keys returning NULL, trailing space leaking into truncated aliases, and store reuse efficiency in alias dedup loops. Result set column names now display cleanly: FNCL 2.5 7/26 instead of "FNCL 2.5 7/26", t.x+1 instead of "t.x+1", and a.b instead of "a.b".

Previously, CREATE MATERIALIZED VIEW queries that had neither a SAMPLE BY clause nor a GROUP BY timestamp_floor(...) column threw the misleading error "TIMESTAMP column is not present in select list", even when the designated timestamp column was clearly present. The actual problem was that no sampling interval could be inferred — a materialized view requires either SAMPLE BY <interval> or GROUP BY timestamp_floor(<interval>, ts) to define the fixed bucket for incremental refresh. This fix distinguishes the two cases: when the timestamp column is present but no interval is inferred, the error message now names the column and points at SAMPLE BY or GROUP BY timestamp_floor(...) as the two supported forms; when no designated timestamp column exists at all, the original generic message is preserved.

When ff.length() returned an error on an existing .pk file during column rename, the error was silently swallowed, causing the hard-link step to fall back to a stale seal transaction number. This left the partition with a .pk chain head referencing a .pv value file that no longer existed, resulting in permanent index corruption where every indexed scan of the partition would fail. This fix now throws a CairoException.critical when length() returns a negative value on an existing .pk file, instead of returning "no chain." The legitimate short-file and absent-file paths retain their existing semantics. Both callers of the affected code path (RENAME COLUMN and CONVERT PARTITION) are covered by this fix.

Suspendable queries can migrate connections and their associated TableReader across worker threads between fragments. A posting index row cursor checked out on one worker could legitimately close on another, but an assertion added in a prior release encoded the premise that a reader is driven by one OS thread between pool acquire and release. This caused assertion failures on multi-worker HTTP parquet exports of posting covering scans, leading to leaked TableReader instances and dropped PostgreSQL Wire Protocol client connections. This fix stamps the reader's operating thread on every getCursor() call, and each cursor close() path re-pools only when the close runs on the stamping thread. An off-thread close falls through to releaseResources(), which frees only cursor-local buffers and touches no reader-shared state. The same gate was extended to bitmap and all-null index readers for consistency. Additionally, this fix corrects single-key covering-index parquet exports that produced all-null covered columns by adding RecordCursorFactory.producesMaterializedPageFrames() to ensure metadata-only producers export row-wise through the record cursor instead of the zero-copy page-frame path.

The DataID class wraps a 128-bit UUID stored as two non-volatile long fields (lo/hi). The value is published by one thread (e.g., the replication WAL downloader) and read by another (e.g., a booting backup scheduler checking isInitialized()). Because the stores and loads were unsynchronized, a reader could observe a torn half-written value or never observe the publish at all due to lack of a happens-before edge. While change() was already synchronized, initialize() and the reader methods were not. This fix synchronizes all accessors (initialize(), isInitialized(), getLo(), getHi(), and toSink()) on the DataID instance. The fix is kept in DataID rather than making Uuid fields volatile, since Uuid is a hot general-purpose value holder and the unsafe publication is specific to DataID's one-writer/other-reader pattern.

This feature introduces two replication capabilities. First, replication.disabled.tables provides a reloadable, comma-separated list of tables to exclude from replication — both the uploader and downloader skip listed tables. Second, replication-aware handling of ALTER TABLE REBASE WAL ensures a rebased table is re-baselined correctly on replicas instead of being rebuilt from an empty state. The uploader detects a rebased table via a permanent _rebase_new marker, skips its empty seed transaction, and uploads from seqTxn 2, recording first_txn=2 in index.msgpack. A replica refuses to build such a table from the object store when it is missing the baseline below first_txn, prompting an operator to copy the table and resume. The previously separate replication status checks are consolidated into a single getReplicationStatus(dirName) callback returning ACTIVE, SUSPENDED, or DISABLED. An additional reloadable switch, replication.primary.pause.upload.on.suspended.tables (default false), allows pausing uploads of WAL-apply-suspended tables. For permissions, ALTER TABLE … SUSPEND WAL and RESUME WAL are both authorized by the single RESUME WAL permission, while REBASE WAL requires SYSTEM ADMIN privilege due to its destructive nature.

This feature adds controls for managing hard-suspended WAL tables and introduces ALTER TABLE <t> REBASE WAL to rebuild a table under a fresh sequencer. The cairo.wal.apply.suspended.tables configuration provides a reloadable, comma-separated list of table names that ApplyWal2TableJob skips, preventing WAL transaction application. When cairo.wal.apply.suspended.write.denied is set to true, writes to a hard-suspended table are rejected instead of being queued. Non-structural ALTER statements and FORCE DROP PARTITION on suspended tables are routed through a WAL-bypass path and applied directly via TableWriter, while structural changes remain denied. REBASE WAL clones the applied table into a new directory via hard links into a .rebase/ staging directory, resets _txn and _meta with a new tableId, seeds two empty transactions, swaps the name registry, and drops the old directory. Preconditions require the table to be a WAL table, hard-suspended, and cairo.wal.apply.suspended.write.denied=true. Rebasing a base table invalidates dependent materialized views (recoverable with REFRESH MATERIALIZED VIEW <v> FULL), while rebasing a materialized view itself re-registers it for a full refresh. Pending unapplied WAL transactions are discarded by a rebase. For permissions, SUSPEND WAL and RESUME WAL share the same authorization path, while REBASE WAL requires system-admin privilege due to its destructive nature.

This feature introduces opt-in memory limits that cap how much native memory a single bounded workload may allocate, throwing at the offending allocation site when the cap is crossed so a runaway is stopped at its source while unrelated workloads keep running. Three independent, dynamically reloadable configuration properties control the limits: cairo.query.memory.limit.bytes for user SQL queries, cairo.mat.view.refresh.memory.limit.bytes for materialized view refreshes, and cairo.wal.apply.memory.limit.bytes for WAL apply. All default to 0 (unlimited). On breach, the engine throws a CairoException with a distinct message identifying the workload type, query ID, limit, and memory tag. A MemoryTracker wraps a 16-byte native block shared with Rust, acquired per workload via QueryRegistry.register/unregister. Coverage includes map family, sort/tree chains, hash-join chains, FastGroupByAllocator and function state, LATEST BY rowid lists and maps, set-operation maps, encoded sort, secondary joins, window/horizon-join aggregations, SAMPLE BY fill, and Parquet decode buffers. The vectorized (Rosti) keyed GROUP BY hash tables remain on the global RSS counter only. The query_activity view gains memory_used and memory_limit columns. Tracker-aware pooled memory classes release their native backing on cursor close and re-allocate on next use to ensure each malloc and its matching free are charged to the same tracker.

This feature introduces statistical aggregate functions for computing kurtosis and skewness. The implementation uses Pébay's one-pass online algorithm (an extension of Welford's) for numerical stability, maintaining running mean and central moments (M2, M3, M4) per group. Partial aggregates merge with pairwise-combine formulas, enabling parallel GROUP BY execution. Sample skewness returns null for fewer than 3 values; sample kurtosis for fewer than 4. The sample variants apply Fisher's bias correction. Population variants return null for empty groups. All variants return null when every observation is equal (zero variance). Example usage:

SELECT skewness(price), kurtosis(price)
FROM trades
WHERE symbol = 'BTC-USD'
SAMPLE BY 1h;

This improvement introduces an internal continuation runtime built on jdk.internal.vm.Continuation that allows SQL functions to yield their carrier worker thread while waiting, so the number of concurrent calls is no longer bounded by the worker pool size. The wait_wal_table(table_name [, seq_txn]) function blocks the current query until the WAL writer has applied transactions up to a target sequencer transaction number, returning true on success. It respects the SQL circuit breaker for query timeout, explicit CANCEL QUERY, and broken client connections. The sleep(seconds) function pauses the current query for a given duration (up to 24 hours) and returns a single TIMESTAMP row. Both functions release their worker carrier while parked. JDK ThreadLocal is replaced with a carrier-keyed CarrierLocal along continuation-critical paths to prevent stale thread identity across yield/resume boundaries. Two new configuration properties are available: cairo.timer.shards controls the number of daemon threads driving deadline-based wakeups, and griffin.query.continuation.wake.interval sets the millisecond interval for circuit breaker probing during park.

INSERT INTO trades VALUES (now(), 'AAPL', 100, 150.0);
SELECT wait_wal_table('trades');

This improvement extends the encoded radix sort path to every column type that ORDER BY accepts and to keys of any width, across both full-sort and serial/parallel top-K paths. Previously, only fixed-width columns fitting within 32 bytes used the fast radix sort; everything else fell back to a red-black tree sort with O(log n) comparator calls per row. The SortKeyEncoder now normalizes each sort column into a byte-comparable segment: VARCHAR uses shifted UTF-8 bytes with a terminator, STRING and non-static SYMBOL use UTF-16BE with escaping, and UUID/LONG256 use unsigned big-endian with an order-preserving null remap. Variable-width keys use a 16-byte inline prefix with overflow into a key heap, and the native kernel finishes tied partitions with pdqsort. Top-K builds entries in an EncodedTopKBuffer that rejects rows up front when their leading word is beyond the kept boundary, skipping full key encoding. Benchmarks on the ClickBench hits table show improvements from 52ms to 37ms at LIMIT 10 and from 100ms to 85ms at LIMIT 100000. The key heap is bounded by cairo.sql.sort.key.max.bytes, and the tree path remains available via cairo.sql.orderby.sort.enabled=false. This change also fixes a pre-existing bug where generateCastFunctions had no LONG128 case, causing UNION queries with both a SYMBOL column requiring casting and a LONG128 column to throw UnsupportedOperationException.

This improvement addresses persistent native memory consumption from cached parallel GROUP BY, top-K, and ASOF/window join factories. Previously, AsyncFilterContext.clear() freed page-frame memory pools but left per-worker DirectLongList row-id buffers at their peak size (up to 8 MB each at the default 1M row page frame), only freeing them on query-cache eviction. Now clear() calls resetCapacity() on the owner and per-worker lists, returning them to the 256-entry initial capacity. The tradeoff is one reallocation when a cached factory is reused for another large scan. This improvement also includes a safety fix for a native memory leak: DirectLongList.resetCapacity() could re-malloc a closed list when clear() was called after close(), which occurred in the four horizon-join factories on a failed getCursor(). The fix adds a guard to skip closed lists and reorders cursor/frameSequence cleanup in horizon-join factories to match the GROUP BY/TopK factories.

This improvement eliminates redundant per-row evaluation of subexpressions whose value is invariant across all rows of a cursor but not known at compile time. A typical example is a time-window threshold like dateadd('d', -30, to_timezone(now(), 'Asia/Kolkata')) inside a CASE branch or filter predicate, where now() is already cached but the surrounding function calls were re-evaluated on every row. A new RuntimeConstFunction wrapper evaluates the subtree once during init(), caches the result in a primitive field, and serves it from every getter. FunctionParser wraps only the maximal runtime-constant subtree at function boundaries, avoiding double-wrapping. Trivial runtime-constant leaves such as bind variables and now() that already cache their values are skipped to avoid unnecessary indirection. Only fixed-width scalar types (timestamp, long, int, double, date, boolean, ipv4, geo, uuid, etc.) are folded; variable-length types are left for future work. The wrapper delegates toPlan() to its argument, so EXPLAIN output is unchanged.

This improvement reduces redundant I/O during checkpoint and snapshot restore by collapsing per-partition Parquet metadata sidecar (_pm) processing from up to two maps and three CRC verifications down to a single map and single CRC per partition, all executed in parallel. Previously, the serial validation pass dominated restore startup on cold, object, or network storage. The recovery pool now submits one worker per thread, each pulling partitions from a shared cursor with dynamic load balancing and reusing native scratch objects across partitions. The committed-size truncation check on data.parquet runs inside the worker, removing the O(N) serial ff.length() loop. Additionally, the non-Parquet bitmap index rebuild now parallelizes across individual (partition, column) work items rather than whole partitions, so a non-partitioned table with several indexed symbol columns no longer serializes all index rebuilds onto one thread. A behavioral tradeoff is that fail-fast detection of truncated captures now occurs at the parallel drain point rather than strictly before sibling work starts, though the first failing worker trips a shared abort latch that short-circuits all other workers.

This fix addresses a bug where a primary snapshot could capture a Parquet partition whose on-disk data.parquet was a later generation than the committed Parquet size in _txn. An in-place O3 rewrite (partitionMutates=false) grows the file before _txn advances. When the replica restored that torn pair and replayed the O3 merge over the partition, the stale _pm caused the merge to decode a column chunk past the committed size, resulting in a "File out of specification: Column chunk range exceeds data length" error that suspended the replica table. This fix regenerates a partition's _pm during snapshot/checkpoint restore when the restored data.parquet is longer than the committed size.

This fix addresses an out-of-memory condition in the posting index seal and rollback paths on memory-constrained instances ingesting skewed symbol columns. The sealIncremental buffers were sized at DENSE_STRIDE (256) * preAllocPerKey, resulting in up to 256x over-allocation on skewed columns. This fix right-sizes the buffers to the actual dirty-stride aggregate, adds RSS pre-flights to both the seal and rollback paths, and streams every rollback cover shape per-key instead of decoding the whole index into one buffer. The fix also closes several latent safety issues including native out-of-bounds writes in unbounded posting decoders on corrupt or torn input, and a seal-purge reuse-race that could delete a live file. The tradeoff is that the incremental-seal pre-flight is conservative and may defer to the slower full seal when the incremental path would have just fit, and the streaming rollback re-derives each surviving key rather than holding the whole index, trading per-key CPU on the rare rollback path for a peak-memory bound.

A snapshot could capture a Parquet partition whose on-disk data.parquet was a later generation than the committed Parquet size recorded in _txn. The restore path trusted the partition's existing _pm sidecar in that state and left it stale, causing the first read or merge after restore to fail with a "Column chunk range exceeds data length" error. On a replica replaying the WAL over the restored partition, this suspended the table and broke replication. The root cause was that the validation only checked whether the sidecar could resolve a footer at the committed size, but a _pm from the later generation still resolved successfully despite being stale. This fix now only trusts an existing _pm when data.parquet is exactly the committed size. When the file is longer (indicating the snapshot captured the partition mid in-place rewrite), the _pm is regenerated from the committed size, which produces a correct compact sidecar for the committed footer.

This fix addresses a production out-of-memory incident on a memory-constrained instance ingesting a skewed symbol column. The incremental seal path (sealIncremental) sized its per-stride merge and trial buffers as if all 256 keys in a stride held the single hottest key's row count, inflating allocation by up to ~256x beyond actual data. The buffers are now sized to the actual aggregate of dirty strides computed from per-key counts, and a pre-flight defers to the full seal (which streams per key) when the correctly sized buffers would still breach the RSS limit. The rollback path previously decoded the entire partition index into one buffer before filtering, which was the exact memory-heavy case that triggered the incident. Rollback now streams per key for every cutoff, bounded by the largest single key rather than the whole index. All covering index shapes (fixed-size, var-size, and addr-based) now stream their sidecar rebuilds. Error handling was hardened across seal and rollback paths: failures after the value-file switch poison the writer so all mutating entry points reject further operations, while failures before the switch clean up staged files and restore writer state. Several latent safety bugs were also fixed, including unbounded native writes in posting decoders on corrupt input (now rejected with a clean error) and a seal-purge reuse race that could delete a live file after a sealTxn was freed and republished.

This fix addresses a remaining out-of-memory path in the posting index incremental seal that was not covered by the prior streaming-fallback RSS pre-flight. The incremental seal snapshots each covered column's entire sealed sidecar into native memory before the stride-merge pre-flight runs, so a skewed symbol with a covered (INCLUDE) column could still trigger an unguarded multi-gigabyte malloc that exceeded the global RSS limit during a WAL fast-lag commit. The fix adds the same headroom pre-flight before the snapshot copy, re-reading RSS usage per cover column to account for snapshots already taken. When the copy would not fit the live headroom, the incremental path is abandoned and the full seal is used instead, which rebuilds every sidecar by streaming per-key from the column files with peak memory bounded by the largest single key rather than the entire sidecar.

This fix resolves an intermittent NullPointerException in CoveringIndexRecordCursorFactory.getCursor that occurred when multiple queries ran concurrently, such as all panels of a Grafana dashboard firing at once over PostgreSQL Wire Protocol. The factory stored a direct reference to the compiler's pooled keyValueFuncs list from IntrinsicModel. Since SQL compilers are pooled and shared across threads, a second thread borrowing the same compiler would clear that shared list (via IntrinsicModel.clear()), nulling out slots while the first thread's factory was still reading them during getCursor(). The fix creates an owned copy of the list in the constructor, preserving the same Function instances which the factory already owns and frees on close. This restores the contract already honored by FilterOnValuesRecordCursorFactory, whose key-value list parameter is consumed at construction rather than retained.

Equality and IN filters on a renamed column of a Parquet partition could return too few rows (often zero), silently dropping valid data. The defect was in Parquet row-group bloom-filter pushdown, which resolved the filtered column against the Parquet file's column names. Those names are frozen when the partition is converted to Parquet, so a later RENAME COLUMN left them stale. When another column already carried the query's current name (common after a chain of renames), the pushdown landed on the wrong Parquet column, checked that column's bloom filter, got a false negative, and skipped the entire row group. This fix makes the native-table pushdown resolve columns by stable column id, the same way the data-decode path already does. read_parquet() cursors retain name-based resolution, which is the correct semantics for arbitrary external files without QuestDB column ids. Pruning effectiveness is unchanged for the common non-renamed case, and the bloom filter format, statistics, and decode paths are untouched.

Many query factories only consulted the execution context's circuit breaker inside their row-processing loop, which never runs for queries that produce no rows (empty tables, no-match filters, empty joins/aggregates), for instant single-row results, or for parallel paths that dispatch nothing when the input is empty. This fix ensures every affected factory consults the breaker at the point its work starts: at the top of hasNext() before the empty/advance guard, before build/aggregate loops, at cursor open for shared empty-cursor singletons, and before dispatchAndAwait() on parallel paths. A new time-throttled breaker variant always tests cancellation and timeout (both cheap, no syscall) and throttles only the heavy connection probe syscall by elapsed wall-clock time, defaulting to a 100 ms window. The throttle state lives on the breaker shared across the whole query, so a CROSS JOIN that re-scans the slave once per master row issues at most one probe per window for the entire query. Every query now pays for one real breaker check on its first consultation, including a single connection-probe syscall. Queries that previously ran to completion under an aggressive timeout can now abort as expected, including catalogue and admin listings.

A SAMPLE BY query using first() or last() aggregation with an indexed SYMBOL column filter and ALIGN TO FIRST OBSERVATION could return a bucket timestamp where an aggregated value was expected. The wrong results appeared only when the designated timestamp column was omitted from the SELECT list, and only for buckets in the middle of the result set — the first and last buckets were always correct. The root cause was that SampleByFirstLastRecordCursorFactory emitted middle rows through a data record that special-cased the bucket-timestamp column using the base-table timestamp index, while the column it was handed is a projection index. The two indexes coincide only when the timestamp is projected at the same position it occupies in the base scan. When the timestamp was not projected, the base index pointed at an unrelated projected column and overwrote it with the bucket timestamp. This fix makes the data record use the same projection index as the boundary record, so when the timestamp is not projected, no column matches the special case and every aggregate reports its stored value.

Subtracting a CHAR or STRING literal from a timestamp, such as SELECT * FROM t WHERE timestamp > now() - '1' LIMIT 1, failed with an internal error. The literal was implicitly bound to the LONG operand slot of the timestamp subtraction operator, but the factory read the right operand with getTimestamp() instead of getLong(). Calling getTimestamp() on a CharFunction throws UnsupportedOperationException, which surfaced as an internal error. This fix changes SubTimestampFunctionFactory to read the right operand with getLong(), mirroring the symmetric AddLongToTimestampFunctionFactory which already did this correctly. As a result, - and + now produce identical, well-typed cast errors for non-numeric literals. For example, now() - '3 day' now reports inconvertible value: '3 day' [STRING -> LONG] instead of the previous [STRING -> TIMESTAMP].

Two malformed expressions involving CASE ... END crashed with internal errors instead of returning syntax errors. A binary operator with a missing right operand directly after CASE ... END (e.g., select sum(case when true then 1 else 0 end & )) caused a NullPointerException because the expression parser's END keyword handler left a stale depth counter that defeated the arity guard, allowing a tree node with a null left-hand side to be built. This fix resets the depth counter to 0 after the flush loop so the arity guard fires correctly, producing the error too few arguments for '&' [found=1,expected=2]. A dot directly after CASE ... END (e.g., select case when true then 1 else 0 end.foo) caused either a NullPointerException or ClassCastException because the dot handler assumed a gluable literal token on the operator stack. This fix extends the token only when the stack top is a literal; otherwise it raises the syntax error '.' is unexpected here. Valid CASE expressions and qualified table.column references are unaffected by both fixes.

This fix addresses a storage-engine corruption that occurred when a WAL REPLACE_RANGE commit in O3 mode appended brand-new partitions above the table's previous last partition. The root cause was in TableWriter.processO3Block, where replace mode derived partitionTimestampHi from o3TimestampMax (the replace-range high boundary) rather than the highest timestamp actually written. For an open-ended range, the boundary is Long.MAX_VALUE - 1, causing getCurrentPartitionMaxTimestamp to overflow. As a result, partitionTimestampHi stayed at the stale pre-commit ceiling, finishO3Commit skipped switching the writer's active column files to the new last partition, and the next commit reused the previous partition's column descriptors, producing rows dated below the new partition's floor or suspending the table. This fix derives partitionTimestampHi from the actual highest written timestamp (replaceMaxTimestamp) in replace mode, leaving the non-replace path unchanged.

This fix resolves a data-ordering bug where a non-WAL writer ingesting out-of-order data could produce out-of-order rows when the in-order prefix crossed a partition boundary. The partition switch sealed the earlier partition in memory but did not persist _txn. On the next lag commit, o3MoveUncommitted reclaimed the active partition's uncommitted in-order tail into the O3 buffer and reset maxTimestamp to the durable _txn value, which predated the switch. The high-water mark ended up below committed data, causing a rollback to reload it and the next O3 commit to reorder the partition tail. This fix ensures maxTimestamp resets to the max timestamp of the last sealed partition by introducing a new variable that is updated whenever a partition switch happens.

This fix addresses a data-ordering bug in the non-WAL out-of-order (O3) commit path where a lag commit (TableWriter.ic()) could leave the table's maxTimestamp below the maximum timestamp actually committed to disk. When uncommitted rows of a lag commit spanned more than the active partition, o3MoveUncommitted() pulled only the active partition's rows into the sorted O3 batch and emptied it. The in-order rows left on disk in the previous partition stayed committed but were not part of the sorted O3 batch, so maxTimestamp was recomputed solely from the O3 batch boundary and could end up lower than the true on-disk maximum. A later O3 commit then merged against that stale boundary and left a single timestamp inversion. This fix reads the earlier partition's actual max timestamp via readPartitionMinMaxTimestamps() and folds it into the committed maxTimestamp computation when the active partition is emptied while uncommitted rows remain in an earlier partition.

This fix resolves a non-deterministic metadata reload failure where MemoryCMRImpl.of() read errno after a cleanup close() call when a file's length could not be read. Since errno is thread-local global state, the intervening close() could overwrite the errno left by the failed length() call. The exception then carried the cleanup call's errno instead of the real one. This mattered because metadata reload classifies failures by errno: TableUtils.handleMetadataLoadException() retries the read only while CairoException.isFileCannotRead() is true. With a clobbered errno, a transient "file does not exist" condition — such as a _meta file briefly unreadable during a concurrent metadata change — was misclassified as fatal, surfacing a "could not get length" error instead of retrying. This fix captures errno immediately after the failed length() call, before close() runs.

This fix encompasses a broad set of query engine hardening changes. Key fixes include: SAMPLE BY FILL(LINEAR) cleanup no longer crashes with NullPointerException on out-of-memory; covering posting index readers now propagate sidecar I/O errors instead of swallowing them; IntervalBwdPartitionFrameCursor.calculateSize() now returns correct counts for backward interval scans with open lower bounds; DATE-argument window value functions (max, min, first_value, last_value, nth_value, lag, lead) no longer throw UnsupportedOperationException; rank() / dense_rank() no longer crash when pass-through columns of unserializable types (UUID, STRING, VARCHAR, LONG256, arrays) are projected; lead() over high-precision DECIMAL no longer crashes; window RANGE-frame functions now return correct values when the designated timestamp is not in the projection; sum() / avg() over Decimal256 now produce correct results over sliding frames by using scale-agnostic subtraction for eviction; JIT filter now correctly widens nested INT products to 64 bits when a FLOAT operand is present in the predicate; LimitedSizeLongTreeChain cleanup no longer crashes on partial construction failure; keyed ASOF JOIN sink-heap no longer leaks on out-of-memory during cursor open; vectorized rosti GROUP BY no longer over-frees NATIVE_ROSTI memory when wrapUp() resizes the map; stale aggregate tasks in the shared vector-aggregate queue no longer crash later queries after a fault aborts a cursor drain; GROUP BY keys referencing aliases of trivial arithmetic expressions now compile correctly; multi-key index scan no longer leaks posting-index block buffers on out-of-memory; and window functions over untyped null literals now raise a clean SqlException suggesting a concrete cast instead of behaving non-deterministically across platforms.

This fix addresses two distinct faults where per-partition code paths iterating posting columns were guarded only against the absent case (columnTop == -1), not the row-less case (columnTop >= partition size). In the first fault, restorePostingIndexersToLastPartition() would crash with "index does not exist" when encountering a row-less column with no .pk key file, causing WAL table suspension or INSERT failures on BYPASS WAL tables. The fix discriminates on the key file's actual presence rather than row-less-ness, correctly re-pointing indexers for row-less columns that do have a .pk (created via ADD COLUMN ... INDEX TYPE POSTING on the active partition) while skipping those without one. In the second fault, linkPartitionIndexFiles() would crash with "index files do not exist" when switching a partition to parquet and encountering a row-less column without a key file. The fix aligns the guard with its sibling copyOrRebuildColumnIndexes() to skip row-less columns on historic partitions where a .pk is never built. The row-less-without-.pk state arises from multi-partition, same-transaction out-of-order writes interleaved with ADD COLUMN. Without the first fix, a blanket row-less skip would silently strand indexers, making rows invisible to indexed predicates with no exception or suspension. The accessor was also changed from getColumnTopQuick to getColumnTop to correctly return -1 for absent columns.

This fix resolves a production bug where a query cancellation could be silently dropped, and the affected query's per-query timeout defeated along with it, when the cancel races query startup. The root cause was that NetworkSqlExecutionCircuitBreaker.cancel() always sets a powerUpTime == Long.MIN_VALUE sentinel but only flips the per-query cancelled flag when that flag is already attached. QueryRegistry.register() publishes the query and fires its listener before binding the per-query flag, so a CancelRequest landing in that window sets only the sentinel. The subsequent register() call then binds a fresh flag with value false, and neither testTimeout() nor testCancelled() consulted the sentinel — testTimeout() would compute an overflowed negative runtime that never trips, and testCancelled() only checked the flag. The query would run to natural completion ignoring both cancel and timeout, pinning workers. The fix makes testCancelled() check the sentinel first so both stateful paths abort a racing cancel, updates getState() to classify the sentinel as STATE_CANCELLED instead of mislabelling it STATE_TIMEOUT, and adds a clearCancelSentinel() call in PGConnectionContext.prepareForNewQuery() to bound the sentinel to a single query and prevent leaking across requests.

This fix addresses several issues where HTTP parser and response objects were not fully resetting their per-request state when connections and contexts were reused across requests. A reused context could hit closed native parsing or compression buffers and crash the server when handling otherwise valid requests. Specifically, the fix reopens the header parser's quoted-value sink when pooled HTTP contexts are reused, clears per-request gzip negotiation state in HttpResponseSink so clients only receive gzip responses when the current request advertises gzip support, clears charset and mapped cookie state in HttpHeaderParser.clear(), fixes multipart false-boundary replay so body bytes are preserved when a boundary-like sequence is followed by non-boundary suffixes, keeps multipart resume pointers anchored to the receive buffer during retry paths, parses Content-Disposition parameters with quote-aware delimiter handling including semicolons and equals signs inside quoted values, and resets Content-Disposition parameter state consistently after known and unknown parameters. This allows uploaded filenames such as a;b.csv and a=b;c.csv to parse correctly.

This fix addresses numerical-stability bugs in the corr() function where the Pearson denominator sqrt(sumXX * sumYY) can overflow or underflow while computing the product of two sums of squared deviations. With large-magnitude inputs (values near +/-1e153), each sum is finite (~1e306) but their product overflows to +Infinity, causing the final division to return 0.0 instead of the true correlation. With small-magnitude inputs (values near +/-1e-150), each sum is finite (~1e-300) but their product underflows to 0.0, causing the division to return NaN. The fix prefers the single-rounding sqrt(a * b) denominator when the product is finite and non-zero, preserving existing bit-exact behavior for normal inputs, and falls back to sqrt(a) * sqrt(b) when the product would overflow or underflow while both factors are non-zero. The final Pearson result is clamped to [-1, 1] to absorb small rounding drift in the fallback path. This applies to CorrGroupByFunctionFactory, AbstractBivariateStatWindowFunctionFactory.computeCorr, and AbstractBivariateStatWindowFunctionFactory.computeCorrWelford. Normal-magnitude inputs are unaffected and keep their prior bit-exact results.