QuestDB Enterprise 4.1: governance for shared clusters

A QuestDB Enterprise cluster rarely stays with the team that brought it in. The trading desk starts with tick data, then risk wants the same history, quants and analysts start exploring it, a reporting job follows, every new application gets its own service account, and sooner or later compliance wants to know who has been reading what.

That's a good problem to have, and QuestDB Enterprise 4.1 is built for it.

Resource groups keep ad hoc work from slowing down the queries that run all day. Quants and analysts exploring a year of data get their own slice of the instance: how many queries they can run at once, how much CPU they get while others are busy, and how much memory their queries may hold. The dashboards and services next to them keep answering in milliseconds.

Audited views record every read of a sensitive view, with the principal and the exact parameter values the read ran with, in a table you query with SQL.

Object visibility shows each user only the tables and views they hold a permission on, and active_permissions() returns the whole access model in one query.

OIDC for services lets applications sign in with tokens from your identity provider, the same way your people do. On Azure, that includes Entra ID managed identities and service principals.

Replication, cold storage and certificate health are now recorded in sys.metrics, and cold storage can keep data on a node's disk for longer.

The release is built on QuestDB 10.1, so it also brings its highlights. Merge-append partitions write late data with much less disk I/O and make it readable sooner. SUBSAMPLE keeps the rows that shape a series, so a chart shows every spike with a fraction of the points. There are also exact percentiles, and GROUP BY over joins now runs on every core.

That's QuestDB Enterprise 4.1 in brief. If it's all you need, read before you upgrade, which includes one replication step some deployments need, and go straight to getting the update. Otherwise, the sections below cover each feature in more detail, with the SQL to try it.

What QuestDB 10.1 brings with it

QuestDB Enterprise 4.1 is built on QuestDB 10.1, our latest open source release, so everything in 10.1 is in QuestDB Enterprise too. The QuestDB 10.1 post has the detail. In short:

• Merge-append partitions: late rows are appended instead of rewriting the partition, so there's much less disk I/O and new rows are readable sooner. It's opt-in.
• SUBSAMPLE: picks the most significant rows of a result, such as the few hundred that keep a chart's shape out of millions.
• Exact percentiles: percentile_disc() and percentile_cont().
• Metrics history: QuestDB writes its own metrics to the sys.metrics table every second. QuestDB Enterprise adds its own columns.
• Lists in variables: a DECLARE variable can hold a list, which audited views use for their parameters.

QuestDB 10.1 also runs GROUP BY over joins on every core, speeds up LATEST ON, and improves HORIZON JOIN, in ways that can change the results of existing queries.

Many teams, one instance: resource groups and memory limits

A quant runs an exploratory query over a year of trades joined to the quotes, grouped several ways. On the same instance, dashboards and services run small queries every second, and while the quant's query runs, those go from milliseconds to seconds.

A resource group is a named policy for what a set of users, ACL groups or service accounts may consume while their queries run. It sets three limits, each with a different guarantee:

• Admission is how many queries a group can run at once, how many can wait, and for how long. It's an exact limit.
• CPU weight is the share of query CPU a group gets while groups compete. It's a share, not a reservation.
• Memory is a budget for the native memory the group's queries hold at once. It's approximate at the margin.

CREATE RESOURCE GROUP dashboards WITH (cpu_weight = 200);

CREATE RESOURCE GROUP adhoc WITH (
    cpu_weight = 50,
    max_active_queries = 4,
    max_queued_queries = 32,
    queue_timeout = '15s',
    memory_limit = '8G'
);

ALTER GROUP trading SET RESOURCE GROUP dashboards MAPPING PRIORITY 20;
ALTER SERVICE ACCOUNT pricing_service SET RESOURCE GROUP dashboards;
ALTER GROUP quants SET RESOURCE GROUP adhoc MAPPING PRIORITY 10;
ALTER GROUP analysts SET RESOURCE GROUP adhoc MAPPING PRIORITY 10;

While both groups are busy, dashboards and services get four times the CPU that ad hoc queries do. Ad hoc work runs at most four queries at a time, queues another 32, and fails any that has waited more than 15 seconds. Everything it runs at once has to fit in 8 GiB.

The CPU share only applies when there's someone to share with. If an analyst is the only person running queries, they get every core on the instance, resource group or not. The concurrency and memory limits are different: they're hard caps, and they hold on a quiet instance too, so set them for the largest query you're happy for one group to run.

A quant who also sits in trading gets the higher priority mapping, and anything not mapped runs in DEFAULT, which has no limits of its own.

The operations guide has six common scenarios as policies you can copy.

Memory limits for a user, a team or a service

QuestDB Enterprise 4.0 added a per-query memory limit, one value for the whole instance. In 4.1 it can also be set on a user, an ACL group or a service account, separately from resource groups. A resource group's memory_limit is a budget for everything the group runs at once. This one caps each single query from that principal.

ALTER USER intern SET MEMORY LIMIT 512M;
ALTER GROUP quants SET MEMORY LIMIT 4G;
ALTER SERVICE ACCOUNT pricing_service SET MEMORY LIMIT 1G;

The most specific limit wins, even when it's larger than the instance default, so the quants can run bigger queries than everyone else. The resource group's budget can still stop a query that would push the group past it. The details are under memory limits.

Audited views: who read what, and with which parameters

Compliance asks who looked at one client's positions last quarter, and for which dates. A list of query texts only gets you halfway there. A statement with bind variables, or a window that starts at now(), doesn't say on its own which data it covered.

An audited view records a row in sys.view_audit for every read: who ran it, when it finished, how long it took, whether it succeeded, and the values its parameters resolved to.

The parameters are DECLARE variables in the view's body. OVERRIDABLE lets the caller set one, AUDITED records the value it resolved to, and since QuestDB 10.1 a variable can hold a list, so a caller can ask for a set of symbols in one read.

CREATE VIEW fx_trades_by_symbol AS (
  DECLARE
    OVERRIDABLE AUDITED @symbols := ('EURUSD', 'GBPUSD'),
    AUDITED @since := dateadd('h', -1, now())
  SELECT timestamp, symbol, side, price, quantity, counterparty
  FROM fx_trades
  WHERE symbol IN @symbols AND timestamp >= @since
) WITH AUDIT;

Grant your analysts the view and not the table, and every read they make goes through it. A read that sets its own symbols is recorded with the symbols it asked for, and with the time @since resolved to on that execution:

DECLARE @symbols := ('USDJPY',)
SELECT * FROM fx_trades_by_symbol;

SELECT ts, principal, view_name, params, status
FROM 'sys.view_audit'
WHERE view_name = 'fx_trades_by_symbol'
ORDER BY ts DESC;

The analyst's read comes back as a row with principal set to analyst, status set to ok, and these parameters:

{"since":"2026-09-19T09:15:02.417950Z","symbols":["USDJPY"]}

Each node records the reads it serves, replicas included, in its own sys.view_audit, and the rows aren't replicated, so query every node that serves reads. What the trail records and what it leaves out, retention, tamper protection and permissions are all in the audited views docs.

Users see only the tables they can use

A new analyst connects a BI tool, and its table browser lists every table in the database, including the many they can't query.

From QuestDB Enterprise 4.1, users see only the tables and views they hold a permission on. That covers tables(), SHOW TABLES, information_schema, pg_catalog, and whatever a JDBC driver or BI tool asks for to fill its browser. Any permission on an object makes it visible, including a grant on a single column or one ON ALL TABLES. Database-level permissions such as CREATE TABLE don't. Administrators see everything.

This is on by default, and it changes an error message, so read this part before the upgrade. A statement that names a table the user can't see now behaves as if the table doesn't exist: table does not exist instead of Access denied, and HTTP 400 instead of 403. Tests or client code that match on "Access denied" need updating. Object visibility has the full rules.

The whole ACL in one query

Two new functions return the access model as rows:

SELECT entity_name, entity_type, permission, column_name
FROM active_permissions()
WHERE table_name = 'trades';

active_permissions() includes what each user inherits from their groups, and active_grants() lists only direct grants. Both cover users, groups and service accounts, but not external SSO identities, whose groups come from the identity provider. Reading either needs LIST USERS and USER DETAILS.

OIDC for services and managed identities

A new service needs to write to QuestDB. Someone creates a service account, generates a password and stores it in a secrets manager, and from then on somebody has to rotate it.

If you run on Azure, that service can now sign in the way your people do, with a token from Entra ID. The claim settings take fallback lists, so one configuration accepts user tokens and app-only tokens from managed identities and service principals:

acl.oidc.sub.claim=preferred_username,oid
acl.oidc.groups.claim=roles,groups

A person is identified by preferred_username and gets their groups from groups. An application has neither, so it's identified by oid and gets its groups from the app roles it was assigned. Grants work the same either way. On the client side, the Java client from 1.3.11 requests and renews tokens itself, so nothing in your code handles them.

One limitation for now: a client that signs in through SSO needs its tables and columns to exist already, since external users can't create them on the fly.

OIDC is also stricter in three ways. Tokens have to come from the issuer you expect, taken from the discovery document or pinned with acl.oidc.issuer. Expired ID tokens are rejected. And an HTTP session now ends as soon as its credential is rejected, revoked or changed, so dropping a user's token or disabling the user takes effect on their open session too. See OIDC for the setup.

Replication, cold storage and certificates in sys.metrics

QuestDB Enterprise adds the sys.metrics columns an operator of a replicated cluster looks at first: how far replication uploads and downloads are behind, how many tables are failing, how many queries are waiting on data from cold storage, and how many seconds each TLS certificate has left.

SELECT ts,
       max(replication_up_lag_txn_max) AS upload_lag_txn,
       max(replication_up_tables_failing) AS tables_failing,
       max(cold_read_parked) AS cold_reads_waiting,
       min(tls_cert_ttl_seconds_http) / 86400 AS cert_days_left
FROM 'sys.metrics'
WHERE ts IN '$now - 1d..$now'
SAMPLE BY 1h;

It's ordinary SQL, so it goes in a notebook cell that refreshes on its own, and a certificate that's about to expire shows up on the same screen as everything else. Each node records its own metrics and the table isn't replicated, so ask the node you care about. Persisted metrics lists the settings.

Cold storage: longer local retention and non-blocking cold reads

Cold storage moves older partitions to object storage on a schedule set by the table's storage policy, and every node follows the same schedule. If one node needs to keep data on local disk for longer, for example a replica that serves the quarterly reporting, it can now hold partitions past the policy. Contact us if you'd like it enabled.

Cold reads also share the workers better. A query waiting for a chunk to come back from the bucket used to hold its worker thread while it waited. It now steps aside, and the worker runs other queries until the data arrives.

Two smaller changes: Azure Blob storage connects through a managed identity without an account_key, and a cold storage manager whose role switch fails can retry without a restart.

Two new connectors: Databento and dbt

Neither is strictly part of this release, and both work from QuestDB Enterprise 4.0 onwards. The Databento connector streams US equity quotes and trades into QuestDB from one config file. A dbt adapter is coming soon.

Before you upgrade to QuestDB Enterprise 4.1

Most of these need nothing from you, but read the list once. QuestDB Enterprise 4.1 also includes the breaking changes in QuestDB 10.1, so read those too.

One replication step, for some deployments. The replication.primary.upload.truncated setting is gone. If your primary never set it to false, there's nothing to do: upgrade the replicas, then the primary. If it did, before upgrading anything:

1. Remove the setting from the primary.
2. Restart the primary, still on the old version.
3. Wait until every replica has caught up.
4. Upgrade the replicas, then the primary.

Hidden objects report "does not exist". Covered above. Update anything that matches on "Access denied".

Stricter OIDC and sessions. Expired ID tokens are rejected, the token issuer has to match the discovery document or acl.oidc.issuer, and a session ends when its credential changes. If you configure the OIDC endpoints by hand, set acl.oidc.issuer. A client that passes a hard-coded ID token will stop working once that token expires.

Administrative functions stay out of views. Materialized and live views can no longer call functions such as active_permissions() or backups(), and catalogue functions like tables() in them need SYSTEM ADMIN. A materialized view that reaches one through a regular view is invalidated on its first refresh after the upgrade.

New columns. SHOW USERS, SHOW GROUPS and SHOW SERVICE ACCOUNTS gain memory_limit and resource_group, and SHOW GROUPS also gains resource_group_priority. Code that reads them by position needs updating.

Cold storage needs conditional requests. The object store has to support them, including If-Match on S3 DeleteObject. Credential discovery on S3 and GCS has changed, and some rarely used root certificates are no longer bundled. The upgrade notes list each one.

Statements that would suspend a table now fail up front. UPDATE on a WAL table with Parquet partitions, and some schema changes on tables with cold partitions, are rejected at compile time rather than suspending the table when applied.

Getting the update

QuestDB Enterprise 4.1 also includes many bug fixes across replication, backups, cold storage and access control. See the release notes page for the details.

Self-managed enterprise customers will find the binaries at the usual download location. BYOC enterprise customers will be contacted for upgrading.

Upgrade replicas before the primary, and if your primary sets replication.primary.upload.truncated=false, follow the steps above first.

Not on QuestDB Enterprise yet? Learn more about QuestDB Enterprise and BYOC, or contact the QuestDB team for a conversation or a demo.