QuestDB Enterprise 4.2.1: improved power-loss durability, ACL audit log

QuestDB Enterprise 4.2.1 brings improved power-loss durability to replicated clusters, records every access control change in an audit log, copies users and grants between clusters, and protects audit tables.

Javier Ramirez
Javier RamirezFast Data Advocate
QuestDB Enterprise is the time-series database behind trading floors, power grids and mission control. This release brings improved power-loss durability to replicated clusters, and keeps a record of every access control change.

QuestDB Enterprise 4.2.1 brings the improved power-loss durability mode to replicated clusters. It also covers what happens around the access control decisions 4.1 introduced: finding out who changed a permission and when, keeping audit records out of reach of the people they record, and setting up another cluster with the same users and grants.

Improved power-loss durability on a replicated cluster protects a primary's acknowledged commits from an operating system crash or a power loss for much less than sync mode costs. Clients can wait until a batch is on the primary's disk, in the object store, or both.

The ACL audit log records every change to users, groups, service accounts and grants in a table you query with SQL, with who made the change and the statement that made it.

Protected audit tables: nobody can write rows into the audit tables, and only the built-in admin can truncate them or change their retention.

Access control in SHOW CREATE DATABASE: a database dump now carries users, groups and grants, so a staging cluster can start with production's access control.

The release is built on QuestDB 10.2, so it also brings GROUPING SETS, passthrough views with row-level retention, and a deduplication mode that keeps the first copy of a row. If you only need the summary, read before you upgrade, which has one object storage requirement to check, and go to getting the update.


Improved power-loss durability on a replicated cluster

By default, QuestDB doesn't wait for the disk before acknowledging a commit, so an operating system crash or a power loss on the primary can lose the latest commits and leave table files damaged. Sync mode already prevents both, by waiting for the disk on every commit and every WAL apply, but it makes commits much slower. QuestDB Enterprise 4.2.1 adds adaptive durability mode, which protects WAL tables from both for much less. Set it on every node that can become primary:

server.conf
cairo.commit.mode=adaptive

Each commit flushes only its own write-ahead log files, and the table files are flushed once a minute. After a crash, the primary rebuilds them from the write-ahead log and restarts with every table consistent.

  • Durable acknowledgements. QWP clients can already wait until a batch reaches the object store, with request_durable_ack=on. They can now also ask for local, on the primary's disk, or local,replicated for both, and the client keeps each batch until it's confirmed.
  • Replicas take no save points, because the object store holds their durable copy. A primary in this mode uploads a transaction only once it's on its own disk, so replicas see the newest commits up to about a second later than before.
  • Role switches flush once when a node is promoted. Demoting a primary flushes each of its tables, about 20 ms per table on Linux.

It still costs some commit latency, and it doesn't cover non-WAL tables or the loss of the disk itself. The durability docs have the details for replicated clusters.


Who changed that grant? The ACL audit log

An auditor asks who gave a contractor access to a production table in March, and who has changed service account credentials since. sys.acl_audit_log has a row for every access control change: users, groups, service accounts, memberships, grants and revokes, passwords, tokens, memory limits and resource group assignments. Each row names the principal that signed in, the principal whose permissions it used, such as an assumed service account, whether the identity came from OIDC, and the statement, with passwords and tokens masked:

Access control changes in the last week
SELECT change_id, session_principal, effective_principal,
is_external, statement_kind, statement_text
FROM 'sys.acl_audit_log'
WHERE change_id IN '$now - 7d..$now'
ORDER BY change_id DESC;

To give an auditor access, grant LIST USERS and USER DETAILS as well as SELECT on the log, since the statements name identities and grants:

Let an auditor read the log
GRANT LIST USERS, USER DETAILS TO auditor;
GRANT SELECT ON 'sys.acl_audit_log' TO auditor;

The log replicates, so every node has it. Its default storage policy, TO PARQUET 1d, never deletes a row, and acl.audit.storage.policy sets another before the log is first created. It doesn't record sign-ins or ordinary queries, and changes made before the upgrade aren't reconstructed.


Audit tables only the built-in admin can change

The ACL audit log, the sys.view_audit trail of audited views, and sys.sp_entries and sys.sp_links, where storage policies are recorded, are now system-managed tables. Reading them works as before, but:

  • Nobody can write their rows, the built-in admin included. Only the server writes them.
  • Only the built-in admin can truncate them, drop their partitions, or change their TTL or storage policy. Grants ON ALL TABLES, DATABASE ADMIN and SYSTEM ADMIN no longer reach these tables.
  • Nobody can drop, rename or restructure the ACL audit log, or skip transactions in its write-ahead log.

That means granting someone TRUNCATE TABLE on every table no longer lets them empty an audit trail. If your deployment disables the built-in admin, nobody can change these tables at all, so set their retention before upgrading. See before you upgrade.


Copying users, groups and grants with SHOW CREATE DATABASE

A team sets up a staging cluster and wants it to have production's users, groups and permissions, so that a query that fails in production fails there too. SHOW CREATE DATABASE now dumps users, groups, service accounts, memberships and grants along with the schema and table storage policies, in the order a replay needs them:

Dump everything, or choose what to copy
SHOW CREATE DATABASE;
SHOW CREATE DATABASE INCLUDE (SCHEMA);
SHOW CREATE DATABASE INCLUDE (ACL);

The full dump and INCLUDE (ACL) need LIST USERS and USER DETAILS. INCLUDE (SCHEMA) needs neither, which makes it the one for seeding a development instance without production's users. Credentials, memory limits and resource groups aren't dumped, so set them on the new cluster before opening it to users. The copy schema recipe shows how to replay a dump.


What QuestDB 10.2 brings with it

QuestDB Enterprise 4.2.1 is built on QuestDB 10.2, our latest open source release, so everything in 10.2 is in QuestDB Enterprise too. The QuestDB 10.2 post has the detail. In short:

  • GROUPING SETS, ROLLUP and CUBE: detail rows, subtotals and a grand total from one query and one scan.
  • Passthrough views and EXPIRE ROWS: a materialized view can be a filtered copy of a table, keeping only the rows you need.
  • DEDUP SKIP: keeps the first row written for a key and discards later copies, replicas included.
  • Faster small QWP messages: one-row messages ingest at about 3.9 million rows per second, up from about 210,000.

Other changes

  • Backups and replication. A single transient object store error no longer fails a backup, and a primary no longer halts with a false ER006 after object store timeouts. Backup progress now follows partition size instead of stalling on a large partition, slow compressions are logged, and WAL pruning no longer gets stuck on tables whose names hold control characters.
  • Restores. Covering indexes no longer return NULLs after a backup restore, and tables with POSTING indexes are no longer suspended after a checkpoint restore taken during ingestion.
  • Cold storage. TO PARQUET storage policies now convert a whole partition into one Parquet file in a single pass, in the partition compaction job.
  • jemalloc. Linux x86-64 packages and the Docker image now start QuestDB with the bundled jemalloc allocator. Set QDB_JEMALLOC=false to opt out, or QDB_JEMALLOC=true to opt in on Linux ARM64.
  • OIDC. Public key reloads are rate-limited, cached logins no longer accept lookalike non-ASCII passwords, and provider response bodies are no longer logged.
  • Fixes in REST token redaction when a token is dropped, and in VACUUM TABLE, which could delete column files of a different table, among others.

Before you upgrade to QuestDB Enterprise 4.2.1

QuestDB Enterprise 4.2.1 also includes the breaking changes in QuestDB 10.2, so read those too.

  • S3-compatible object stores must support or ignore If-Match and If-None-Match on PutObject. On a store that rejects them, a primary can't start or be promoted, and every backup fails. AWS S3, Azure Blob Storage and Google Cloud Storage support them. See S3-compatible stores.
  • Audit and storage policy tables. Scripts and jobs that truncate sys.view_audit or change its retention must run as the built-in admin. If you disable the built-in admin, set the trail's retention before upgrading.
  • ACL audit log. Set acl.audit.storage.policy on every node before the first start if you want something other than the default. An existing object named sys.acl_audit_log with other columns stops the start.
  • SHOW CREATE. The full dump needs LIST USERS and USER DETAILS, no SHOW CREATE output carries OWNED BY anymore, and materialized and live views accept only the schema-only form.
  • Views with PIVOT ... IN (SELECT ...) can no longer call administrative functions in that subquery. Existing views that do turn invalid at their next compile.
  • CREATE SERVICE ACCOUNT IF NOT EXISTS leaves an existing account unchanged. It used to assign the account to the statement's owner, with grant option.
  • A replicating primary in the default commit mode now refuses to start after a power loss if its log of a table ends before the last transaction in the object store.
  • DEDUP SKIP. Upgrade every node before using it.
  • Embedded Java integrations that implement QuestDB Enterprise ACL or security context interfaces need updating.
  • Going back to 4.1 or earlier. QuestDB Enterprise 4.2.1 locks every table it writes, and earlier releases refuse to open a locked table. See Downgrade to an earlier release.

The upgrade guide has every step, including what happens to a database restored from an older backup.


Getting the update

QuestDB Enterprise 4.2.1 also includes many bug fixes across replication, backups, cold storage and access control. See the release notes page for the details.

Self-managed enterprise customers will find the binaries at the usual download location. BYOC enterprise customers will be contacted for upgrading. Upgrade the replicas before the primary.

Not on QuestDB Enterprise yet? Learn more about QuestDB Enterprise and BYOC, or contact the QuestDB team for a conversation or a demo.

Subscribe to stay up to date with all things QuestDB.