Energetech powers commodity trading strategies with QuestDB

Energetech uses QuestDB as the backbone of their commodity trading strategies, managing real-time prices and forecasts for energy markets.

Handling vast energy data

Energetech stores two main types of time-series data: market data (prices, order book snapshots) and forecasts.

They ingest transaction and order book data from various venues and perform 'latest by' or aggregating queries for downstream applications.

With multiple external forecast providers publishing data several times a day, Energetech requires efficient data deduplication and high ingestion performance to handle short bursts of high-volume data.

Cost-saving data architecture

Energetech's architecture efficiently ingests and processes vast amounts of energy data from multiple providers. QuestDB plays a central role in storing and querying time-series data with high performance and reliability.

The built-in deduplication and out-of-order handling capabilities of QuestDB simplify the data pipeline, enabling Energetech to focus on delivering value to their clients. It also significantly limits database growth, which results in both immediate and sustained cost savings.

Industry leading ingestion capabilities

Energetech's pipeline ingests hundreds of GBs of data daily, achieving peak ingestion rates of over 10 million messages per minute.

Thanks to QuestDB's powerful deduplication and compression, database growth remains minimal at around 1% per week.

When you combine both massive performance and extreme hardware efficiency, the end result is significant cost and time savings for your team (in both systems costs and total maintenance and development costs over time). High performing teams need the highest performing tools.

WITH
latest_data AS (
  SELECT meta_id, published_at_utc
  FROM /table/
  WHERE meta_id IN /list of 100-500 meta_ids/
    AND published_at_utc < '/cutoff for data/'
  LATEST ON published_at_utc PARTITION BY meta_id
),
period_remit AS (
  SELECT meta_id, published_at_utc, event_start_utc, event_end_utc, value AS availability
  FROM /table/
  WHERE event_start_utc < '/period_start/'
    AND event_end_utc > '/period_end/'
)
SELECT
  period_remit.meta_id,
  period_remit.published_at_utc,
  period_remit.event_start_utc,
  period_remit.event_end_utc,
  period_remit.availability AS availability
FROM period_remit
JOIN latest_data ON (meta_id, published_at_utc);

Incremental Adoption

After a successful Proof of Concept, Energetech switched to a Bring Your Own Cloud (BYOC) deployment, and started adopting some of the newer features, like read-replicas for increased availability, or materialized views for query efficiency.