When aggregating time-series metrics or event tables in Apache IoTDB, calculating approximate cardinality using APPROX_COUNT_DISTINCT is standard practice for maintaining high throughput across large datasets. However, in Apache IoTDB (particularly in table model versions around 2.0.x), running APPROX_COUNT_DISTINCT over a time window with zero matching rows can trigger an unexpected internal error:

Msg: org.apache.iotdb.jdbc.IoTDBSQLException: 301: Cannot invoke "...HyperLogLog.cardinality()" because the return value of "...SingleHyperLogLogState.getHyperLogLog()" is null

In this article, we will examine why this error happens, why exact distinct counting (COUNT(DISTINCT ...)) behaves differently, and how to work around it in your dashboards and queries.

The Problem: Reproducing the Error

Consider a minimal scenario where you track alarm events in an industrial IoT setting using Apache IoTDB's table model:

CREATE DATABASE alarm_summary;
USE alarm_summary;
CREATE TABLE alarm_events(line STRING TAG, alarm_code INT32 FIELD);

INSERT INTO alarm_events(time, line, alarm_code) VALUES
  (1000, 'press_a', 10),
  (2000, 'press_a', 20),
  (3000, 'press_a', 10);

When querying a time window that contains data, everything works as intended:

SELECT COUNT(*) AS events, APPROX_COUNT_DISTINCT(alarm_code) AS kinds
FROM alarm_events
WHERE time >= 1000 AND time < 4000;

The output returns events = 3 and kinds = 2. However, when querying an empty window (for example, between timestamp 4000 and 5000), IoTDB crashes with an unhandled exception:

Cannot invoke "org.apache.iotdb.calc.execution.operator.source.relational.aggregation.approximate.HyperLogLog.cardinality()" 
because the return value of "org.apache.iotdb.calc.execution.operator.source.relational.aggregation.approximate.HyperLogLogStateFactory$SingleHyperLogLogState.getHyperLogLog()" is null

Why Does This Happen?

This issue stems from a lazy initialization bug in IoTDB's relational aggregation engine:

  • State Initialization: The engine allocates a SingleHyperLogLogState object to maintain the sketch state for the column.
  • Row Ingestion: To conserve memory, the underlying HyperLogLog structure is lazily initialized on the first non-null input row passed to the aggregator.
  • Finalization Step: When the query finishes scanning rows, the engine calls .cardinality() on the state's inner HyperLogLog instance. If no records matched the WHERE clause, the sketch was never initialized, remaining null. When the engine attempts to read its cardinality, Java throws a NullPointerException.

By contrast, standard aggregates like COUNT(DISTINCT ...) initialize their accumulator or set structure to an empty collection at creation time, safely returning 0 when evaluating empty partitions.

Workarounds and Solutions

1. Use Exact Distinct Count (COUNT(DISTINCT))

If your windowed query frequently encounters empty intervals (such as real-time sliding windows or sparse alert streams), use COUNT(DISTINCT ...) until an upstream patch is applied:

SELECT COUNT(*) AS events,
       COUNT(DISTINCT alarm_code) AS kinds
FROM alarm_events
WHERE time >= 4000 AND time < 5000;

Because alarm counts in short time windows are typically small to moderate, the memory and computational overhead of an exact distinct count is usually negligible compared to HyperLogLog.

2. Fallback via CASE WHEN or Pre-flight Checks

If you are processing massive batches where approximate distinct counts are critical for performance, you can prevent the aggregator from evaluating an empty set by structuring queries or application logic to verify row counts prior to triggering approximate aggregations.

3. Keep IoTDB Updated

This is a known engine edge case in IoTDB's relational aggregation layer. A fix involves checking state.getHyperLogLog() == null and returning 0 before invoking cardinality(). Check recent releases of Apache IoTDB (post-2.0.10) to see if this bug has been patched in newer releases.

Summary

While APPROX_COUNT_DISTINCT is powerful for approximate cardinality at scale, empty input streams in IoTDB 2.0.x cause uninitialized HyperLogLog states to throw a NullPointerException. For empty or sparse time windows, switch to COUNT(DISTINCT ...) to ensure your dashboards always render zero values reliably.