Skip to content

ClickHouse Output

The clickhouse output sends selected Adder events to the versioned adder_events_v1 table in ClickHouse. Adder sends rows in JSONEachRow batches over the ClickHouse HTTP interface, and ClickHouse receives each event as a separate row after a successful insert.

See the Adder integration overview and the command reference for related configuration and output options.

Configure the output in the Adder YAML file:

output: clickhouse
plugins:
output:
clickhouse:
url: https://clickhouse.example.org:8443
database: analytics
username: adder
password: YOUR_PASSWORD
event-types: blocks,transactions,rollbacks

The output uses these defaults:

OptionDefaultDescription
urlhttp://127.0.0.1:8123ClickHouse HTTP origin.
databasedefaultClickHouse database name.
usernamedefaultClickHouse user.
passwordEmptyOptional ClickHouse password.
event-typesblocks,transactions,rollbacks,governance,drep-registration,drep-update,drep-retirementComma-separated event selections.

The url value must use http or https, include a host, and identify only an origin. Do not include credentials, a query, a fragment, or a path other than /. The database value must be nonempty and cannot contain whitespace, /, or a backslash. The username value must be nonempty. When configuration sets a password, use https for a remote ClickHouse server; plain HTTP with a password works only for localhost or another loopback address. Protect the configuration and password with the same access controls used for ClickHouse credentials.

Adder does not follow HTTP redirects for ClickHouse requests. This prevents ClickHouse credentials from being sent to another origin.

The event-types value must name at least one supported selection. If the configuration omits the value, the output uses the default shown in the table.

At startup, Adder creates adder_events_v1 in the configured database when the table does not already exist. The table uses the ClickHouse MergeTree engine and stores each event as one JSONEachRow object with these fields:

FieldClickHouse typeContents
schema_versionUInt8Schema version 1.
event_hashStringSHA-256 of the complete event JSON.
event_typeLowCardinality(String)The Adder event type, such as input.block or input.rollback.
event_timeDateTime64(9, 'UTC')The event timestamp in UTC with nanosecond precision.
event_jsonStringThe complete event JSON.

ClickHouse partitions the table by event month and orders rows by event_time, event_type, and event_hash. The event_hash supports comparison; it does not identify a unique row.

Adder sends a batch when it reaches 100 selected events or when one second passes, whichever occurs first. Each flush uses one synchronous INSERT request with async_insert=0, wait_end_of_query=1, deduplicate_insert=disable, and insert_deduplicate=0. These settings make the insert synchronous and disable ClickHouse insert deduplication. Adder acknowledges the batch only after the HTTP response succeeds and the query completes.

When Adder stops, it drains the input and flushes events already accepted into the output buffer. The final request has a ten-second deadline. Adder does not maintain a durable queue, so events in a failed batch do not persist locally for later delivery.

Adder checks each event timestamp before it adds the event to a batch. A timestamp outside the supported DateTime64(9) range produces an individual error, while other valid events continue through batching. Adder logs and reports the batch when the HTTP request or ClickHouse query fails, including an error discovered while reading the response body, and then discards the batch locally. Adder does not automatically retry an ambiguous failed insert because repeating the request can create duplicate rows.

ClickHouse keeps every row that it accepts successfully, including identical event duplicates. It also keeps rollback events as ordinary input.rollback rows; Adder does not delete earlier rows when a rollback occurs. Query the retained rows with the event fields, for example:

SELECT event_type, event_time, event_json
FROM adder_events_v1
ORDER BY event_time;

Doc Holiday logo

Docs authored by Doc Holiday