ClickHouse

Description

This ClickHouse destination can be used to send metrics, logs, and traces to a ClickHouse server.

Supported Types

MetricsLogsTracesBindplane Agent
v1.41.0+

Configuration

FieldDescription
Telemetry TypesThe kind of telemetry that will be sent to the ClickHouse server. Can be any combination of metrics, logs, and traces.
ProtocolThe kind of protocol to be used when sending to the ClickHouse server. Can be TCP, HTTP, or HTTPS. See this ClickHouse documentation for more information.
EndpointThe endpoint to use to send telemetry data to ClickHouse. Supports multiple endpoints. See this ClickHouse documentation for more information.
UsernameUsername to use to authenticate with the ClickHouse server. See this ClickHouse documentation for more information.
PasswordPassword to use to authenticate with the ClickHouse server. See this ClickHouse documentation for more information.
DatabaseName of the database to use when interacting with the ClickHouse server. Creates the database if it does not already exist.
Logs Table NameName of the table inside Database in ClickHouse to store log data in. Creates the table if it does not already exist.
Metrics Table NameName of the table inside Database in ClickHouse to store metric data in. Creates the table if it does not already exist.
Traces Table NameName of the table inside Database in ClickHouse to store traces data in. Creates the table if it does not already exist.
TTLThe data time-to-live, for example '30m'. 0 means no TTL. Make sure the telemetry sent has a timestamp field.
TimeoutTimeout for each attempt to send data to ClickHouse.
Connection ParametersAdditional connection parameters with map format. Used as query parameters in the URL. See this ClickHouse documentation for more information.

important

Metrics stored in ClickHouse are grouped by their type (sum, gauge, etc.) and stored in tables specific to that type by appending _type to the table name. For example, using the example configuration below, gauge metrics would be stored in bpop.bp_metrics_gauge. To read more, see this documentation.

note

TLS can be configured using Connection Parameters. To do so, add an entry called secure with the value true. An example can be seen below.

warning

The agent will fail to start if it cannot reach the ClickHouse server on start up. If the ClickHouse server becomes unreachable while the agent is already running, it will continue running while reporting an error.

The agent can also fail to start if configured to use TTL, but the telemetry being sent lacks a Timestamp field.

This destination supports the following retry and queuing settings:

Sending QueuePersistent QueueRetry on Failure

Example Configuration

Here we set a single endpoint that uses the TCP protocol. We are sending metrics, logs, and traces, so we have them selected and table names for them. We also utilize authentication and are setting compression and TLS via the connection parameters. Finally, we have the sending queue, persistent queue, and retry on failure enabled as well.

Web Interface

observIQ docs - ClickHouse Destination - image 1 observIQ docs - ClickHouse Destination - image 2

Standalone Destination

yaml
1apiVersion: bindplane.observiq.com/v1
2kind: Destination
3metadata:
4  id: clickhouse
5  name: clickhouse
6spec:
7  type: clickhouse
8  parameters:
9    - name: telemetry_types
10      value: ['Logs', 'Metrics', 'Traces']
11    - name: protocol
12      value: tcp
13    - name: endpoint
14      value: ['0.0.0.0:9000']
15    - name: username
16      value: admin
17    - name: password
18      value: admin
19    - name: database
20      value: bpop
21    - name: logs_table_name
22      value: bp_logs
23    - name: metrics_table_name
24      value: bp_metrics
25    - name: traces_table_name
26      value: bp_traces
27    - name: connection_params
28      value:
29        compress: gzip
30        secure: true
31    - name: retry_on_failure_enabled
32      value: 'true'
33    - name: sending_queue_enabled
34      value: 'true'
35    - name: persistent_queue_enabled
36      value: 'true'