Kafka | ObservIQ

Kafka

Supported Types

Logs	Metrics	Traces
✓	✓	✓

Configuration Table

Parameter	Type	Default	Description
telemetry_types	`telemetrySelector`	`["Logs", "Metrics", "Traces"]`	Specifies which types of telemetry to export.
protocol_version	`enum`	"2.0.0"	The Kafka protocol version to use when communicating with brokers. Valid values are: `"2.2.1"`, `"2.2.0"`, `"2.0.0"`, or `"1.0.0"`.
brokers	`strings`	localhost:9092	A list of the brokers to connect to when sending metrics, traces, and logs.
timeout	`int`	5	Timeout (seconds) for every attempt to send data to the backend.
log_topic	`string`	otlp_logs	The name of the topic to export logs to.
log_encoding	`enum`	otlp_proto	The encoding to use when publishing logs to Kafka. Options are otlp_proto, otlp_json, and raw.
metric_topic	`string`	otlp_metrics	The name of the topic to export metrics to.
metric_encoding	`enum`	otlp_proto	The encoding to use when publishing metrics to Kafka. Options are otlp_proto and otlp_json.
trace_topic	`string`	otlp_spans	The name of the topic to export traces to.
trace_encoding	`enum`	oltp_proto	The encoding to use when publishing traces to Kafka. Options are otlp_proto, otlp_json, jaeger_proto, jeager_json, zipkin_proto, and zipkin_json.
compression	`enum`	gzip	The compression algorithm to use when publishing data to Kafka. Options are gzip, snappy, lz4, and none.
enable_auth	`bool`	false
auth_type	`enum`	basic	`basic`, `sasl`, or `kerberos`
basic_username	`string`
basic_password	`string`
sasl_username	`string`
sasl_password	`enum`
sasl_mechanism	`string`	SCRAM-SHA-256	`SCRAM-SHA-256`, `SCRAM-SHA-512`, or `PLAIN`
kerberos_service_name	`string`
kerberos_realm	`string`
kerberos_config_file	`string`	/etc/krb5.conf
kerberos_auth_type	`enum`	keytab	`keytab` or `basic`
kerberos_keytab_file	`string`	/etc/security/kafka.keytab
kerberos_username	`string`
kerberos_password	`string`

Supported Retry and Queuing Settings

This destination supports the following retry and queuing settings:

Sending Queue	Persistent Queue	Retry on Failure
✓	✓	✓

Example Configuration

Basic Configuration

Web Interface

observIQ docs - Jaeger OTLP Destination - image 1

Standalone Destination

yaml

1apiVersion: bindplane.observiq.com/v1
2kind: Destination
3metadata:
4    name: kafka
5spec:
6    type: kafka_otlp_destination
7    parameters:
8        - name: telemetry_types
9          value:
10            - Metrics
11            - Logs
12            - Traces
13        - name: protocol_version
14          value: 2.0.0
15        - name: brokers
16          value:
17            - localhost:3333
18        - name: timeout
19          value: 50
20        - name: metric_topic
21          value: my_metrics
22        - name: metric_encoding
23          value: otlp_json
24        - name: log_topic
25          value: my_logs
26        - name: log_encoding
27          value: raw
28        - name: trace_topic
29          value: my_spans
30        - name: trace_encoding
31          value: zipkin_proto
32        - name: compression
33          value: gzip

Updated 1 month ago