Kafka | observIQ

Kafka

Supported Types

Logs	Metrics	Traces
✓	✓	✓

Configuration Table

Parameter	Type	Default	Description
telemetry_types	`telemetrySelector`	`["Logs", "Metrics", "Traces"]`	Specifies which types of telemetry to export.
protocol_version	`enum`	"2.0.0"	The Kafka protocol version to use when communicating with brokers. Valid values are: `"2.2.1"`, `"2.2.0"`, `"2.0.0"`, or `"1.0.0"`.
brokers	`strings`	localhost:9092	A list of the brokers to connect to when sending metrics, traces, and logs.
timeout	`int`	5	Timeout (seconds) for every attempt to send data to the backend.
log_topic	`string`	otlp_logs	The name of the topic to export logs to.
log_encoding	`enum`	otlp_proto	The encoding to use when publishing logs to Kafka. Options are otlp_proto, otlp_json, and raw.
metric_topic	`string`	otlp_metrics	The name of the topic to export metrics to.
metric_encoding	`enum`	otlp_proto	The encoding to use when publishing metrics to Kafka. Options are otlp_proto and otlp_json.
trace_topic	`string`	otlp_spans	The name of the topic to export traces to.
trace_encoding	`enum`	oltp_proto	The encoding to use when publishing traces to Kafka. Options are otlp_proto, otlp_json, jaeger_proto, jeager_json, zipkin_proto, and zipkin_json.
compression	`enum`	gzip	The compression algorithm to use when publishing data to Kafka. Options are gzip, snappy, lz4, and none.
enable_auth	`bool`	false
auth_type	`enum`	basic	`basic`, `sasl`, or `kerberos`
basic_username	`string`
basic_password	`string`
sasl_username	`string`
sasl_password	`enum`
sasl_mechanism	`string`	SCRAM-SHA-256	`SCRAM-SHA-256`, `SCRAM-SHA-512`, or `PLAIN`
kerberos_service_name	`string`
kerberos_realm	`string`
kerberos_config_file	`string`	/etc/krb5.conf
kerberos_auth_type	`enum`	keytab	`keytab` or `basic`
kerberos_keytab_file	`string`	/etc/security/kafka.keytab
kerberos_username	`string`
kerberos_password	`string`

Supported Retry and Queuing Settings

This destination supports the following retry and queuing settings:

Sending Queue	Persistent Queue	Retry on Failure
✓	✓	✓

Example Configuration

Basic Configuration

Web Interface

observIQ docs - Jaeger OTLP Destination - image 1

observIQ docs - Jaeger OTLP Destination - image 2

Standalone Destination

yaml

1apiVersion: bindplane.observiq.com/v1
2kind: Destination
3metadata:
4  name: kafka
5spec:
6  type: kafka_otlp_destination
7  parameters:
8    - name: telemetry_types
9      value:
10        - Metrics
11        - Logs
12        - Traces
13    - name: protocol_version
14      value: 2.0.0
15    - name: brokers
16      value:
17        - localhost:3333
18    - name: timeout
19      value: 50
20    - name: metric_topic
21      value: my_metrics
22    - name: metric_encoding
23      value: otlp_json
24    - name: log_topic
25      value: my_logs
26    - name: log_encoding
27      value: raw
28    - name: trace_topic
29      value: my_spans
30    - name: trace_encoding
31      value: zipkin_proto
32    - name: compression
33      value: gzip

Updated 2 months ago