Delete Empty Values

Description

The Delete Empty Values processor can be used to delete null and other empty values from telemetry resource attributes, telemetry attributes, or a log record's body.

Supported Types

Metrics	Logs	Traces
✓	✓	✓

Configuration Table

Parameter	Type	Default	Description
telemetry_types	`telemetrySelector`	`["Logs", "Metrics", "Traces"]`	The list of telemetry types the processor will act on.
deleted_values	`enums`	`["Null Values"]`	List of values types to delete. May include "Null Values", "Empty Lists", and "Empty Maps". Enabling "Null Values" will remove both empty (zero-length) strings and entirely empty values. Enabling "Empty Lists" will delete empty (no element) list values. Enabling "Empty Maps" will delete empty (no key) map values.
exclude_resource_keys	`strings`	`[]`	List of resource keys to exclude from deletion.
exclude_attribute_keys	`strings`	`[]`	List of attribute keys to exclude from deletion.
exclude_body_keys	`string`	`[]`	List of body keys to exclude from deletion.
empty_string_values	`strings`	`[]`	List of string values that are considered "empty". String fields will be deleted if they match any of the strings in this list.

Example Configuration

This example configuration removes empty values from NGINX logs, where "-" is used to denote an empty field.

Web Interface

observIQ docs - Delete Empty Values - image 1

Standalone Processor

yaml

1apiVersion: bindplane.observiq.com/v1
2kind: Processor
3metadata:
4  id: remove-empty-attributes
5  name: remove-empty-attributes
6spec:
7  type: remove_empty_values
8  parameters:
9    - name: telemetry_types
10      value:
11        - Logs
12        - Metrics
13        - Traces
14    - name: empty_string_values
15      value:
16        - '-'

Configuration with Embedded Processor

yaml

1apiVersion: bindplane.observiq.com/v1
2kind: Configuration
3metadata:
4  id: remove-empty-attributes
5  name: remove-empty-attributes
6  labels:
7    platform: linux
8spec:
9  sources:
10    - type: nginx
11      parameters:
12        - name: telemetry_types
13          value:
14            - Logs
15            - Metrics
16        - name: endpoint
17          value: http://localhost:80/status
18        - name: disable_metrics
19          value: []
20        - name: enable_tls
21          value: false
22        - name: insecure_skip_verify
23          value: false
24        - name: ca_file
25          value: ''
26        - name: cert_file
27          value: ''
28        - name: key_file
29          value: ''
30        - name: collection_interval
31          value: 60
32        - name: data_flow
33          value: high
34        - name: log_format
35          value: default
36        - name: access_log_paths
37          value:
38            - /var/log/nginx/access.log*
39        - name: error_log_paths
40          value:
41            - /var/log/nginx/error.log*
42        - name: start_at
43          value: end
44      processors:
45        - type: delete_empty_values
46          parameters:
47            - name: telemetry_types
48              value:
49                - Logs
50                - Metrics
51                - Traces
52            - name: deleted_values
53              value:
54                - Null Values
55            - name: exclude_resource_keys
56              value: []
57            - name: exclude_attribute_keys
58              value: []
59            - name: exclude_body_keys
60              value: []
61            - name: empty_string_values
62              value:
63                - '-'
64          disabled: false
65      disabled: false
66  selector:
67    matchLabels:
68      configuration: remove-empty-attributes

Updated 6 months ago