Live Workshop: Integrate Google SecOps with Bindplane - Join Us on January 29th at 11 AM ET!Sign Up Now

Parse Key Value

MetricsLogsTracesBindPlane Agent
✓✓✓v1.45.0+

Description

The Parse Key Value processor is utilized to parse key value pairs from specified fields within log, metric, or trace data. The processor supports specifying the source field and the target field for the parsed key value data, offering flexibility in handling diverse data structures.

Use

When dealing with telemetry data that includes key value pairs embedded within logs, metrics, or traces, the Parse Key Value Processor becomes instrumental. For instance, logs from certain applications or systems might contain key value pairs representing specific attributes or metadata. By utilizing the Parse Key Value Processor, these key value pairs can be parsed and converted into structured data, enhancing readability and facilitating more complex queries and analyses.

Configuration

FieldDescription
Telemetry TypesThe types of telemetry to apply the processor to.
ConditionThe condition to apply the Key Value parsing. It supports OTTL expressions for logs, metrics, and traces. This field determines which telemetry data entries are processed based on their content and attributes.
Source Field TypeDetermines the type of source field for logs, metrics, or traces. This can be Resource, Attribute, Body, or Custom for logs and Resource, Attribute, or Custom for metrics and traces. It defines where the processor should look to find the key value pairs to parse.
Source FieldSpecifies the exact field where the key value pairs are located, based on the selected Source Field Type. For instance, if the Source Field Type is Attribute, this field should specify the particular attribute containing the key value pairs.
Target Field TypeLike the Source Field Type, this field determines the type of target field for logs, metrics, or traces where the parsed key value pairs will be stored. The options are similar, allowing users to store the parsed data as a resource, attribute, body, or in a custom field.
Target FieldSpecifies the exact field where the parsed key value pairs data will be stored, based on the selected Target Field Type. This allows users to organize and structure the parsed data in a manner that facilitates easy querying and analysis.
DelimiterSpecifies the string that should be used to split a key value pair. The default is =.
Pair DelimiterSpecifies the string that should be used to separate multiple pairs from each other. The default is a single space(" ").

note

  • It is not supported to parse from the "Body" or "Attributes" field into the "Resource" field.

Example Configurations

Parse Key Value Pairs from Logs

In this example, we are looking to parse key value pairs from a log's attribute field and store the parsed data in another attribute field. The logs contain key value pairs detailing additional information about log events, and we want to make this data more accessible.

observIQ docs - Parse Key Value - image 1
JSON
1{
2    "level": "info",
3    "eventDetails": "action:login!status:success"
4}

We want to parse the key value pairs from the eventDetails attribute and store them as structured data within the log entry. The configuration for the Parse Key Value Processor would be:

  • Condition: "attributes['eventDetails'] != nil"
  • Source Field Type: Attribute
  • Source Field: eventDetails
  • Target Field Type: Attribute
  • Target Field: parsedEventDetails
  • Delimiter: :
  • Pair Delimiter: !

The resulting log entry after processing would be:

JSON
1{
2    "level": "info",
3    "eventDetails": "action:login!status:success",
4    "parsedEventDetails": {
5        "action": "login",
6        "status": "success"
7    }
8}

This structured format makes it easier to filter and analyze the log data based on the action and status fields.