Parse XML

MetricsLogsTracesBindPlane Agent
v1.46.0+

Description

The Parse XML Processor is utilized to parse XML document strings from specified fields within log, metric, or trace data. It's particularly useful when your telemetry data contains a serialized XML document, and you need to convert them into a structured format for easier analysis and filtering. The processor supports specifying the source field and the target field for the parsed XML data, offering flexibility in handling diverse data structures.

Use

When dealing with telemetry data that includes an XML document embedded within logs, metrics, or traces, the Parse XML Processor becomes instrumental. For instance, logs from certain applications or systems might contain XML documents representing specific attributes or metadata. By utilizing the Parse XML Processor, these XML documents can be parsed and converted into structured data, enhancing readability and facilitating more complex queries and analyses.

important

Multi-line XML

It's common for XML to be formatted to span multiple lines. When reading XML logs from a file, make sure to configure the multiline section of the File source to properly read the whole XML document.

The parsed XML is structured as follows:

  1. All character data for an XML element is trimmed and placed in the content field.
  2. The tag for an XML element is trimmed and placed in a tag field.
  3. The attributes for an XML element is placed as a mapping of attribute name to attribute value in the attribute field.
  4. Processing instructions, directives, and comments are ignored and not represented in the parsed XML.
  5. All child XML elements are parsed as above, and placed in an array in a children field.

As an example, see the following XML:

xml
1<?xml version="1.0" encoding="UTF-8" ?>
2<Log>
3  <User>
4    <ID>00001</ID>
5    <Name type="first">Joe</Name>
6    <Email>joe.smith@example.com</Email>
7  </User>
8  <Text>User fired alert A</Text>
9</Log>

This XML, when parsed, becomes:

json
1{
2  "tag": "Log",
3  "children": [
4    {
5      "tag": "User",
6      "children": [
7        {
8          "tag": "ID",
9          "content": "00001"
10        },
11        {
12          "tag": "Name",
13          "content": "Joe",
14          "attributes": {
15            "type": "first"
16          }
17        },
18        {
19          "tag": "Email",
20          "content": "joe.smith@example.com"
21        }
22      ]
23    },
24    {
25      "tag": "Text",
26      "content": "User fired alert A"
27    }
28  ]
29}

Configuration

FieldDescription
ConditionThe condition to apply the XML parsing. It supports OTTL expressions for logs, metrics, and traces. This field determines which telemetry data entries are processed based on their content and attributes.
Source Field TypeDetermines the type of source field for logs, metrics, or traces. This can be Resource, Attribute, Body, or Custom for logs and Resource, Attribute, or Custom for metrics and traces. It defines where the processor should look to find the XML document to parse.
Source FieldSpecifies the exact field where the XML document is located, based on the selected Source Field Type. For instance, if the Source Field Type is Attribute, this field should specify the particular attribute containing the XML document.
Target Field TypeLike the Source Field Type, this field determines the type of target field for logs, metrics, or traces where the parsed XML data will be stored. The options are similar, allowing users to store the parsed data as a resource, attribute, body, or in a custom field.
Target FieldSpecifies the exact field where the parsed XML data will be stored, based on the selected Target Field Type. This allows users to organize and structure the parsed data in a manner that facilitates easy querying and analysis.

Example Configurations

Parse XML from Logs

In this example, we have a basic log that details an action and the user that triggered the action, like an audit log. This log is in XML format, and we'd like to parse the content into a structured log.

observIQ docs - Parse XML - image 1

Here is a sample log record:

json
1{
2  "body": "<Log><User><ID>00001</ID><Name><First>Joe</First></Name></User><Text>User did a thing</Text></Log>"
3}

In order to parse the body of the log record, and store it on the parsed_xml attribute, we can configure the Parse XML processor as follows:

  • Telemetry: Logs
  • Condition: true
  • Source Field Type: Body
  • Source Field: Left empty
  • Target Field Type: Attribute
  • Target Field: parsed_xml

After parsing, the log record looks like this:

json
1{
2  "body": "<Log><User><ID>00001</ID><Name><First>Joe</First></Name></User><Text>User did a thing</Text></Log>",
3  "attributes": {
4    "parsed_xml": {
5      "children": [
6        {
7          "children": [
8            {
9              "content": "00001",
10              "tag": "ID"
11            },
12            {
13              "children": [
14                {
15                  "content": "Joe",
16                  "tag": "First"
17                }
18              ],
19              "tag": "Name"
20            }
21          ],
22          "tag": "User"
23        },
24        {
25          "content": "User did a thing",
26          "tag": "Text"
27        }
28      ],
29      "tag": "Log"
30    }
31  }
32}