Routing Telemetry

Routing Telemetry to a specific destination

There are many ways to include or exclude logs sent to a particular destination. Two ways we will be walking through are:

1. Excluding logs based on a shared attribute, so they will not be sent to an individual destination.
2. Only sending logs that meet a criteria, in this example a added attribute.

For this exercise we will start with excluding the logs.

Excluding Logs

We will start by using the Filter by Field Processor. First we will select the 'Destination' Processor on the right side.

Now we will identify a shared attribute across all of the logs we would like to exclude. We can do that by expanding entries in the telemetry example in the left hand column.

Now we can add the Filter by Field Processor.

We will want to make sure we change the match type to regex, if we will be using a fuzzy search. This enables you to use regex to grab something specific. If you select 'strict', it will need to be verbatim. In this example I will be excluding every log with the attribute key of 'log_type' and a value of 'bindplane*' based on the values pulled from the sample on the left column.

After saving, we will only need to rollout the change to make the change take effect.

Now we can verify our change worked. As you can see in this example, entries of 'log_type bindplane*' are being filtered out of the right hand side.

Including only tagged Logs

The other way that logs can be sent to a single destination is by manually tagging the log file, then only moving the tagged logs to a single destination. First we will add a incoming processor to a log that we wish to tag.

Now we will select the Add Fields Processor

Here we will be using a 'upsert' attribute action, and for this example a field of 'source' and a value of 'syslog'.

We can save that processor now, and move on to the Destination Processor on the right hand side, next to the Destination you would like to send these logs to.

For this we will be using the Filter by Field Processor. We will want to configure it for this example by specifying the Action of 'include' and a Match Type of 'strict'.

Below that in the Attribute Fields section, we will specify the field as 'Source' and the value as 'tomcat'.

After you save and roll out the configuration to the agents, you can verify it is working by going back in to the destination processor which will have what is being sent to the destination in the far right column.