BindPlane OP

Configuring OpenTelemetry Agents to Enrich Data and Reduce Observability Costs

Paul Stefanski
Paul Stefanski

BindPlane OP is a powerful, open source tool that makes it easy to build and manage telemetry pipelines to ship data from IT environments of any kind and size to any analysis tool or storage destination. BindPlane OP installs and configures OpenTelemetry agents, which support a wide variety of sources and can be configured to ship data to multiple destinations while enriching or reducing data simultaneously. The vendor-agnostic toolset is excellent for reducing data costs and getting the most out of your data. OpenTelemetry agent configurations are formatted as yaml files that are always editable without the need to uninstall and reinstall the agent. BindPlane OP makes creating and editing configurations even easier with a codeless visual interface and step by step options for collecting and managing your data. This blog covers how to create and edit a configuration in BindPlane OP, add sources and destinations, add processors to reduce or enrich data, and apply that configuration to one or many agents with a few clicks.

What are Configs in BindPlane OP?

Configurations in BindPlane OP are instructions for directing OpenTelemetry agents. The Configs tab is where you create, save, and apply agent configurations in BindPlane OP. Existing configurations appear in a sortable, searchable list, and new configurations can be created by clicking the “New Configuration” button at the top. Clicking on a configuration will open a page where you can see details for that configuration, including the sources, destinations, and a topographical map representing the real time data flow and processing of agents using that configuration.

You can edit the sources and destinations by clicking them, and add new ones by clicking the respective buttons. New configurations created on the Configs page exist as templates that do not affect your agents unless explicitly applied, so don’t worry about affecting your environment while creating or editing new configurations, but if you edit an existing configuration that is already applied to agents in your environment, those changes will be pushed to the appropriate agents when you click “Save” after an edit is made.

At the bottom of any configuration’s page, you can see a searchable list of agents using that configuration, their operating system, and their connection status. You can also apply the configuration to new agents by clicking the “+” button at the top of this section.

Creating an OpenTelemetry Agent Configuration in BindPlane OP

Creating a configuration is a three step process.

  • Give your configuration a name, select your intended operating system, and write a brief description.
    • It’s good practice to give enough detail in the description so that other users can quickly identify the intended use of the configuration.
  • Add your intended source(s)
    • Sources appear in a searchable list with” log”, “metrics”, and “traces” labels.
    • You can adjust your basic settings for the source, like the hostname and port. You can also adjust advanced settings, such as the collection interval. Advanced settings vary by source.
  • Add your destination(s)
    • Destinations appear in a searchable list. Click a destination to select it.
    • Give your destination a name for use in BindPlane OP (the name does not have to match anything outside of BindPlane OP, but that can make organization easier)
    • Add the necessary routing and authentication information for the destination. The specific requirements vary by destination, but typically include a region and an access key or password. Tooltips in BindPlane OP can help you find your authentication information if you don’t have it.
    • You can add as many destinations as you wish by repeating the previous steps. OpenTelemetry agents can ship data to any number of destinations.
    • Once you add a destination to a configuration, it is saved as a component, so you can skip the routing and authentication when you use it in another configuration.

Adding Processors to Enrich Data and Reduce Data Costs

Processors are components in configurations that manipulate data at the source before it’s sent to your destinations. There are many ways data can be manipulated, but the two general categories of data manipulation are enrichment and reduction. Enrichment means you add additional information to data at the source that helps you use the data more effectively at your destination. Reduction means you filter or eliminate data to reduce data flow and save on data costs. Below are the steps for adding a processor, and list of processors and their uses.

  1. On the Configs page, click on the configuration you want to add a processor to, then click on the source you want the processor to affect data from.
  2. Click “Add Processor” at the bottom of the source details.
  3. Select the processor you want from the list, or create a custom processor.
    1. Input the details for how you want the processor to manipulate your data, then click “Save”. The processor will be applied to any agents using the configuration automatically.
Add AttributeThe Add Log Record Attribute processor can be used to enrich logs by adding attributes to all log records in the pipeline.
Add ResourceThe Add Resource Attribute processor can be used to enrich telemetry by adding resources to all metrics, traces, and logs in the pipeline.
Custom ProcessorThe Custom processor can be used to inject a custom processor configuration into a pipeline. The Custom processor is useful for solving use cases not covered by BindPlane OP's other processor types.
Delete AttributeThe Delete Log Record Attribute processor can be used to remove attributes from all log records in the pipeline.
Delete ResourceThe Delete Resource processor can be used to remove resources from metrics, traces, and logs.
Filter SeverityThe Severity Filter processor can be used to filter out logs that do not meet a given severity threshold.
Filter Log Record AttributeThe Log Record Attribute Filter processor can be used to include or exclude logs based on matched attributes.
Filter Resource AttributeThe Resource Attribute Filter processor can be used to include or exclude logs based on matched resources.
Filter Metric NameThe Metric Name Filter processor can be used to include or exclude metrics based on their name.
Log SamplingThe Log Sampling processor can be used to filter out logs with a configured "drop ratio".
Extract MetricThe Extract Metric Processor can look at all logs matching a filter, extract a numerical value from a field, and then create a metric with that value. Both the name and units of the created metric can be configured. Additionally, fields from matching logs can be preserved as metric attributes.
Count LogsThe Count Logs Processor can count the number of logs matching some filter, and create a metric with that value. Both the name and units of the created metric can be configured. Additionally, fields from matching logs can be preserved as metric attributes.

Paul Stefanski
Paul Stefanski

Related posts

All posts

Get our latest content
in your inbox every week

By subscribing to our Newsletter, you agreed to our Privacy Notice

Community Engagement

Join the Community

Become a part of our thriving community, where you can connect with like-minded individuals, collaborate on projects, and grow together.

Ready to Get Started

Deploy in under 20 minutes with our one line installation script and start configuring your pipelines.

Try it now