Reduce Data Costs: Log Sampling with OpenTelemetry and BindPlane OP

Redundant logs are a common nuisance in observability pipelines of all kinds. In large environments, excess logs can multiply data costs to unsustainable amounts. Log sampling is the process of randomly sampling logs to produce the same valuable insight with dramatically reduced data flow.

Configuring agents in a pipeline to appropriately sample logs can be a pain. Pipeline managers, like BindPlane OP, make that process simple and scalable.

Using BindPlane OP to Manage Observability Pipelines

BindPlane OP is the first observability pipeline management system built for OpenTelemetry. It’s a vendor agnostic solution for managing agents and connecting sources to any or multiple destinations. Agents can be configured to enrich, filter, or sample data at the source, saving on ingestions costs. This blog details the steps for sampling logs. The motto is “instrument once, send anywhere,” because BindPlane OP was built for the purpose of managing environments of any size, even with thousands of agents. It’s available as an open source solution, and a full scale enterprise solution.

You can find the open source version on GitHub and refer to the BindPlane OP documentation to get started. The BindPlane OP Enterprise Edition comes with all the power and flexibility of the open source version as well as a long list of features built for large organizations with environments of any size. Contact observIQ to schedule a BindPlane OP Enterprise Demo, or join the BindPlane OP Slack to talk to us and other BindPlane OP users.

Once your BindPlane OP instance is running, load it up on any device with access to your environment and jump to the Agents tab.

Install OpenTelemetry Agents to Sources

• With BindPlane OP, installing agents to sources is simple. From the Agents tab, just click the “Install Agents” button on the top right.
  

• Select the operating system of the host you want to install the agent on. For this guide, we’re installing an agent on MacOS devices.

• Copy the command from BindPlane OP and paste it into a command prompt with administrator access to the host.

• After a successful installation, the agent will appear on the agents tab. You can use the search bar at the top to locate it quickly. If the agent doesn’t appear within a few seconds, refresh the BindPlane OP window.

Create a Custom Configuration to Sample Logs

• Custom configurations are the easiest way to optimize your telemetry pipelines. Create a new configuration by clicking the “New Configuration” button on the top right of the Configurations tab.

• Name your configuration and select the appropriate platform. Add a description if you wish, then click “Next.” Select the source type for the config. observIQ regularly expands BindPlane OP support for new sources, so if you don’t see something you’re looking for, let us know.

• Select the destination type for your configuration. Destinations you’ve connected with other configurations will appear here. If you need to add a new destination, click “Create New” at the bottom and input the requested information for your destination. You can add as many destinations to a configuration as you like.

• Click “Save” and your new configuration will appear in the configurations tab. Select it to see a detailed visualization of its pipelines for logs, metrics, and traces, as well as configured sources and destinations. Click on the source you want to sample logs from.

• A window will appear where you can edit the details for the source within the configuration. Scroll to the bottom and click “Add processor.”

• Find and select the “Log Sampling” processor.

• Select the ratio of logs you want to sample, then click “Save.” Save again, and the configuration is updated to include the log sampling processor.

Apply the Configuration to the Agents

• Stay on the configuration overview window, or find it again in the configurations tab. Scroll to the bottom and click the “+” button in the Agents section.

• Select the agents you want to apply the configuration to and click save.

• They will appear in the Agents field, where you can see their status. The configuration will be applied to disconnected agents when they are back online.

If you want to learn more about how to use BindPlane OP, check out the observability blog and the BindPlane OP page at observIQ.com. We are committed to developing the next generation of open source telemetry tools. If you want to join, learn more, or just silently observe, join our community of enthusiasts, developers, and professionals on the BindPlane OP Slack.