New Features

• Configurable Measurement Interval
  • Review the Upgrade Notes for upgrade requirements
• Prometheus Metrics
• CLI Completion Command
• Multiple performance improvements for Postgres and Bolt Store
• Added the ability to specify a selector or query to the /agents/labels endpoint

Bug Fixes

• Fixed Configurations page telemetry sorting

Upgrade Notes

Prometheus

If you are not using BindPlane with high availability, no action is required.

BindPlane High Availability users should upgrade their Prometheus Linux Package. The latest version of the package will contain the new recording rules that enable Configurable Measurement Interval. After upgrading, no further action is required.

If you are managing Prometheus manually, you will need to re-review the documentation. Make sure to update your recording rules.

The new recording rules look like this.

1groups:
2- name: configuration-rollups
3  interval: 1m
4  rules:
5  - record: bindplane_agent_measurements:rollup:rate:1m
6    expr: sum without (agent) (rate(bindplane_agent_measurements{}[1m9s999ms] offset 10s))
7- name: 5m-configuration-rollups
8  interval: 5m
9  rules:
10  - record: bindplane_agent_measurements:rollup:rate:5m
11    expr: sum without (agent) (rate(bindplane_agent_measurements:1m{}[5m59s999ms] offset 10s))
12- name: 1h-configuration-rollups
13  interval: 1h
14  rules:
15  - record: bindplane_agent_measurements:rollup:rate:1h
16    expr: sum without (agent) (rate(bindplane_agent_measurements:15m{}[1h14m59s999ms] offset 10s))

Configurable Measurement Interval

Agents send configuration throughput metrics to BindPlane every 10 seconds. This interval is now configurable, supporting 10s, 1m, and 15m. It is recommended to use 1m or 15m intervals when managing large numbers of agents. At scale, 10s intervals can be responsible for high overhead.

The interval can be set by modifying a configuration's advanced settings. Select the gear icon on and choose "Advanced Configuration Options".

See the Upgrade Notes for important details regarding configurable measurement intervals.

Prometheus Metrics

BindPlane can expose APM metrics using a built-in Prometheus exporter. Metrics are disabled by default and can be enabled by updating your configuration file.

1type: prometheus
2interval: 10s
3prometheus:
4    endpoint: /metrics

CLI Completion Command

The CLI supports shell completion for bash, zsh, fish and powershell. After upgrading your CLI, use the bindplane completion -h command for more information.

New Features

• Add max log size option to Kubernetes Container source
• New Parse Severity processor
  • Supports flexible parsing, such as parsing HTTP status codes.
  • Parse Severity v1 has been deprecated and should be replaced by the new processor.
• New command bindplane init auth for updating the BindPlane server's authentication configuration

Bug Fixes

• Fail to save resources if all pipelines are disabled
  • The web interface will not allow you to save a resource that does not have logs, metrics, or traces enabled.

Bug Fixes

• Fix rendering Windows Events sources using remote collection and custom channels.

New Features

• Support multi line Kubernetes container logs
• Add remote option to Windows events log

Bug Fixes

• Fix deleting a Library Source from configuration

New Features

• New init ldap CLI command, for configuring and migrating authentication to ldap

Bug Fixes

• Return 401 for missing account header (measurements)
• Enforce NATS mutual TLS
• Conditional Builder Regex escaping
  • See Upgrade Procedure section.

Init LDAP

When operating BindPlane OP on Linux, you can migrate to LDAP by executing the following command on the Linux server.

1sudo BINDPLANE_CONFIG_HOME=/var/lib/bindplane \
2    /usr/local/bin/bindplane init ldap \
3    --config /etc/bindplane/config.yaml

The command will prompt for your ldap server information, and migrate your BindPlane accounts to the new authentication mechanism.

Upgrade Procedure

After upgrading to BindPlane v1.62.0, review your configurations for pending rollouts. Update any configurations that contain processors with conditions built using the condition builder. Previous versions of BindPlane will have required the user to handle regex escaping on their own. v1.62.0 handles the escaping for you, however, existing conditions will need to be updated to take this into account.

New Features

• NATS Event Bus
• Filter By Regex processor, available for Google licenses
• Agent's page is now sortable by labels
• SQL Query Source

Bug Fixes

• Add validation to guard against applying resources without metadata.name
• Correctly handle deleting multiple configs
• Handle duplicating configs without a current version

Other

• Rebrand Google Chronicle to Google SecOps

NATS Event Bus

Enterprise users operating BindPlane in High Availability can now use the NATS Event Bus. The NATS Event Bus eliminates the need for an external event bus such as Google Pub/Sub or Apache Kafka.

New Features

• Support setting agent metrics port with global config option
• OpAMP agent authentication configuration

Bug Fixes

• Fix transform processor in Couchbase source
• Fix small rollout segment width when configurations have a low agent count

Other

• Optimize agents GraphQL queries
• Return 452 status when there is no license for the organization
• Include error in log for BindPlane Transform Agent process
• Upgrade to opamp-go v0.15.0

Agent Metrics Port

Agents managed by BindPlane expose metrics on TCP port 8888 by default. The port is now configurable using a global setting. Modifying the port will affect all agents.

The BindPlane configuration file's advanced section can be updated to include the agent.telemetryPort option.

1advanced:
2  agent:
3    telemetryPort: 8888

Modifying advanced options should be considered an advanced operation, and is not recommended unless guided by BindPlane support.

OpAMP Authentication

A new configuration section agents allow users to configure OpAMP authentication. Currently, secretKey is the only supported authentication type. It supports one or more headers. In this example, the BindPlane control plane will check for X-Bindplane-Authorization first and then Authorization if the first header is not present.

1agents:
2  auth:
3    type: secretKey
4    secretKey:
5      headers: 
6      - X-Bindplane-Authorization
7      - Authorization

Modifying the agent authentication options should be considered an advanced operation, and is not recommended unless guided by BindPlane support.

New Features

• Organizations and Projects
• New Kubernetes Prometheus source
  • Detect and scrape Prometheus exporters using Prometheus Kubernetes service discovery.

Bug Fixes

• Google Cloud destination: Set location resource if not already set
  • Resolves an issue where Google rejects metric types that require a location.
• Audit log CSV exporter now respects users timezone
• Require configuration match labels
  • If match labels are not present, configuration=<configuration name> will be added automatically when creating a configuration using the API.

Other

• Agent's telemetry port 8888 will now bind to localhost on Linux, Windows, and macOS.
• Deprecated Filter by Field processor in favor of Filter by Condition processor

Organizations and Projects

Starting in 1.59 each deployment of BindPlane OP will have a new top-level Organization made up of all the previous existing Accounts on the deployment. Accounts will now be referred to as Projects, which still contain all of the Agents and Resources associated with the account.

Some things to note:

• Any previous Admin level users now also receive the Organization Admin role. Organization admins can see all projects in an organization and any user that uses those projects.
• Organization Admins can access the new Organization page found in the settings menu.
• New Project creation is now limited to Organization Admins only.

New Features

• Add metrics support to bindplane-agent source
• Progressive Rollouts

Bug Fixes

• Support nested fields in Add fields processor

Progressive Rollouts

With Progressive Rollouts you can now deploy your configuration changes incrementally, one stage at a time. Tag specific agents as belonging to dev, stage, and prod environments and then BindPlane will pause the rollout after it completes each stage.

The stages themselves are entirely configurable, so you can create the exact rollout that works best for your environment.

Get started

New Features

Updated telemetry displayed in the Edit Processors (Live Preview) dialog to show more detail.

• Log
  • Added fields
    • observed_time
    • severity_number
    • severity_text
    • span_id.string
    • trace_id.string
• Metrics
  • Support for histogram and exponential histogram metrics
  • Added fields
    • value_int
    • value_double
    • quantile_values
    • explicit_bounds
    • bucket_counts
    • count
    • sum
    • description
    • type
    • aggregation_temporality
    • is_monotonic
• Traces
  • Added fields
    • span_id.string
    • parent_span_id
    • trace_id.string
    • kind
    • kind.string
    • status.code
    • status.message

New Features

• Additional TLS options for Postgres
• An option to change the default gRPC max incoming message size on the OTLP and Bindplane Gateway sources
• Storing attributes specified on OpAMP AgentDescription
• Additional processor recommendations for parsing sev, severity field
• Better visibility of deleted telemetry fields in Live Preview

Fixes

• Delete fields properly deletes nested fields

New Features

• New AWS S3 Rehydration Source
• New Filter By Condition Processor
• Additional MYSQL Source Metrics
• Add date picker to Azure Blob Rehydration source

AWS S3 Rehydration Source

v1.55.0 adds a new source for rehydrating data stored in an AWS S3 bucket. Read the source documentation here.

Filter By Condition Processor

Filter By Condition is a new processor introduced in v1.55.0. It can be used to include or exclude telemetry based on a condition that is evaluated against the telemetry data. Read the processor documentation here.

New Features

• New nop destination
• Processor Recommendations
• Secret key management

Processor Recommendations

v1.54.0 introduces processor recommendations. BindPlane will recommend processors when viewing Live Preview.

The initial release recommendation feature set includes the following recommendations:

• Parse Log Body as JSON
• Parse Severity
• Remove Debug Logs
• Remove Empty Values

Secret Key Management

New CLI command for rotating secret keys. Run bindplane secret --help for details.

1Manage secret keys
2
3Usage:
4  bindplane secret [command]
5
6Available Commands:
7  add         Add secret key
8  default     Set default secret key
9  delete      Delete secret key
10  get         Get secret keys

New Features

• New Telemetry Selector for components that support multiple telemetry types
• Chronicle destinations are now supported when using a Google license

Fixes

• Improved agent heartbeat
  • Resolves an issue where agents could become disconnected when using a Kubernetes Service mesh such as Istio

New Features

• New processor Condition Builder
  • Easily create OTTL conditions for most processors
  • See Upgrade and Downgrade for upgrade considerations
• New Gateway source and destination
• Add force_flush_period and max_log_size parameters to file log source

Upgrade and Downgrade

Downgrades from 1.52.0 to previous versions of BindPlane are not supported if you make use of the new OTTL condition builder feature. BindPlane 1.52.0 introduces a new parameter type that will not render

Condition Builder

The new condition builder allows the user to easily create OTTL.

Bug Fixes

• Minor UI fixes

New Features

• New Resource Detection processor. The previous processor is deprecated, with this new one matching OTel capabilities.
  • New Detectors:
    • Consul
    • AWS EKS
    • Kubernetes
    • OpenShift
  • Adds support for user-provided list of detectors
• Synthetic Telemetry Generator Source Type

Bug Fixes

• Various fixes for large-scale environments (100k agents)
• Fix for connected agents sometimes appearing as disconnected in the UI

Other

• Improvements to resource display in Configuration topology

New Features

• Agent labeling
• New Logstash source

Labels

We added new functionality that allows users to assign custom labels to agents. Agent labels can be managed either individually or in bulk. To manage labels individually, users can click on the labels in the details of the agent. To manage labels in bulk, users can select multiple agents from the agents table and click the new “Manage Labels” button that will appear. In addition to this, labels are now displayed in the agents table. If labels are truncated in this view, users can use the hover interaction to display all labels.

Bug Fixes

• VMware vCenter: Resolved an issue where disabling TLS verification would not work
• Move Fields Processor: Fixed OTTL exists condition

New Features

• Splunk HEC source support for Kubernetes Gateway
• Parse XML processor
• Initial support for OpenTelemetry Contrib collector
  • See Connecting Other OpenTelemetry Collectors Using the OpAMP Extension

Bug Fixes

• Corrupted concurrent downloads of offline agent updates
• Prevent scroll jitter in snapshot console

New Features

• Resource Library adds the ability to unlink Sources and Processors
• Application onboarding
• Determining Agent capabilities from OpAMP

New Features

• New Resource Library page for reusable Sources, Destinations, and Processors.
• New Destination: Azure Monitor
• New Destination: Snowflake
• New Processor: Parse CSV
• New Processor: Key Value processor
• Add endpoint parameter to Honeycomb Destination.

Upgrade Notes

As of 1.46.0, Prometheus is required to store agent throughput measurements. Throughput measurement support has been removed from Bolt Store and PostgreSQL.

Single-instance users do not need to take action. BindPlane will spawn a Prometheus sub-process automatically.

High-availability users will need to make sure Prometheus is configured in their environment. Documentation for configuring Prometheus in a BindPlane HA architecture can be found here.

Helm users should upgrade to chart version 1.2.0. Consult the chart release notes before upgrading. Once upgraded to Chart version 1.2.0, it is safe to upgrade to Chart version 1.2.1, which contains BindPlane v1.46.0.

New Features

• Added Splunk TCP source to Kubernetes Gateway Agent
• Added Encoding param to Kafka destination
• Added Optional Path Prefix parameter to OTLP HTTP
• New Lookup Table processor
• Agent health monitoring
• Terraform Provider: Added support for Extensions

Bug Fixes

• Added missing TLS options to CLI

New Features

• New Dynatrace Destination

New Features

• Introduced search capability to processor live preview and agent recent telemetry
• Added support for metrics to Azure Event Hub Source
• Added TLS server name override option to OTLP destination

Bug Fixes

• Removed compression option from Splunk Cloud destination
  • Compression is always enabled

Bug Fixes

Fixes an issue where changing the system account username would cause the user to lose access to their account.

It is recommended that all users upgrade from 1.43.0 to 1.43.1.

New Features

• Introduced the ability to delete old summary rollups to improve system performance and data management.

Bug Fixes

• Fix an issue where unavailable processors were shown as available for Google Edition

Other

• BindPlane OP Server architecture docs
• BindPlane OP HA Prometheus Linux Packages docs

New Features

• Azure Event Hub Source

Bug Fixes and Improvements

• Improved OpAMP debugging

Migration Notes

This release includes a change that will break future downgrade compatibility for users with a single account boltstore installation of BindPlane. Before upgrading, those users should create a backup of their storage file, which can be used in the event of a downgrade.*

To downgrade, those users should follow these steps:

1. Stop the BindPlane service
2. Replace the storage file with the backed-up version
3. Restart the BindPlane service

Changes made after upgrading will not be persisted when performing this downgrade.

* If, instead, BindPlane is configured with either multi-account or postgres, this release is backward compatible and no action is needed to downgrade safely. BindPlane is multi-account if accounts.enable is true in its configuration:

1accounts:
2  enable: true

New Features

• New destinations
  • Observe, Inc
  • Chronicle Forwarder
• Source changes
  • TCP Logs: Added new encoding option

Bug Fixes and Improvements

• Resolved an issue where Prometheus backed agent measurements were inconsistent in some cases
  • See the Upgrade Procedure notes for upgrade information

Upgrade Procedure

Prometheus

Agent measurement values will be inconsistent in some situations. Upgrading to v1.41.0 solves this issue.

If you are using BindPlane's built in Prometheus service, no action is required. If you are unsure, generally this means your BindPlane server is managing it's own Prometheus service behind the scenes. You can skip this procedure.

If you are using a self managed Prometheus instance for your measurements backend, you must update your Prometheus recording rule at /etc/prometheus/rules.yml.

1groups:
2- name: configuration-rollups
3  interval: 1m
4  rules:
5  - record: bindplane_agent_measurements:rollup:rate:1m
6    expr: sum without (agent) (rate(bindplane_agent_measurements{}[1m9s999ms] offset 10s))

Once updated, restart the service.

1sudo systemctl restart prometheus

New Features

• UI will link out to resource (source, destination, processor, extension) documentation when viewing a given resource.
• Add Compression Param to Chronicle
• Support headers for http protocol on OTLP exporters
• Extension: Go Performance Profiler (pprof)
• Extension: Health Check

Bug Fixes and Improvements

• Improved live preview loading times

New Features

• Summary Page Beta
• Updated Configuration Editing flow
• Kubernetes Gateway agent installation as deployment
• File Source sort by mtime option.
• Extension Audit Events
• Automatic removal of unused deprecated resources
• Improved error messages for CLI

New Features

• Amazon Managed Prometheus destination
• Honeycomb destination
• Rename metric processor
• Add sort by mtime option to file source
• Extensions UI
• Added TTL parameter to ClickHouse destination
• Include support bundle script in releases
  • See /var/lib/bindplane/scripts/create-support-bundle.sh after upgrading

New Features

• ClickHouse Destination Type
• Jaeger OTLP Destination Type
• Sumo Logic Destination Type
• HTTPCheck Source Type
• Extension Support
• Optional Prometheus measurements database, promising much faster and more reliable measurements

Bug Fixes

• Ongoing UI polish
• Severity processor now handles numeric values
• Deleting configuration doesn’t leave zombie agents
• Named source measurement fixes
• Unmasking sensitive parameters in sources