Technical “How-To’s”

How to monitor Zookeeper with OpenTelemetry

Deepa Ramachandra
Deepa Ramachandra

We are back with a simplified configuration for another critical open-source component, Zookeeper. Monitoring Zookeeper applications helps to ensure that the data sets are distributed as expected across the cluster. Although Zookeeper is considered to be very resilient to network mishaps, monitoring is inevitable. To do so, we’ll set up monitoring using the Zookeeper receiver from OpenTelemetry.

The configuration detailed in this post uses observIQ’s distribution of the OpenTelemetry collector. We are simplifying the use of OpenTelemetry for all users. If you are as excited as we are, take a look at the details of this support in our repo.

You can utilize this receiver in conjunction with any OTel collector: including the OpenTelemetry Collector and observIQ’s distribution of the collector.

Monitoring performance metrics for Zookeper is necessary to ensure that all the jobs are running as expected and the clusters are humming. The following categories of metrics are monitored using this configuration:


Automatically discover Zookeeper Clusters, monitor memory (heap and non-heap) on the Znode and get alerts of changes in resource consumption. Automatically collect, graph and get alerts on garbage collection iterations, heap size and usage, and threads. ZooKeeper hosts are deployed in a cluster and, as long as a majority of hosts are up, the service will be available. Make sure the total node count inside the ZooKeeper tree is consistent.

Latency and throughput:

Consistent view of the performance of your servers, regardless of whether they change roles from Followers to Leader or back – you’ll get a meaningful view of the history.

Configuring the Zookeeper Receiver

After the installation, the config file for the collector can be found at:

  • C:\Program Files\observIQ OpenTelemetry Collector\config.yaml (Windows)
  • /opt/observiq-otel-collector/config.yaml(Linux)

Receiver Configuration:

  1. Configure the collection_interval attribute. It is set to 60 seconds in this sample configuration.
  2. Setup the endpoint attribute as the system that is running the Hadoop instance
2  zookeeper:
3    collection_interval: 30s
4    endpoint: localhost:2181

Processor Configuration:

  1. The resource detection processor is used to create a distinction between metrics received from multiple Hadoop systems. This helps with filtering metrics from specific Redis hosts in the monitoring tool, in this case, Google Cloud operations.
  2. Add the batch processor to bundle the metrics from multiple receivers. We highly recommend using this processor in the configuration, especially for the benefit of the logging component of the collector. To learn more about this processor check the documentation.
2  resourcedetection:
3    detectors: ["system"]
4    system:
5      hostname_sources: ["os"]
7  batch:

Exporter Configuration:

In this example, the metrics are exported to New Relic using the OTLP exporter. If you would like to forward your metrics to a different destination, check the destinations that OpenTelemetry supports at this time, here.

2  otlp:
3    endpoint:
4    headers:
5      api-key: 00000-00000-00000
6    tls:
7      insecure: false

Set up the pipeline:

2  pipelines:
3    metrics:
4      receivers:
5      - zookeeper
6      processors:
7      - resourcedetection
8      - batch
9      exporters:
10      - otlp

Viewing the metrics

All the metrics the Zookeeper receiver scrapes are listed below.

zookeeper.connection.activeThe number of active connections.
zookeeper.data_tree…hemeral_node.countThe number of ephemeral nodes.
zookeeper.data_tree.sizeThe size of the data tree.
zookeeper.file_descriptor.limitThe limit set for the file descriptor.
zookeeper.file_descriptor.openThe number of open file descriptors
zookeeper.latency.maxThe maximum latency
zookeeper.latency.minThe minimum latency set.
zookeeper.packet.countThe packet count
zookeeper.request.activeThe number of active requests watch count
zookeeper.znode.countThe total number of znode.


Now that you have the metrics gathered and exported to the destination of your choice, you may want to explore how to effectively configure alerts for these metrics. Here are some alerting possibilities for ZooKeeper:

ZooKeeper server is downcritical
Too many znodes createdwarning
Too many connections createdwarning
Memory occupied by znode is too largewarning
Set too many watchwarning
Too many files openwarning
Average latency is too highwarning
JVM memory almost fullwarning

observIQ’s distribution is a game-changer for companies looking to implement the OpenTelemetry standards. The single line installer, seamlessly integrated receivers, exporter, and processor pool make working with this collector simple. Follow this space to keep up with all our future posts and simplified configurations for various sources. For questions, requests, and suggestions, reach out to our support team at or join the conversation on Slack!

Deepa Ramachandra
Deepa Ramachandra

Related posts

All posts

Get our latest content
in your inbox every week

By subscribing to our Newsletter, you agreed to our Privacy Notice

Community Engagement

Join the Community

Become a part of our thriving community, where you can connect with like-minded individuals, collaborate on projects, and grow together.

Ready to Get Started

Deploy in under 20 minutes with our one line installation script and start configuring your pipelines.

Try it now