An Upgrade to observIQ’s Log Agent – Incorporating OpenTelemetry

Our log agent is powerful, efficient, and highly adaptable. With OpenTelemetry setting new standards in the observability space, we wanted to incorporate that collaboration into our log agent and offer our users the ability to take advantage of the OpenTelemetry ecosystem. Starting today, you can upgrade the log agents in your observIQ account to the new Open Telemetry-based observIQ log agent with a single click.

OpenTelemetry’s logging USP adheres to the “textbook” definition of a log. By aligning our agent to the OpenTelemtry collector, we aim to attain textbook perfection for our log management capabilities. To understand why this is a game-changer, let’s dive into the basic architecture of the OpenTelemetry log collector.

Log Collector’s Architecture

The open telemetry collector is designed to support logs from legacy systems, logging libraries, and cloud-native applications. The problem the open telemetry collector addresses is the incohesive logging solutions and libraries that have an incomplete correlation between the aspects of observability data, namely, metrics, logs, and traces. By implementing a standardization in how observability data is parsed, ingested, distributed, and consumed, OpenTelemetry has made the telemetry data very relevant and highly informative.

Open Telemetry has standardized data models for all logs, metrics, and traces they parse. Once parsed, the OpenTelemetry collector enriches the data further to create more correlation between the data. The most notable factor here is that the enrichment across logs, traces, and metrics has the same attribute names and values, maintaining uniformity across all observability data. OpenTelemetry’s log collector follows a defined log data model that dictates the information that should or should not be recorded in the log data. This log data model is created to have log data transmitted, saved, and analyzed in a standardized manner. The existing log libraries are expected to align with this log data model in future versions.

The log collector’s architecture components are Receivers, Processors, and Exporters. A pipeline in the collector defines the trajectory of the traces and metrics data but not the log data. The receiver is essentially the entry point for the log data, where the data is collected, assimilated, and forwarded to the processor, which enriches and correlates the data. Once enriched, the data is transmitted to the destination path/ applications via the exporters.

Related Content: How to Manage Sensitive Log Data

How Does Correlation Work?

The primary aspect of correlation, according to OpenTelemetry standards, is the time-based correlation, where logs, traces, and metrics are mapped to each other based on the time or period of execution. The next level of correlation is based on the execution. Logs and traces in the same execution context are associated using TraceID and SpanID. Another significant correlation factor is the resource name included in the traces and metrics data.

What Difference Does this Upgrade Make to Your Log Management?

• It leverages the benefits of the evolving OpenTelemetry log collector’s components.
• Now, restarting an agent would be a unique event, not necessitating restarting the application or the other components of your infrastructure.
• There are more refined log level standardizations, mirroring the log levels/ service text in OpenTelemetry, such as trace, debug, error, warn, info, and fatal.
• observIQ’s log agent now adapts a new log rotation and checkpoint resuming capability using OpenTelemetry’s file log receiver.

Related Content: Turning Logs into Metrics with OpenTelemetry and BindPlane OP

Steps to Upgrade to the Log Agent

Technically, the upgrade to the new log agent works at the click of a button. This is in line with our mantra – keep everything simple.

The gif above is unedited to show how simple the upgrade works. It takes under 15 seconds to navigate, initiate, and complete the upgrade. You can try this out in your observIQ account or sign up for an account.

The upgraded log agent and the vast set of source plugins that observIQ offers make your log management a breeze. As always, we're around to help you with any log management questions or requests. Please reach out to our excellent support team. Stay observant with observIQ.