Parse Timestamp
| Metrics | Logs | Traces | BindPlane Agent |
|---|---|---|---|
| ✓ | ✓ | ✓ | v1.36.0+ |
Description
The Parse Timestamp Processor is designed to extract and standardize timestamps from specified fields in log, metric, or trace data. It ensures uniformity of timestamp data across different sources and formats, facilitating seamless analysis and correlation of time-series data.
Use
In environments where telemetry data comes in various timestamp formats or from different fields, uniformity in timestamp data is crucial for accurate analysis and monitoring. The Parse Timestamp Processor addresses this by allowing users to specify the source field and format, enabling the extraction and standardization of timestamps across diverse data types and sources.
Configuration
| Field | Description |
|---|---|
| Telemetry Types | The types of telemetry to apply the processor to. |
| Condition | A condition expressed in OTTL that must be true for the processor to be applied. It supports dynamic evaluation, allowing for conditional application of the processor based on the content or attributes of the telemetry data. |
| Source Field Type | Determines the type of source field to parse the timestamp from. It can be either Resource, Attribute, or Body for logs, and Resource or Attribute for metrics and traces. |
| Source Field | The specific field from which to parse the timestamp. |
| Time Format | The format of the timestamp in the source field. Supported formats include RFC3339, ISO8601, Epoch, and Manual, catering to a wide range of timestamp formats encountered in real-world data. |
| Epoch Layout | Applicable when the Time Format is set to Epoch. It determines the layout of the epoch timestamp, ensuring accurate parsing of timestamps represented as epoch time. |
| Manual Layout | Required when the Time Format is set to Manual. It defines the strptime layout for parsing timestamps, offering flexibility to handle custom timestamp formats beyond the standard RFC3339 and ISO8601 formats. |
Example Configurations
Standardizing Log Timestamps
In this example, we configure the Parse Timestamp Processor to extract and standardize timestamps embedded within log messages. The log entries contain timestamps in various formats, and the goal is to normalize them for consistent analysis.
Sample log entry with a non-standard timestamp format:
The configuration for the Parse Timestamp Processor is set as follows:
- Condition:
"attributes['timestamp'] != nil" - Source FieldType:
Attribute - Source Field:
timestamp - Time Format:
Manual - Manual Layout:
%d/%m/%Y %H:%M:%S
As a result, the log entry is processed to extract and standardize the timestamp, transforming it into a consistent, machine-readable format for enhanced querying and analysis.
Processed log entry:
This setup ensures that all timestamps, regardless of their original format, are standardized to facilitate accurate and efficient data analysis.