Live Workshop: Integrate Google SecOps with Bindplane - Join Us on January 29th at 11 AM ET!Sign Up Now

File

warning

This source offers a delete_after_read option that can be hazardous. When this option is combined with file globbing, it will delete every file that matches the globbing pattern. Use with caution and care.


Supported Platforms

PlatformMetricsLogsTraces
Linux
Windows
macOS

Configuration Table

ParameterTypeDefaultDescription
file_path*stringsFile or directory paths to tail for logs.
exclude_file_pathstrings""File or directory paths to exclude.
log_typestring"file"A friendly name that will be added to each log entry as an attribute.
parse_formatenumnoneMethod to use when parsing. Valid values are none, json, and regex. When regex is selected, 'Regex Pattern' must be set.
regex_patternstringThe regex pattern that is used when parsing log entries.
multiline_line_start_patternstringRegex pattern that matches the beginning of a log entry for handling multiline logs.
multiline_line_end_patternstringRegex pattern that matches the end of a log entry, useful for terminating parsing of multiline logs.
parse_timestampboolfalseWhether to parse the timestamp from the log entry.
timestamp_fieldstringtimestampThe field containing the timestamp in the log entry.
parse_timestamp_formatenumISO8601The format of the timestamp in the log entry. Choose a common format, or specify a custom format. Options include "ISO8601", "RFC3339", "Epoch", and "Manual".
epoch_timestamp_formatenumsThe layout of the epoch-based timestamp. It's required when parse_timestamp_format is set to "Epoch". Options include "s", "ms", "us", "ns", "s.ms", "s.us", "s.ns".
manual_timestamp_formatstring'%Y-%m-%dT%H:%M:%S.%f%z'The strptime layout of the timestamp. It's used when parse_timestamp_format is set to "Manual".
timezonetimezoneUTCThe timezone to use if the Timestamp Format doesn't include a timezone. Otherwise, the timezone in the Timestamp Format will be respected. NOTE: This is also required to parse timezone abbreviations due to their ambiguity.
parse_severityboolfalseWhether to parse severity from the log entry.
severity_fieldstringseverityThe field containing the severity in the log entry.
encodingenumutf-8The encoding of the file being read. Valid values are nop, utf-8, utf-16le, utf-16be, ascii, and big5.
include_file_name_attributebooltrueWhether to add the file name as the attribute log.file.name.
include_file_path_attributeboolfalseWhether to add the file path as the attribute log.file.path.
include_file_name_resolvedboolfalseWhether to add the file name after symlinks resolution as the attribute log.file.name_resolved.
include_file_path_resolvedboolfalseWhether to add the file path after symlinks resolution as the attribute log.file.path_resolved.
delete_after_readboolfalseWhether to delete the file(s) after reading. Only valid in combination start_at: beginning.
offset_storage_dirstring$OIQ_OTEL_COLLECTOR_HOME/storageThe directory where the offset storage file will be created. It is okay if multiple receivers use the same directory. By default, the observIQ Distro for OpenTelemetry Collector sets $OIQ_OTEL_COLLECTOR_HOME in its runtime.
poll_intervalint200The duration of time in milliseconds between filesystem polls.
max_concurrent_filesint1024The maximum number of log files from which logs will be read concurrently. If the number of files matched exceeds this number, then files will be processed in batches.
parse_tostringbodyThe field that the log will be parsed to. Some exporters handle logs favorably when parsed to attributes over body and vice versa.
start_atenumendStart reading the file from the 'beginning' or 'end'.
*required field