How to Collect Metrics and Logs for NGINX Using the OpsAgent

Why use OpsAgent?

The Ops agent is Google’s recommended agent for collating your application’s telemetry data and forwarding them to GCP for visualization, alerting, and monitoring. The Ops agent collates logs and a metrics collector into one single powerhouse. Some of the key advantages of using the Ops agent are outlined below:

• Ability to monitor and parse logs that are written to file or console.
• It is a lightweight agent.
• The installation and configuration are straightforward and seamless.

How to install it?

Before you install:

1. Identify the most recent version of the Ops agent from Google Cloud documentation. Please note that the Ops agent version installed on the VM to monitor Nginx should be ver 2.1.0 or higher.
2. Get the deployment link for the VM’s operating system.

You may install the Ops agent in the following methods:

• Use the gcloud/ agent policies to install the agent to a fleet of VMs simultaneously.
• Use automation tools such as Ansible, Terraform, etc., to install the agent of a fleet of VMs.
• Install the agent to a single VM from the Google Cloud Console.

Related Content: Getting Started with BindPlane OP and Google Cloud Operations

This post details the steps to install the Ops agent on a single VM(that has Nginx installed) from the Google Cloud Console.

Copy the installation code from Google’s documentation, which is linked here. Please ensure that you have the installation code for the VM’s operating system. In this case, for a Linux Debian 10 VM, the installation code is:

curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh

sudo bash add-google-cloud-ops-agent-repo.sh –also-install

Configuring the Ops agent to collect Nginx logs and metrics

These components must be enabled in the ops agent config to collect logs and metrics.

Access the Ops agent’s config file in the /etc/google-cloud-ops-agent/config.yaml directory and add the following configuration:

1logging: 
2 receivers:
3  nginx_access:
4   type:nginx_access
5  nginx_error:
6   type:nginx_error
7 service:
8  pipelines:
9   nginx:
10    receivers:
11     - nginx_access
12     - nginx_error
13metrics:
14 receivers:
15  nginx:
16   type:nginx
17   stub_status_url: http://127.0.0.1:80/status
18  pipelines:
19   nginx:
20    receivers:
21   - nginx

Broken image

Viewing Nginx logs and metrics

You can view the logs ingested from Nginx under the Logs Explorer.

View the metrics forwarded to Google Cloud Monitoring.

What logs are ingested from Nginx?

The Ops agents ingest and forward the following logs to the Logs viewer. There are two types of Nginx logs: access logs and error logs.

1. Request type: The protocol used in every request sent is logged.
2. Referrer header: Nginx blocks access to a site if the request received has invalid values in the header field. The contents of this header field are logged.
3. Client IP address: This log contains the IP address of the client from which the request is received.
4. HTTP Method: The HTTP method in the request header.
5. Request URL: The URL in the request received.
6. Response size: The response sent to the client from Nginx.
7. HTTP status code: The HTTP status code is sent as part of the response to the client.
8. User-agent header: The contents of the user-agent proxy sent to the client.
9. JSON payload username: The authenticated username sending the request
10. Timestamp: The time that the request is received

Related Content: How to monitor Vault with Google Cloud Platform

What metrics does the Ops agent collect from the instance?

Nginx collects and forwards the following metrics from Nginx.

1. Requests received
2. Connections accepted based on requests
3. Connections handled by Nginx
4. The current active connections

Our telemetry solutions' ease of use is a must for any organization looking to refine and expand their observability. Please get in touch with our customer support team with your questions and requests.