How to Set Up Stanza as the Log Agent for Your GCP

Stanza is a robust log agent. GCP users can use Stanza for ingesting large volumes of log data. Before we dive into the configuration steps, here’s a matrix detailing the functional differences between all the common log agents GCP users use.

Stanza was built as a modernized version of FluentD, Fluentbit, and Logstash. GCP users can now install Stanza to their VMs/ GKE clusters to ingest logs and route them to GCP log explorer. This post details the steps for installing Stanza to Linux, Windows, and Kubernetes environments and viewing the logs in the GCP log explorer.

Stanza for a Linux VM in GCP:

1. Single line installation command: In your VM, run the following single line installation command for Stanza

sh -c "$(curl -fsSlL https://github.com/observiq/stanza/releases/latest/download/unix-install.sh)" unix-install.sh

Related Content: Getting Started with BindPlane OP and Google Cloud Operations

Stanza installation for Linux

2. Once the installation is complete, the following message displays. It provides the commands for starting and stopping Stanza. In addition, it shows the path for the config file.

Post-installation instructions

3. Check if Stanza is running using

ps -ef | grep stanza

Check if Stanza is available on the VM

4. Open the config.yaml using the command vi config.yaml. In this example, we use the vi editor. Alternatively, any Linux/unix editor could edit and save the config file.

5. Comment everything except the following in the config.yaml and save the file

type: file_input include: file path output: example_output id: example_output type: google_cloud_output

Change the config.yaml to route logs to the GCP Logs Explorer.

6. After saving the config file, stop and start Stanza using the following commands and verify if the service is running.

systemctl stop stanza systemctl start stanza ps -ef | grep stanza

Related Content: Exploring & Remediating Consumption Costs with Google Billing and BindPlane OP

Stop and Start Stanza after changing the config file.

7. Run a search query as shown below to verify if the logs are sent to the GCP cloud and if they are available for viewing in GCP’s log Explorer.

Stanza for a Windows VM in GCP:

1. Access the command line and run the single-line installation command for Windows.

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 ; Invoke-Expression ((New-Object net.webclient).DownloadString('https://github.com/observiq/stanza/releases/latest/download/windows-install.ps1')); Log-Agent-Install

Windows VM in GCP log agent installation

2. Check if Stanza is running using

ps -ef | grep stanza

3. Make the following edits to the config.yaml

type: file_input include: file path output: example_output id: example_output type: google_cloud_output

4. Stop and start Stanza using the following commands After the config file is saved.

1net stop stanza
2net start stanza

5. Verify if the logs are routed to the correct path and are available in GCP’s log explorer. To do this, in the Log Explorer console, enter the following query:

label.file.name: ”stanza.log” stanza

Windows logs in GCP

Stanza for ingesting logs from GKE clusters:

1. As a prerequisite, follow the instructions in the link to create credentials secret on a JSON file.

2. Download and add the following files to the bucket in GCP. The files are available at

https://github.com/observIQ/stanza/tree/7537876b255cf727106e8b6ca6622c8a07f36c57/examples/k8s/events

3. Run the following command in the GKE cluster to create the service account. Our sample application is running an e-commerce application. For test purposes, you can use this application linked here.

kubectl apply -f service_account.yaml

4. Run the following command to create the config map.

kubectl apply -f config.yaml

5. Run the following command to deploy the agent to the GKE cluster

kubectl apply -f deployment.yaml

6. Here’s a sample of the expected configuration if you do it manually.

https://github.com/observIQ/stanza/blob/master/examples/k8s/daemonset/daemonset_gke.yaml

7. Verify in the logs explorer that Stanza ingests GKE cluster logs for the application or the test application.

Verify that Stanza ingests the GKE cluster logs.

Stanza for ingesting logs from MySQL databases in GCP:

1. Use the following default configurations to begin ingesting logs.

- enable_error_log: true enable_general_log: false enable_mariadb_audit_log: false enable_slow_log: false error_log_path: /var/log/mysql/error.log start_at: end type: mysql - type: google_cloud_output

2. Verify that the logs are ingested from the database in the logs explorer.

View the MySQL logs ingested in Log Explorer.

Stanza is a lightweight log transporter and processing agent. Clubbed with the great features of GCP, you should be able to set up flawless end-to-end observability for your applications. Try this out and write to us with your requests/ suggestions.