Snowflake

Description

This destination can send logs, metrics, and traces to Snowflake, a cloud data warehouse service.

Supported Types

Prerequisites

• Snowflake data warehouse
• Snowflake user with appropriate privileges

The following guide will detail how to get a user initialized in Snowflake that can be used with this destination. It is meant to get telemetry flowing with limited time spent configuring. If you'd like to take an alternative approach, check out the exporter documentation on this topic here.

Snowflake has a variety of ways to connect to it, but these instructions will be tailored for "Classic Console" as all accounts have access to it.

Before starting, log in to Classic Console using a user that has access to the ACCOUNTADMIN role or another role in you Snowflake account that has permission to grant privileges and create users. If the default role is not the required one, then you'll need to assume that role using this SQL command (replace the role as needed):

1ASSUME ROLE "ACCOUNTADMIN";

These instructions will grant privileges to one of the default roles Snowflake is initialized with, SYSADMIN. If you want to grant privileges to a different role then just switch out SYSADMIN for your role in the SQL commands.

1. Grant Warehouse Usage

First, we need to grant the USAGE privilege to the SYSADMIN role on the data warehouse telemetry data will be stored in. Run this SQL command next (replace TEST with your warehouse name):

1GRANT USAGE ON WAREHOUSE "TEST" TO ROLE "SYSADMIN";

2. Grant Create Database Privilege

Next the SYSADMIN role needs to be granted the ability to create databases in the Snowflake account. Run the following SQL to do so:

1GRANT CREATE DATABASE ON ACCOUNT TO ROLE "SYSADMIN";

3. Create New User For BindPlane

Now a new user needs to be created that the BindPlane Agent can login as. The user should also have the default role assigned as SYSADMIN, although it isn't necessary.

Note: If the default role is not assigned, then the exporter will need to be configured with the correct role to work.

Remember the login name and password you use and configure the destination with these values. Replace the user, password, and login name in the following SQL to match yours:

1CREATE USER BP_AGENT PASSWORD="password" LOGIN_NAME="BP_AGENT" DEFAULT_ROLE="SYSADMIN";

4. Grant Privilege to SYSADMIN Role

Even though the default role was set as SYSADMIN we still need to grant the new account permission to it. This can be done using the next SQL command (replace user as needed):

1GRANT ROLE "SYSADMIN" TO USER BP_AGENT;

Now we have a Snowflake user with the correct permissions to be able to create a database, schemas, and tables and also use the configured warehouse to store telemetry data in.

Configuration

This destination supports the following retry and queuing settings:

Example Configuration

Here we will configure this destination to send all telemetry types to a Snowflake account using the default values for database resources. We'll also configure the sending queue, persistent queue, and retry on failure.

Web Interface

Standalone Destination

1apiVersion: BindPlane.observiq.com/v1
2kind: Destination
3metadata:
4  id: snowflake
5  name: snowflake
6spec:
7  type: snowflake
8  parameters:
9    - name: telemetry_types
10      value: ['Logs', 'Metrics', 'Traces']
11    - name: account_id
12      value: 'my_account_id'
13    - name: warehouse
14      value: 'my_warehouse'
15    - name: username
16      value: 'my_user'
17    - name: password
18      value: 'my_pass'
19    - name: sending_queue_enabled
20      value: true
21    - name: persistent_queue_enabled
22      value: true
23    - name: retry_on_failure_enabled
24      value: true