Amazon Managed Prometheus

Description

This Amazon Managed Prometheus destination can be used to send metrics to an Amazon Managed Prometheus (AMP) workspace in AWS.

Supported Types

Prerequisites

Getting the BindPlane Agent authenticated and authorized with AWS requires completing the following steps. Before starting, make sure you are logged in to AWS Console and have permission to create users, create roles, and generate access tokens.

1. Create an AWS User the exporter can use.

To start, head to the Identity and Access Management (IAM) service and under Access management navigate to Users. There should be an option to Create user on this page. This is the user the exporter and BindPlane Agent will be authenticated as, so name it appropriately. This new user will not need access to AWS Management Console, so leave this option unselected. For permissions, you don't need to specify any at this point since the user will be assuming a role with the required permissions. Once the user is created, navigate to its summary page and copy the ARN (we will need it in the next step).

2. Create an AWS Role the exporter can assume.

Now we need to create an AWS Role the exporter will assume. This is what will allow the exporter/BindPlane Agent to send data to the AMP workspace. To do this, head to IAM and under Access management navigate to Roles and select Create role. Under the first step, Select trusted entity, we are defining which AWS resource will be allowed to use this role, which we want to be the user we created in the first step. Select Custom trust policy. We want to edit just the Principal JSON field to contain the ARN copied in the first step. It should look similar to this:

1"Principal": {
2    "AWS": "arn:aws:iam::123456789:user/BindPlane"
3}

The next step in creating the role is Add permissions. Search for "Prometheus" in the search bar and a number of permissions should show up. The exporter only needs AmazonPrometheusRemoteWriteAccess, so select that. In the next area, give the role an appropriate name and description and finish with Create role.

As an alternative to the AWS pre-defined AmazonPrometheusRemoteWriteAccess permission policy, you may also create your own permission policy. To read more, see this AWS documentation. If taking this approach, be sure the exporter will still have write access to the desired AMP Workspace or the BindPlane Agent will fail.

3. Give BindPlane Agent access keys.

Now that the BindPlane Agent will have access to the AMP workspace, it needs access keys so it can access AWS as the user that was created for it in step 1. Head back to IAM and Users and select the user created in step 1. Now select the Security credentials tab and find the Access keys section. For the first step of creating a key, select Other and continue to the next step. Provide an appropriate description and select Create access key. Make note of or download the Access key and Secret access key values as AWS won't show these again.

Now that the access keys have been created, we need to create environment variables for them on the same machine as the BindPlane Agent. The environment variables to add are AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, which correspond to the values retrieved early in this step.

You can modify the collector's environment variables by configuring a systemd override. Run sudo systemctl edit observiq-otel-collector and add the keys:

1[Service]
2Environment=AWS_ACCESS_KEY_ID=******
3Environment=AWS_SECRET_ACCESS_KEY=******

When finished, reload Systemd.

1sudo systemctl daemon-reload

Now the BindPlane Agent will be able to authenticate with AWS as the user that was created back in step 1.

For more information on specifying the AWS credentials for the BindPlane Agent, please see this AWS documentation. See this AWS documentation for alternative ways of using AWS environment variables for you specific environment.

4. Retrieve Amazon Managed Prometheus workspace endpoint.

The last pre-requisite step to configuring the Amazon Managed Prometheus destination is retrieving the remote write endpoint. If you haven't already created an Amazon Managed Prometheus workspace, search for "Amazon Prometheus" in the AWS console search bar and initialize one. With a workspace created, navigate to All workspaces inside of the Amazon Prometheus AWS service. Select the workspace you wish to send metrics to. On the summary page, copy the value for Endpoint - remote write URL. This is the endpoint you'll use when configuring the destination.

Configuration

This destination supports the following retry and queuing settings:

Example Configuration

Here we configure the destination to send to AMP by providing the endpoint, region, and arn. Some advanced configuration options we make use of include compression, namespace, max batch size, convert attributes, unit and type suffixes, created metric, and external labels by defining a label called "bp_agent" with a value of "agent1". We also enable TLS, Strict TLS, and Mutual TLS and provide a certificate authority file, client certificate file, and client key file. We also have a remote write queue and Retry on Failure enabled with their respective default configurations.

Web Interface

Standalone Destination

1apiVersion: BindPlane.observiq.com/v1
2kind: Destination
3metadata:
4  id: aws_managed_prometheus
5  name: aws_managed_prometheus
6spec:
7  type: aws_managed_prometheus
8  parameters:
9    - name: endpoint
10      value: some_aws_amp_workspace_endpoint
11    - name: region
12      value: us-east-2
13    - name: arn
14      value: some_aws_role_arn
15    - name: compression
16      value: snappy
17    - name: namespace
18      value: bp_amp
19    - name: max_batch_size_bytes
20      value: 3000000
21    - name: enable_resource_to_telemetry_conversion
22      value: true
23    - name: add_unit_and_type
24      value: true
25    - name: export_created_metric
26      value: true
27    - name: external_labels
28      value:
29        bp_agent: agent1
30    - name: enable_tls
31      value: true
32    - name: strict_tls_verify
33      value: true
34    - name: ca_file
35      value: some_ca_file
36    - name: mutual_tls
37      value: true
38    - name: cert_file
39      value: some_cert_file
40    - name: key_file
41      value: some_key_file
42    - name: enable_remote_write_queue
43      value: true
44    - name: retry_on_failure_enabled
45      value: true