Amazon Managed Prometheus
This Amazon Managed Prometheus destination can be used to send metrics to an Amazon Managed Prometheus (AMP) workspace in AWS.
Getting the BindPlane Agent authenticated and authorized with AWS requires completing the following steps. Before starting, make sure you are logged in to AWS Console and have permission to create users, create roles, and generate access tokens.
1. Create an AWS User the exporter can use.
To start, head to the
Identity and Access Management (IAM) service and under
Access management navigate to
Users. There should be an option to
Create user on this page. This is the user the exporter and BindPlane Agent will be authenticated as, so name it appropriately. This new user will not need access to AWS Management Console, so leave this option unselected. For permissions, you don't need to specify any at this point since the user will be assuming a role with the required permissions. Once the user is created, navigate to its summary page and copy the ARN (we will need it in the next step).
2. Create an AWS Role the exporter can assume.
Now we need to create an AWS Role the exporter will assume. This is what will allow the exporter/BindPlane Agent to send data to the AMP workspace. To do this, head to
IAM and under
Access management navigate to
Roles and select
Create role. Under the first step,
Select trusted entity, we are defining which AWS resource will be allowed to use this role, which we want to be the user we created in the first step. Select
Custom trust policy. We want to edit just the
Principal JSON field to contain the ARN copied in the first step. It should look similar to this:
2 "AWS": "arn:aws:iam::123456789:user/BindPlane"
The next step in creating the role is
Add permissions. Search for "Prometheus" in the search bar and a number of permissions should show up. The exporter only needs
AmazonPrometheusRemoteWriteAccess, so select that. In the next area, give the role an appropriate name and description and finish with
As an alternative to the AWS pre-defined
AmazonPrometheusRemoteWriteAccess permission policy, you may also create your own permission policy. To read more, see this AWS documentation. If taking this approach, be sure the exporter will still have write access to the desired AMP Workspace or the BindPlane Agent will fail.
3. Give BindPlane Agent access keys.
Now that the BindPlane Agent will have access to the AMP workspace, it needs access keys so it can access AWS as the user that was created for it in step 1. Head back to
Users and select the user created in step 1. Now select the
Security credentials tab and find the
Access keys section. For the first step of creating a key, select
Other and continue to the next step. Provide an appropriate description and select
Create access key. Make note of or download the
Access key and
Secret access key values as AWS won't show these again.
Now that the access keys have been created, we need to create environment variables for them on the same machine as the BindPlane Agent. The environment variables to add are
AWS_SECRET_ACCESS_KEY, which correspond to the values retrieved early in this step.
You can modify the collector's environment variables by configuring a
systemd override. Run
sudo systemctl edit observiq-otel-collector and add the keys:
Now the BindPlane Agent will be able to authenticate with AWS as the user that was created back in step 1.
For more information on specifying the AWS credentials for the BindPlane Agent, please see this AWS documentation. See this AWS documentation for alternative ways of using AWS environment variables for you specific environment.
4. Retrieve Amazon Managed Prometheus workspace endpoint.
The last pre-requisite step to configuring the Amazon Managed Prometheus destination is retrieving the remote write endpoint. If you haven't already created an Amazon Managed Prometheus workspace, search for "Amazon Prometheus" in the AWS console search bar and initialize one. With a workspace created, navigate to
All workspaces inside of the Amazon Prometheus AWS service. Select the workspace you wish to send metrics to. On the summary page, copy the value for
Endpoint - remote write URL. This is the endpoint you'll use when configuring the destination.
|The remote write endpoint to send metrics to the Amazon Managed Prometheus workspace. See this AMP documentation for more.
|The region the Amazon Managed Prometheus workspace is located in. One location it can be found is in the given endpoint. See this AMP documentation for more.
|The Amazon Resource Name (ARN) for the AWS role the exporter will assume. The role should have permission to write to Amazon Managed Prometheus. See this AMP documentation for more.
|The region to use for assuming the
AWS Role. Useful for cross region authentication (i.e. if the agent is located in a different region than the AMP workspace). See this AMP documentation for more.
|Optional name to give the session when the exporter assumes the
AWS Role. Useful to differentiate sessions when multiple principals could assume the given role. See this AWS Documentation for more.
|Compression algorithm to use when sending data to Amazon Managed Prometheus.
|Prefix to attach to each metric name. See this Prometheus documentation for more.
|Max Batch Size
|Maximum size, in bytes, for a batch of metrics to be sent to the AMP endpoint. If a batch is larger than this limit, it will be broken up into multiple batches.
|Whether or not to convert all resource attributes to metric attributes.
|Unit & Type Suffixes
|Whether or not to attach the metric unit and metric type to the end of the metric name.
|Whether or not a "_created" metric is exported for Summary, Histogram, and Monotonic Sum metric points when "StartTimeUnixNano" is set.
|Additional headers to attach to each HTTP Request. The following headers cannot be changed: Content-Encoding, Content-Type, X-Prometheus-Remote-Write-Version, and User-Agent.
|Label names and values to be attached as metric attributes. See this Prometheus documentation for more.
|Whether or not to use TLS.
|Strict TLS Verify
|Whether or not to use Strict TLS Certification Verification.
|Certificate Authority File
|Certificate authority used to validate TLS certificates. Not required if the collector's operating system already trusts the certificate authority.
|Enable Mutual TLS
|Whether ot not to use Mutual TLS.
|Client Certification File
|A TLS certificate used for client authentication, if Mutual TLS is enabled.
|Client Private Key File
|A TLS private key used for client authentication, if Mutual TLS is enabled.
|Enable Remote Queue
|Whether or not to enable a remote write queue. This helps balance outgoing results.
|Remote Write Queue Size
|The number of metrics that can be queued.
|Remote Write Queue Consumers
|The minimum number of workers to use to fan out the outgoing requests.
This destination supports the following retry and queuing settings:
|Retry on Failure
Here we configure the destination to send to AMP by providing the endpoint, region, and arn. Some advanced configuration options we make use of include compression, namespace, max batch size, convert attributes, unit and type suffixes, created metric, and external labels by defining a label called "bp_agent" with a value of "agent1". We also enable TLS, Strict TLS, and Mutual TLS and provide a certificate authority file, client certificate file, and client key file. We also have a remote write queue and Retry on Failure enabled with their respective default configurations.
4 id: aws_managed_prometheus
5 name: aws_managed_prometheus
7 type: aws_managed_prometheus
9 - name: endpoint
10 value: some_aws_amp_workspace_endpoint
11 - name: region
12 value: us-east-2
13 - name: arn
14 value: some_aws_role_arn
15 - name: compression
16 value: snappy
17 - name: namespace
18 value: bp_amp
19 - name: max_batch_size_bytes
20 value: 3000000
21 - name: enable_resource_to_telemetry_conversion
22 value: true
23 - name: add_unit_and_type
24 value: true
25 - name: export_created_metric
26 value: true
27 - name: external_labels
29 bp_agent: agent1
30 - name: enable_tls
31 value: true
32 - name: strict_tls_verify
33 value: true
34 - name: ca_file
35 value: some_ca_file
36 - name: mutual_tls
37 value: true
38 - name: cert_file
39 value: some_cert_file
40 - name: key_file
41 value: some_key_file
42 - name: enable_remote_write_queue
43 value: true
44 - name: retry_on_failure_enabled
45 value: true