The Observability Blog

Categories:
  • Uncategorized

BindPlane OP Build Process – Using Goreleaser

Joe Sirianni Headshot
by Joe Sirianni on
July 28, 2022

Intro

BindPlane OP is written in Go. It is a single http webserver, serving REST, Websocket, and Graphql clients. It includes embedded react applications for serving the user interface.

Go provides us with the ability to produce a single binary program that has no external dependencies. The binary is not dynamically linked to external libraries, meaning it is easy to build, deploy, and run on any platform supported by the Go compiler.

BindPlane OP officially supports Linux, Windows, and macOS. Unofficially, BindPlane OP can be built and run on any platform supported by Go.

Generally, a Go program’s build process is very simple. Simply run “go build” to quickly build your application. BindPlane OP’s build process is more complex. Because of this, we leverage a tool called Goreleaser, for our build platform.

Why Goreleaser?

Why is BindPlane OP’s build process so complex? Technically it is not. Nothing stops a developer from running `go build` within the `cmd/bindplane` directory. This will output a perfectly functional `bindplane` binary assuming they have already built the `ui` portion of the application.

We leverage Goreleaser to streamline this build process. We have several requirements:

  • Multiple binaries: BindPlane consists of the `bindplane` and `bindplanectl` commands. Goreleaser can build both of these.
  • Pre build commands: Goreleaser can run `make` targets that build the embedded react application (BindPlane OP’s web interface).
  • Packages: Goreleaser gives us the ability to easily build Debian (deb) and Red Hat (rpm) packages.
  • macOS (Homebrew) packages.
  • Container images: Gorelease can build and tag images for multiple CPU architectures. Each BindPlane release contains tags for major, major.minor, and major.minor.patch tags. BindPlane OP’s container images are multi architecture (amd64 / arm64) capable.
  • Generated release notes: Gorelease can generate release notes based on pull request names.
  • Automatic release publishing: Goreleaser can create a Github release when a new tag is pushed. This release will contain the generated release notes, archives containing both commands, and linux packages.

Without a tool like Goreleaser, we would need to maintain complicated `make` targets and shell scripts.

Goreleaser Usage

BindPlane OP’s Goreleaser configuration can be found here.

Goreleaser has many features, some of the features I find notable will be detailed below

  • Before hooks
  • Build
  • Linux Packages
  • Container Images
  • Homebrew

Before Hooks

At the top of the configuration, you will find `before.hooks`. This section allows Goreleaser to run arbitrary commands to ensure the build environment is prepared for the actual build. In this case, Goreleaser will run `make ci` and `make ui-build`. These commands will generate the Node JS web interface, which will then be bundled into the binary and served by the Go web server. You can learn more about this process here.

before:
  hooks:
    - make ci
    - make ui-build

Build

The builds section is for building binaries. BindPlane OP has two binaries: `bindplane` (the server component), and `bindplanectl`, the command line interface. The build’s section allows us to specify a GOOS and GOARCH matrix, as well as specify exclusion rules. For example, we want to build for linux, windows, and darwin and arm, arm64, and amd64. We exclude arm on Windows builds.

Additionally, we can specify an environment and compiler flags.

builds:
- id: bindplane
  main: ./cmd/bindplane
  env:
    - CGO_ENABLED=0
  mod_timestamp: '{{ .CommitTimestamp }}'
  goos:
    - windows
    - linux
    - darwin
  goarch:
    - amd64
    - arm64
    - arm
  ignore:
    - goos: windows
      goarch: arm
  binary: 'bindplane'
  ldflags:
    - -X github.com/observiq/bindplane-op/internal/version.gitTag=v{{ .Version }}
- id: bindplanectl
  main: ./cmd/bindplanectl
  env:
    - CGO_ENABLED=0
  mod_timestamp: '{{ .CommitTimestamp }}'
  goos:
    - windows
    - linux
    - darwin
  goarch:
    - amd64
    - arm64
    - arm
  ignore:
    - goos: windows
      goarch: arm
  binary: 'bindplanectl'
  ldflags:
    - -X github.com/observiq/bindplane-op/internal/version.gitTag=v{{ .Version }}

Linux Packages

The `nfpms` section allows us to build Linux packages. Nfpm supports Debian (deb), Red Hat (rpm), and Alpine (apk) packages.

Bindplanectl is simple to package. It includes only the compiled binary, and places it in `/usr/local/bin`.

Bindplane’s binary is much more complicated, as it includes pre and post install scripts, directory creation, default configuration, and a systemd service file. The pre install script handles setting up a system user, while the post installation script handles setting up the systemd service. Packages are a great way to distribute a service that is intended to run on Linux.

nfpms:
- id: bindplanectl
  package_name: bindplanectl
  builds:
    - bindplanectl
  vendor: observIQ, Inc
  homepage: https://github.com/observIQ/bindplane-op
  maintainer: observIQ, Inc
  description: Next generation agent management platform
  license: Apache 2.0
  formats:
  - rpm
  - deb
  - apk
  bindir: /usr/local/bin

- id: bindplane
  package_name: bindplane
  builds:
    - bindplane
  vendor: observIQ, Inc
  homepage: https://github.com/observIQ/bindplane-op
  maintainer: observIQ, Inc
  description: Next generation agent management platform
  license: Apache 2.0
  formats:
  - rpm
  - deb
  bindir: /usr/local/bin
  contents:
  - dst: /var/lib/bindplane
    type: dir
    file_info:
      owner: bindplane
      group: bindplane
      mode: 0750
  - dst: /var/lib/bindplane/storage
    type: dir
    file_info:
      owner: bindplane
      group: bindplane
      mode: 0750
  - dst: /var/lib/bindplane/downloads
    type: dir
    file_info:
      owner: bindplane
      group: bindplane
      mode: 0750
  - dst: /var/log/bindplane
    type: dir
    file_info:
      owner: bindplane
      group: bindplane
      mode: 0750
  - src: scripts/systemd/bindplane.service
    dst: /usr/lib/systemd/system/bindplane.service
    type: "config"
    file_info:
      owner: root
      group: root
      mode: 0640
  - dst: /etc/bindplane
    type: dir
    file_info:
      owner: bindplane
      group: bindplane
      mode: 0750
  - src: scripts/package/bindplane.example.yaml
    dst: /etc/bindplane/config.yaml
    type: "config|noreplace"
    file_info:
      owner: bindplane
      group: bindplane
      mode: 0640
  scripts:
    preinstall: "./scripts/package/preinstall.sh"
    postinstall: ./scripts/package/postinstall.sh

Container Images

Goreleaser has great support for building container images that support multiple architectures. This is done by building independent images “observiq/bindplane-amd64” and “observiq/bindplane-arm64”. These images contain the correct binary for their architecture.

Next, Goreleaser uses the `docker_manifest` to define a multi architecture container manifest which contains both amd64 and arm64 images. When your container runtime pulls the image “observiq/bindplane”, it will detect the correct underlying image for your cpu architecture.

The configuration for building BindPlane OP’s container images looks like this:

docker:


- goos: linux

 goarch: amd64

 ids:

 - bindplane

 image_templates:

 - "observiq/bindplane-amd64:latest"
 - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
 - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}"
 - "observiq/bindplane-amd64:{{ .Major }}"
 # ShortCommit: git rev-parse --short HEAD
 - "observiq/bindplane-amd64:{{ .ShortCommit }}"

 dockerfile: ./Dockerfile

 use: buildx

 build_flag_templates:

 - "--label=created={{.Date}}"
 - "--label=title={{.ProjectName}}"
 - "--label=revision={{.FullCommit}}"
 - "--label=version={{.Version}}"
 - "--platform=linux/amd64"

- goos: linux

 goarch: arm64

 ids:

 - bindplane

 image_templates:

 - "observiq/bindplane-arm64:latest"
 - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
 - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}"
 - "observiq/bindplane-arm64:{{ .Major }}"
 # ShortCommit: git rev-parse --short HEAD
 - "observiq/bindplane-arm64:{{ .ShortCommit }}"

 dockerfile: ./Dockerfile

 use: buildx

 build_flag_templates:

 - "--label=created={{.Date}}"
 - "--label=title={{.ProjectName}}"
 - "--label=revision={{.FullCommit}}"
 - "--label=version={{.Version}}"
 - "--platform=linux/arm64"


docker_manifests:

 - name_template: "observiq/bindplane:latest"

   image_templates:

     - "observiq/bindplane-amd64:latest"
     - "observiq/bindplane-arm64:latest"

   skip_push: false

 - name_template: "observiq/bindplane:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"

   image_templates:

     - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
     - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"

   skip_push: false

 - name_template: "observiq/bindplane:{{ .Major }}.{{ .Minor }}"

   image_templates:

     - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}"
     - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}"

   skip_push: false

 - name_template: "observiq/bindplane:{{ .Major }}"

   image_templates:

     - "observiq/bindplane-amd64:{{ .Major }}"
     - "observiq/bindplane-arm64:{{ .Major }}"

   skip_push: false

 # ShortCommit: git rev-parse --short HEAD

 - name_template: "observiq/bindplane:{{ .ShortCommit }}"

   image_templates:

     - "observiq/bindplane-amd64:{{ .ShortCommit }}"
     - "observiq/bindplane-arm64:{{ .ShortCommit }}"

Note that the tagging strategy involves tagging `major`, `major.minor`, and `major.minor.patch`. This allows users to pin to a given major or minor release, without needing to rely on something like “latest”. If the user wishes to pin to a given release, they can simply use the `major.minor.patch` tag to prevent automatic updates.

Homebrew

Goreleaser supports generating Homebrew configurations. The configuration is simple, just point Goreleaser to a repository and let it generate the correct ruby code.

brews:
- name: bindplane
  tap:
    owner: observIQ
    name: homebrew-bindplane-op
    branch: main
  folder: Formula
  url_template: https://github.com/observIQ/bindplane-op/releases/download/{{ .Tag }}/{{ .ArtifactName }}
  commit_author:
    name: bindplane
    email: support@observiq.com
  homepage: "https://github.com/observIQ/bindplane-op"
  license: "Apache 2.0"

Challenges

  • Some nice features are hidden behind a paywall (Goreleaser Pro). In my experience, the Goreleaser developers have been responsive to issues and feature requests. Supporting their efforts with a Github sponsorship or by paying for Goreleaser Pro seems reasonable to me.
  • Homebrew support is great, but if you have a private homebrew repo, you will need to take extra steps. Homebrew removed the built-in ability to download from a private repository. This is not a Goreleaser issue, but it does complicate things.
  • Building within CI is time consuming. If you want to test packages / container images within your CI pipeline, you must perform a full build. This is correct as of 7/12/2022.

Conclusion

Building a Go application is generally a simple process. Goreleaser brings value in allowing you to standardize your build across many applications. At observIQ, we use Goreleaser for many public and private repositories. Goreleaser makes it easy to generate Linux packages and container images. The configuration is so simple, we have a hard time justifying not building these extra artifacts.

Useful links