BindPlane OP Build Process Using Goreleaser

Intro

BindPlane OP is written in Go. It is a single http webserver that serves REST, Websocket, and Graphql clients. It includes embedded react applications for serving the user interface.

Go provides us with the ability to produce a single binary program that has no external dependencies. The binary is not dynamically linked to external libraries, making it easy to build, deploy, and run on any platform supported by the Go compiler.

BindPlane OP officially supports Linux, Windows, and macOS. Unofficially, BindPlane OP can be built and run on any platform supported by Go.

Generally, the build process of a Go program is straightforward. Run “go build” to quickly build your application. BindPlane OP’s build process is more complex. Because of this, we leverage a tool called Goreleaser for our build platform.

Why Goreleaser?

Why is BindPlane OP’s build process so complex? Technically, it is not. Nothing stops developers from running `go build` within the `cmd/bindplane` directory. This will output a perfectly functional `bindplane` binary, assuming they have already built the `ui` portion of the application.

We leverage Goreleaser to streamline this build process. We have several requirements:

• Multiple binaries: BindPlane consists of the `bindplane` and `bindplanectl` commands. Goreleaser can build both of these.
• Pre-build commands: Goreleaser can run `make` targets that build the embedded react application (BindPlane OP’s web interface).
• Packages: Goreleaser lets us quickly build Debian (deb) and Red Hat (rpm) packages.
• macOS (Homebrew) packages.
• Container images: Gorelease can build and tag images for multiple CPU architectures. Each BindPlane release contains tags for major, major.minor, and major.minor.patch tags. BindPlane OP’s container images can multi-architecture (amd64 / arm64).
• Generated release notes: Gorelease can create release notes based on pull request names.
• Automatic release publishing: Goreleaser can create a Github release when a new tag is pushed. This release will contain the generated release notes and archives containing both commands and Linux packages.

Without a tool like Goreleaser, we would need to maintain complicated `make` targets and shell scripts.

Goreleaser Usage

BindPlane OP’s Goreleaser configuration can be found here.

Goreleaser has many features; some of the features I find notable will be detailed below.

• Before hooks
• Build
• Linux Packages
• Container Images
• Homebrew

Before Hooks

At the top of the configuration, you will find `before.hooks`. This section allows Goreleaser to run arbitrary commands to ensure the build environment is prepared for the actual build. Goreleaser will run `make ci` and `make ui-build` in this case. These commands will generate the Node JS web interface, which will then be bundled into the binary and served by the Go web server. You can learn more about this process here.

1before:
2  hooks:
3    - make ci
4    - make ui-build

Build

The builds section is for building binaries. BindPlane OP has two binaries: `bindplane` (the server component), and `bindplanectl`, the command line interface. The build’s section allows us to specify a GOOS and GOARCH matrix and exclusion rules. For example, we want to build for Linux, Windows, Darwin, ARM, ARM64, and AMD64. We exclude ARM on Windows builds.

Additionally, we can specify an environment and compiler flags.

1builds:
2- id: bindplane
3  main: ./cmd/bindplane
4  env:
5    - CGO_ENABLED=0
6  mod_timestamp: '{{ .CommitTimestamp }}'
7  goos:
8    - windows
9    - linux
10    - darwin
11  goarch:
12    - amd64
13    - arm64
14    - arm
15  ignore:
16    - goos: windows
17      goarch: arm
18  binary: 'bindplane'
19  ldflags:
20    - -X github.com/observiq/bindplane-op/internal/version.gitTag=v{{ .Version }}
21- id: bindplanectl
22  main: ./cmd/bindplanectl
23  env:
24    - CGO_ENABLED=0
25  mod_timestamp: '{{ .CommitTimestamp }}'
26  goos:
27    - windows
28    - linux
29    - darwin
30  goarch:
31    - amd64
32    - arm64
33    - arm
34  ignore:
35    - goos: windows
36      goarch: arm
37  binary: 'bindplanectl'
38  ldflags:
39    - -X github.com/observiq/bindplane-op/internal/version.gitTag=v{{ .Version }}

Linux Packages

The `nfpms` section allows us to build Linux packages. Nfpm supports Debian (deb), Red Hat (rpm), and Alpine (apk) packages.

Bindplanectl is straightforward to package. It includes only the compiled binary, and places it in `/usr/local/bin`.

Bindplane’s binary is much more complicated, including pre and post-install scripts, directory creation, default configuration, and a system service file. The pre-install script handles setting up a system user, while the post-installation script handles setting up the system service. Packages are a great way to distribute a service intended to run on Linux.

1nfpms:
2- id: bindplanectl
3  package_name: bindplanectl
4  builds:
5    - bindplanectl
6  vendor: observIQ, Inc
7  homepage: https://github.com/observIQ/bindplane-op
8  maintainer: observIQ, Inc
9  description: Next generation agent management platform
10  license: Apache 2.0
11  formats:
12  - rpm
13  - deb
14  - apk
15  bindir: /usr/local/bin
16
17- id: bindplane
18  package_name: bindplane
19  builds:
20    - bindplane
21  vendor: observIQ, Inc
22  homepage: https://github.com/observIQ/bindplane-op
23  maintainer: observIQ, Inc
24  description: Next generation agent management platform
25  license: Apache 2.0
26  formats:
27  - rpm
28  - deb
29  bindir: /usr/local/bin
30  contents:
31  - dst: /var/lib/bindplane
32    type: dir
33    file_info:
34      owner: bindplane
35      group: bindplane
36      mode: 0750
37  - dst: /var/lib/bindplane/storage
38    type: dir
39    file_info:
40      owner: bindplane
41      group: bindplane
42      mode: 0750
43  - dst: /var/lib/bindplane/downloads
44    type: dir
45    file_info:
46      owner: bindplane
47      group: bindplane
48      mode: 0750
49  - dst: /var/log/bindplane
50    type: dir
51    file_info:
52      owner: bindplane
53      group: bindplane
54      mode: 0750
55  - src: scripts/systemd/bindplane.service
56    dst: /usr/lib/systemd/system/bindplane.service
57    type: "config"
58    file_info:
59      owner: root
60      group: root
61      mode: 0640
62  - dst: /etc/bindplane
63    type: dir
64    file_info:
65      owner: bindplane
66      group: bindplane
67      mode: 0750
68  - src: scripts/package/bindplane.example.yaml
69    dst: /etc/bindplane/config.yaml
70    type: "config|noreplace"
71    file_info:
72      owner: bindplane
73      group: bindplane
74      mode: 0640
75  scripts:
76    preinstall: "./scripts/package/preinstall.sh"
77    postinstall: ./scripts/package/postinstall.sh

Container Images

Goreleaser has excellent support for building container images that support multiple architectures. This is done by building independent images “observiq/bindplane-amd64” and “observiq/bindplane-arm64”. These images contain the correct binary for their architecture.

Next, Goreleaser uses the `docker_manifest` to define a multi-architecture container manifest containing AMD64 and ARM64 images. When your container runtime pulls the image “observiq/bindplane”, it will detect the correct underlying image for your cpu architecture.

The configuration for building BindPlane OP’s container images looks like this:

1docker:
2
3
4- goos: linux
5
6 goarch: amd64
7
8 ids:
9
10 - bindplane
11
12 image_templates:
13
14 - "observiq/bindplane-amd64:latest"
15 - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
16 - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}"
17 - "observiq/bindplane-amd64:{{ .Major }}"
18 # ShortCommit: git rev-parse --short HEAD
19 - "observiq/bindplane-amd64:{{ .ShortCommit }}"
20
21 dockerfile: ./Dockerfile
22
23 use: buildx
24
25 build_flag_templates:
26
27 - "--label=created={{.Date}}"
28 - "--label=title={{.ProjectName}}"
29 - "--label=revision={{.FullCommit}}"
30 - "--label=version={{.Version}}"
31 - "--platform=linux/amd64"
32
33- goos: linux
34
35 goarch: arm64
36
37 ids:
38
39 - bindplane
40
41 image_templates:
42
43 - "observiq/bindplane-arm64:latest"
44 - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
45 - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}"
46 - "observiq/bindplane-arm64:{{ .Major }}"
47 # ShortCommit: git rev-parse --short HEAD
48 - "observiq/bindplane-arm64:{{ .ShortCommit }}"
49
50 dockerfile: ./Dockerfile
51
52 use: buildx
53
54 build_flag_templates:
55
56 - "--label=created={{.Date}}"
57 - "--label=title={{.ProjectName}}"
58 - "--label=revision={{.FullCommit}}"
59 - "--label=version={{.Version}}"
60 - "--platform=linux/arm64"
61
62
63docker_manifests:
64
65 - name_template: "observiq/bindplane:latest"
66
67   image_templates:
68
69     - "observiq/bindplane-amd64:latest"
70     - "observiq/bindplane-arm64:latest"
71
72   skip_push: false
73
74 - name_template: "observiq/bindplane:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
75
76   image_templates:
77
78     - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
79     - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}.{{ .Patch }}"
80
81   skip_push: false
82
83 - name_template: "observiq/bindplane:{{ .Major }}.{{ .Minor }}"
84
85   image_templates:
86
87     - "observiq/bindplane-amd64:{{ .Major }}.{{ .Minor }}"
88     - "observiq/bindplane-arm64:{{ .Major }}.{{ .Minor }}"
89
90   skip_push: false
91
92 - name_template: "observiq/bindplane:{{ .Major }}"
93
94   image_templates:
95
96     - "observiq/bindplane-amd64:{{ .Major }}"
97     - "observiq/bindplane-arm64:{{ .Major }}"
98
99   skip_push: false
100
101 # ShortCommit: git rev-parse --short HEAD
102
103 - name_template: "observiq/bindplane:{{ .ShortCommit }}"
104
105   image_templates:
106
107     - "observiq/bindplane-amd64:{{ .ShortCommit }}"
108     - "observiq/bindplane-arm64:{{ .ShortCommit }}"

Note that the tagging strategy involves tagging `major`, `major.minor`, and `major.minor.patch`. This allows users to pin to a given major or minor release without relying on something like “latest”. Users wishing to pin to a given release can use the `major.minor.patch` tag to prevent automatic updates.

Homebrew

Goreleaser supports generating Homebrew configurations. The configuration is straightforward: point Goreleaser to a repository and let it generate the correct ruby code.

1brews:
2- name: bindplane
3  tap:
4    owner: observIQ
5    name: homebrew-bindplane-op
6    branch: main
7  folder: Formula
8  url_template: https://github.com/observIQ/bindplane-op/releases/download/{{ .Tag }}/{{ .ArtifactName }}
9  commit_author:
10    name: bindplane
11    email: support@observiq.com
12  homepage: "https://github.com/observIQ/bindplane-op"
13  license: "Apache 2.0"

Challenges

• Some excellent features are hidden behind a paywall (Goreleaser Pro). In my experience, the Goreleaser developers have been responsive to issues and feature requests. Supporting their efforts with a Github sponsorship or paying for Goreleaser Pro is reasonable.
• Homebrew support is great, but you must take extra steps if you have a private homebrew repo. Homebrew removed the built-in ability to download from a private repository. This is not a Goreleaser issue, but it does complicate things.
• Building within CI is time consuming. You'll need to perform a full build to test packages/container images within your CI pipeline. This is correct as of 7/12/2022.

Conclusion

Building a Go application is generally a simple process. Goreleaser brings value in allowing you to standardize your build across many applications. At observIQ, we use Goreleaser for many public and private repositories. Goreleaser makes it easy to generate Linux packages and container images. The configuration is so simple that we have a hard time justifying not building these extra artifacts.

Useful links

• Goreleaser github: https://github.com/goreleaser/goreleaser
• Goreleaser docs: https://goreleaser.com/
• Goreleaser pro: https://goreleaser.com/pro/