Cloud Build wishlist: Mountable Golang Modules Proxy

- 3 minutes read - 508 words

I think it would be valuable if Google were to provide volumes in Cloud Build of package registries (e.g. Go Modules; PyPi; Maven; NPM etc.).

Google provides a mirror of a subset of Docker Hub. This confers several benefits: Google’s imprimatur; speed (latency); bandwidth; and convenience.

The same benefits would apply to package registries.

In the meantime, there’s a hacky way to gain some of the benefits of these when using Cloud Build.

In the following example, I’ll show an approach using Golang Modules and Google’s module proxy aka proxy.golang.org.

Example

In the following Cloud Build configuration, I have a (not uncommon) build of multiple Golang binaries. This is contrived because the build could be combined into a single step but, for the sake of the argument, let’s assume, I need multiple steps. I’m using a recent Golang runtime (1.14-rc-buster) in which Go Modules are enabled by default.

NB If you’re using an earlier (<1.13?) version of Go, you may enable Modules using GO111MODULE=on and GOPROXY=https://proxy.golang.org.

When invoked on the following configuration, Cloud Build will pull dependencies twice; first to build the server and then again to build the client.

substitutions:
  _GOLANG_VERSION: "1.14-rc-buster"

steps:
  # Build Server
  - name: golang:${_GOLANG_VERSION}
    env:
      - CGO_ENABLED=0
      - GOOS=linux
    args:
      - go
      - build
      - -a
      - -installsuffix
      - cgo
      - -o
      - ./bin/server
      - github.com/DazWilkin/some-project/server

  # Build Client
  - name: golang:${_GOLANG_VERSION}
    env:
      - CGO_ENABLED=0
      - GOOS=linux
    args:
      - go
      - build
      - -a
      - -installsuffix
      - cgo
      - -o
      - ./bin/client
      - github.com/DazWilkin/some-project/client

However, one promise (commitment?) with Go Modules is that a given package@version will never change; once I’ve pulled a package@version once, I need (should) never pull it again. It won’t change. While it’s non-trivial to enforce this across distinct invocations of Cloud Build, it’s possible to implement it within a build using volumes.

In the following configuration, the Golang build steps share (!) a volume called go-modules. This volume is mounted as /go. This has the benefit of sharing /go/bin, /go/src and, of course /go/pkg. /go/pkg is used by Go Modules to persist package versions. Using this mechanism, once a package@version is pulled by an earlier step, it is available to be used by subsequent steps.

# options:
#   env:
#   - GOPATH=/go
#   - GO111MODULE=on
#   - GOPROXY=https://proxy.golang.org
#   volumes:
#   - name: go-modules
#     path: /go

substitutions:
  _GOLANG_VERSION: "1.14-rc-buster"

steps:
  # Build Server
  - name: golang:${_GOLANG_VERSION}
    env:
      - CGO_ENABLED=0
      - GOOS=linux
    args:
      - go
      - build
      - -a
      - -installsuffix
      - cgo
      - -o
      - ./bin/server
      - github.com/DazWilkin/some-project/server
    volumes:
      - name: go-modules
        path: /go

  # Build Client
  - name: golang:${_GOLANG_VERSION}
    env:
      - CGO_ENABLED=0
      - GOOS=linux
    args:
      - go
      - build
      - -a
      - -installsuffix
      - cgo
      - -o
      - ./bin/client
      - github.com/DazWilkin/some-project/client
    volumes:
      - name: go-modules
        path: /go

NB The options section is commented but provides a way to set global (for all step) properties including environment (e.g. GOPROXY) and volumes (e.g. go-modules).

NB It would be possible to replace the repeated volumes sections in each golang step with the global volumes definition defined in options.