Julia’s post Blog about what you’ve struggled with resonates because I’ve been struggling with Golang structs in a project. Not the definitions of structs but seemingly needing to reproduce them across the project. I realize that each instance of these resources differs from the others but I’m particularly concerned by having to duplicate method implementations on them.

I’m kinda hoping that I see the solution to my problem by writing it out. If you’re reading this, I didn’t :-(

I’ve written a basic discoverer of Google Cloud Run services. This is for a project and it extends work done in some previous posts to Multiplex gRPC and Prometheus with Cloud Run and to use Consul for Prometheus service discovery.

This solution:

• Accepts a set of Google Cloud Platform (GCP) projects
• Trawls them for Cloud Run services
• Assumes that the services expose Prometheus metrics on :443/metrics
• Relabels the services
• Surfaces any discovered Cloud Run services’ metrics in Prometheus

You’ll need Docker and Docker Compose.

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another.

Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port!

TL;DR See the cmux Limitations and use:

grpcl := m.MatchWithWriters(
   cmux.HTTP2MatchHeaderFieldSendSettings("content-type", "application/grpc"))

Extending the example from the cmux repo:

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quickly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum.

I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus.

This is one solution.

NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use.

Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation. Unfortunately, Cloud Run is not explicitly supported. The alternative appears to be to use file-based discovery but this seems ‘challenging’; it requires, for example, reloading Prometheus on file changes.

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language.

I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.16. Come on Google!

I spent some time over the weekend understanding Fly.io. It’s always fascinating to me how many smart people are building really neat solutions. Fly.io is subtly different to other platforms that I use (Kubernetes, GCP, DO, Linode) and I’ve found the Fly.io team to be highly responsive and helpful to my noob questions.

One of the team’s posts, Docker without Docker surfaced in my Feedly feed (hackernews) and it piqued my interest.

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang).

Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form:

package main

import (
  "context"
  "crypto/sha256"
  "fmt"
  "log"
  "os"
  "time"

  "cloud.google.com/go/firestore"
)

func hash(s string) string {
  h := sha256.New()
  h.Write([]byte(s))
  return fmt.Sprintf("%x", h.Sum(nil))
}

type Dog struct {
  Name    string                 `firestore:"name"`
  Age     int                    `firestore:"age"`
  Human   *firestore.DocumentRef `firestore:"human"`
  Created time.Time              `firestore:"created"`
}

func NewDog(name string, age int, human *firestore.DocumentRef) Dog {
  return Dog{
    Name:    name,
    Age:     age,
    Human:   human,
    Created: time.Now(),
  }
}

func (d *Dog) ID() string {
  return hash(d.Name)
}

type Human struct {
  Name string `firestore:"name"`
}

func (h *Human) ID() string {
  return hash(h.Name)
}

func main() {
  ctx := context.Background()
  project := os.Getenv("PROJECT")

  client, err := firestore.NewClient(ctx, project)

  if value := os.Getenv("FIRESTORE_EMULATOR_HOST"); value != "" {
    log.Printf("Using Firestore Emulator: %s", value)
  }

  if err != nil {
    log.Fatal(err)
  }
  defer client.Close()

  me := Human{
    Name: "me",
  }
  meDocRef := client.Collection("humans").Doc(me.ID())
  if _, err := meDocRef.Set(ctx, me); err != nil {
    log.Fatal(err)
  }

  freddie := NewDog("Freddie", 2, meDocRef)
  freddieDocRef := client.Collection("dogs").Doc(freddie.ID())
  if _, err := freddieDocRef.Set(ctx, freddie); err != nil {
    log.Fatal(err)
  }
}

Then you can interact instead with the Firestore Emulator.

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so.

My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work.

Goal

Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run.

i.e. achieve this:

gcloud run deploy ${NAME} \
--image=${IMAGE} \
--platform=managed \
--no-allow-unauthenticated \
--region=${REGION} \
--project=${PROJECT}

TRICK --log-http is your friend

Phew!

For Want of a Nail

I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered:

• kubectl top
• Vertical Pod Autoscaler
• A (valuable) digression through PodMonitor
• kube-state-metrics
• `kubectl-patch
• Created a Graph
• References

Kubernetes Resources

Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests.

E.g.:

resources:
  limits:
    cpu: "1"
    memory: "512Mi"

I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.