(p)retired

Posts

May 6, 2021

Firestore Golang Timestamps & Merging

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quickly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum.

I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

read more
April 19, 2021

Prometheus Service Discovery w/ Consul for Cloud Run

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus.

This is one solution.

NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use.

Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation. Unfortunately, Cloud Run is not explicitly supported. The alternative appears to be to use file-based discovery but this seems ‘challenging’; it requires, for example, reloading Prometheus on file changes.

read more
April 12, 2021

Cloud Firestore Triggers in Golang

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language.

I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.16. Come on Google!

read more
April 12, 2021

Fly.io

I spent some time over the weekend understanding Fly.io. It’s always fascinating to me how many smart people are building really neat solutions. Fly.io is subtly different to other platforms that I use (Kubernetes, GCP, DO, Linode) and I’ve found the Fly.io team to be highly responsive and helpful to my noob questions.

One of the team’s posts, Docker without Docker surfaced in my Feedly feed (hackernews) and it piqued my interest.

read more
March 17, 2021

Using Golang with the Firestore Emulator

Firestore

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang).

Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form:

package main

import (
  "context"
  "crypto/sha256"
  "fmt"
  "log"
  "os"
  "time"

  "cloud.google.com/go/firestore"
)

func hash(s string) string {
  h := sha256.New()
  h.Write([]byte(s))
  return fmt.Sprintf("%x", h.Sum(nil))
}

type Dog struct {
  Name    string                 `firestore:"name"`
  Age     int                    `firestore:"age"`
  Human   *firestore.DocumentRef `firestore:"human"`
  Created time.Time              `firestore:"created"`
}

func NewDog(name string, age int, human *firestore.DocumentRef) Dog {
  return Dog{
    Name:    name,
    Age:     age,
    Human:   human,
    Created: time.Now(),
  }
}

func (d *Dog) ID() string {
  return hash(d.Name)
}

type Human struct {
  Name string `firestore:"name"`
}

func (h *Human) ID() string {
  return hash(h.Name)
}

func main() {
  ctx := context.Background()
  project := os.Getenv("PROJECT")

  client, err := firestore.NewClient(ctx, project)

  if value := os.Getenv("FIRESTORE_EMULATOR_HOST"); value != "" {
    log.Printf("Using Firestore Emulator: %s", value)
  }

  if err != nil {
    log.Fatal(err)
  }
  defer client.Close()

  me := Human{
    Name: "me",
  }
  meDocRef := client.Collection("humans").Doc(me.ID())
  if _, err := meDocRef.Set(ctx, me); err != nil {
    log.Fatal(err)
  }

  freddie := NewDog("Freddie", 2, meDocRef)
  freddieDocRef := client.Collection("dogs").Doc(freddie.ID())
  if _, err := freddieDocRef.Set(ctx, freddie); err != nil {
    log.Fatal(err)
  }
}

Then you can interact instead with the Firestore Emulator.

read more
March 16, 2021

Programmatically deploying Cloud Run services (Golang|Python)

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so.

My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work.

Goal

Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run.

i.e. achieve this:

gcloud run deploy ${NAME} \
--image=${IMAGE} \
--platform=managed \
--no-allow-unauthenticated \
--region=${REGION} \
--project=${PROJECT}

TRICK --log-http is your friend

read more
March 5, 2021

Prometheus VPA Recommendations

Phew!

For Want of a Nail

I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered:

  • kubectl top
  • Vertical Pod Autoscaler
  • A (valuable) digression through PodMonitor
  • kube-state-metrics
  • `kubectl-patch
  • Created a Graph
  • References

Kubernetes Resources

Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests.

E.g.:

resources:
  limits:
    cpu: "1"
    memory: "512Mi"

I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

read more
February 22, 2021

Dapr

It’s a good name, I read it as “dapper” but I frequently type “darp” :-(

Was interested to read that Dapr is now v1.0 and decided to check it out. I was initially confused between Dapr and service mesh functionality. But, having used Dapr, it appears to be more focused in aiding the development of (cloud-native) (distributed) apps by providing developers with abstractions for e.g. service discovery, eventing, observability whereas service meshes feel (!) more oriented towards simplifying the deployment of existing apps. Both use the concept of proxies, deployed alongside app components (as sidecars on Kubernetes) to provide their functionality to apps.

read more
January 22, 2021

Krustlet on DO Managed Kubernetes

I’ve spent time this week returning to the interesting Deislabs project Krustlet. Since the last time, the bootstrapping process has been simplified using Kubernetes Bootstrap Tokens. I know this updated process works with MicroK8s. Unfortunately, I’m struggling with it on GKE and thought I’d try DigitalOcean Managed Kubernetes.

It worked first time!

In the following, we run both the Kubernetes cluster and the Krustlet Droplet on DigitalOcean but, as long as the cluster and the VM are able to communicate with one another, you should be able to run these anywhere.

read more
January 8, 2021

Kubernetes cert-manager

I developed an admission webhook for Akri, twice (Golang, Rust). I naively followed other examples for the generation of the certificates, created a 1.20 cluster and broke that process.

I’d briefly considered using cert-manager recently but quickly abandoned the idea thinking it would be onerous and unnecessary complexity for little-old-me. I was wrong. It’s excellent and I recommend it highly.

I won’t reproduce the v1beta1 and v1 examples from the Stackoverflow question as they should be self-explanatory. I suspect (!?) that I should not have used Kubernete’s (API Server’s) CA for the Webhook but it could well be that I just don’t understand the correct approach.

read more
  • ««
  • «
  • 7
  • 8
  • 9
  • 10
  • 11
  • »
  • »»
© (p)retired 2025