Posts

Using Google's Public Certificate Authority with Golang autocert

Last year, I wrote about using Automatic Certs w/ Golang gRPC service on Compute Engine. That solution uses ACME with (the wonderful) Let’s Encrypt. Google is offering a private preview of Automate Public Certificates Lifecycle Management via RFC 8555 (ACME) and, because I’m using Google Cloud Platform extensively to build a “thing” and I think it would be useful to have a backup to Let’s Encrypt, I thought I’d give the solution a try.

Posts

Automatic Certs w/ Golang gRPC service on Compute Engine

I needed to deploy a healthcheck-enabled gRPC TLS-enabled service. Fortunately, most (all?) of the SDKs include an implementation, e.g. Golang has grpc-go/health. I learned in my travels that: DigitalOcean [App] platform does not (link) work with TLS-based gRPC apps. Fly has a regression (link) that breaks gRPC So, I resorted to Google Cloud Platform (GCP). Although Cloud Run would be well-suited to running the gRPC app, it uses a proxy|sidecar to provision a cert for the app and I wanted to be able to (easily use a custom domain) and give myself a somewhat general-purpose solution.

Posts

Actions SDK Conversational Quickstart

Google’s tutorial didn’t work for me. In this post, I’ll help you get this working. https://developers.google.com/assistant/conversational/quickstart Create and set up a project This mostly works. I recommend using the Actions Console as described to create the project. I chose “Custom” and “Blank Project” You need not enable Actions API as this is done automatically: For the console work, I’m going to use Google’s excellent Cloud Shell. You may access this through the browser or through a terminal:

Posts

Actions SDK Conversational Quickstart

Google’s tutorial didn’t work for me. In this post, I’ll help you get this working. https://developers.google.com/assistant/conversational/quickstart Create and set up a project This mostly works. I recommend using the Actions Console as described to create the project. I chose “Custom” and “Blank Project” You need not enable Actions API as this is done automatically: For the console work, I’m going to use Google’s excellent Cloud Shell. You may access this through the browser or through a terminal:

Posts

Kubernetes cert-manager

I developed an admission webhook for Akri, twice (Golang, Rust). I naively followed other examples for the generation of the certificates, created a 1.20 cluster and broke that process. I’d briefly considered using cert-manager recently but quickly abandoned the idea thinking it would be onerous and unnecessary complexity for little-old-me. I was wrong. It’s excellent and I recommend it highly. I won’t reproduce the v1beta1 and v1 examples from the Stackoverflow question as they should be self-explanatory.

Posts

Kubernetes Webhooks

I spent some time last week writing my first admission webhook for Kubernetes. I wrote the handler in Golang because I’m most familiar with Golang and because, as Kubernetes’ native language, I was more confident that the necessary SDKs would exist and that the documentation would likely use Golang by default. I struggled to find useful documentation and so this post is to help you (and me!) remember how to do this next time!

Posts

Kubernetes Device Plugins

I’m debugging an issue with Akri Zeroconf protocol in which Instance environment variables are no longer (!) being surfaced within the Broker pods. In my adventures, it seemed useful to better understand how Akri works and specifically, how Akri uses Kubernetes Device Plugins. IIUC plugins register with the Kubelet (!) via a gRPC service (Registration) that the Kubelet exposes on a UNIX socket at /var/lib/kubelet/device-plugins/kubelet.sock Then (!) if successful, devices should be reported by the Node’s metadata (spec) and available to be bound to Pods.

Posts

Akri

For the past couple of weeks, I’ve been playing around with Akri, a Microsoft (DeisLabs) project for building a connected edge with Kubernetes. Kubernetes, IoT, Rust (and Golang) make this all compelling to me. Initially, I deployed an Akri End-to-End to MicroK8s on Google Compute Engine (link) and Digital Ocean (link). But I was interested to create me own example and so have proposed a very (!) simple HTTP-based protocol.

Posts

GitHub Actions && GitHub Container Registry

You know when you start something and then regret it!? I think I’ll be sticking with Google Cloud Build; GitHub Actions appears functional and useful but I found the documentation to be confusing and limited and struggled to get a simple container image build|push working. I’ve long used Docker Hub but am planning to use it less as a result of the planned changes. I want to see Docker succeed and to do so it needs to find a way to make money but, there are free alternatives including the new GitHub Container Registry and the very very cheap Google Container Registry.

Posts

akri

I was very interested to read about Microsoft’s DeisLab’s latest (rust-based) Kubernetes project: akri. If I understand it correctly, it provides a mechanism to make any (IoT) device accessible to containers running within a cluster. I need to spend more time playing around with it so that I can fully understand it. I had some problems getting the End-to-End demo running on Google Compute Engine (and then I tried DigitalOcean droplet) instances.

Posts

Automating HackMD documents

I was introduced to HackMD while working on an open-source project. It’s a collaborative editing tool for Markdown documents and there’s an API I wanted to be able to programmatically edit one of my documents with a daily update. The API is easy-to-use and my only challenge was futzing with escape characters in bash strips representing the document Markdown content. You’ll need an account with HackMD and an to Create API Token that I’ll refer to as TOKEN.

Posts

Actions SDK Conversational Quickstart

Google’s tutorial didn’t work for me. In this post, I’ll help you get this working. https://developers.google.com/assistant/conversational/quickstart Create and set up a project This mostly works. I recommend using the Actions Console as described to create the project. I chose “Custom” and “Blank Project” You need not enable Actions API as this is done automatically: For the console work, I’m going to use Google’s excellent Cloud Shell. You may access this through the browser or through a terminal:

Posts

Infrastructure as Code

Problem I’m building an application that comprises: Kubernetes¹ Kubernetes Operator Cloud Firestore Cloud Functions Cloud Run Cloud Endpoints Stripe Firebase Authentication ¹ - I’m using Google Kubernetes Engine (GKE) but may include other managed Kubernetes offerings (e.g. Digital Ocean, Linode, Oracle). GKE clusters are manageable by gcloud but other platforms require other CLI tools. All are accessible from bash but are these supported by e.g. Terraform (see below)?

Posts

Minimizing WASM binaries

I’ve spent time recently playing around with WebAssembly (WASM) and waPC. Rust and WASM were born at Mozilla and there’s a natural affinity with writing WASM binaries in Rust. In the WASM examples I’ve been using for WASM Transparency, waPC and MsgPack and waPC and Protobufs. I’ve created 3 WASM binaries: complex.wasm, simplex.wasm and fabcar.wasm and each is about 2.5MB when: cargo build --target=wasm32-unknown-unknown --release The Rust and WebAssembly book has an excellent section titled Shrinking .

Posts

OriginStamp: Verifying Proofs

Recently, I wrote about some initial adventures with OriginStamp Using OriginStamp’s UI or API, submitting a hash results in transactions being submitted to Bitcoin, Ethereum and a German newspaper. Using the API, it’s possible to query OriginStamp’s service for a proof. This post explains how to verify such a proof. The diligent reader among you (Hey Mom!) will recall that I submitted a hash for the message: Frederik Jack is a bubbly Border Collie The SHA-256 hash of this message is:

Posts

Golang Xiaomi Bluetooth Temperature|Humidity (LYWSD03MMC) 2nd Gen

Well, this became more of an adventure that I’d originally wanted but, after learning some BLE and, with the help of others (Thanks Jonatha, JsBergbau), I’ve sample code that connects to 4 Xiaomi 2nd gen. Thermometers, subscribes to readings and publishes the data to MQTT. From there, I’m scraping it using Inuits MQTTGateway into Prometheus. Repo: https://github.com/DazWilkin/gomijia2 Thanks|Credit: Jonathan McDowell for gomijia and help JsBergbau for help Background I’ve been playing around with ESPHome and blogged around my very positive experience ESPHome, MQTT, Prometheus and almost Cloud IoT.

Posts

Adventures around BPF

I think (!?) this interesting learning experience started with Envoy Go Extensions. IIUC, Cilium contributed this mechanism (Envoy Go Extensions) by which it’s possible to extend Envoy using Golang. The documentation references BPF. Cilium and BNF were both unfamiliar technologies to me and so began my learning. This part of the journey concludes with Weave Scope. I learned that Cilium is a CNI implementation that may be used with Kubernetes. GKE defaults (but is not limited to) to Google’s own CNI implementation (link).

Posts

Run cAdvisor when using Docker Compose

cAdvisor has long been a favorite monitoring tool of mine. I’m using Docker Compose for local testing and have begun including a cAdvisor in my docker-compose.yaml files. cadvisor: restart: always image: google/cadvisor:${CADVISOR_VERSION} container_name: cadvisor # command: # - --prometheus_endpoint="/metrics" # Default volumes: - "/:/rootfs:ro" - "/var/run:/var/run:rw" - "/sys:/sys:ro" - "/var/snap/docker/current:/var/lib/docker:ro" #- "/var/lib/docker/:/var/lib/docker:ro" expose: - "8080" ports: - 8080:8080 I’d not realized until recently, that cAdvisor also surfaces a Prometheus metrics endpoint and so, if you do follow this path and you’re also using Prometheus, don’t forget to add cAdvisor to your Prometheus targets.

Posts

Kubernetes cert-manager

I developed an admission webhook for Akri, twice (Golang, Rust). I naively followed other examples for the generation of the certificates, created a 1.20 cluster and broke that process. I’d briefly considered using cert-manager recently but quickly abandoned the idea thinking it would be onerous and unnecessary complexity for little-old-me. I was wrong. It’s excellent and I recommend it highly. I won’t reproduce the v1beta1 and v1 examples from the Stackoverflow question as they should be self-explanatory.

Posts

Adventures around BPF

I think (!?) this interesting learning experience started with Envoy Go Extensions. IIUC, Cilium contributed this mechanism (Envoy Go Extensions) by which it’s possible to extend Envoy using Golang. The documentation references BPF. Cilium and BNF were both unfamiliar technologies to me and so began my learning. This part of the journey concludes with Weave Scope. I learned that Cilium is a CNI implementation that may be used with Kubernetes. GKE defaults (but is not limited to) to Google’s own CNI implementation (link).

Posts

Vultr CLI and JSON output

I’ve begun exploring Vultr after the company announced a managed Kubernetes offering Vultr Kubernetes Engine (VKE). In my brief experience, it’s a decent platform and its CLI vultr-cli is mostly (!) good. The CLI has a limitation in that command output is text formatted and this makes it challenging to parse the output when scripting. NOTE The Vultr developers have a branch rewrite that includes a solution to this problem.

Posts

Rube Goldberg Cloud Build machine for solving Quadratic equations

Google Cloud Build is described by Google as a “CI/CD platform” but it’s fundamentally a service that permits a series of containers to be chained together in a pipeline, optionally leveraging shared data. As a CI/CD platform, it can be used to lint, test, compile and build software but, if you were looking for a way to explain its basic awesomeness, you could… I don’t know… build a Rube Goldberg style machine that solves Quadratic equations using it 😄

Posts

Cloud Build wishlist: Mountable Golang Modules Proxy

I think it would be valuable if Google were to provide volumes in Cloud Build of package registries (e.g. Go Modules; PyPi; Maven; NPM etc.). Google provides a mirror of a subset of Docker Hub. This confers several benefits: Google’s imprimatur; speed (latency); bandwidth; and convenience. The same benefits would apply to package registries. In the meantime, there’s a hacky way to gain some of the benefits of these when using Cloud Build.

Posts

Renewing Firebase Authentication ID tokens with gRPC

I’ve written before about a project in which I’m using Firebase Authentication in combination with Google Cloud Endpoints and a gRPC service running on Cloud Run: Firebase Authentication, Cloud Endpoints and gRPC (1of2) Firebase Authentication, Cloud Endpoints and gRPC (2of2) This works well with one caveat, the ID tokens (JWTs) minted by Firebase Authentication have a 3600 second (one hour) lifetime. The user flow in my app is that whenever the user invokes the app’s CLI:

Posts

Firebase Authentication, Cloud Endpoints and gRPC (2of2)

Earlier this week, I wrote about using Firebase Authentcation, Cloud Endpoints and gRPC (1of2). Since then, I learned some more and added a gRPC interceptor to implement basic authorization for the service. ESPv2 --allow-unauthenticated The Cloud Enpoints (ESPv2) proxy must be run as --allow-unauthenticated on Cloud Run to ensure that requests make it to the proxy where the request is authenticated and only authenticated requests make it on to the backend service.

Posts

Firebase Authentication, Cloud Endpoints and gRPC (1of2)

I’m building a service that requires user authentication. The primary endpoint is a gRPC-based service. I would like to consider using certificate-based auth but this feels… challenging. Instead, I have been aware of, but never used, Firebase Authentication and was interested to see that Cloud Endpoints includes Firebase Authentication as one of its supported auth mechanisms. Curiosity piqued, I confirmed that gRPC supports Google token-based authentication. The following is a summary of what I did but I’ll leave the extensive documentation to Google, (Google’s) Firebase and gRPC, all of which, in this case, provide really good explanations.

Posts

Cloud Endpoints combine OpenAPI and gRPC... or not!

See: Multiplexing gRPC and HTTP endpoints with Cloud Run gRPC, Cloud Run & Endpoints ESPv2: Configure Cloud Endpoints to proxy traffic to a Cloud Run multiplexed (gRPC|HTTP) service Challenges: Cloud Run permits single port Cloud Run services publishing e.g. gRPC and Prometheus, must multiplex transports Cloud Run services publishing multiplexed transports are challenging to expose using Cloud Endpoints Hypothesis #1: Multiplexed transports work with Cloud Run See: Multiplexing gRPC and HTTP endpoints with Cloud Run

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Cloud Firestore Triggers in Golang

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language. I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.

Posts

webmention

Let’s see if this works!? I’ve added the following link to this site’s baseof.html so that it is now rendered for each page: <link href="https://us-central1-webmention.cloudfunctions.net/webmention" rel="webmention" /> I discovered indieweb yesterday reading about webmention. Who knows what got me to webmention!? The principles of both are sound. Instead of relying on come-go behemoths to run our digital world, indieweb seeks to provide technologies that enable us to achieve things without them.

Posts

WASM Cloud Functions

Following on from waPC & Protobufs and a question on Stack Overflow about Cloud Functions, I was compelled to try running WASM on Cloud Functions no JavaScript. I wanted to reuse the WASM waPC functions that I’d written in Rust as described in the other post. Cloud Functions does not (yet!?) provide a Rust runtime and so I’m using the waPC Host for Go in this example. It works! PARAMS=$(printf '{"a":{"real":39,"imag":3},"b":{"real":39,"imag":3}}' | base64) TOKEN=$(gcloud auth print-identity-token) echo "{ \"filename\":\"complex.

Posts

ESPHome, MQTT, Prometheus and almost Cloud IoT

ESPHome is a very interesting project. I’d not heard of it until this week and am surprised that it isn’t more newsworthy. I’m always tinkering with IoT stuff, have a couple of Wemos D1 ESP8266s. They are brought out occasionally for learning. I’ve been using them this week with ESPHome. I’m looking to buy some Xiaomi BLE temperature sensors and thinking I could read the temperatures from these using the ESPs (thanks to ESPHome) and publish the data to MQTT.

Posts

Golang Structured Logging w/ Google Cloud Logging (2)

UPDATE There’s an issue with my naive implementation of RenderValuesHook as described in this post. I summarized the problem in this issue where I’ve outlined (hopefully) a more robust solution. Recently, I described how to configure Golang logging so that user-defined key-values applied to the logs are parsed when ingested by Google Cloud Logging. Here’s an example of what we’re trying to achieve. This is an example Cloud Logging log entry that incorporates user-defined labels (see dog:freddie and foo:bar) and a readily-querable jsonPayload:

Posts

Golang Structured Logging w/ Google Cloud Logging

I’ve multiple components in an app and these are deployed across multiple Google Cloud Platform (GCP) services: Kubernetes Engine, Cloud Functions, Cloud Run, etc. Almost everything is written in Golang and I started the project using go-logr. logr is in two parts: a Logger that you use to write log entries; a LogSink (adaptor) that consumes log entries and outputs them to a specific log implementation. Initially, I defaulted to using stdr which is a LogSink for Go’s standard logging implementation.

Posts

Using Google Monitoring Alerting to send Pushover notifications

Table of Contents Artifacts Pushover Caveat Cloud Monitoring Webhook Cloud Functions Cloud Run Artifacts GitHub: go-gcp-pushover-notificationchannel Image: ghcr.io/dazwilkin/go-gcp-pushover-notificationchannel:220515 Pushover Logging in to your Pushover account, you will be presented with a summary|dashboard page that includes Your User Key. Copy the value of this key into a variable called PUSHOVER_USER Create New Application|API Token Pushover API has a Pushing Messages method. The documentation describes the format of the HTTP Request.

Posts

Cloud Run custom domain mappings

I have several Cloud Run services that I want to map to a domain. During development, I create a Google Cloud Platform (GCP) project each day into which everything is deployed. This means that, every day, the Cloud Run services have newly non-inferable (to me) URLs. I thought this would be tedious to manage because: My DNS service isn’t programmable (I know!) Cloud Run services have non-inferable (by me) URLs i.

Posts

Prometheus HTTP Service Discovery of Cloud Run services

Some time ago, I wrote about using Prometheus Service Discovery w/ Consul for Cloud Run and also Scraping metrics exposed by Google Cloud Run services that require authentication. Both solutions remain viable but they didn’t address another use case for Prometheus and Cloud Run services that I have with a “thing” that I’ve been building. In this scenario, I want to: Configure Prometheus to scrape Cloud Run service metrics Discover Cloud Run services dynamically Authenticate to Cloud Run using Firebase Auth ID tokens These requirements and – one other – present several challenges:

Posts

Firebase Auth authorized domains

I’m using Firebase Authentication in a project to authenticate users of various OAuth2 identity systems. Firebase Authentication requires a set of Authorized Domains. The (web) app that interacts with Firebase Authentication is deployed to Cloud Run. The Authorized Domains list must include the app’s Cloud Run service URL. Cloud Run service URLs vary by Project (ID). They are a combination of the service name, a hash (?) of the Project (ID) and .

Posts

Scraping metrics exposed by Google Cloud Run services that require authentication

I’ve written a solution (gcp-oidc-token-proxy) that can be used in conjunction with Prometheus OAuth2 to authenticate requests so that Prometheus can scrape metrics exposed by e.g. Cloud Run services that require authentication. The solution resulted from my question on Stack overflow. Problem #1: Endpoint requires authentication Given a Cloud Run service URL for which: ENDPOINT="my-server-blahblah-wl.a.run.app" # Returns 200 when authentication w/ an ID token TOKEN="$(gcloud auth print-identity-token)" curl \ --silent \ --request GET \ --header "Authorization: Bearer ${TOKEN}" \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics # Returns 403 otherwise curl \ --silent \ --request GET \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics Problem #2: Prometheus OAuth2 configuration is constrained

Posts

`gcloud beta run services replace`

TL;DR I’m working on a project that includes multiple Cloud Run services. I’ve been putting my gcloud head on to deploy these services thinking that it’s curious there’s no way to write the specs as YAML configs. Today, I learned that there is: gcloud beta run services replace. What prompted the discovery was some frustration trying to deploy a JSON-valued environment variable to Cloud Run: local FIREBASE_CONFIG="{ apiKey: ${FIREBASE_API_KEY}, authDomain: ${FIREBASE_AUTH_DOMAIN}, projectId: ${FIREBASE_PROJECT}, storageBucket: ${FIREBASE_STORAGE_BUCKET}, messagingSenderId: ${FIREBASE_MESSAGING_SENDER}, appId: ${FIREBASE_APP}}" gcloud run deploy ${SRV_NAME} \ --image=${IMAGE} \ --command="/server" \ --args="--endpoint=:${PORT}" \ --set-env-vars=FIREBASE_CONFIG="${FIREBASE_CONFIG}" \ --max-instances=1 \ --memory=256Mi \ --ingress=all \ --platform=managed \ --port=${PORT} \ --allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} gcloud balks at this.

Posts

Cloud Endpoints combine OpenAPI and gRPC... or not!

See: Multiplexing gRPC and HTTP endpoints with Cloud Run gRPC, Cloud Run & Endpoints ESPv2: Configure Cloud Endpoints to proxy traffic to a Cloud Run multiplexed (gRPC|HTTP) service Challenges: Cloud Run permits single port Cloud Run services publishing e.g. gRPC and Prometheus, must multiplex transports Cloud Run services publishing multiplexed transports are challenging to expose using Cloud Endpoints Hypothesis #1: Multiplexed transports work with Cloud Run See: Multiplexing gRPC and HTTP endpoints with Cloud Run

Posts

Consul discovers Google Cloud Run

I’ve written a basic discoverer of Google Cloud Run services. This is for a project and it extends work done in some previous posts to Multiplex gRPC and Prometheus with Cloud Run and to use Consul for Prometheus service discovery. This solution: Accepts a set of Google Cloud Platform (GCP) projects Trawls them for Cloud Run services Assumes that the services expose Prometheus metrics on :443/metrics Relabels the services Surfaces any discovered Cloud Run services’ metrics in Prometheus You’ll need Docker and Docker Compose.

Posts

Multiplexing gRPC and HTTP (Prometheus) endpoints with Cloud Run

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another. Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port! TL;DR See the cmux Limitations and use:

Posts

Prometheus Service Discovery w/ Consul for Cloud Run

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus. This is one solution. NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use. Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation.

Posts

Programmatically deploying Cloud Run services (Golang|Python)

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so. My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work. Goal Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run. i.e. achieve this: gcloud run deploy ${NAME} \ --image=${IMAGE} \ --platform=managed \ --no-allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} TRICK --log-http is your friend

Posts

Google Trillian on Cloud Run

I’ve written previously (Google Trillian for Noobs) about Google’s interesting project Trillian and about some of the “personalities” (e.g. PyPi Transparency) that I’ve build using it. Having gone slight cra-cra on Cloud Run and gRPC this week with Golang gRPC Cloud Run and gRPC, Cloud Run & Endpoints, I thought it’d be fun to deploy Trillian and a personality to Cloud Run. It mostly (!) works :-) At the end of the post, I’ve summarized creating a Cloud SQL instance to host the Trillian data(base).

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Golang gRPC Cloud Run

Update: 2020-03-24: Since writing this post, I’ve contributed Golang and Rust samples to Google’s project. I recommend you start there. Google explained how to run gRPC servers with Cloud Run. The examples are good but only Python and Node.JS: gRPC comes to Cloud Run gRPC in Google Cloud Run Missing Golang…. until now ;-) I had problems with 1.14 and so I’m using 1.13. Project structure I’ll tidy up my repo but the code may be found:

Posts

Automating Scheduled Firestore Exports

For my “thing”, I use Firestore to persist state. I like Firestore a lot and, having been around Google for almost (!) a decade, I much prefer it to Datastore. Firestore has a managed export|import service and I use this to backup Firestore collections|documents. I’d been doing backups manually (using gcloud) and decided today to take the plunge and use Cloud Scheduler for the first time. I’d been reluctant to do this until now because I’d assumed incorrectly that I’d need to write a wrapping service to invoke the export.

Posts

Visual Studio Code plus Google Cloud Shell

Update: 2020-09-24 Three updates since I wrote the post: gcloud alpha cloud-shell get-mount-command ${DIR} It’s possible to use sshfs to mount the Cloud Shell home directory locally: DIR=/path/to/dir gcloud alpha cloud-shell get-mount-command ${DIR} Which generates something of the form: sshfs [[USERNAME]]@[[HOST]]: ${DIR} \ -p [[PORT]] \ -oIdentityFile=~/.ssh/google_compute_engine \ -oStrictHostKeyChecking=no You may then code --new-window ${DIR} curl command may lack .sshHost curl’ing the cloudshell.googleapis.com endpoint will result in a null value for .

Posts

Hugo and Google Cloud Storage

I’m using Hugo as a static site generator for this blog. I’m using Firebase (for free) to host lefsilver. I have other domains that I want to promote and decided to use Google Cloud Storage buckets for these sites. Using Google Cloud Storage for Hosting a static website and using Hugo to deploy sites to Google Cloud Storage (GCS) are documented but, I didn’t find a location where this is combined into a single tutorial and I wanted to add an explanation for ensuring your sites are included in Google’s and Bing’s search indexes.

Posts

Using Google's Public Certificate Authority with Golang autocert

Last year, I wrote about using Automatic Certs w/ Golang gRPC service on Compute Engine. That solution uses ACME with (the wonderful) Let’s Encrypt. Google is offering a private preview of Automate Public Certificates Lifecycle Management via RFC 8555 (ACME) and, because I’m using Google Cloud Platform extensively to build a “thing” and I think it would be useful to have a backup to Let’s Encrypt, I thought I’d give the solution a try.

Posts

Automatic Certs w/ Golang gRPC service on Compute Engine

I needed to deploy a healthcheck-enabled gRPC TLS-enabled service. Fortunately, most (all?) of the SDKs include an implementation, e.g. Golang has grpc-go/health. I learned in my travels that: DigitalOcean [App] platform does not (link) work with TLS-based gRPC apps. Fly has a regression (link) that breaks gRPC So, I resorted to Google Cloud Platform (GCP). Although Cloud Run would be well-suited to running the gRPC app, it uses a proxy|sidecar to provision a cert for the app and I wanted to be able to (easily use a custom domain) and give myself a somewhat general-purpose solution.

Posts

Akri

For the past couple of weeks, I’ve been playing around with Akri, a Microsoft (DeisLabs) project for building a connected edge with Kubernetes. Kubernetes, IoT, Rust (and Golang) make this all compelling to me. Initially, I deployed an Akri End-to-End to MicroK8s on Google Compute Engine (link) and Digital Ocean (link). But I was interested to create me own example and so have proposed a very (!) simple HTTP-based protocol.

Posts

akri

I was very interested to read about Microsoft’s DeisLab’s latest (rust-based) Kubernetes project: akri. If I understand it correctly, it provides a mechanism to make any (IoT) device accessible to containers running within a cluster. I need to spend more time playing around with it so that I can fully understand it. I had some problems getting the End-to-End demo running on Google Compute Engine (and then I tried DigitalOcean droplet) instances.

Posts

Setting up a GCE Instance as an Inlets Exit Node

The prolific Alex Ellis has a new project, Inlets. Here’s a quick tutorial using Google Compute Platform’s (GCP) Compute Engine (GCE). NB I’m using one of Google’s “Always free” f1-micro instances but you may still pay for network *gress and storage Assumptions I’m assuming you’ve a Google account, have used GCP and have a billing account established, i.e. the following returns at least one billing account: gcloud beta billing accounts list If you’ve only one billing account and it’s the one you wish to use, then you can:

Posts

Google Cloud Platform (GCP) Exporter

Earlier this week I discussed a Linode Prometheus Exporter. I added metrics for Digital Ocean’s Managed Kubernetes service to @metalmatze’s Digital Ocean Exporter. This left, metrics for Google Cloud Platform (GCP) which has, for many years, been my primary cloud platform. So, today I wrote Prometheus Exporter for Google Cloud Platform. All 3 of these exporters follow the template laid down by @metalmatze and, because each of these services has a well-written Golang SDK, it’s straightforward to implement an exporter for each of them.

Posts

Consul discovers Google Cloud Run

I’ve written a basic discoverer of Google Cloud Run services. This is for a project and it extends work done in some previous posts to Multiplex gRPC and Prometheus with Cloud Run and to use Consul for Prometheus service discovery. This solution: Accepts a set of Google Cloud Platform (GCP) projects Trawls them for Cloud Run services Assumes that the services expose Prometheus metrics on :443/metrics Relabels the services Surfaces any discovered Cloud Run services’ metrics in Prometheus You’ll need Docker and Docker Compose.

Posts

Prometheus Service Discovery w/ Consul for Cloud Run

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus. This is one solution. NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use. Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation.

Posts

Playing with GitHub Container registry REST API

I’ve a day to catch up on blogging. I’m building a “thing” and getting this near to the finish line consumes my time and has meant that I’m not originating anything particularly new. However, there are a couple of tricks in my deployment process that may be of interest to others. I’ve been a long-term using of Google’s [Cloud Build] and like the simplicity (everything’s a container, alot!). Because I’m using GitHub repos, I’ve been using GitHub Actions to (re)build containers on pushes and GitHub Container registry (GHCR) to store the results.

Posts

Using Google's Public Certificate Authority with Golang autocert

Last year, I wrote about using Automatic Certs w/ Golang gRPC service on Compute Engine. That solution uses ACME with (the wonderful) Let’s Encrypt. Google is offering a private preview of Automate Public Certificates Lifecycle Management via RFC 8555 (ACME) and, because I’m using Google Cloud Platform extensively to build a “thing” and I think it would be useful to have a backup to Let’s Encrypt, I thought I’d give the solution a try.

Posts

Automatic Certs w/ Golang gRPC service on Compute Engine

I needed to deploy a healthcheck-enabled gRPC TLS-enabled service. Fortunately, most (all?) of the SDKs include an implementation, e.g. Golang has grpc-go/health. I learned in my travels that: DigitalOcean [App] platform does not (link) work with TLS-based gRPC apps. Fly has a regression (link) that breaks gRPC So, I resorted to Google Cloud Platform (GCP). Although Cloud Run would be well-suited to running the gRPC app, it uses a proxy|sidecar to provision a cert for the app and I wanted to be able to (easily use a custom domain) and give myself a somewhat general-purpose solution.

Posts

Accessing GCR repos from Kubernetes

Until today, I’d not accessed a Google Container Registry repo from a non-GKE Kubernetes deployment. It turns out that it’s pretty well-documented (link) but, here’s an end-end example. Assuming: BILLING=[[YOUR-BILLING]] PROJECT=[[YOUR-PROJECT]] SERVER="us.gcr.io" If not already: gcloud projects create {$PROJECT} gcloud beta billing projects link ${PROJECT} \ --billing-account=${BILLING} gcloud services enable containerregistry.googleapis.com \ --project=${PROJECT} Container Registry IMAGE="busybox" # Or ... docker pull ${IMAGE} docker tag \ ${IMAGE} \ ${SERVER}/${PROJECT}/${IMAGE} docker push ${SERVER}/${PROJECT}/${IMAGE} gcloud container images list-tags ${SERVER}/${PROJECT}/${IMAGE} Service Account Create a service account that’s permitted to download (read-only) images from this project’s registry

Posts

Prometheus Service Discovery w/ Consul for Cloud Run

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus. This is one solution. NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use. Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation.

Posts

Rust implementation of Crate Transparency using Google Trillian

I’ve been hacking on a Rust-based transparent application for Google Trillian. As appears to be my fixation, this personality is for another package manager. This time, Rust’s Crates often found in crates.io which is Rust’s Package Registry. I discussed this project earlier this month Rust Crate Transparency && Rust SDK for Google Trillian and and earlier approach for Python’s packages with pypi-transparency. This time, of course, I’m using Rust. And, by way of a first for me, for the gRPC server implementation (aka “personality”).

Posts

Playing with GitHub Container registry REST API

I’ve a day to catch up on blogging. I’m building a “thing” and getting this near to the finish line consumes my time and has meant that I’m not originating anything particularly new. However, there are a couple of tricks in my deployment process that may be of interest to others. I’ve been a long-term using of Google’s [Cloud Build] and like the simplicity (everything’s a container, alot!). Because I’m using GitHub repos, I’ve been using GitHub Actions to (re)build containers on pushes and GitHub Container registry (GHCR) to store the results.

Posts

Dapr

It’s a good name, I read it as “dapper” but I frequently type “darp” :-( Was interested to read that Dapr is now v1.0 and decided to check it out. I was initially confused between Dapr and service mesh functionality. But, having used Dapr, it appears to be more focused in aiding the development of (cloud-native) (distributed) apps by providing developers with abstractions for e.g. service discovery, eventing, observability whereas service meshes feel (!

Posts

Trendnet TEW-812DRU and DD-WRT

The FBI Portland published an interesting advisory with several, sensible recommendations including firewalling IoT devices from other devices on a home network. I decided to deploy a redundant Trendnet TEW-812DRU version 2.0 for this purpose. Caveat Developer: Before I go further, I don’t recommend installing DD-WRT on a Trendnet TEW-812DRU unless you’re willing to brick the device irrecoverably. I read the DD-WRT instructions several times (“peacock thread”,router database – do not use v3.

Posts

Krustlet on DO Managed Kubernetes

I’ve spent time this week returning to the interesting Deislabs project Krustlet. Since the last time, the bootstrapping process has been simplified using Kubernetes Bootstrap Tokens. I know this updated process works with MicroK8s. Unfortunately, I’m struggling with it on GKE and thought I’d try DigitalOcean Managed Kubernetes. It worked first time! In the following, we run both the Kubernetes cluster and the Krustlet Droplet on DigitalOcean but, as long as the cluster and the VM are able to communicate with one another, you should be able to run these anywhere.

Posts

GitHub help with dependency management

This is very useful: I am building an application that comprises multiple repos. I continue to procrastinate on whether using multiple repos vs. a monorepo was a good idea but, an issue that I have (had) is the need to ensure that the repos’ contents are using current|latest modules. GitHub can help. Most of the application is written in Golang with a smattering of Rust and some JavaScript. Aside Even though I’m more proficient with Golang, I want to become more proficient in Rust and I contemplated trying to write the entire application in Rust.

Posts

GitHub help with dependency management

This is very useful: I am building an application that comprises multiple repos. I continue to procrastinate on whether using multiple repos vs. a monorepo was a good idea but, an issue that I have (had) is the need to ensure that the repos’ contents are using current|latest modules. GitHub can help. Most of the application is written in Golang with a smattering of Rust and some JavaScript. Aside Even though I’m more proficient with Golang, I want to become more proficient in Rust and I contemplated trying to write the entire application in Rust.

Posts

Kubernetes Device Plugins

I’m debugging an issue with Akri Zeroconf protocol in which Instance environment variables are no longer (!) being surfaced within the Broker pods. In my adventures, it seemed useful to better understand how Akri works and specifically, how Akri uses Kubernetes Device Plugins. IIUC plugins register with the Kubelet (!) via a gRPC service (Registration) that the Kubelet exposes on a UNIX socket at /var/lib/kubelet/device-plugins/kubelet.sock Then (!) if successful, devices should be reported by the Node’s metadata (spec) and available to be bound to Pods.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

Krustlet on DO Managed Kubernetes

I’ve spent time this week returning to the interesting Deislabs project Krustlet. Since the last time, the bootstrapping process has been simplified using Kubernetes Bootstrap Tokens. I know this updated process works with MicroK8s. Unfortunately, I’m struggling with it on GKE and thought I’d try DigitalOcean Managed Kubernetes. It worked first time! In the following, we run both the Kubernetes cluster and the Krustlet Droplet on DigitalOcean but, as long as the cluster and the VM are able to communicate with one another, you should be able to run these anywhere.

Posts

Deploying a Rust HTTP server to DigitalOcean App Platform

DigitalOcean launched an App Platform with many Supported Languages and Frameworks. I used Golang first, then wondered how to use non-natively-supported languages, i.e. Rust. The good news is that Docker is a supported framework and so, you can run pretty much anything. Repo: https://github.com/DazWilkin/do-apps-rust Rust I’m a Rust noob. I’m always receptive to feedback on improvements to the code. I looked to mirror the Golang example. I’m using rocket and rocket-prometheus for the first time:

Posts

Cloud Run custom domain mappings

I have several Cloud Run services that I want to map to a domain. During development, I create a Google Cloud Platform (GCP) project each day into which everything is deployed. This means that, every day, the Cloud Run services have newly non-inferable (to me) URLs. I thought this would be tedious to manage because: My DNS service isn’t programmable (I know!) Cloud Run services have non-inferable (by me) URLs i.

Posts

Run cAdvisor when using Docker Compose

cAdvisor has long been a favorite monitoring tool of mine. I’m using Docker Compose for local testing and have begun including a cAdvisor in my docker-compose.yaml files. cadvisor: restart: always image: google/cadvisor:${CADVISOR_VERSION} container_name: cadvisor # command: # - --prometheus_endpoint="/metrics" # Default volumes: - "/:/rootfs:ro" - "/var/run:/var/run:rw" - "/sys:/sys:ro" - "/var/snap/docker/current:/var/lib/docker:ro" #- "/var/lib/docker/:/var/lib/docker:ro" expose: - "8080" ports: - 8080:8080 I’d not realized until recently, that cAdvisor also surfaces a Prometheus metrics endpoint and so, if you do follow this path and you’re also using Prometheus, don’t forget to add cAdvisor to your Prometheus targets.

Posts

Run cAdvisor when using Docker Compose

cAdvisor has long been a favorite monitoring tool of mine. I’m using Docker Compose for local testing and have begun including a cAdvisor in my docker-compose.yaml files. cadvisor: restart: always image: google/cadvisor:${CADVISOR_VERSION} container_name: cadvisor # command: # - --prometheus_endpoint="/metrics" # Default volumes: - "/:/rootfs:ro" - "/var/run:/var/run:rw" - "/sys:/sys:ro" - "/var/snap/docker/current:/var/lib/docker:ro" #- "/var/lib/docker/:/var/lib/docker:ro" expose: - "8080" ports: - 8080:8080 I’d not realized until recently, that cAdvisor also surfaces a Prometheus metrics endpoint and so, if you do follow this path and you’re also using Prometheus, don’t forget to add cAdvisor to your Prometheus targets.

Posts

Using Golang with the Firestore Emulator

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang). Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form: package main import ( "context" "crypto/sha256" "fmt" "log" "os" "time" "cloud.

Posts

Envoy WASM filters in Rust

A digression thanks to Sal Rashid who’s exploring WASM filters w/ Envoy. The documentation is sparse but: How to write WASM filters for Envoy… There is a Rust SDK but it’s not documented: proxy-wasm-rust-sdk I found two useful posts by Rustaceans who were able to make use of it: Extending Envoy with WASM and Rust Extending Istio with Rust and WebAssembly Here’s my simple use of the SDK’s examples.

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

ESPHome, MQTT, Prometheus and almost Cloud IoT

ESPHome is a very interesting project. I’d not heard of it until this week and am surprised that it isn’t more newsworthy. I’m always tinkering with IoT stuff, have a couple of Wemos D1 ESP8266s. They are brought out occasionally for learning. I’ve been using them this week with ESPHome. I’m looking to buy some Xiaomi BLE temperature sensors and thinking I could read the temperatures from these using the ESPs (thanks to ESPHome) and publish the data to MQTT.

Posts

ESPHome, MQTT, Prometheus and almost Cloud IoT

ESPHome is a very interesting project. I’d not heard of it until this week and am surprised that it isn’t more newsworthy. I’m always tinkering with IoT stuff, have a couple of Wemos D1 ESP8266s. They are brought out occasionally for learning. I’ve been using them this week with ESPHome. I’m looking to buy some Xiaomi BLE temperature sensors and thinking I could read the temperatures from these using the ESPs (thanks to ESPHome) and publish the data to MQTT.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

Firebase Authentication, Cloud Endpoints and gRPC (2of2)

Earlier this week, I wrote about using Firebase Authentcation, Cloud Endpoints and gRPC (1of2). Since then, I learned some more and added a gRPC interceptor to implement basic authorization for the service. ESPv2 --allow-unauthenticated The Cloud Enpoints (ESPv2) proxy must be run as --allow-unauthenticated on Cloud Run to ensure that requests make it to the proxy where the request is authenticated and only authenticated requests make it on to the backend service.

Posts

Firebase Authentication, Cloud Endpoints and gRPC (1of2)

I’m building a service that requires user authentication. The primary endpoint is a gRPC-based service. I would like to consider using certificate-based auth but this feels… challenging. Instead, I have been aware of, but never used, Firebase Authentication and was interested to see that Cloud Endpoints includes Firebase Authentication as one of its supported auth mechanisms. Curiosity piqued, I confirmed that gRPC supports Google token-based authentication. The following is a summary of what I did but I’ll leave the extensive documentation to Google, (Google’s) Firebase and gRPC, all of which, in this case, provide really good explanations.

Posts

Firebase Auth authorized domains

I’m using Firebase Authentication in a project to authenticate users of various OAuth2 identity systems. Firebase Authentication requires a set of Authorized Domains. The (web) app that interacts with Firebase Authentication is deployed to Cloud Run. The Authorized Domains list must include the app’s Cloud Run service URL. Cloud Run service URLs vary by Project (ID). They are a combination of the service name, a hash (?) of the Project (ID) and .

Posts

Renewing Firebase Authentication ID tokens with gRPC

I’ve written before about a project in which I’m using Firebase Authentication in combination with Google Cloud Endpoints and a gRPC service running on Cloud Run: Firebase Authentication, Cloud Endpoints and gRPC (1of2) Firebase Authentication, Cloud Endpoints and gRPC (2of2) This works well with one caveat, the ID tokens (JWTs) minted by Firebase Authentication have a 3600 second (one hour) lifetime. The user flow in my app is that whenever the user invokes the app’s CLI:

Posts

Firebase Authentication, Cloud Endpoints and gRPC (2of2)

Earlier this week, I wrote about using Firebase Authentcation, Cloud Endpoints and gRPC (1of2). Since then, I learned some more and added a gRPC interceptor to implement basic authorization for the service. ESPv2 --allow-unauthenticated The Cloud Enpoints (ESPv2) proxy must be run as --allow-unauthenticated on Cloud Run to ensure that requests make it to the proxy where the request is authenticated and only authenticated requests make it on to the backend service.

Posts

Firebase Authentication, Cloud Endpoints and gRPC (1of2)

I’m building a service that requires user authentication. The primary endpoint is a gRPC-based service. I would like to consider using certificate-based auth but this feels… challenging. Instead, I have been aware of, but never used, Firebase Authentication and was interested to see that Cloud Endpoints includes Firebase Authentication as one of its supported auth mechanisms. Curiosity piqued, I confirmed that gRPC supports Google token-based authentication. The following is a summary of what I did but I’ll leave the extensive documentation to Google, (Google’s) Firebase and gRPC, all of which, in this case, provide really good explanations.

Posts

Automating Scheduled Firestore Exports

For my “thing”, I use Firestore to persist state. I like Firestore a lot and, having been around Google for almost (!) a decade, I much prefer it to Datastore. Firestore has a managed export|import service and I use this to backup Firestore collections|documents. I’d been doing backups manually (using gcloud) and decided today to take the plunge and use Cloud Scheduler for the first time. I’d been reluctant to do this until now because I’d assumed incorrectly that I’d need to write a wrapping service to invoke the export.

Posts

Firestore Golang Timestamps & Merging

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quikcly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum. I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

Posts

Cloud Firestore Triggers in Golang

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language. I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.

Posts

Using Golang with the Firestore Emulator

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang). Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form: package main import ( "context" "crypto/sha256" "fmt" "log" "os" "time" "cloud.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

Fly.io

I spent some time over the weekend understanding Fly.io. It’s always fascinating to me how many smart people are building really neat solutions. Fly.io is subtly different to other platforms that I use (Kubernetes, GCP, DO, Linode) and I’ve found the Fly.io team to be highly responsive and helpful to my noob questions. One of the team’s posts, Docker without Docker surfaced in my Feedly feed (hackernews) and it piqued my interest.

Posts

Setting up a GCE Instance as an Inlets Exit Node

The prolific Alex Ellis has a new project, Inlets. Here’s a quick tutorial using Google Compute Platform’s (GCP) Compute Engine (GCE). NB I’m using one of Google’s “Always free” f1-micro instances but you may still pay for network *gress and storage Assumptions I’m assuming you’ve a Google account, have used GCP and have a billing account established, i.e. the following returns at least one billing account: gcloud beta billing accounts list If you’ve only one billing account and it’s the one you wish to use, then you can:

Posts

Using `gcloud ... --format` with arbitrary returned data

If you use jq, you’ll know that, its documentation uses examples that you can try locally or using the excellent jqplay: printf "[1,2,3]" | jq .[1:] [ 2, 3 ] And here If you use Google Cloud Platform (GCP) CLI, gcloud, this powerful tool includes JSON output formatting of results (--format=json) and YAML (--format=yaml) etc. and includes a set of so-called projections that you can use to format the returned data.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

Using Google Monitoring Alerting to send Pushover notifications

Table of Contents Artifacts Pushover Caveat Cloud Monitoring Webhook Cloud Functions Cloud Run Artifacts GitHub: go-gcp-pushover-notificationchannel Image: ghcr.io/dazwilkin/go-gcp-pushover-notificationchannel:220515 Pushover Logging in to your Pushover account, you will be presented with a summary|dashboard page that includes Your User Key. Copy the value of this key into a variable called PUSHOVER_USER Create New Application|API Token Pushover API has a Pushing Messages method. The documentation describes the format of the HTTP Request.

Posts

Cloud Run custom domain mappings

I have several Cloud Run services that I want to map to a domain. During development, I create a Google Cloud Platform (GCP) project each day into which everything is deployed. This means that, every day, the Cloud Run services have newly non-inferable (to me) URLs. I thought this would be tedious to manage because: My DNS service isn’t programmable (I know!) Cloud Run services have non-inferable (by me) URLs i.

Posts

Automating Scheduled Firestore Exports

For my “thing”, I use Firestore to persist state. I like Firestore a lot and, having been around Google for almost (!) a decade, I much prefer it to Datastore. Firestore has a managed export|import service and I use this to backup Firestore collections|documents. I’d been doing backups manually (using gcloud) and decided today to take the plunge and use Cloud Scheduler for the first time. I’d been reluctant to do this until now because I’d assumed incorrectly that I’d need to write a wrapping service to invoke the export.

Posts

Using Google's Public Certificate Authority with Golang autocert

Last year, I wrote about using Automatic Certs w/ Golang gRPC service on Compute Engine. That solution uses ACME with (the wonderful) Let’s Encrypt. Google is offering a private preview of Automate Public Certificates Lifecycle Management via RFC 8555 (ACME) and, because I’m using Google Cloud Platform extensively to build a “thing” and I think it would be useful to have a backup to Let’s Encrypt, I thought I’d give the solution a try.

Posts

Infrastructure as Code

Problem I’m building an application that comprises: Kubernetes¹ Kubernetes Operator Cloud Firestore Cloud Functions Cloud Run Cloud Endpoints Stripe Firebase Authentication ¹ - I’m using Google Kubernetes Engine (GKE) but may include other managed Kubernetes offerings (e.g. Digital Ocean, Linode, Oracle). GKE clusters are manageable by gcloud but other platforms require other CLI tools. All are accessible from bash but are these supported by e.g. Terraform (see below)?

Posts

Cloud Firestore Triggers in Golang

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language. I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.

Posts

WASM Cloud Functions

Following on from waPC & Protobufs and a question on Stack Overflow about Cloud Functions, I was compelled to try running WASM on Cloud Functions no JavaScript. I wanted to reuse the WASM waPC functions that I’d written in Rust as described in the other post. Cloud Functions does not (yet!?) provide a Rust runtime and so I’m using the waPC Host for Go in this example. It works! PARAMS=$(printf '{"a":{"real":39,"imag":3},"b":{"real":39,"imag":3}}' | base64) TOKEN=$(gcloud auth print-identity-token) echo "{ \"filename\":\"complex.

Posts

Setting up a GCE Instance as an Inlets Exit Node

The prolific Alex Ellis has a new project, Inlets. Here’s a quick tutorial using Google Compute Platform’s (GCP) Compute Engine (GCE). NB I’m using one of Google’s “Always free” f1-micro instances but you may still pay for network *gress and storage Assumptions I’m assuming you’ve a Google account, have used GCP and have a billing account established, i.e. the following returns at least one billing account: gcloud beta billing accounts list If you’ve only one billing account and it’s the one you wish to use, then you can:

Posts

Google Cloud Platform (GCP) Exporter

Earlier this week I discussed a Linode Prometheus Exporter. I added metrics for Digital Ocean’s Managed Kubernetes service to @metalmatze’s Digital Ocean Exporter. This left, metrics for Google Cloud Platform (GCP) which has, for many years, been my primary cloud platform. So, today I wrote Prometheus Exporter for Google Cloud Platform. All 3 of these exporters follow the template laid down by @metalmatze and, because each of these services has a well-written Golang SDK, it’s straightforward to implement an exporter for each of them.

Posts

Google Container Registry w/ OCI

I’ve been spending some time this week with Krustlet. I’m working on documenting how to run Krustlet(s) alongside GKE. I’ve been running a Krustlet with MicroK8s. The Krustlet demos reference WASM assemblines stored in Azure Container Registry as OCI containers. Google Container Registry supports OCI format and so I tried (successfully) using GCR instead of AZR. There may be an easier approach but this is how I got this working.

Posts

Accessing GCR repos from Kubernetes

Until today, I’d not accessed a Google Container Registry repo from a non-GKE Kubernetes deployment. It turns out that it’s pretty well-documented (link) but, here’s an end-end example. Assuming: BILLING=[[YOUR-BILLING]] PROJECT=[[YOUR-PROJECT]] SERVER="us.gcr.io" If not already: gcloud projects create {$PROJECT} gcloud beta billing projects link ${PROJECT} \ --billing-account=${BILLING} gcloud services enable containerregistry.googleapis.com \ --project=${PROJECT} Container Registry IMAGE="busybox" # Or ... docker pull ${IMAGE} docker tag \ ${IMAGE} \ ${SERVER}/${PROJECT}/${IMAGE} docker push ${SERVER}/${PROJECT}/${IMAGE} gcloud container images list-tags ${SERVER}/${PROJECT}/${IMAGE} Service Account Create a service account that’s permitted to download (read-only) images from this project’s registry

Posts

Hugo and Google Cloud Storage

I’m using Hugo as a static site generator for this blog. I’m using Firebase (for free) to host lefsilver. I have other domains that I want to promote and decided to use Google Cloud Storage buckets for these sites. Using Google Cloud Storage for Hosting a static website and using Hugo to deploy sites to Google Cloud Storage (GCS) are documented but, I didn’t find a location where this is combined into a single tutorial and I wanted to add an explanation for ensuring your sites are included in Google’s and Bing’s search indexes.

Posts

Playing with GitHub Container registry REST API

I’ve a day to catch up on blogging. I’m building a “thing” and getting this near to the finish line consumes my time and has meant that I’m not originating anything particularly new. However, there are a couple of tricks in my deployment process that may be of interest to others. I’ve been a long-term using of Google’s [Cloud Build] and like the simplicity (everything’s a container, alot!). Because I’m using GitHub repos, I’ve been using GitHub Actions to (re)build containers on pushes and GitHub Container registry (GHCR) to store the results.

Posts

Sigstore

I’ve been on a digression (gcp-oidc-token-proxy) this week. Yesterday I began exploring Podman and wrote briefly about running gcp-oidc-token-proxy on my localhost using it. This morning while walking with my dog, I listened to Google’s Dan Lorenc explain Sigstore (blog](https://blog.sigstore.dev/)) on The Kubelist Podcast1 The plan today is to try to sign the gcp-oidc-token-proxy container images in GitHub Container Registry. NOTE I decided against trying the hardware key approach. I have a Google Titan key and only Yubikeys are well-tested by go-piv

Posts

GitHub Actions && GitHub Container Registry

You know when you start something and then regret it!? I think I’ll be sticking with Google Cloud Build; GitHub Actions appears functional and useful but I found the documentation to be confusing and limited and struggled to get a simple container image build|push working. I’ve long used Docker Hub but am planning to use it less as a result of the planned changes. I want to see Docker succeed and to do so it needs to find a way to make money but, there are free alternatives including the new GitHub Container Registry and the very very cheap Google Container Registry.

Posts

GitHub help with dependency management

This is very useful: I am building an application that comprises multiple repos. I continue to procrastinate on whether using multiple repos vs. a monorepo was a good idea but, an issue that I have (had) is the need to ensure that the repos’ contents are using current|latest modules. GitHub can help. Most of the application is written in Golang with a smattering of Rust and some JavaScript. Aside Even though I’m more proficient with Golang, I want to become more proficient in Rust and I contemplated trying to write the entire application in Rust.

Posts

GitHub Actions && GitHub Container Registry

You know when you start something and then regret it!? I think I’ll be sticking with Google Cloud Build; GitHub Actions appears functional and useful but I found the documentation to be confusing and limited and struggled to get a simple container image build|push working. I’ve long used Docker Hub but am planning to use it less as a result of the planned changes. I want to see Docker succeed and to do so it needs to find a way to make money but, there are free alternatives including the new GitHub Container Registry and the very very cheap Google Container Registry.

Posts

GitHub Actions && GitHub Container Registry

You know when you start something and then regret it!? I think I’ll be sticking with Google Cloud Build; GitHub Actions appears functional and useful but I found the documentation to be confusing and limited and struggled to get a simple container image build|push working. I’ve long used Docker Hub but am planning to use it less as a result of the planned changes. I want to see Docker succeed and to do so it needs to find a way to make money but, there are free alternatives including the new GitHub Container Registry and the very very cheap Google Container Registry.

Posts

Kubernetes Engine and Free Tier

Google Cloud Platform Free Tier appears (please verify this for yourself) to provide the ability to run a(n admittedly miniscule) Kubernetes cluster for free. So, why do this? It provides a definitive Kubernetes (Engine) experience on Google Cloud Platform that you may use for learning and testing. Kubernetes Engine the master node(s) and the control plane are free. Kubernetes (i.e. Compute Engine) nodes potentially incur charges including for the VM runtime and any attached storage, snapshots etc.

Posts

Golang Structured Logging w/ Google Cloud Logging (2)

UPDATE There’s an issue with my naive implementation of RenderValuesHook as described in this post. I summarized the problem in this issue where I’ve outlined (hopefully) a more robust solution. Recently, I described how to configure Golang logging so that user-defined key-values applied to the logs are parsed when ingested by Google Cloud Logging. Here’s an example of what we’re trying to achieve. This is an example Cloud Logging log entry that incorporates user-defined labels (see dog:freddie and foo:bar) and a readily-querable jsonPayload:

Posts

Golang Structured Logging w/ Google Cloud Logging (2)

UPDATE There’s an issue with my naive implementation of RenderValuesHook as described in this post. I summarized the problem in this issue where I’ve outlined (hopefully) a more robust solution. Recently, I described how to configure Golang logging so that user-defined key-values applied to the logs are parsed when ingested by Google Cloud Logging. Here’s an example of what we’re trying to achieve. This is an example Cloud Logging log entry that incorporates user-defined labels (see dog:freddie and foo:bar) and a readily-querable jsonPayload:

Posts

Golang Structured Logging w/ Google Cloud Logging

I’ve multiple components in an app and these are deployed across multiple Google Cloud Platform (GCP) services: Kubernetes Engine, Cloud Functions, Cloud Run, etc. Almost everything is written in Golang and I started the project using go-logr. logr is in two parts: a Logger that you use to write log entries; a LogSink (adaptor) that consumes log entries and outputs them to a specific log implementation. Initially, I defaulted to using stdr which is a LogSink for Go’s standard logging implementation.

Posts

GitHub help with dependency management

This is very useful: I am building an application that comprises multiple repos. I continue to procrastinate on whether using multiple repos vs. a monorepo was a good idea but, an issue that I have (had) is the need to ensure that the repos’ contents are using current|latest modules. GitHub can help. Most of the application is written in Golang with a smattering of Rust and some JavaScript. Aside Even though I’m more proficient with Golang, I want to become more proficient in Rust and I contemplated trying to write the entire application in Rust.

Posts

gRPC Interceptors and in-memory gRPC connections

For… reasons, I wanted to pre-filter gRPC requests to check for authorization. Authorization is implemented as a ‘micro-service’ and I wanted the authorization server to run in the same process as the gRPC client. TL;DR: Shiju’s “Writing gRPC Interceptors in Go” is great This Stack overflow answer ostensibly for writing unit tests for gRPC got me an in-process server What follows stands on these folks’ shoulders… A key motivator for me to write blog posts is that helps me ensure that I understand things.

Posts

Stripe

It’s been almost a month since my last post. I’ve been occupied learning Stripe and integrating it into an application that I’m developing. The app benefits from a billing mechanism for prospective customers and, as far as I can tell, Stripe is the solution. I’d be interested in hearing perspectives on alternatives. As with any platform, there’s good and bad and I’ll summarize my perspective on Stripe here. It’s been some time since I developed in JavaScript and this lack of familiarity has meant that the solution took longer than I wanted to develop.

Posts

Struggling with Golang structs

Julia’s post Blog about what you’ve struggled with resonates because I’ve been struggling with Golang structs in a project. Not the definitions of structs but seemingly needing to reproduce them across the project. I realize that each instance of these resources differs from the others but I’m particularly concerned by having to duplicate method implementations on them. I’m kinda hoping that I see the solution to my problem by writing it out.

Posts

Firestore Golang Timestamps & Merging

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quikcly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum. I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

Posts

Cloud Firestore Triggers in Golang

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language. I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.

Posts

Using Golang with the Firestore Emulator

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang). Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form: package main import ( "context" "crypto/sha256" "fmt" "log" "os" "time" "cloud.

Posts

Programmatically deploying Cloud Run services (Golang|Python)

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so. My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work. Goal Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run. i.e. achieve this: gcloud run deploy ${NAME} \ --image=${IMAGE} \ --platform=managed \ --no-allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} TRICK --log-http is your friend

Posts

Dapr

It’s a good name, I read it as “dapper” but I frequently type “darp” :-( Was interested to read that Dapr is now v1.0 and decided to check it out. I was initially confused between Dapr and service mesh functionality. But, having used Dapr, it appears to be more focused in aiding the development of (cloud-native) (distributed) apps by providing developers with abstractions for e.g. service discovery, eventing, observability whereas service meshes feel (!

Posts

Kubernetes Webhooks

I spent some time last week writing my first admission webhook for Kubernetes. I wrote the handler in Golang because I’m most familiar with Golang and because, as Kubernetes’ native language, I was more confident that the necessary SDKs would exist and that the documentation would likely use Golang by default. I struggled to find useful documentation and so this post is to help you (and me!) remember how to do this next time!

Posts

Akri

For the past couple of weeks, I’ve been playing around with Akri, a Microsoft (DeisLabs) project for building a connected edge with Kubernetes. Kubernetes, IoT, Rust (and Golang) make this all compelling to me. Initially, I deployed an Akri End-to-End to MicroK8s on Google Compute Engine (link) and Digital Ocean (link). But I was interested to create me own example and so have proposed a very (!) simple HTTP-based protocol.

Posts

GitHub Actions && GitHub Container Registry

You know when you start something and then regret it!? I think I’ll be sticking with Google Cloud Build; GitHub Actions appears functional and useful but I found the documentation to be confusing and limited and struggled to get a simple container image build|push working. I’ve long used Docker Hub but am planning to use it less as a result of the planned changes. I want to see Docker succeed and to do so it needs to find a way to make money but, there are free alternatives including the new GitHub Container Registry and the very very cheap Google Container Registry.

Posts

Trillian Map Mode

Chatting with one of Google’s Trillian team, I was encouraged to explore Trillian’s Map Mode. The following is the result of some spelunking through this unfamiliar cave. I can’t provide any guarantee that this usage is correct or sufficient. Here’s the repo: https://github.com/DazWilkin/go-trillian-map I’ve written about Trillian Log Mode elsewhere. I uncovered use of Trillian Map Mode through Trillian’s integration tests. I’m unclear on the distinction between TrillianMapClient and TrillianMapWriteClient but the latter served most of my needs.

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

waPC and MsgPack (Rust|Golang)

As my reader will know (Hey Mom!), I’ve been noodling around with WASM and waPC. I’ve been exploring ways to pass structured messages across the host:guest boundary. Protobufs was my first choice. @KevinHoffman created waPC and waSCC and he explained to me and that wSCC uses Message Pack. It’s slightly surprising to me (still) that technologies like this exist with everyone else seemingly using them and I’ve not heard of them.

Posts

Golang Protobuf APIv2

Google has a new Golang Protobuf API, APIv2 (google.golang.org/protobuf) superseding APIv1 (github.com/golang/protobuf). If your code is importing github.com/golang/protobuf, you’re using APIv2. Otherwise, you should consult with the docs because Google reimplemented APIv1 atop APIv2. One challenge this caused me, as someone who does not use protobufs and gRPC on a daily basis, is that gRPC (code-generation) is being removed from the (Golang) protoc-gen-go, the Golang plugin that generates gRPC service bindings.

Posts

Golang Xiaomi Bluetooth Temperature|Humidity (LYWSD03MMC) 2nd Gen

Well, this became more of an adventure that I’d originally wanted but, after learning some BLE and, with the help of others (Thanks Jonatha, JsBergbau), I’ve sample code that connects to 4 Xiaomi 2nd gen. Thermometers, subscribes to readings and publishes the data to MQTT. From there, I’m scraping it using Inuits MQTTGateway into Prometheus. Repo: https://github.com/DazWilkin/gomijia2 Thanks|Credit: Jonathan McDowell for gomijia and help JsBergbau for help Background I’ve been playing around with ESPHome and blogged around my very positive experience ESPHome, MQTT, Prometheus and almost Cloud IoT.

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Golang gRPC Cloud Run

Update: 2020-03-24: Since writing this post, I’ve contributed Golang and Rust samples to Google’s project. I recommend you start there. Google explained how to run gRPC servers with Cloud Run. The examples are good but only Python and Node.JS: gRPC comes to Cloud Run gRPC in Google Cloud Run Missing Golang…. until now ;-) I had problems with 1.14 and so I’m using 1.13. Project structure I’ll tidy up my repo but the code may be found:

Posts

Google's New Golang SDK for Protobufs

Google has released a new Golang SDK for protobuf. In the [announcement], a useful tool to redact properties is described. If like me, this is somewhat novel to you, here’s a mashup using Google’s Protocol Buffer Basics w/ redaction. To be very clear, as it’s an important distinction: Version Repo Docs v2 google.golang.org/protobuf Docs v1 github.com/golang/protobuf Docs Project Here’s my project structure:

Posts

OriginStamp: Verifying Proofs

Recently, I wrote about some initial adventures with OriginStamp Using OriginStamp’s UI or API, submitting a hash results in transactions being submitted to Bitcoin, Ethereum and a German newspaper. Using the API, it’s possible to query OriginStamp’s service for a proof. This post explains how to verify such a proof. The diligent reader among you (Hey Mom!) will recall that I submitted a hash for the message: Frederik Jack is a bubbly Border Collie The SHA-256 hash of this message is:

Posts

FreeTSA & Digitorus' Timestamp SDK

I wrote recently about some exploration of Timestamping with OriginStamp. Since writing that post, I had some supportive feedback from the helpful folks at OriginStamp and plan to continue exploring that solution. Meanwhile, OriginStamp exposed me to timestamping and trusted timestamping and I discovered freeTSA.org. What’s the point? These services provide authoritative proof of the existence of a digital asset before some point in time; OriginStamp provides a richer service and uses multiple timestamp authorities including Bitcoin, Ethereum and rather interestingly a German Newspaper’s Trusted Timestamp.

Posts

OriginStamp Python|Golang SDK Examples

A friend mentioned OriginStamp to me. NB There are 2 sites: originstamp.com and originstamp.org. It’s an interesting project. It’s a solution for providing auditable proof that you had a(ccess to) some digital thing before a certain date. OriginStamp provides user-|developer-friendly means to submit files|hashes (of your content) and have these bundled into transactions that are submitted to e.g. bitcoin. I won’t attempt to duplicate the narrative here, review OriginStamp’s site and some of its content.

Posts

Cloud Build wishlist: Mountable Golang Modules Proxy

I think it would be valuable if Google were to provide volumes in Cloud Build of package registries (e.g. Go Modules; PyPi; Maven; NPM etc.). Google provides a mirror of a subset of Docker Hub. This confers several benefits: Google’s imprimatur; speed (latency); bandwidth; and convenience. The same benefits would apply to package registries. In the meantime, there’s a hacky way to gain some of the benefits of these when using Cloud Build.

Posts

Setting up a GCE Instance as an Inlets Exit Node

The prolific Alex Ellis has a new project, Inlets. Here’s a quick tutorial using Google Compute Platform’s (GCP) Compute Engine (GCE). NB I’m using one of Google’s “Always free” f1-micro instances but you may still pay for network *gress and storage Assumptions I’m assuming you’ve a Google account, have used GCP and have a billing account established, i.e. the following returns at least one billing account: gcloud beta billing accounts list If you’ve only one billing account and it’s the one you wish to use, then you can:

Posts

Google Fit

I’ve spent a few days exploring [Google Fit SDK] as I try to wean myself from my obsession with metrics (of all forms). A quick Googling got me to Robert’s Exporter Google Fit Daily Steps, Weight and Distance to a Google Sheet. This works and is probably where I should have stopped… avoiding the rabbit hole that I’ve been down… I threw together a simple Golang implementation of the SDK using Google’s Golang API Client Library.

Posts

Google Home Exporter

I’m obsessing over Prometheus exporters. First came Linode Exporter, then GCP Exporter and, on Sunday, I stumbled upon a reverse-engineered API for Google Home devices and so wrote a very basic Google Home SDK and a similarly basic Google Home Exporter: The SDK only implements /setup/eureka_info and then only some of the returned properties. There’s not a lot of metric-like data to use besides SignalLevel (signal_level) and NoiseLevel (noise_level). I’m not clear on the meaning of some of the properties.

Posts

Google Cloud Platform (GCP) Exporter

Earlier this week I discussed a Linode Prometheus Exporter. I added metrics for Digital Ocean’s Managed Kubernetes service to @metalmatze’s Digital Ocean Exporter. This left, metrics for Google Cloud Platform (GCP) which has, for many years, been my primary cloud platform. So, today I wrote Prometheus Exporter for Google Cloud Platform. All 3 of these exporters follow the template laid down by @metalmatze and, because each of these services has a well-written Golang SDK, it’s straightforward to implement an exporter for each of them.

Posts

Linode Prometheus Exporter

I enjoy using Prometheus and have toyed around with it for some time particularly in combination with Kubernetes. I signed up with Linode [referral] compelled by the addition of a managed Kubernetes service called Linode Kubernetes Engine (LKE). I have an anxiety that I’ll inadvertently leave resources running (unused) on a cloud platform. Instead of refreshing the relevant billing page, it struck me that Prometheus may (not yet proven) help.

Posts

PyPi Transparency

I’ve been noodling around with another Trillian personality. Another in a theme that interests me in providing tamperproof logs for the packages in the popular package management registries. The Golang team recently announced Go Module Mirror which is built atop Trillian. It seems to me that all the package registries (Go Modules, npm, Maven, NuGet etc.) would benefit from tamperproof logs hosted by a trusted 3rd-party. As you may have guessed, PyPi Transparency is a log for PyPi packages.

Posts

Cloud Functions Simple(st) HTTP Multi-host Proxy

Tweaked yesterday’s solution so that it will randomly select one from several hosts with which it’s configured. package proxy import ( "log" "math/rand" "net/http" "net/url" "os" "strings" "time" ) func robin() { hostsList := os.Getenv("PROXY_HOST") if hostsList == "" { log.Fatal("'PROXY_HOST' environment variable should contain comma-separated list of hosts") } // Comma-separated lists of hosts hosts := strings.

Posts

Cloud Functions Simple(st) HTTP Proxy

I’m investigating the use of LetsEncrypt for gRPC services. I found this straightforward post by Scott Devoid and am going to try this approach. Before I can do that, I need to be able to publish services (make them Internet-accessible) and would like to try to continue to use GCP for free. Some time ago, I wrote about using the excellent Microk8s on GCP. Using an f1-micro, I’m hoping (!) to stay within the Compute Engine free tier.

Posts

pypi-transparency

The goal of pypi-transparency is very similar to the underlying motivation for the Golang team’s Checksum Database (also built with Trillian). Even though, PyPi provides hashes of the content of packages it hosts, the developer must trust that PyPi’s data is consistent. One ambition with pypi-transparency is to provide a companion, tamperproof log of PyPi package files in order to provide a double-check of these hashes. It is important to understand what this does (and does not) provide.

Posts

Automatic Certs w/ Golang gRPC service on Compute Engine

I needed to deploy a healthcheck-enabled gRPC TLS-enabled service. Fortunately, most (all?) of the SDKs include an implementation, e.g. Golang has grpc-go/health. I learned in my travels that: DigitalOcean [App] platform does not (link) work with TLS-based gRPC apps. Fly has a regression (link) that breaks gRPC So, I resorted to Google Cloud Platform (GCP). Although Cloud Run would be well-suited to running the gRPC app, it uses a proxy|sidecar to provision a cert for the app and I wanted to be able to (easily use a custom domain) and give myself a somewhat general-purpose solution.

Posts

Consul discovers Google Cloud Run

I’ve written a basic discoverer of Google Cloud Run services. This is for a project and it extends work done in some previous posts to Multiplex gRPC and Prometheus with Cloud Run and to use Consul for Prometheus service discovery. This solution: Accepts a set of Google Cloud Platform (GCP) projects Trawls them for Cloud Run services Assumes that the services expose Prometheus metrics on :443/metrics Relabels the services Surfaces any discovered Cloud Run services’ metrics in Prometheus You’ll need Docker and Docker Compose.

Posts

Multiplexing gRPC and HTTP (Prometheus) endpoints with Cloud Run

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another. Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port! TL;DR See the cmux Limitations and use:

Posts

Firestore Golang Timestamps & Merging

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quikcly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum. I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

Posts

Cloud Firestore Triggers in Golang

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language. I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.

Posts

Using Golang with the Firestore Emulator

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang). Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form: package main import ( "context" "crypto/sha256" "fmt" "log" "os" "time" "cloud.

Posts

Programmatically deploying Cloud Run services (Golang|Python)

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so. My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work. Goal Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run. i.e. achieve this: gcloud run deploy ${NAME} \ --image=${IMAGE} \ --platform=managed \ --no-allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} TRICK --log-http is your friend

Posts

webmention

Let’s see if this works!? I’ve added the following link to this site’s baseof.html so that it is now rendered for each page: <link href="https://us-central1-webmention.cloudfunctions.net/webmention" rel="webmention" /> I discovered indieweb yesterday reading about webmention. Who knows what got me to webmention!? The principles of both are sound. Instead of relying on come-go behemoths to run our digital world, indieweb seeks to provide technologies that enable us to achieve things without them.

Posts

Hugo and Google Cloud Storage

I’m using Hugo as a static site generator for this blog. I’m using Firebase (for free) to host lefsilver. I have other domains that I want to promote and decided to use Google Cloud Storage buckets for these sites. Using Google Cloud Storage for Hosting a static website and using Hugo to deploy sites to Google Cloud Storage (GCS) are documented but, I didn’t find a location where this is combined into a single tutorial and I wanted to add an explanation for ensuring your sites are included in Google’s and Bing’s search indexes.

Posts

Actions SDK Conversational Quickstart

Google’s tutorial didn’t work for me. In this post, I’ll help you get this working. https://developers.google.com/assistant/conversational/quickstart Create and set up a project This mostly works. I recommend using the Actions Console as described to create the project. I chose “Custom” and “Blank Project” You need not enable Actions API as this is done automatically: For the console work, I’m going to use Google’s excellent Cloud Shell. You may access this through the browser or through a terminal:

Posts

WASM Cloud Functions

Following on from waPC & Protobufs and a question on Stack Overflow about Cloud Functions, I was compelled to try running WASM on Cloud Functions no JavaScript. I wanted to reuse the WASM waPC functions that I’d written in Rust as described in the other post. Cloud Functions does not (yet!?) provide a Rust runtime and so I’m using the waPC Host for Go in this example. It works! PARAMS=$(printf '{"a":{"real":39,"imag":3},"b":{"real":39,"imag":3}}' | base64) TOKEN=$(gcloud auth print-identity-token) echo "{ \"filename\":\"complex.

Posts

Rust Crate Transparency && Rust SDK for Google Trillian

I’m noodling the utility of a Transparency solution for Rust Crates. When developers push crates to Cargo, a bunch of metadata is associated with the crate. E.g. protobuf. As with Golang Modules, Python packages on PyPi etc., there appears to be utility in making tamperproof recordings of these publications. Then, other developers may confirm that a crate pulled from cates.io is highly unlikely to have been changed. On Linux, Cargo stores downloaded crates under ${HOME}/.

Posts

Google Trillian on Cloud Run

I’ve written previously (Google Trillian for Noobs) about Google’s interesting project Trillian and about some of the “personalities” (e.g. PyPi Transparency) that I’ve build using it. Having gone slight cra-cra on Cloud Run and gRPC this week with Golang gRPC Cloud Run and gRPC, Cloud Run & Endpoints, I thought it’d be fun to deploy Trillian and a personality to Cloud Run. It mostly (!) works :-) At the end of the post, I’ve summarized creating a Cloud SQL instance to host the Trillian data(base).

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Golang gRPC Cloud Run

Update: 2020-03-24: Since writing this post, I’ve contributed Golang and Rust samples to Google’s project. I recommend you start there. Google explained how to run gRPC servers with Cloud Run. The examples are good but only Python and Node.JS: gRPC comes to Cloud Run gRPC in Google Cloud Run Missing Golang…. until now ;-) I had problems with 1.14 and so I’m using 1.13. Project structure I’ll tidy up my repo but the code may be found:

Posts

Accessing GCR repos from Kubernetes

Until today, I’d not accessed a Google Container Registry repo from a non-GKE Kubernetes deployment. It turns out that it’s pretty well-documented (link) but, here’s an end-end example. Assuming: BILLING=[[YOUR-BILLING]] PROJECT=[[YOUR-PROJECT]] SERVER="us.gcr.io" If not already: gcloud projects create {$PROJECT} gcloud beta billing projects link ${PROJECT} \ --billing-account=${BILLING} gcloud services enable containerregistry.googleapis.com \ --project=${PROJECT} Container Registry IMAGE="busybox" # Or ... docker pull ${IMAGE} docker tag \ ${IMAGE} \ ${SERVER}/${PROJECT}/${IMAGE} docker push ${SERVER}/${PROJECT}/${IMAGE} gcloud container images list-tags ${SERVER}/${PROJECT}/${IMAGE} Service Account Create a service account that’s permitted to download (read-only) images from this project’s registry

Posts

Cloud Build wishlist: Mountable Golang Modules Proxy

I think it would be valuable if Google were to provide volumes in Cloud Build of package registries (e.g. Go Modules; PyPi; Maven; NPM etc.). Google provides a mirror of a subset of Docker Hub. This confers several benefits: Google’s imprimatur; speed (latency); bandwidth; and convenience. The same benefits would apply to package registries. In the meantime, there’s a hacky way to gain some of the benefits of these when using Cloud Build.

Posts

Setting up a GCE Instance as an Inlets Exit Node

The prolific Alex Ellis has a new project, Inlets. Here’s a quick tutorial using Google Compute Platform’s (GCP) Compute Engine (GCE). NB I’m using one of Google’s “Always free” f1-micro instances but you may still pay for network *gress and storage Assumptions I’m assuming you’ve a Google account, have used GCP and have a billing account established, i.e. the following returns at least one billing account: gcloud beta billing accounts list If you’ve only one billing account and it’s the one you wish to use, then you can:

Posts

Kubernetes Engine and Free Tier

Google Cloud Platform Free Tier appears (please verify this for yourself) to provide the ability to run a(n admittedly miniscule) Kubernetes cluster for free. So, why do this? It provides a definitive Kubernetes (Engine) experience on Google Cloud Platform that you may use for learning and testing. Kubernetes Engine the master node(s) and the control plane are free. Kubernetes (i.e. Compute Engine) nodes potentially incur charges including for the VM runtime and any attached storage, snapshots etc.

Posts

Google Fit

I’ve spent a few days exploring [Google Fit SDK] as I try to wean myself from my obsession with metrics (of all forms). A quick Googling got me to Robert’s Exporter Google Fit Daily Steps, Weight and Distance to a Google Sheet. This works and is probably where I should have stopped… avoiding the rabbit hole that I’ve been down… I threw together a simple Golang implementation of the SDK using Google’s Golang API Client Library.

Posts

Google Home Exporter

I’m obsessing over Prometheus exporters. First came Linode Exporter, then GCP Exporter and, on Sunday, I stumbled upon a reverse-engineered API for Google Home devices and so wrote a very basic Google Home SDK and a similarly basic Google Home Exporter: The SDK only implements /setup/eureka_info and then only some of the returned properties. There’s not a lot of metric-like data to use besides SignalLevel (signal_level) and NoiseLevel (noise_level). I’m not clear on the meaning of some of the properties.

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

Secure (TLS) gRPC services with VKE

NOTE cert-manager is a better solution to what follows. I’ve a need to deploy a Vultr Kubernetes Engine (VKE) cluster on a daily basis (create and delete within a few hours) and expose (securely|TLS) a gRPC service. I have an existing solution Automatic Certs w/ Golang gRPC service on Compute Engine that combines a gRPC Healthchecking and an ACME service and decided to reuse this. In order for it work, we need:

Posts

Using Google's Public Certificate Authority with Golang autocert

Last year, I wrote about using Automatic Certs w/ Golang gRPC service on Compute Engine. That solution uses ACME with (the wonderful) Let’s Encrypt. Google is offering a private preview of Automate Public Certificates Lifecycle Management via RFC 8555 (ACME) and, because I’m using Google Cloud Platform extensively to build a “thing” and I think it would be useful to have a backup to Let’s Encrypt, I thought I’d give the solution a try.

Posts

Automatic Certs w/ Golang gRPC service on Compute Engine

I needed to deploy a healthcheck-enabled gRPC TLS-enabled service. Fortunately, most (all?) of the SDKs include an implementation, e.g. Golang has grpc-go/health. I learned in my travels that: DigitalOcean [App] platform does not (link) work with TLS-based gRPC apps. Fly has a regression (link) that breaks gRPC So, I resorted to Google Cloud Platform (GCP). Although Cloud Run would be well-suited to running the gRPC app, it uses a proxy|sidecar to provision a cert for the app and I wanted to be able to (easily use a custom domain) and give myself a somewhat general-purpose solution.

Posts

Renewing Firebase Authentication ID tokens with gRPC

I’ve written before about a project in which I’m using Firebase Authentication in combination with Google Cloud Endpoints and a gRPC service running on Cloud Run: Firebase Authentication, Cloud Endpoints and gRPC (1of2) Firebase Authentication, Cloud Endpoints and gRPC (2of2) This works well with one caveat, the ID tokens (JWTs) minted by Firebase Authentication have a 3600 second (one hour) lifetime. The user flow in my app is that whenever the user invokes the app’s CLI:

Posts

gRPC Interceptors and in-memory gRPC connections

For… reasons, I wanted to pre-filter gRPC requests to check for authorization. Authorization is implemented as a ‘micro-service’ and I wanted the authorization server to run in the same process as the gRPC client. TL;DR: Shiju’s “Writing gRPC Interceptors in Go” is great This Stack overflow answer ostensibly for writing unit tests for gRPC got me an in-process server What follows stands on these folks’ shoulders… A key motivator for me to write blog posts is that helps me ensure that I understand things.

Posts

Firebase Authentication, Cloud Endpoints and gRPC (2of2)

Earlier this week, I wrote about using Firebase Authentcation, Cloud Endpoints and gRPC (1of2). Since then, I learned some more and added a gRPC interceptor to implement basic authorization for the service. ESPv2 --allow-unauthenticated The Cloud Enpoints (ESPv2) proxy must be run as --allow-unauthenticated on Cloud Run to ensure that requests make it to the proxy where the request is authenticated and only authenticated requests make it on to the backend service.

Posts

Firebase Authentication, Cloud Endpoints and gRPC (1of2)

I’m building a service that requires user authentication. The primary endpoint is a gRPC-based service. I would like to consider using certificate-based auth but this feels… challenging. Instead, I have been aware of, but never used, Firebase Authentication and was interested to see that Cloud Endpoints includes Firebase Authentication as one of its supported auth mechanisms. Curiosity piqued, I confirmed that gRPC supports Google token-based authentication. The following is a summary of what I did but I’ll leave the extensive documentation to Google, (Google’s) Firebase and gRPC, all of which, in this case, provide really good explanations.

Posts

Cloud Endpoints combine OpenAPI and gRPC... or not!

See: Multiplexing gRPC and HTTP endpoints with Cloud Run gRPC, Cloud Run & Endpoints ESPv2: Configure Cloud Endpoints to proxy traffic to a Cloud Run multiplexed (gRPC|HTTP) service Challenges: Cloud Run permits single port Cloud Run services publishing e.g. gRPC and Prometheus, must multiplex transports Cloud Run services publishing multiplexed transports are challenging to expose using Cloud Endpoints Hypothesis #1: Multiplexed transports work with Cloud Run See: Multiplexing gRPC and HTTP endpoints with Cloud Run

Posts

Multiplexing gRPC and HTTP (Prometheus) endpoints with Cloud Run

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another. Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port! TL;DR See the cmux Limitations and use:

Posts

Fly.io

I spent some time over the weekend understanding Fly.io. It’s always fascinating to me how many smart people are building really neat solutions. Fly.io is subtly different to other platforms that I use (Kubernetes, GCP, DO, Linode) and I’ve found the Fly.io team to be highly responsive and helpful to my noob questions. One of the team’s posts, Docker without Docker surfaced in my Feedly feed (hackernews) and it piqued my interest.

Posts

Dapr

It’s a good name, I read it as “dapper” but I frequently type “darp” :-( Was interested to read that Dapr is now v1.0 and decided to check it out. I was initially confused between Dapr and service mesh functionality. But, having used Dapr, it appears to be more focused in aiding the development of (cloud-native) (distributed) apps by providing developers with abstractions for e.g. service discovery, eventing, observability whereas service meshes feel (!

Posts

Remotely invoking WASM functions using gRPC and waPC

Following on from waPC & Protobufs, I can now remotely invoke (arbitrary) WASM functions: Client: The logging isn’t perfectly clear but, the client gets (a previously added) WASM binary from the server (using the SHA-256 of the WASM binary as a unique identifier). The result includes metadata that includes a protobuf descriptor of the WASM binary’s functions. The descriptor defines gRPC services (that represent the WASM functions) with input (parameters) and output (results) messages.

Posts

Rust implementation of Crate Transparency using Google Trillian

I’ve been hacking on a Rust-based transparent application for Google Trillian. As appears to be my fixation, this personality is for another package manager. This time, Rust’s Crates often found in crates.io which is Rust’s Package Registry. I discussed this project earlier this month Rust Crate Transparency && Rust SDK for Google Trillian and and earlier approach for Python’s packages with pypi-transparency. This time, of course, I’m using Rust. And, by way of a first for me, for the gRPC server implementation (aka “personality”).

Posts

Rust Crate Transparency && Rust SDK for Google Trillian

I’m noodling the utility of a Transparency solution for Rust Crates. When developers push crates to Cargo, a bunch of metadata is associated with the crate. E.g. protobuf. As with Golang Modules, Python packages on PyPi etc., there appears to be utility in making tamperproof recordings of these publications. Then, other developers may confirm that a crate pulled from cates.io is highly unlikely to have been changed. On Linux, Cargo stores downloaded crates under ${HOME}/.

Posts

Google Trillian on Cloud Run

I’ve written previously (Google Trillian for Noobs) about Google’s interesting project Trillian and about some of the “personalities” (e.g. PyPi Transparency) that I’ve build using it. Having gone slight cra-cra on Cloud Run and gRPC this week with Golang gRPC Cloud Run and gRPC, Cloud Run & Endpoints, I thought it’d be fun to deploy Trillian and a personality to Cloud Run. It mostly (!) works :-) At the end of the post, I’ve summarized creating a Cloud SQL instance to host the Trillian data(base).

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Golang gRPC Cloud Run

Update: 2020-03-24: Since writing this post, I’ve contributed Golang and Rust samples to Google’s project. I recommend you start there. Google explained how to run gRPC servers with Cloud Run. The examples are good but only Python and Node.JS: gRPC comes to Cloud Run gRPC in Google Cloud Run Missing Golang…. until now ;-) I had problems with 1.14 and so I’m using 1.13. Project structure I’ll tidy up my repo but the code may be found:

Posts

Cloud Functions Simple(st) HTTP Multi-host Proxy

Tweaked yesterday’s solution so that it will randomly select one from several hosts with which it’s configured. package proxy import ( "log" "math/rand" "net/http" "net/url" "os" "strings" "time" ) func robin() { hostsList := os.Getenv("PROXY_HOST") if hostsList == "" { log.Fatal("'PROXY_HOST' environment variable should contain comma-separated list of hosts") } // Comma-separated lists of hosts hosts := strings.

Posts

Cloud Functions Simple(st) HTTP Proxy

I’m investigating the use of LetsEncrypt for gRPC services. I found this straightforward post by Scott Devoid and am going to try this approach. Before I can do that, I need to be able to publish services (make them Internet-accessible) and would like to try to continue to use GCP for free. Some time ago, I wrote about using the excellent Microk8s on GCP. Using an f1-micro, I’m hoping (!) to stay within the Compute Engine free tier.

Posts

Google Trillian on Cloud Run

I’ve written previously (Google Trillian for Noobs) about Google’s interesting project Trillian and about some of the “personalities” (e.g. PyPi Transparency) that I’ve build using it. Having gone slight cra-cra on Cloud Run and gRPC this week with Golang gRPC Cloud Run and gRPC, Cloud Run & Endpoints, I thought it’d be fun to deploy Trillian and a personality to Cloud Run. It mostly (!) works :-) At the end of the post, I’ve summarized creating a Cloud SQL instance to host the Trillian data(base).

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Golang gRPC Cloud Run

Update: 2020-03-24: Since writing this post, I’ve contributed Golang and Rust samples to Google’s project. I recommend you start there. Google explained how to run gRPC servers with Cloud Run. The examples are good but only Python and Node.JS: gRPC comes to Cloud Run gRPC in Google Cloud Run Missing Golang…. until now ;-) I had problems with 1.14 and so I’m using 1.13. Project structure I’ll tidy up my repo but the code may be found:

Posts

Automating HackMD documents

I was introduced to HackMD while working on an open-source project. It’s a collaborative editing tool for Markdown documents and there’s an API I wanted to be able to programmatically edit one of my documents with a daily update. The API is easy-to-use and my only challenge was futzing with escape characters in bash strips representing the document Markdown content. You’ll need an account with HackMD and an to Create API Token that I’ll refer to as TOKEN.

Posts

Fly.io

I spent some time over the weekend understanding Fly.io. It’s always fascinating to me how many smart people are building really neat solutions. Fly.io is subtly different to other platforms that I use (Kubernetes, GCP, DO, Linode) and I’ve found the Fly.io team to be highly responsive and helpful to my noob questions. One of the team’s posts, Docker without Docker surfaced in my Feedly feed (hackernews) and it piqued my interest.

Posts

Multiplexing gRPC and HTTP (Prometheus) endpoints with Cloud Run

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another. Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port! TL;DR See the cmux Limitations and use:

Posts

Hugo and Google Cloud Storage

I’m using Hugo as a static site generator for this blog. I’m using Firebase (for free) to host lefsilver. I have other domains that I want to promote and decided to use Google Cloud Storage buckets for these sites. Using Google Cloud Storage for Hosting a static website and using Hugo to deploy sites to Google Cloud Storage (GCS) are documented but, I didn’t find a location where this is combined into a single tutorial and I wanted to add an explanation for ensuring your sites are included in Google’s and Bing’s search indexes.

Posts

Renewing Firebase Authentication ID tokens with gRPC

I’ve written before about a project in which I’m using Firebase Authentication in combination with Google Cloud Endpoints and a gRPC service running on Cloud Run: Firebase Authentication, Cloud Endpoints and gRPC (1of2) Firebase Authentication, Cloud Endpoints and gRPC (2of2) This works well with one caveat, the ID tokens (JWTs) minted by Firebase Authentication have a 3600 second (one hour) lifetime. The user flow in my app is that whenever the user invokes the app’s CLI:

Posts

gRPC Interceptors and in-memory gRPC connections

For… reasons, I wanted to pre-filter gRPC requests to check for authorization. Authorization is implemented as a ‘micro-service’ and I wanted the authorization server to run in the same process as the gRPC client. TL;DR: Shiju’s “Writing gRPC Interceptors in Go” is great This Stack overflow answer ostensibly for writing unit tests for gRPC got me an in-process server What follows stands on these folks’ shoulders… A key motivator for me to write blog posts is that helps me ensure that I understand things.

Posts

webmention

Let’s see if this works!? I’ve added the following link to this site’s baseof.html so that it is now rendered for each page: <link href="https://us-central1-webmention.cloudfunctions.net/webmention" rel="webmention" /> I discovered indieweb yesterday reading about webmention. Who knows what got me to webmention!? The principles of both are sound. Instead of relying on come-go behemoths to run our digital world, indieweb seeks to provide technologies that enable us to achieve things without them.

Posts

NGINX Ingress

I’ve written a couple of deployment options (Google Compute Engine; Kubernetes) for an open-source project. The Kubernetes deployment provides NodePort and (TCP) LoadBalancer options and I’ve been trying (unsuccessfully) to add HTTPS Load-balancing. I should (!) try to deploy to Google Kubernetes Engine (GKE) but I’ve been using microk8s, Digital Ocean Managed Kubernetes and the Linode LKE Beta. Each of these requires an implementation of Ingress controller. For GKE, GCP’s HTTP/S Load-balancer (GCLB) is used.

Posts

Setting up a GCE Instance as an Inlets Exit Node

The prolific Alex Ellis has a new project, Inlets. Here’s a quick tutorial using Google Compute Platform’s (GCP) Compute Engine (GCE). NB I’m using one of Google’s “Always free” f1-micro instances but you may still pay for network *gress and storage Assumptions I’m assuming you’ve a Google account, have used GCP and have a billing account established, i.e. the following returns at least one billing account: gcloud beta billing accounts list If you’ve only one billing account and it’s the one you wish to use, then you can:

Posts

gRPC Interceptors and in-memory gRPC connections

For… reasons, I wanted to pre-filter gRPC requests to check for authorization. Authorization is implemented as a ‘micro-service’ and I wanted the authorization server to run in the same process as the gRPC client. TL;DR: Shiju’s “Writing gRPC Interceptors in Go” is great This Stack overflow answer ostensibly for writing unit tests for gRPC got me an in-process server What follows stands on these folks’ shoulders… A key motivator for me to write blog posts is that helps me ensure that I understand things.

Posts

ESPHome, MQTT, Prometheus and almost Cloud IoT

ESPHome is a very interesting project. I’d not heard of it until this week and am surprised that it isn’t more newsworthy. I’m always tinkering with IoT stuff, have a couple of Wemos D1 ESP8266s. They are brought out occasionally for learning. I’ve been using them this week with ESPHome. I’m looking to buy some Xiaomi BLE temperature sensors and thinking I could read the temperatures from these using the ESPs (thanks to ESPHome) and publish the data to MQTT.

Posts

Stripe

It’s been almost a month since my last post. I’ve been occupied learning Stripe and integrating it into an application that I’m developing. The app benefits from a billing mechanism for prospective customers and, as far as I can tell, Stripe is the solution. I’d be interested in hearing perspectives on alternatives. As with any platform, there’s good and bad and I’ll summarize my perspective on Stripe here. It’s been some time since I developed in JavaScript and this lack of familiarity has meant that the solution took longer than I wanted to develop.

Posts

Vultr CLI and JSON output

I’ve begun exploring Vultr after the company announced a managed Kubernetes offering Vultr Kubernetes Engine (VKE). In my brief experience, it’s a decent platform and its CLI vultr-cli is mostly (!) good. The CLI has a limitation in that command output is text formatted and this makes it challenging to parse the output when scripting. NOTE The Vultr developers have a branch rewrite that includes a solution to this problem.

Posts

Renewing Firebase Authentication ID tokens with gRPC

I’ve written before about a project in which I’m using Firebase Authentication in combination with Google Cloud Endpoints and a gRPC service running on Cloud Run: Firebase Authentication, Cloud Endpoints and gRPC (1of2) Firebase Authentication, Cloud Endpoints and gRPC (2of2) This works well with one caveat, the ID tokens (JWTs) minted by Firebase Authentication have a 3600 second (one hour) lifetime. The user flow in my app is that whenever the user invokes the app’s CLI:

Posts

Krustlet on DO Managed Kubernetes

I’ve spent time this week returning to the interesting Deislabs project Krustlet. Since the last time, the bootstrapping process has been simplified using Kubernetes Bootstrap Tokens. I know this updated process works with MicroK8s. Unfortunately, I’m struggling with it on GKE and thought I’d try DigitalOcean Managed Kubernetes. It worked first time! In the following, we run both the Kubernetes cluster and the Krustlet Droplet on DigitalOcean but, as long as the cluster and the VM are able to communicate with one another, you should be able to run these anywhere.

Posts

waPC & Protobufs

I’m hacking around with a solution that combines WASM and Google Trillian. Ultimately, I’d like to be able to ship WASM (binaries) to a Trillian personality and then invoke (exported) functions on them. Some this was borne from the interesting exploration of Krustlet and its application of wascc. I’m still booting into WASM but it’s a very interesting technology that has most interesting potential outside the browser. Some folks have been trailblazing the technology and I have been reading Kevin Hoffman’s medium and wascc (nee waxosuit) work.

Posts

Prometheus VPA Recommendations

Phew! For Want of a Nail I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered: kubectl top Vertical Pod Autoscaler A (valuable) digression through PodMonitor kube-state-metrics `kubectl-patch Created a Graph References Kubernetes Resources Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests. E.g.: resources: limits: cpu: "1" memory: "512Mi" I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

Posts

Prometheus VPA Recommendations

Phew! For Want of a Nail I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered: kubectl top Vertical Pod Autoscaler A (valuable) digression through PodMonitor kube-state-metrics `kubectl-patch Created a Graph References Kubernetes Resources Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests. E.g.: resources: limits: cpu: "1" memory: "512Mi" I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

Posts

`gcloud beta run services replace`

TL;DR I’m working on a project that includes multiple Cloud Run services. I’ve been putting my gcloud head on to deploy these services thinking that it’s curious there’s no way to write the specs as YAML configs. Today, I learned that there is: gcloud beta run services replace. What prompted the discovery was some frustration trying to deploy a JSON-valued environment variable to Cloud Run: local FIREBASE_CONFIG="{ apiKey: ${FIREBASE_API_KEY}, authDomain: ${FIREBASE_AUTH_DOMAIN}, projectId: ${FIREBASE_PROJECT}, storageBucket: ${FIREBASE_STORAGE_BUCKET}, messagingSenderId: ${FIREBASE_MESSAGING_SENDER}, appId: ${FIREBASE_APP}}" gcloud run deploy ${SRV_NAME} \ --image=${IMAGE} \ --command="/server" \ --args="--endpoint=:${PORT}" \ --set-env-vars=FIREBASE_CONFIG="${FIREBASE_CONFIG}" \ --max-instances=1 \ --memory=256Mi \ --ingress=all \ --platform=managed \ --port=${PORT} \ --allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} gcloud balks at this.

Posts

Secure (TLS) gRPC services with VKE

NOTE cert-manager is a better solution to what follows. I’ve a need to deploy a Vultr Kubernetes Engine (VKE) cluster on a daily basis (create and delete within a few hours) and expose (securely|TLS) a gRPC service. I have an existing solution Automatic Certs w/ Golang gRPC service on Compute Engine that combines a gRPC Healthchecking and an ACME service and decided to reuse this. In order for it work, we need:

Posts

Vultr CLI and JSON output

I’ve begun exploring Vultr after the company announced a managed Kubernetes offering Vultr Kubernetes Engine (VKE). In my brief experience, it’s a decent platform and its CLI vultr-cli is mostly (!) good. The CLI has a limitation in that command output is text formatted and this makes it challenging to parse the output when scripting. NOTE The Vultr developers have a branch rewrite that includes a solution to this problem.

Posts

Prometheus VPA Recommendations

Phew! For Want of a Nail I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered: kubectl top Vertical Pod Autoscaler A (valuable) digression through PodMonitor kube-state-metrics `kubectl-patch Created a Graph References Kubernetes Resources Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests. E.g.: resources: limits: cpu: "1" memory: "512Mi" I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

Posts

Krustlet on DO Managed Kubernetes

I’ve spent time this week returning to the interesting Deislabs project Krustlet. Since the last time, the bootstrapping process has been simplified using Kubernetes Bootstrap Tokens. I know this updated process works with MicroK8s. Unfortunately, I’m struggling with it on GKE and thought I’d try DigitalOcean Managed Kubernetes. It worked first time! In the following, we run both the Kubernetes cluster and the Krustlet Droplet on DigitalOcean but, as long as the cluster and the VM are able to communicate with one another, you should be able to run these anywhere.

Posts

Kubernetes cert-manager

I developed an admission webhook for Akri, twice (Golang, Rust). I naively followed other examples for the generation of the certificates, created a 1.20 cluster and broke that process. I’d briefly considered using cert-manager recently but quickly abandoned the idea thinking it would be onerous and unnecessary complexity for little-old-me. I was wrong. It’s excellent and I recommend it highly. I won’t reproduce the v1beta1 and v1 examples from the Stackoverflow question as they should be self-explanatory.

Posts

Kubernetes Webhooks

I spent some time last week writing my first admission webhook for Kubernetes. I wrote the handler in Golang because I’m most familiar with Golang and because, as Kubernetes’ native language, I was more confident that the necessary SDKs would exist and that the documentation would likely use Golang by default. I struggled to find useful documentation and so this post is to help you (and me!) remember how to do this next time!

Posts

Kubernetes Device Plugins

I’m debugging an issue with Akri Zeroconf protocol in which Instance environment variables are no longer (!) being surfaced within the Broker pods. In my adventures, it seemed useful to better understand how Akri works and specifically, how Akri uses Kubernetes Device Plugins. IIUC plugins register with the Kubelet (!) via a gRPC service (Registration) that the Kubelet exposes on a UNIX socket at /var/lib/kubelet/device-plugins/kubelet.sock Then (!) if successful, devices should be reported by the Node’s metadata (spec) and available to be bound to Pods.

Posts

Akri

For the past couple of weeks, I’ve been playing around with Akri, a Microsoft (DeisLabs) project for building a connected edge with Kubernetes. Kubernetes, IoT, Rust (and Golang) make this all compelling to me. Initially, I deployed an Akri End-to-End to MicroK8s on Google Compute Engine (link) and Digital Ocean (link). But I was interested to create me own example and so have proposed a very (!) simple HTTP-based protocol.

Posts

akri

I was very interested to read about Microsoft’s DeisLab’s latest (rust-based) Kubernetes project: akri. If I understand it correctly, it provides a mechanism to make any (IoT) device accessible to containers running within a cluster. I need to spend more time playing around with it so that I can fully understand it. I had some problems getting the End-to-End demo running on Google Compute Engine (and then I tried DigitalOcean droplet) instances.

Posts

Accessing GCR repos from Kubernetes

Until today, I’d not accessed a Google Container Registry repo from a non-GKE Kubernetes deployment. It turns out that it’s pretty well-documented (link) but, here’s an end-end example. Assuming: BILLING=[[YOUR-BILLING]] PROJECT=[[YOUR-PROJECT]] SERVER="us.gcr.io" If not already: gcloud projects create {$PROJECT} gcloud beta billing projects link ${PROJECT} \ --billing-account=${BILLING} gcloud services enable containerregistry.googleapis.com \ --project=${PROJECT} Container Registry IMAGE="busybox" # Or ... docker pull ${IMAGE} docker tag \ ${IMAGE} \ ${SERVER}/${PROJECT}/${IMAGE} docker push ${SERVER}/${PROJECT}/${IMAGE} gcloud container images list-tags ${SERVER}/${PROJECT}/${IMAGE} Service Account Create a service account that’s permitted to download (read-only) images from this project’s registry

Posts

NGINX Ingress

I’ve written a couple of deployment options (Google Compute Engine; Kubernetes) for an open-source project. The Kubernetes deployment provides NodePort and (TCP) LoadBalancer options and I’ve been trying (unsuccessfully) to add HTTPS Load-balancing. I should (!) try to deploy to Google Kubernetes Engine (GKE) but I’ve been using microk8s, Digital Ocean Managed Kubernetes and the Linode LKE Beta. Each of these requires an implementation of Ingress controller. For GKE, GCP’s HTTP/S Load-balancer (GCLB) is used.

Posts

Kubernetes Engine and Free Tier

Google Cloud Platform Free Tier appears (please verify this for yourself) to provide the ability to run a(n admittedly miniscule) Kubernetes cluster for free. So, why do this? It provides a definitive Kubernetes (Engine) experience on Google Cloud Platform that you may use for learning and testing. Kubernetes Engine the master node(s) and the control plane are free. Kubernetes (i.e. Compute Engine) nodes potentially incur charges including for the VM runtime and any attached storage, snapshots etc.

Posts

Google Cloud Platform (GCP) Exporter

Earlier this week I discussed a Linode Prometheus Exporter. I added metrics for Digital Ocean’s Managed Kubernetes service to @metalmatze’s Digital Ocean Exporter. This left, metrics for Google Cloud Platform (GCP) which has, for many years, been my primary cloud platform. So, today I wrote Prometheus Exporter for Google Cloud Platform. All 3 of these exporters follow the template laid down by @metalmatze and, because each of these services has a well-written Golang SDK, it’s straightforward to implement an exporter for each of them.

Posts

Kubernetes Engine and Free Tier

Google Cloud Platform Free Tier appears (please verify this for yourself) to provide the ability to run a(n admittedly miniscule) Kubernetes cluster for free. So, why do this? It provides a definitive Kubernetes (Engine) experience on Google Cloud Platform that you may use for learning and testing. Kubernetes Engine the master node(s) and the control plane are free. Kubernetes (i.e. Compute Engine) nodes potentially incur charges including for the VM runtime and any attached storage, snapshots etc.

Posts

Automatic Certs w/ Golang gRPC service on Compute Engine

I needed to deploy a healthcheck-enabled gRPC TLS-enabled service. Fortunately, most (all?) of the SDKs include an implementation, e.g. Golang has grpc-go/health. I learned in my travels that: DigitalOcean [App] platform does not (link) work with TLS-based gRPC apps. Fly has a regression (link) that breaks gRPC So, I resorted to Google Cloud Platform (GCP). Although Cloud Run would be well-suited to running the gRPC app, it uses a proxy|sidecar to provision a cert for the app and I wanted to be able to (easily use a custom domain) and give myself a somewhat general-purpose solution.

Posts

Prometheus AlertManager

Yesterday I discussed a Linode Prometheus Exporter and tantalized use of Prometheus AlertManager. Success: Configure The process is straightforward although I found the Prometheus (config) documentation slightly unwieldy to navigate :-( The overall process is documented. Here are the steps I took: Configure Prometheus Configure AlertManager Revise Docker Compose Configure Prometheus Added the following to prometheus.yml: rule_files: - "/etc/alertmanager/rules/linode.yml" alerting: alertmanagers: - scheme: http static_configs: - targets: - "alertmanager:9093" Rules must be defined in separate rules files.

Posts

Linode Prometheus Exporter

I enjoy using Prometheus and have toyed around with it for some time particularly in combination with Kubernetes. I signed up with Linode [referral] compelled by the addition of a managed Kubernetes service called Linode Kubernetes Engine (LKE). I have an anxiety that I’ll inadvertently leave resources running (unused) on a cloud platform. Instead of refreshing the relevant billing page, it struck me that Prometheus may (not yet proven) help.

Posts

Golang Structured Logging w/ Google Cloud Logging (2)

UPDATE There’s an issue with my naive implementation of RenderValuesHook as described in this post. I summarized the problem in this issue where I’ve outlined (hopefully) a more robust solution. Recently, I described how to configure Golang logging so that user-defined key-values applied to the logs are parsed when ingested by Google Cloud Logging. Here’s an example of what we’re trying to achieve. This is an example Cloud Logging log entry that incorporates user-defined labels (see dog:freddie and foo:bar) and a readily-querable jsonPayload:

Posts

Golang Structured Logging w/ Google Cloud Logging

I’ve multiple components in an app and these are deployed across multiple Google Cloud Platform (GCP) services: Kubernetes Engine, Cloud Functions, Cloud Run, etc. Almost everything is written in Golang and I started the project using go-logr. logr is in two parts: a Logger that you use to write log entries; a LogSink (adaptor) that consumes log entries and outputs them to a specific log implementation. Initially, I defaulted to using stdr which is a LogSink for Go’s standard logging implementation.

Posts

Golang Xiaomi Bluetooth Temperature|Humidity (LYWSD03MMC) 2nd Gen

Well, this became more of an adventure that I’d originally wanted but, after learning some BLE and, with the help of others (Thanks Jonatha, JsBergbau), I’ve sample code that connects to 4 Xiaomi 2nd gen. Thermometers, subscribes to readings and publishes the data to MQTT. From there, I’m scraping it using Inuits MQTTGateway into Prometheus. Repo: https://github.com/DazWilkin/gomijia2 Thanks|Credit: Jonathan McDowell for gomijia and help JsBergbau for help Background I’ve been playing around with ESPHome and blogged around my very positive experience ESPHome, MQTT, Prometheus and almost Cloud IoT.

Posts

Automating HackMD documents

I was introduced to HackMD while working on an open-source project. It’s a collaborative editing tool for Markdown documents and there’s an API I wanted to be able to programmatically edit one of my documents with a daily update. The API is easy-to-use and my only challenge was futzing with escape characters in bash strips representing the document Markdown content. You’ll need an account with HackMD and an to Create API Token that I’ll refer to as TOKEN.

Posts

Firestore Golang Timestamps & Merging

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quikcly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum. I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

Posts

OriginStamp: Verifying Proofs

Recently, I wrote about some initial adventures with OriginStamp Using OriginStamp’s UI or API, submitting a hash results in transactions being submitted to Bitcoin, Ethereum and a German newspaper. Using the API, it’s possible to query OriginStamp’s service for a proof. This post explains how to verify such a proof. The diligent reader among you (Hey Mom!) will recall that I submitted a hash for the message: Frederik Jack is a bubbly Border Collie The SHA-256 hash of this message is:

Posts

waPC and MsgPack (Rust|Golang)

As my reader will know (Hey Mom!), I’ve been noodling around with WASM and waPC. I’ve been exploring ways to pass structured messages across the host:guest boundary. Protobufs was my first choice. @KevinHoffman created waPC and waSCC and he explained to me and that wSCC uses Message Pack. It’s slightly surprising to me (still) that technologies like this exist with everyone else seemingly using them and I’ve not heard of them.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

Multiplexing gRPC and HTTP (Prometheus) endpoints with Cloud Run

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another. Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port! TL;DR See the cmux Limitations and use:

Posts

Prometheus VPA Recommendations

Phew! For Want of a Nail I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered: kubectl top Vertical Pod Autoscaler A (valuable) digression through PodMonitor kube-state-metrics `kubectl-patch Created a Graph References Kubernetes Resources Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests. E.g.: resources: limits: cpu: "1" memory: "512Mi" I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

Posts

Dapr

It’s a good name, I read it as “dapper” but I frequently type “darp” :-( Was interested to read that Dapr is now v1.0 and decided to check it out. I was initially confused between Dapr and service mesh functionality. But, having used Dapr, it appears to be more focused in aiding the development of (cloud-native) (distributed) apps by providing developers with abstractions for e.g. service discovery, eventing, observability whereas service meshes feel (!

Posts

Kubernetes Device Plugins

I’m debugging an issue with Akri Zeroconf protocol in which Instance environment variables are no longer (!) being surfaced within the Broker pods. In my adventures, it seemed useful to better understand how Akri works and specifically, how Akri uses Kubernetes Device Plugins. IIUC plugins register with the Kubelet (!) via a gRPC service (Registration) that the Kubelet exposes on a UNIX socket at /var/lib/kubelet/device-plugins/kubelet.sock Then (!) if successful, devices should be reported by the Node’s metadata (spec) and available to be bound to Pods.

Posts

Akri

For the past couple of weeks, I’ve been playing around with Akri, a Microsoft (DeisLabs) project for building a connected edge with Kubernetes. Kubernetes, IoT, Rust (and Golang) make this all compelling to me. Initially, I deployed an Akri End-to-End to MicroK8s on Google Compute Engine (link) and Digital Ocean (link). But I was interested to create me own example and so have proposed a very (!) simple HTTP-based protocol.

Posts

akri

I was very interested to read about Microsoft’s DeisLab’s latest (rust-based) Kubernetes project: akri. If I understand it correctly, it provides a mechanism to make any (IoT) device accessible to containers running within a cluster. I need to spend more time playing around with it so that I can fully understand it. I had some problems getting the End-to-End demo running on Google Compute Engine (and then I tried DigitalOcean droplet) instances.

Posts

akri

I was very interested to read about Microsoft’s DeisLab’s latest (rust-based) Kubernetes project: akri. If I understand it correctly, it provides a mechanism to make any (IoT) device accessible to containers running within a cluster. I need to spend more time playing around with it so that I can fully understand it. I had some problems getting the End-to-End demo running on Google Compute Engine (and then I tried DigitalOcean droplet) instances.

Posts

GitHub help with dependency management

This is very useful: I am building an application that comprises multiple repos. I continue to procrastinate on whether using multiple repos vs. a monorepo was a good idea but, an issue that I have (had) is the need to ensure that the repos’ contents are using current|latest modules. GitHub can help. Most of the application is written in Golang with a smattering of Rust and some JavaScript. Aside Even though I’m more proficient with Golang, I want to become more proficient in Rust and I contemplated trying to write the entire application in Rust.

Posts

Cloud Build wishlist: Mountable Golang Modules Proxy

I think it would be valuable if Google were to provide volumes in Cloud Build of package registries (e.g. Go Modules; PyPi; Maven; NPM etc.). Google provides a mirror of a subset of Docker Hub. This confers several benefits: Google’s imprimatur; speed (latency); bandwidth; and convenience. The same benefits would apply to package registries. In the meantime, there’s a hacky way to gain some of the benefits of these when using Cloud Build.

Posts

ESPHome, MQTT, Prometheus and almost Cloud IoT

ESPHome is a very interesting project. I’d not heard of it until this week and am surprised that it isn’t more newsworthy. I’m always tinkering with IoT stuff, have a couple of Wemos D1 ESP8266s. They are brought out occasionally for learning. I’ve been using them this week with ESPHome. I’m looking to buy some Xiaomi BLE temperature sensors and thinking I could read the temperatures from these using the ESPs (thanks to ESPHome) and publish the data to MQTT.

Posts

ESPHome, MQTT, Prometheus and almost Cloud IoT

ESPHome is a very interesting project. I’d not heard of it until this week and am surprised that it isn’t more newsworthy. I’m always tinkering with IoT stuff, have a couple of Wemos D1 ESP8266s. They are brought out occasionally for learning. I’ve been using them this week with ESPHome. I’m looking to buy some Xiaomi BLE temperature sensors and thinking I could read the temperatures from these using the ESPs (thanks to ESPHome) and publish the data to MQTT.

Posts

waPC and MsgPack (Rust|Golang)

As my reader will know (Hey Mom!), I’ve been noodling around with WASM and waPC. I’ve been exploring ways to pass structured messages across the host:guest boundary. Protobufs was my first choice. @KevinHoffman created waPC and waSCC and he explained to me and that wSCC uses Message Pack. It’s slightly surprising to me (still) that technologies like this exist with everyone else seemingly using them and I’ve not heard of them.

Posts

NGINX Ingress

I’ve written a couple of deployment options (Google Compute Engine; Kubernetes) for an open-source project. The Kubernetes deployment provides NodePort and (TCP) LoadBalancer options and I’ve been trying (unsuccessfully) to add HTTPS Load-balancing. I should (!) try to deploy to Google Kubernetes Engine (GKE) but I’ve been using microk8s, Digital Ocean Managed Kubernetes and the Linode LKE Beta. Each of these requires an implementation of Ingress controller. For GKE, GCP’s HTTP/S Load-balancer (GCLB) is used.

Posts

Scraping metrics exposed by Google Cloud Run services that require authentication

I’ve written a solution (gcp-oidc-token-proxy) that can be used in conjunction with Prometheus OAuth2 to authenticate requests so that Prometheus can scrape metrics exposed by e.g. Cloud Run services that require authentication. The solution resulted from my question on Stack overflow. Problem #1: Endpoint requires authentication Given a Cloud Run service URL for which: ENDPOINT="my-server-blahblah-wl.a.run.app" # Returns 200 when authentication w/ an ID token TOKEN="$(gcloud auth print-identity-token)" curl \ --silent \ --request GET \ --header "Authorization: Bearer ${TOKEN}" \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics # Returns 403 otherwise curl \ --silent \ --request GET \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics Problem #2: Prometheus OAuth2 configuration is constrained

Posts

Scraping metrics exposed by Google Cloud Run services that require authentication

I’ve written a solution (gcp-oidc-token-proxy) that can be used in conjunction with Prometheus OAuth2 to authenticate requests so that Prometheus can scrape metrics exposed by e.g. Cloud Run services that require authentication. The solution resulted from my question on Stack overflow. Problem #1: Endpoint requires authentication Given a Cloud Run service URL for which: ENDPOINT="my-server-blahblah-wl.a.run.app" # Returns 200 when authentication w/ an ID token TOKEN="$(gcloud auth print-identity-token)" curl \ --silent \ --request GET \ --header "Authorization: Bearer ${TOKEN}" \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics # Returns 403 otherwise curl \ --silent \ --request GET \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics Problem #2: Prometheus OAuth2 configuration is constrained

Posts

Google Container Registry w/ OCI

I’ve been spending some time this week with Krustlet. I’m working on documenting how to run Krustlet(s) alongside GKE. I’ve been running a Krustlet with MicroK8s. The Krustlet demos reference WASM assemblines stored in Azure Container Registry as OCI containers. Google Container Registry supports OCI format and so I tried (successfully) using GCR instead of AZR. There may be an easier approach but this is how I got this working.

Posts

Scraping metrics exposed by Google Cloud Run services that require authentication

I’ve written a solution (gcp-oidc-token-proxy) that can be used in conjunction with Prometheus OAuth2 to authenticate requests so that Prometheus can scrape metrics exposed by e.g. Cloud Run services that require authentication. The solution resulted from my question on Stack overflow. Problem #1: Endpoint requires authentication Given a Cloud Run service URL for which: ENDPOINT="my-server-blahblah-wl.a.run.app" # Returns 200 when authentication w/ an ID token TOKEN="$(gcloud auth print-identity-token)" curl \ --silent \ --request GET \ --header "Authorization: Bearer ${TOKEN}" \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics # Returns 403 otherwise curl \ --silent \ --request GET \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics Problem #2: Prometheus OAuth2 configuration is constrained

Posts

Cloud Endpoints combine OpenAPI and gRPC... or not!

See: Multiplexing gRPC and HTTP endpoints with Cloud Run gRPC, Cloud Run & Endpoints ESPv2: Configure Cloud Endpoints to proxy traffic to a Cloud Run multiplexed (gRPC|HTTP) service Challenges: Cloud Run permits single port Cloud Run services publishing e.g. gRPC and Prometheus, must multiplex transports Cloud Run services publishing multiplexed transports are challenging to expose using Cloud Endpoints Hypothesis #1: Multiplexed transports work with Cloud Run See: Multiplexing gRPC and HTTP endpoints with Cloud Run

Posts

OriginStamp Rust SDK Example

I wrote recently describing Python and Golang clients for OriginStamp based on OriginStamp’s API’s swagger spec. As a way to pursue learning rust, I’ve been forcing myself to write examples using rust. I’m honestly finding learning rust tough going and think I’m probably better to revert to the “Learning Rust” tutorials. That said, herewith an explanation of building a rust client using an OpenAPI (!) generated SDK from OriginStamp’s swagger spec.

Posts

OriginStamp: Verifying Proofs

Recently, I wrote about some initial adventures with OriginStamp Using OriginStamp’s UI or API, submitting a hash results in transactions being submitted to Bitcoin, Ethereum and a German newspaper. Using the API, it’s possible to query OriginStamp’s service for a proof. This post explains how to verify such a proof. The diligent reader among you (Hey Mom!) will recall that I submitted a hash for the message: Frederik Jack is a bubbly Border Collie The SHA-256 hash of this message is:

Posts

FreeTSA & Digitorus' Timestamp SDK

I wrote recently about some exploration of Timestamping with OriginStamp. Since writing that post, I had some supportive feedback from the helpful folks at OriginStamp and plan to continue exploring that solution. Meanwhile, OriginStamp exposed me to timestamping and trusted timestamping and I discovered freeTSA.org. What’s the point? These services provide authoritative proof of the existence of a digital asset before some point in time; OriginStamp provides a richer service and uses multiple timestamp authorities including Bitcoin, Ethereum and rather interestingly a German Newspaper’s Trusted Timestamp.

Posts

OriginStamp Python|Golang SDK Examples

A friend mentioned OriginStamp to me. NB There are 2 sites: originstamp.com and originstamp.org. It’s an interesting project. It’s a solution for providing auditable proof that you had a(ccess to) some digital thing before a certain date. OriginStamp provides user-|developer-friendly means to submit files|hashes (of your content) and have these bundled into transactions that are submitted to e.g. bitcoin. I won’t attempt to duplicate the narrative here, review OriginStamp’s site and some of its content.

Posts

pest: parsing in Rust

A Microsoft engineer introduced me to pest as a way to introduce service filtering in a ZeroConf plugin that I’m prototyping for Akri. It’s been fun to learn but I worry that, because I won’t use it frequently, I’m going to quickly forget what I’ve done. So, here are my notes. Here’s the problem, I’d like to be able to provide users of the ZeroConf plugin with a string-based filter that permits them to filter the services discovered when the Akri agent browses a network.

Posts

Podman

I’ve read about Podman and been intrigued by it but never taken the time to install it and play around. This morning, walking with my dog, I listened to the almost-always-interesting Kubernetes Podcast and two of the principals behind Podman were on the show to discuss it. I decided to install it and use it in this week’s project. Here’s a working Podman deployment for gcp-oidc-token-proxy ACCOUNT="..." ENDPOINT="..." # Can't match container name i.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

Prometheus HTTP Service Discovery of Cloud Run services

Some time ago, I wrote about using Prometheus Service Discovery w/ Consul for Cloud Run and also Scraping metrics exposed by Google Cloud Run services that require authentication. Both solutions remain viable but they didn’t address another use case for Prometheus and Cloud Run services that I have with a “thing” that I’ve been building. In this scenario, I want to: Configure Prometheus to scrape Cloud Run service metrics Discover Cloud Run services dynamically Authenticate to Cloud Run using Firebase Auth ID tokens These requirements and – one other – present several challenges:

Posts

Scraping metrics exposed by Google Cloud Run services that require authentication

I’ve written a solution (gcp-oidc-token-proxy) that can be used in conjunction with Prometheus OAuth2 to authenticate requests so that Prometheus can scrape metrics exposed by e.g. Cloud Run services that require authentication. The solution resulted from my question on Stack overflow. Problem #1: Endpoint requires authentication Given a Cloud Run service URL for which: ENDPOINT="my-server-blahblah-wl.a.run.app" # Returns 200 when authentication w/ an ID token TOKEN="$(gcloud auth print-identity-token)" curl \ --silent \ --request GET \ --header "Authorization: Bearer ${TOKEN}" \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics # Returns 403 otherwise curl \ --silent \ --request GET \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics Problem #2: Prometheus OAuth2 configuration is constrained

Posts

Consul discovers Google Cloud Run

I’ve written a basic discoverer of Google Cloud Run services. This is for a project and it extends work done in some previous posts to Multiplex gRPC and Prometheus with Cloud Run and to use Consul for Prometheus service discovery. This solution: Accepts a set of Google Cloud Platform (GCP) projects Trawls them for Cloud Run services Assumes that the services expose Prometheus metrics on :443/metrics Relabels the services Surfaces any discovered Cloud Run services’ metrics in Prometheus You’ll need Docker and Docker Compose.

Posts

Multiplexing gRPC and HTTP (Prometheus) endpoints with Cloud Run

Google Cloud Run is useful but, each service is limited to exposing a single port. This caused me problems with a gRPC service that serves (non-gRPC) Prometheus metrics because customarily, you would serve gRPC on one port and the Prometheus metrics on another. Fortunately, cmux provides a solution by providing a mechanism that multiplexes both services (gRPC and HTTP) on a single port! TL;DR See the cmux Limitations and use:

Posts

Prometheus Service Discovery w/ Consul for Cloud Run

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus. This is one solution. NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use. Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation.

Posts

Prometheus VPA Recommendations

Phew! For Want of a Nail I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered: kubectl top Vertical Pod Autoscaler A (valuable) digression through PodMonitor kube-state-metrics `kubectl-patch Created a Graph References Kubernetes Resources Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests. E.g.: resources: limits: cpu: "1" memory: "512Mi" I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

Posts

Deploying a Rust HTTP server to DigitalOcean App Platform

DigitalOcean launched an App Platform with many Supported Languages and Frameworks. I used Golang first, then wondered how to use non-natively-supported languages, i.e. Rust. The good news is that Docker is a supported framework and so, you can run pretty much anything. Repo: https://github.com/DazWilkin/do-apps-rust Rust I’m a Rust noob. I’m always receptive to feedback on improvements to the code. I looked to mirror the Golang example. I’m using rocket and rocket-prometheus for the first time:

Posts

Google Home Exporter

I’m obsessing over Prometheus exporters. First came Linode Exporter, then GCP Exporter and, on Sunday, I stumbled upon a reverse-engineered API for Google Home devices and so wrote a very basic Google Home SDK and a similarly basic Google Home Exporter: The SDK only implements /setup/eureka_info and then only some of the returned properties. There’s not a lot of metric-like data to use besides SignalLevel (signal_level) and NoiseLevel (noise_level). I’m not clear on the meaning of some of the properties.

Posts

Google Cloud Platform (GCP) Exporter

Earlier this week I discussed a Linode Prometheus Exporter. I added metrics for Digital Ocean’s Managed Kubernetes service to @metalmatze’s Digital Ocean Exporter. This left, metrics for Google Cloud Platform (GCP) which has, for many years, been my primary cloud platform. So, today I wrote Prometheus Exporter for Google Cloud Platform. All 3 of these exporters follow the template laid down by @metalmatze and, because each of these services has a well-written Golang SDK, it’s straightforward to implement an exporter for each of them.

Posts

Prometheus AlertManager

Yesterday I discussed a Linode Prometheus Exporter and tantalized use of Prometheus AlertManager. Success: Configure The process is straightforward although I found the Prometheus (config) documentation slightly unwieldy to navigate :-( The overall process is documented. Here are the steps I took: Configure Prometheus Configure AlertManager Revise Docker Compose Configure Prometheus Added the following to prometheus.yml: rule_files: - "/etc/alertmanager/rules/linode.yml" alerting: alertmanagers: - scheme: http static_configs: - targets: - "alertmanager:9093" Rules must be defined in separate rules files.

Posts

Linode Prometheus Exporter

I enjoy using Prometheus and have toyed around with it for some time particularly in combination with Kubernetes. I signed up with Linode [referral] compelled by the addition of a managed Kubernetes service called Linode Kubernetes Engine (LKE). I have an anxiety that I’ll inadvertently leave resources running (unused) on a cloud platform. Instead of refreshing the relevant billing page, it struck me that Prometheus may (not yet proven) help.

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

Remotely invoking WASM functions using gRPC and waPC

Following on from waPC & Protobufs, I can now remotely invoke (arbitrary) WASM functions: Client: The logging isn’t perfectly clear but, the client gets (a previously added) WASM binary from the server (using the SHA-256 of the WASM binary as a unique identifier). The result includes metadata that includes a protobuf descriptor of the WASM binary’s functions. The descriptor defines gRPC services (that represent the WASM functions) with input (parameters) and output (results) messages.

Posts

Golang Protobuf APIv2

Google has a new Golang Protobuf API, APIv2 (google.golang.org/protobuf) superseding APIv1 (github.com/golang/protobuf). If your code is importing github.com/golang/protobuf, you’re using APIv2. Otherwise, you should consult with the docs because Google reimplemented APIv1 atop APIv2. One challenge this caused me, as someone who does not use protobufs and gRPC on a daily basis, is that gRPC (code-generation) is being removed from the (Golang) protoc-gen-go, the Golang plugin that generates gRPC service bindings.

Posts

WASM Cloud Functions

Following on from waPC & Protobufs and a question on Stack Overflow about Cloud Functions, I was compelled to try running WASM on Cloud Functions no JavaScript. I wanted to reuse the WASM waPC functions that I’d written in Rust as described in the other post. Cloud Functions does not (yet!?) provide a Rust runtime and so I’m using the waPC Host for Go in this example. It works! PARAMS=$(printf '{"a":{"real":39,"imag":3},"b":{"real":39,"imag":3}}' | base64) TOKEN=$(gcloud auth print-identity-token) echo "{ \"filename\":\"complex.

Posts

waPC & Protobufs

I’m hacking around with a solution that combines WASM and Google Trillian. Ultimately, I’d like to be able to ship WASM (binaries) to a Trillian personality and then invoke (exported) functions on them. Some this was borne from the interesting exploration of Krustlet and its application of wascc. I’m still booting into WASM but it’s a very interesting technology that has most interesting potential outside the browser. Some folks have been trailblazing the technology and I have been reading Kevin Hoffman’s medium and wascc (nee waxosuit) work.

Posts

Rust implementation of Crate Transparency using Google Trillian

I’ve been hacking on a Rust-based transparent application for Google Trillian. As appears to be my fixation, this personality is for another package manager. This time, Rust’s Crates often found in crates.io which is Rust’s Package Registry. I discussed this project earlier this month Rust Crate Transparency && Rust SDK for Google Trillian and and earlier approach for Python’s packages with pypi-transparency. This time, of course, I’m using Rust. And, by way of a first for me, for the gRPC server implementation (aka “personality”).

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

Golang gRPC Cloud Run

Update: 2020-03-24: Since writing this post, I’ve contributed Golang and Rust samples to Google’s project. I recommend you start there. Google explained how to run gRPC servers with Cloud Run. The examples are good but only Python and Node.JS: gRPC comes to Cloud Run gRPC in Google Cloud Run Missing Golang…. until now ;-) I had problems with 1.14 and so I’m using 1.13. Project structure I’ll tidy up my repo but the code may be found:

Posts

Google's New Golang SDK for Protobufs

Google has released a new Golang SDK for protobuf. In the [announcement], a useful tool to redact properties is described. If like me, this is somewhat novel to you, here’s a mashup using Google’s Protocol Buffer Basics w/ redaction. To be very clear, as it’s an important distinction: Version Repo Docs v2 google.golang.org/protobuf Docs v1 github.com/golang/protobuf Docs Project Here’s my project structure:

Posts

Google's New Golang SDK for Protobufs

Google has released a new Golang SDK for protobuf. In the [announcement], a useful tool to redact properties is described. If like me, this is somewhat novel to you, here’s a mashup using Google’s Protocol Buffer Basics w/ redaction. To be very clear, as it’s an important distinction: Version Repo Docs v2 google.golang.org/protobuf Docs v1 github.com/golang/protobuf Docs Project Here’s my project structure:

Posts

Cloud Build wishlist: Mountable Golang Modules Proxy

I think it would be valuable if Google were to provide volumes in Cloud Build of package registries (e.g. Go Modules; PyPi; Maven; NPM etc.). Google provides a mirror of a subset of Docker Hub. This confers several benefits: Google’s imprimatur; speed (latency); bandwidth; and convenience. The same benefits would apply to package registries. In the meantime, there’s a hacky way to gain some of the benefits of these when using Cloud Build.

Posts

Using Google Monitoring Alerting to send Pushover notifications

Table of Contents Artifacts Pushover Caveat Cloud Monitoring Webhook Cloud Functions Cloud Run Artifacts GitHub: go-gcp-pushover-notificationchannel Image: ghcr.io/dazwilkin/go-gcp-pushover-notificationchannel:220515 Pushover Logging in to your Pushover account, you will be presented with a summary|dashboard page that includes Your User Key. Copy the value of this key into a variable called PUSHOVER_USER Create New Application|API Token Pushover API has a Pushing Messages method. The documentation describes the format of the HTTP Request.

Posts

pypi-transparency

The goal of pypi-transparency is very similar to the underlying motivation for the Golang team’s Checksum Database (also built with Trillian). Even though, PyPi provides hashes of the content of packages it hosts, the developer must trust that PyPi’s data is consistent. One ambition with pypi-transparency is to provide a companion, tamperproof log of PyPi package files in order to provide a double-check of these hashes. It is important to understand what this does (and does not) provide.

Posts

Programmatically deploying Cloud Run services (Golang|Python)

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so. My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work. Goal Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run. i.e. achieve this: gcloud run deploy ${NAME} \ --image=${IMAGE} \ --platform=managed \ --no-allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} TRICK --log-http is your friend

Posts

OriginStamp Python|Golang SDK Examples

A friend mentioned OriginStamp to me. NB There are 2 sites: originstamp.com and originstamp.org. It’s an interesting project. It’s a solution for providing auditable proof that you had a(ccess to) some digital thing before a certain date. OriginStamp provides user-|developer-friendly means to submit files|hashes (of your content) and have these bundled into transactions that are submitted to e.g. bitcoin. I won’t attempt to duplicate the narrative here, review OriginStamp’s site and some of its content.

Posts

PyPi Transparency

I’ve been noodling around with another Trillian personality. Another in a theme that interests me in providing tamperproof logs for the packages in the popular package management registries. The Golang team recently announced Go Module Mirror which is built atop Trillian. It seems to me that all the package registries (Go Modules, npm, Maven, NuGet etc.) would benefit from tamperproof logs hosted by a trusted 3rd-party. As you may have guessed, PyPi Transparency is a log for PyPi packages.

Posts

Renewing Firebase Authentication ID tokens with gRPC

I’ve written before about a project in which I’m using Firebase Authentication in combination with Google Cloud Endpoints and a gRPC service running on Cloud Run: Firebase Authentication, Cloud Endpoints and gRPC (1of2) Firebase Authentication, Cloud Endpoints and gRPC (2of2) This works well with one caveat, the ID tokens (JWTs) minted by Firebase Authentication have a 3600 second (one hour) lifetime. The user flow in my app is that whenever the user invokes the app’s CLI:

Posts

Deploying a Rust HTTP server to DigitalOcean App Platform

DigitalOcean launched an App Platform with many Supported Languages and Frameworks. I used Golang first, then wondered how to use non-natively-supported languages, i.e. Rust. The good news is that Docker is a supported framework and so, you can run pretty much anything. Repo: https://github.com/DazWilkin/do-apps-rust Rust I’m a Rust noob. I’m always receptive to feedback on improvements to the code. I looked to mirror the Golang example. I’m using rocket and rocket-prometheus for the first time:

Posts

Programmatically deploying Cloud Run services (Golang|Python)

Phew! Programmitcally deploying Cloud Run services should be easy but it didn’t find it so. My issues were that the Cloud Run Admin (!) API is poorly documented and it uses non-standard endpoints (thanks Sal!). Here, for others who may struggle with this, is how I got this to work. Goal Programmatically (have Golang, Python, want Rust) deploy services to Cloud Run. i.e. achieve this: gcloud run deploy ${NAME} \ --image=${IMAGE} \ --platform=managed \ --no-allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} TRICK --log-http is your friend

Posts

pest: parsing in Rust

A Microsoft engineer introduced me to pest as a way to introduce service filtering in a ZeroConf plugin that I’m prototyping for Akri. It’s been fun to learn but I worry that, because I won’t use it frequently, I’m going to quickly forget what I’ve done. So, here are my notes. Here’s the problem, I’d like to be able to provide users of the ZeroConf plugin with a string-based filter that permits them to filter the services discovered when the Akri agent browses a network.

Posts

Akri

For the past couple of weeks, I’ve been playing around with Akri, a Microsoft (DeisLabs) project for building a connected edge with Kubernetes. Kubernetes, IoT, Rust (and Golang) make this all compelling to me. Initially, I deployed an Akri End-to-End to MicroK8s on Google Compute Engine (link) and Digital Ocean (link). But I was interested to create me own example and so have proposed a very (!) simple HTTP-based protocol.

Posts

Deploying a Rust HTTP server to DigitalOcean App Platform

DigitalOcean launched an App Platform with many Supported Languages and Frameworks. I used Golang first, then wondered how to use non-natively-supported languages, i.e. Rust. The good news is that Docker is a supported framework and so, you can run pretty much anything. Repo: https://github.com/DazWilkin/do-apps-rust Rust I’m a Rust noob. I’m always receptive to feedback on improvements to the code. I looked to mirror the Golang example. I’m using rocket and rocket-prometheus for the first time:

Posts

Minimizing WASM binaries

I’ve spent time recently playing around with WebAssembly (WASM) and waPC. Rust and WASM were born at Mozilla and there’s a natural affinity with writing WASM binaries in Rust. In the WASM examples I’ve been using for WASM Transparency, waPC and MsgPack and waPC and Protobufs. I’ve created 3 WASM binaries: complex.wasm, simplex.wasm and fabcar.wasm and each is about 2.5MB when: cargo build --target=wasm32-unknown-unknown --release The Rust and WebAssembly book has an excellent section titled Shrinking .

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

waPC and MsgPack (Rust|Golang)

As my reader will know (Hey Mom!), I’ve been noodling around with WASM and waPC. I’ve been exploring ways to pass structured messages across the host:guest boundary. Protobufs was my first choice. @KevinHoffman created waPC and waSCC and he explained to me and that wSCC uses Message Pack. It’s slightly surprising to me (still) that technologies like this exist with everyone else seemingly using them and I’ve not heard of them.

Posts

Envoy WASM filters in Rust

A digression thanks to Sal Rashid who’s exploring WASM filters w/ Envoy. The documentation is sparse but: How to write WASM filters for Envoy… There is a Rust SDK but it’s not documented: proxy-wasm-rust-sdk I found two useful posts by Rustaceans who were able to make use of it: Extending Envoy with WASM and Rust Extending Istio with Rust and WebAssembly Here’s my simple use of the SDK’s examples.

Posts

Rust implementation of Crate Transparency using Google Trillian

I’ve been hacking on a Rust-based transparent application for Google Trillian. As appears to be my fixation, this personality is for another package manager. This time, Rust’s Crates often found in crates.io which is Rust’s Package Registry. I discussed this project earlier this month Rust Crate Transparency && Rust SDK for Google Trillian and and earlier approach for Python’s packages with pypi-transparency. This time, of course, I’m using Rust. And, by way of a first for me, for the gRPC server implementation (aka “personality”).

Posts

Rust Crate Transparency && Rust SDK for Google Trillian

I’m noodling the utility of a Transparency solution for Rust Crates. When developers push crates to Cargo, a bunch of metadata is associated with the crate. E.g. protobuf. As with Golang Modules, Python packages on PyPi etc., there appears to be utility in making tamperproof recordings of these publications. Then, other developers may confirm that a crate pulled from cates.io is highly unlikely to have been changed. On Linux, Cargo stores downloaded crates under ${HOME}/.

Posts

gRPC, Cloud Run & Endpoints

<3 Google but there’s quite often an assumption that we’re all sitting around the engineering table and, of course, we’re not. Cloud Endpoints is a powerful offering but – IMO – it’s super confusing to understand and complex to deploy. If you’re familiar with the motivations behind service meshes (e.g. Istio), Cloud Endpoints fits in a similar niche (“neesh” or “nitch”?). The underlying ambition is that, developers can take existing code and by adding a proxy (or sidecar), general-purpose abstractions, security, logging etc.

Posts

OriginStamp Rust SDK Example

I wrote recently describing Python and Golang clients for OriginStamp based on OriginStamp’s API’s swagger spec. As a way to pursue learning rust, I’ve been forcing myself to write examples using rust. I’m honestly finding learning rust tough going and think I’m probably better to revert to the “Learning Rust” tutorials. That said, herewith an explanation of building a rust client using an OpenAPI (!) generated SDK from OriginStamp’s swagger spec.

Posts

PyPi Transparency Client (Rust)

I’ve finally being able to hack my way through to a working Rust gRPC client (for PyPi Transparency). It’s not very good: poorly structured, hacky etc. but it serves the purpose of giving me a foothold into Rust development so that I can evolve it as I learn the language and its practices. There are several Rust crates (SDK) for gRPC. There’s no sanctioned SDK for Rust on grpc.io. I chose stepancheg’s grpc-rust because it’s a pure Rust implementation (not built atop the C implementation).

Posts

Prometheus HTTP Service Discovery of Cloud Run services

Some time ago, I wrote about using Prometheus Service Discovery w/ Consul for Cloud Run and also Scraping metrics exposed by Google Cloud Run services that require authentication. Both solutions remain viable but they didn’t address another use case for Prometheus and Cloud Run services that I have with a “thing” that I’ve been building. In this scenario, I want to: Configure Prometheus to scrape Cloud Run service metrics Discover Cloud Run services dynamically Authenticate to Cloud Run using Firebase Auth ID tokens These requirements and – one other – present several challenges:

Posts

Consul discovers Google Cloud Run

I’ve written a basic discoverer of Google Cloud Run services. This is for a project and it extends work done in some previous posts to Multiplex gRPC and Prometheus with Cloud Run and to use Consul for Prometheus service discovery. This solution: Accepts a set of Google Cloud Platform (GCP) projects Trawls them for Cloud Run services Assumes that the services expose Prometheus metrics on :443/metrics Relabels the services Surfaces any discovered Cloud Run services’ metrics in Prometheus You’ll need Docker and Docker Compose.

Posts

Prometheus Service Discovery w/ Consul for Cloud Run

I’m working on a project that will programmatically create Google Cloud Run services and I want to be able to dynamically discover these services using Prometheus. This is one solution. NOTE Google Cloud Run is the service I’m using, but the principle described herein applies to any runtime service that you’d wish to use. Why is this challenging? IIUC, it’s primarily because Prometheus has a limited set of plugins for service discovery, see the sections that include _sd_ in Prometheus Configuration documentation.

Posts

Playing with GitHub Container registry REST API

I’ve a day to catch up on blogging. I’m building a “thing” and getting this near to the finish line consumes my time and has meant that I’m not originating anything particularly new. However, there are a couple of tricks in my deployment process that may be of interest to others. I’ve been a long-term using of Google’s [Cloud Build] and like the simplicity (everything’s a container, alot!). Because I’m using GitHub repos, I’ve been using GitHub Actions to (re)build containers on pushes and GitHub Container registry (GHCR) to store the results.

Posts

Sigstore

I’ve been on a digression (gcp-oidc-token-proxy) this week. Yesterday I began exploring Podman and wrote briefly about running gcp-oidc-token-proxy on my localhost using it. This morning while walking with my dog, I listened to Google’s Dan Lorenc explain Sigstore (blog](https://blog.sigstore.dev/)) on The Kubelist Podcast1 The plan today is to try to sign the gcp-oidc-token-proxy container images in GitHub Container Registry. NOTE I decided against trying the hardware key approach. I have a Google Titan key and only Yubikeys are well-tested by go-piv

Posts

Secure (TLS) gRPC services with VKE

NOTE cert-manager is a better solution to what follows. I’ve a need to deploy a Vultr Kubernetes Engine (VKE) cluster on a daily basis (create and delete within a few hours) and expose (securely|TLS) a gRPC service. I have an existing solution Automatic Certs w/ Golang gRPC service on Compute Engine that combines a gRPC Healthchecking and an ACME service and decided to reuse this. In order for it work, we need:

Posts

Stripe

It’s been almost a month since my last post. I’ve been occupied learning Stripe and integrating it into an application that I’m developing. The app benefits from a billing mechanism for prospective customers and, as far as I can tell, Stripe is the solution. I’d be interested in hearing perspectives on alternatives. As with any platform, there’s good and bad and I’ll summarize my perspective on Stripe here. It’s been some time since I developed in JavaScript and this lack of familiarity has meant that the solution took longer than I wanted to develop.

Posts

OriginStamp Rust SDK Example

I wrote recently describing Python and Golang clients for OriginStamp based on OriginStamp’s API’s swagger spec. As a way to pursue learning rust, I’ve been forcing myself to write examples using rust. I’m honestly finding learning rust tough going and think I’m probably better to revert to the “Learning Rust” tutorials. That said, herewith an explanation of building a rust client using an OpenAPI (!) generated SDK from OriginStamp’s swagger spec.

Posts

OriginStamp Python|Golang SDK Examples

A friend mentioned OriginStamp to me. NB There are 2 sites: originstamp.com and originstamp.org. It’s an interesting project. It’s a solution for providing auditable proof that you had a(ccess to) some digital thing before a certain date. OriginStamp provides user-|developer-friendly means to submit files|hashes (of your content) and have these bundled into transactions that are submitted to e.g. bitcoin. I won’t attempt to duplicate the narrative here, review OriginStamp’s site and some of its content.

Posts

Infrastructure as Code

Problem I’m building an application that comprises: Kubernetes¹ Kubernetes Operator Cloud Firestore Cloud Functions Cloud Run Cloud Endpoints Stripe Firebase Authentication ¹ - I’m using Google Kubernetes Engine (GKE) but may include other managed Kubernetes offerings (e.g. Digital Ocean, Linode, Oracle). GKE clusters are manageable by gcloud but other platforms require other CLI tools. All are accessible from bash but are these supported by e.g. Terraform (see below)?

Posts

Trendnet TEW-812DRU and DD-WRT

The FBI Portland published an interesting advisory with several, sensible recommendations including firewalling IoT devices from other devices on a home network. I decided to deploy a redundant Trendnet TEW-812DRU version 2.0 for this purpose. Caveat Developer: Before I go further, I don’t recommend installing DD-WRT on a Trendnet TEW-812DRU unless you’re willing to brick the device irrecoverably. I read the DD-WRT instructions several times (“peacock thread”,router database – do not use v3.

Posts

Firestore Golang Timestamps & Merging

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quikcly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum. I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

Posts

OriginStamp: Verifying Proofs

Recently, I wrote about some initial adventures with OriginStamp Using OriginStamp’s UI or API, submitting a hash results in transactions being submitted to Bitcoin, Ethereum and a German newspaper. Using the API, it’s possible to query OriginStamp’s service for a proof. This post explains how to verify such a proof. The diligent reader among you (Hey Mom!) will recall that I submitted a hash for the message: Frederik Jack is a bubbly Border Collie The SHA-256 hash of this message is:

Posts

FreeTSA & Digitorus' Timestamp SDK

I wrote recently about some exploration of Timestamping with OriginStamp. Since writing that post, I had some supportive feedback from the helpful folks at OriginStamp and plan to continue exploring that solution. Meanwhile, OriginStamp exposed me to timestamping and trusted timestamping and I discovered freeTSA.org. What’s the point? These services provide authoritative proof of the existence of a digital asset before some point in time; OriginStamp provides a richer service and uses multiple timestamp authorities including Bitcoin, Ethereum and rather interestingly a German Newspaper’s Trusted Timestamp.

Posts

Secure (TLS) gRPC services with VKE

NOTE cert-manager is a better solution to what follows. I’ve a need to deploy a Vultr Kubernetes Engine (VKE) cluster on a daily basis (create and delete within a few hours) and expose (securely|TLS) a gRPC service. I have an existing solution Automatic Certs w/ Golang gRPC service on Compute Engine that combines a gRPC Healthchecking and an ACME service and decided to reuse this. In order for it work, we need:

Posts

Scraping metrics exposed by Google Cloud Run services that require authentication

I’ve written a solution (gcp-oidc-token-proxy) that can be used in conjunction with Prometheus OAuth2 to authenticate requests so that Prometheus can scrape metrics exposed by e.g. Cloud Run services that require authentication. The solution resulted from my question on Stack overflow. Problem #1: Endpoint requires authentication Given a Cloud Run service URL for which: ENDPOINT="my-server-blahblah-wl.a.run.app" # Returns 200 when authentication w/ an ID token TOKEN="$(gcloud auth print-identity-token)" curl \ --silent \ --request GET \ --header "Authorization: Bearer ${TOKEN}" \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics # Returns 403 otherwise curl \ --silent \ --request GET \ --write-out "%{response_code}" \ --output /dev/null \ https://${ENDPOINT}/metrics Problem #2: Prometheus OAuth2 configuration is constrained

Posts

Trendnet TEW-812DRU and DD-WRT

The FBI Portland published an interesting advisory with several, sensible recommendations including firewalling IoT devices from other devices on a home network. I decided to deploy a redundant Trendnet TEW-812DRU version 2.0 for this purpose. Caveat Developer: Before I go further, I don’t recommend installing DD-WRT on a Trendnet TEW-812DRU unless you’re willing to brick the device irrecoverably. I read the DD-WRT instructions several times (“peacock thread”,router database – do not use v3.

Posts

Trillian Map Mode

Chatting with one of Google’s Trillian team, I was encouraged to explore Trillian’s Map Mode. The following is the result of some spelunking through this unfamiliar cave. I can’t provide any guarantee that this usage is correct or sufficient. Here’s the repo: https://github.com/DazWilkin/go-trillian-map I’ve written about Trillian Log Mode elsewhere. I uncovered use of Trillian Map Mode through Trillian’s integration tests. I’m unclear on the distinction between TrillianMapClient and TrillianMapWriteClient but the latter served most of my needs.

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

Rust implementation of Crate Transparency using Google Trillian

I’ve been hacking on a Rust-based transparent application for Google Trillian. As appears to be my fixation, this personality is for another package manager. This time, Rust’s Crates often found in crates.io which is Rust’s Package Registry. I discussed this project earlier this month Rust Crate Transparency && Rust SDK for Google Trillian and and earlier approach for Python’s packages with pypi-transparency. This time, of course, I’m using Rust. And, by way of a first for me, for the gRPC server implementation (aka “personality”).

Posts

Rust Crate Transparency && Rust SDK for Google Trillian

I’m noodling the utility of a Transparency solution for Rust Crates. When developers push crates to Cargo, a bunch of metadata is associated with the crate. E.g. protobuf. As with Golang Modules, Python packages on PyPi etc., there appears to be utility in making tamperproof recordings of these publications. Then, other developers may confirm that a crate pulled from cates.io is highly unlikely to have been changed. On Linux, Cargo stores downloaded crates under ${HOME}/.

Posts

Google Trillian on Cloud Run

I’ve written previously (Google Trillian for Noobs) about Google’s interesting project Trillian and about some of the “personalities” (e.g. PyPi Transparency) that I’ve build using it. Having gone slight cra-cra on Cloud Run and gRPC this week with Golang gRPC Cloud Run and gRPC, Cloud Run & Endpoints, I thought it’d be fun to deploy Trillian and a personality to Cloud Run. It mostly (!) works :-) At the end of the post, I’ve summarized creating a Cloud SQL instance to host the Trillian data(base).

Posts

PyPi Transparency Client (Rust)

I’ve finally being able to hack my way through to a working Rust gRPC client (for PyPi Transparency). It’s not very good: poorly structured, hacky etc. but it serves the purpose of giving me a foothold into Rust development so that I can evolve it as I learn the language and its practices. There are several Rust crates (SDK) for gRPC. There’s no sanctioned SDK for Rust on grpc.io. I chose stepancheg’s grpc-rust because it’s a pure Rust implementation (not built atop the C implementation).

Posts

PyPi Transparency

I’ve been noodling around with another Trillian personality. Another in a theme that interests me in providing tamperproof logs for the packages in the popular package management registries. The Golang team recently announced Go Module Mirror which is built atop Trillian. It seems to me that all the package registries (Go Modules, npm, Maven, NuGet etc.) would benefit from tamperproof logs hosted by a trusted 3rd-party. As you may have guessed, PyPi Transparency is a log for PyPi packages.

Posts

pypi-transparency

The goal of pypi-transparency is very similar to the underlying motivation for the Golang team’s Checksum Database (also built with Trillian). Even though, PyPi provides hashes of the content of packages it hosts, the developer must trust that PyPi’s data is consistent. One ambition with pypi-transparency is to provide a companion, tamperproof log of PyPi package files in order to provide a double-check of these hashes. It is important to understand what this does (and does not) provide.

Posts

FreeTSA & Digitorus' Timestamp SDK

I wrote recently about some exploration of Timestamping with OriginStamp. Since writing that post, I had some supportive feedback from the helpful folks at OriginStamp and plan to continue exploring that solution. Meanwhile, OriginStamp exposed me to timestamping and trusted timestamping and I discovered freeTSA.org. What’s the point? These services provide authoritative proof of the existence of a digital asset before some point in time; OriginStamp provides a richer service and uses multiple timestamp authorities including Bitcoin, Ethereum and rather interestingly a German Newspaper’s Trusted Timestamp.

Posts

Minimizing WASM binaries

I’ve spent time recently playing around with WebAssembly (WASM) and waPC. Rust and WASM were born at Mozilla and there’s a natural affinity with writing WASM binaries in Rust. In the WASM examples I’ve been using for WASM Transparency, waPC and MsgPack and waPC and Protobufs. I’ve created 3 WASM binaries: complex.wasm, simplex.wasm and fabcar.wasm and each is about 2.5MB when: cargo build --target=wasm32-unknown-unknown --release The Rust and WebAssembly book has an excellent section titled Shrinking .

Posts

Prometheus VPA Recommendations

Phew! For Want of a Nail I was interested in learning how to Manage Resources for Containers. On the way, I learned and discovered: kubectl top Vertical Pod Autoscaler A (valuable) digression through PodMonitor kube-state-metrics `kubectl-patch Created a Graph References Kubernetes Resources Visual Studio Code has begun to bug me (reasonably) to add resources to Kubernetes manifests. E.g.: resources: limits: cpu: "1" memory: "512Mi" I’ve been spending time with Deislab’s Akri and decided to determine whether Akri’s primary resources (Agent, Controller) and some of my creations HTTP Device and Discovery, were being suitably constrained.

Posts

Visual Studio Code plus Google Cloud Shell

Update: 2020-09-24 Three updates since I wrote the post: gcloud alpha cloud-shell get-mount-command ${DIR} It’s possible to use sshfs to mount the Cloud Shell home directory locally: DIR=/path/to/dir gcloud alpha cloud-shell get-mount-command ${DIR} Which generates something of the form: sshfs [[USERNAME]]@[[HOST]]: ${DIR} \ -p [[PORT]] \ -oIdentityFile=~/.ssh/google_compute_engine \ -oStrictHostKeyChecking=no You may then code --new-window ${DIR} curl command may lack .sshHost curl’ing the cloudshell.googleapis.com endpoint will result in a null value for .

Posts

Secure (TLS) gRPC services with VKE

NOTE cert-manager is a better solution to what follows. I’ve a need to deploy a Vultr Kubernetes Engine (VKE) cluster on a daily basis (create and delete within a few hours) and expose (securely|TLS) a gRPC service. I have an existing solution Automatic Certs w/ Golang gRPC service on Compute Engine that combines a gRPC Healthchecking and an ACME service and decided to reuse this. In order for it work, we need:

Posts

Vultr CLI and JSON output

I’ve begun exploring Vultr after the company announced a managed Kubernetes offering Vultr Kubernetes Engine (VKE). In my brief experience, it’s a decent platform and its CLI vultr-cli is mostly (!) good. The CLI has a limitation in that command output is text formatted and this makes it challenging to parse the output when scripting. NOTE The Vultr developers have a branch rewrite that includes a solution to this problem.

Posts

Visual Studio Code: gopls and YAML

The Go team is developing a Language Server Protocol [LSP] implementation) called gopls. Visual Studio Code (and others) support LSP. Other languages (e.g. Python have LSP implementations too). I’ve been using gopls for some time. It works (mostly) very well and replaces multiple, indepedent tools with two (gopls and delve). My Visual Studio Code settings that include gopls is: "go.autocompleteUnimportedPackages": true, "go.useLanguageServer": true, "[go]": { "editor.snippetSuggestions": "none", "editor.

Posts

Secure (TLS) gRPC services with VKE

NOTE cert-manager is a better solution to what follows. I’ve a need to deploy a Vultr Kubernetes Engine (VKE) cluster on a daily basis (create and delete within a few hours) and expose (securely|TLS) a gRPC service. I have an existing solution Automatic Certs w/ Golang gRPC service on Compute Engine that combines a gRPC Healthchecking and an ACME service and decided to reuse this. In order for it work, we need:

Posts

Vultr CLI and JSON output

I’ve begun exploring Vultr after the company announced a managed Kubernetes offering Vultr Kubernetes Engine (VKE). In my brief experience, it’s a decent platform and its CLI vultr-cli is mostly (!) good. The CLI has a limitation in that command output is text formatted and this makes it challenging to parse the output when scripting. NOTE The Vultr developers have a branch rewrite that includes a solution to this problem.

Posts

Prometheus Exporters for fly.io and Vultr

I’ve been on a roll building utilities this week. I developed a Service Health dashboard for my “thing”, a Prometheus Exporter for Fly.io and today, a Prometheus Exporter for Vultr. This is motivated by the fear that I will forget a deployed Cloud resource and incur a horrible bill. I’ve no written several Prometheus Exporters for cloud platforms: Prometheus Exporter for GCP Prometheus Exporter for Linode Prometheus Exporter for Fly.

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

waPC and MsgPack (Rust|Golang)

As my reader will know (Hey Mom!), I’ve been noodling around with WASM and waPC. I’ve been exploring ways to pass structured messages across the host:guest boundary. Protobufs was my first choice. @KevinHoffman created waPC and waSCC and he explained to me and that wSCC uses Message Pack. It’s slightly surprising to me (still) that technologies like this exist with everyone else seemingly using them and I’ve not heard of them.

Posts

Remotely invoking WASM functions using gRPC and waPC

Following on from waPC & Protobufs, I can now remotely invoke (arbitrary) WASM functions: Client: The logging isn’t perfectly clear but, the client gets (a previously added) WASM binary from the server (using the SHA-256 of the WASM binary as a unique identifier). The result includes metadata that includes a protobuf descriptor of the WASM binary’s functions. The descriptor defines gRPC services (that represent the WASM functions) with input (parameters) and output (results) messages.

Posts

WASM Cloud Functions

Following on from waPC & Protobufs and a question on Stack Overflow about Cloud Functions, I was compelled to try running WASM on Cloud Functions no JavaScript. I wanted to reuse the WASM waPC functions that I’d written in Rust as described in the other post. Cloud Functions does not (yet!?) provide a Rust runtime and so I’m using the waPC Host for Go in this example. It works! PARAMS=$(printf '{"a":{"real":39,"imag":3},"b":{"real":39,"imag":3}}' | base64) TOKEN=$(gcloud auth print-identity-token) echo "{ \"filename\":\"complex.

Posts

waPC & Protobufs

I’m hacking around with a solution that combines WASM and Google Trillian. Ultimately, I’d like to be able to ship WASM (binaries) to a Trillian personality and then invoke (exported) functions on them. Some this was borne from the interesting exploration of Krustlet and its application of wascc. I’m still booting into WASM but it’s a very interesting technology that has most interesting potential outside the browser. Some folks have been trailblazing the technology and I have been reading Kevin Hoffman’s medium and wascc (nee waxosuit) work.

Posts

waPC & Protobufs

I’m hacking around with a solution that combines WASM and Google Trillian. Ultimately, I’d like to be able to ship WASM (binaries) to a Trillian personality and then invoke (exported) functions on them. Some this was borne from the interesting exploration of Krustlet and its application of wascc. I’m still booting into WASM but it’s a very interesting technology that has most interesting potential outside the browser. Some folks have been trailblazing the technology and I have been reading Kevin Hoffman’s medium and wascc (nee waxosuit) work.

Posts

Minimizing WASM binaries

I’ve spent time recently playing around with WebAssembly (WASM) and waPC. Rust and WASM were born at Mozilla and there’s a natural affinity with writing WASM binaries in Rust. In the WASM examples I’ve been using for WASM Transparency, waPC and MsgPack and waPC and Protobufs. I’ve created 3 WASM binaries: complex.wasm, simplex.wasm and fabcar.wasm and each is about 2.5MB when: cargo build --target=wasm32-unknown-unknown --release The Rust and WebAssembly book has an excellent section titled Shrinking .

Posts

WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution. Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented. 2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Message 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [random:New] Float32 2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.

Posts

waPC and MsgPack (Rust|Golang)

As my reader will know (Hey Mom!), I’ve been noodling around with WASM and waPC. I’ve been exploring ways to pass structured messages across the host:guest boundary. Protobufs was my first choice. @KevinHoffman created waPC and waSCC and he explained to me and that wSCC uses Message Pack. It’s slightly surprising to me (still) that technologies like this exist with everyone else seemingly using them and I’ve not heard of them.

Posts

Envoy WASM filters in Rust

A digression thanks to Sal Rashid who’s exploring WASM filters w/ Envoy. The documentation is sparse but: How to write WASM filters for Envoy… There is a Rust SDK but it’s not documented: proxy-wasm-rust-sdk I found two useful posts by Rustaceans who were able to make use of it: Extending Envoy with WASM and Rust Extending Istio with Rust and WebAssembly Here’s my simple use of the SDK’s examples.

Posts

Remotely invoking WASM functions using gRPC and waPC

Following on from waPC & Protobufs, I can now remotely invoke (arbitrary) WASM functions: Client: The logging isn’t perfectly clear but, the client gets (a previously added) WASM binary from the server (using the SHA-256 of the WASM binary as a unique identifier). The result includes metadata that includes a protobuf descriptor of the WASM binary’s functions. The descriptor defines gRPC services (that represent the WASM functions) with input (parameters) and output (results) messages.

Posts

WASM Cloud Functions

Following on from waPC & Protobufs and a question on Stack Overflow about Cloud Functions, I was compelled to try running WASM on Cloud Functions no JavaScript. I wanted to reuse the WASM waPC functions that I’d written in Rust as described in the other post. Cloud Functions does not (yet!?) provide a Rust runtime and so I’m using the waPC Host for Go in this example. It works! PARAMS=$(printf '{"a":{"real":39,"imag":3},"b":{"real":39,"imag":3}}' | base64) TOKEN=$(gcloud auth print-identity-token) echo "{ \"filename\":\"complex.

Posts

waPC & Protobufs

I’m hacking around with a solution that combines WASM and Google Trillian. Ultimately, I’d like to be able to ship WASM (binaries) to a Trillian personality and then invoke (exported) functions on them. Some this was borne from the interesting exploration of Krustlet and its application of wascc. I’m still booting into WASM but it’s a very interesting technology that has most interesting potential outside the browser. Some folks have been trailblazing the technology and I have been reading Kevin Hoffman’s medium and wascc (nee waxosuit) work.

Posts

Google Container Registry w/ OCI

I’ve been spending some time this week with Krustlet. I’m working on documenting how to run Krustlet(s) alongside GKE. I’ve been running a Krustlet with MicroK8s. The Krustlet demos reference WASM assemblines stored in Azure Container Registry as OCI containers. Google Container Registry supports OCI format and so I tried (successfully) using GCR instead of AZR. There may be an easier approach but this is how I got this working.

Posts

Adventures around BPF

I think (!?) this interesting learning experience started with Envoy Go Extensions. IIUC, Cilium contributed this mechanism (Envoy Go Extensions) by which it’s possible to extend Envoy using Golang. The documentation references BPF. Cilium and BNF were both unfamiliar technologies to me and so began my learning. This part of the journey concludes with Weave Scope. I learned that Cilium is a CNI implementation that may be used with Kubernetes. GKE defaults (but is not limited to) to Google’s own CNI implementation (link).

Posts

Adventures around BPF

I think (!?) this interesting learning experience started with Envoy Go Extensions. IIUC, Cilium contributed this mechanism (Envoy Go Extensions) by which it’s possible to extend Envoy using Golang. The documentation references BPF. Cilium and BNF were both unfamiliar technologies to me and so began my learning. This part of the journey concludes with Weave Scope. I learned that Cilium is a CNI implementation that may be used with Kubernetes. GKE defaults (but is not limited to) to Google’s own CNI implementation (link).

Posts

Using Google Monitoring Alerting to send Pushover notifications

Table of Contents Artifacts Pushover Caveat Cloud Monitoring Webhook Cloud Functions Cloud Run Artifacts GitHub: go-gcp-pushover-notificationchannel Image: ghcr.io/dazwilkin/go-gcp-pushover-notificationchannel:220515 Pushover Logging in to your Pushover account, you will be presented with a summary|dashboard page that includes Your User Key. Copy the value of this key into a variable called PUSHOVER_USER Create New Application|API Token Pushover API has a Pushing Messages method. The documentation describes the format of the HTTP Request.

Posts

Kubernetes cert-manager

I developed an admission webhook for Akri, twice (Golang, Rust). I naively followed other examples for the generation of the certificates, created a 1.20 cluster and broke that process. I’d briefly considered using cert-manager recently but quickly abandoned the idea thinking it would be onerous and unnecessary complexity for little-old-me. I was wrong. It’s excellent and I recommend it highly. I won’t reproduce the v1beta1 and v1 examples from the Stackoverflow question as they should be self-explanatory.

Posts

Kubernetes Webhooks

I spent some time last week writing my first admission webhook for Kubernetes. I wrote the handler in Golang because I’m most familiar with Golang and because, as Kubernetes’ native language, I was more confident that the necessary SDKs would exist and that the documentation would likely use Golang by default. I struggled to find useful documentation and so this post is to help you (and me!) remember how to do this next time!

Posts

webmention

Let’s see if this works!? I’ve added the following link to this site’s baseof.html so that it is now rendered for each page: <link href="https://us-central1-webmention.cloudfunctions.net/webmention" rel="webmention" /> I discovered indieweb yesterday reading about webmention. Who knows what got me to webmention!? The principles of both are sound. Instead of relying on come-go behemoths to run our digital world, indieweb seeks to provide technologies that enable us to achieve things without them.

Posts

Golang Xiaomi Bluetooth Temperature|Humidity (LYWSD03MMC) 2nd Gen

Well, this became more of an adventure that I’d originally wanted but, after learning some BLE and, with the help of others (Thanks Jonatha, JsBergbau), I’ve sample code that connects to 4 Xiaomi 2nd gen. Thermometers, subscribes to readings and publishes the data to MQTT. From there, I’m scraping it using Inuits MQTTGateway into Prometheus. Repo: https://github.com/DazWilkin/gomijia2 Thanks|Credit: Jonathan McDowell for gomijia and help JsBergbau for help Background I’ve been playing around with ESPHome and blogged around my very positive experience ESPHome, MQTT, Prometheus and almost Cloud IoT.

Posts

`gcloud beta run services replace`

TL;DR I’m working on a project that includes multiple Cloud Run services. I’ve been putting my gcloud head on to deploy these services thinking that it’s curious there’s no way to write the specs as YAML configs. Today, I learned that there is: gcloud beta run services replace. What prompted the discovery was some frustration trying to deploy a JSON-valued environment variable to Cloud Run: local FIREBASE_CONFIG="{ apiKey: ${FIREBASE_API_KEY}, authDomain: ${FIREBASE_AUTH_DOMAIN}, projectId: ${FIREBASE_PROJECT}, storageBucket: ${FIREBASE_STORAGE_BUCKET}, messagingSenderId: ${FIREBASE_MESSAGING_SENDER}, appId: ${FIREBASE_APP}}" gcloud run deploy ${SRV_NAME} \ --image=${IMAGE} \ --command="/server" \ --args="--endpoint=:${PORT}" \ --set-env-vars=FIREBASE_CONFIG="${FIREBASE_CONFIG}" \ --max-instances=1 \ --memory=256Mi \ --ingress=all \ --platform=managed \ --port=${PORT} \ --allow-unauthenticated \ --region=${REGION} \ --project=${PROJECT} gcloud balks at this.

Posts

pest: parsing in Rust

A Microsoft engineer introduced me to pest as a way to introduce service filtering in a ZeroConf plugin that I’m prototyping for Akri. It’s been fun to learn but I worry that, because I won’t use it frequently, I’m going to quickly forget what I’ve done. So, here are my notes. Here’s the problem, I’d like to be able to provide users of the ZeroConf plugin with a string-based filter that permits them to filter the services discovered when the Akri agent browses a network.

Posts

Golang Structured Logging w/ Google Cloud Logging

I’ve multiple components in an app and these are deployed across multiple Google Cloud Platform (GCP) services: Kubernetes Engine, Cloud Functions, Cloud Run, etc. Almost everything is written in Golang and I started the project using go-logr. logr is in two parts: a Logger that you use to write log entries; a LogSink (adaptor) that consumes log entries and outputs them to a specific log implementation. Initially, I defaulted to using stdr which is a LogSink for Go’s standard logging implementation.