I’ve written before about Ackal’s use of Firestore and subscribing to Firestore document CRUD events:

• Routing Firestore events to GKE with Eventarc
• Cloud Firestore Triggers in Golang using Firestore triggers

I find Google’s Eventarc documentation to be confusing and, in typical Google fashion, even though open-sourced, you often need to do some legwork to find relevant sources, viz:

• Google’s Protobufs for Eventarc (using cloudevents) google-cloudevents1
• Convenience (since you can generate these using protoc) language-specific types generated from the above e.g. google-cloudevents-go; google-cloudevents-python etc.

1 – IIUC EventArc is the Google service. It carries Google Events that are CloudEvents. These are defined by protocol buffers schemas.

Google announced Firestore … integration with Eventarc. Ackal uses Firestore to persist Customer and Check information and it uses Google Cloud Firestore Triggers to handle events on these document types.

Eventarc feels like the strategic future of eventing in Google Cloud and I’ve been concerned since adopting the technology that Google would abandon Google Cloud Firestore Triggers.

For this reason, when I saw last week’s announcement, I thought I should evaluate the mechanism and this blog post is a summary of that work.

While testing Firestore export and import yesterday and checking the IAM permissions on a Cloud Storage Bucket, I noticed some Member (member) values (I think Google refers to these as Principals) were logical but unfamiliar to me:

deleted:serviceAccount:{email}?uid={uid}

I was using gsutil iam get gs://${BUCKET} because I’d realized (and this is another useful lesson) that, as I’ve been creating daily test projects, I’ve been binding each project’s Firestore Service Account (service-{project-number}@gcp-sa-firestore.iam.gserviceaccount.com) to a Bucket owned by another Project but I hadn’t been deleting the binding when I deleted the Project.

For my “thing”, I use Firestore to persist state. I like Firestore a lot and, having been around Google for almost (!) a decade, I much prefer it to Datastore.

Firestore has a managed export|import service and I use this to backup Firestore collections|documents.

I’d been doing backups manually (using gcloud) and decided today to take the plunge and use Cloud Scheduler for the first time. I’d been reluctant to do this until now because I’d assumed incorrectly that I’d need to write a wrapping service to invoke the export.

I’m using Google’s Golang SDK for Firestore. The experience is excellent and I’m quickly becoming a fan of Firestore. However, as a Golang Firestore developer, I’m feeling less loved and some of the concepts in the database were causing me a conundrum.

I’m still not entirely certain that I have Timestamps nailed but… I learned an important lesson on the auto-creation of Timestamps in documents and how to retain these values.

I was pleased to discover that Google provides a non-Node.JS mechanism to subscribe to and act upon Firestore triggers, Google Cloud Firestore Triggers. I’ve nothing against Node.JS but, for the project i’m developing, everything else is written in Golang. It’s good to keep it all in one language.

I’m perplexed that Cloud Functions still (!) only supports Go 1.13 (03-Sep-2019). Even Go 1.14 (25-Feb-2020) was released pre-pandemic and we’re now running on 1.16. Come on Google!

This works great but it wasn’t clearly documented for non-Firebase users. I assume it will work, as well, for any of the client libraries (not just Golang).

Assuming you have some (Golang) code (in this case using the Google Cloud Client Library) that interacts with a Firestore database. Something of the form:

package main

import (
  "context"
  "crypto/sha256"
  "fmt"
  "log"
  "os"
  "time"

  "cloud.google.com/go/firestore"
)

func hash(s string) string {
  h := sha256.New()
  h.Write([]byte(s))
  return fmt.Sprintf("%x", h.Sum(nil))
}

type Dog struct {
  Name    string                 `firestore:"name"`
  Age     int                    `firestore:"age"`
  Human   *firestore.DocumentRef `firestore:"human"`
  Created time.Time              `firestore:"created"`
}

func NewDog(name string, age int, human *firestore.DocumentRef) Dog {
  return Dog{
    Name:    name,
    Age:     age,
    Human:   human,
    Created: time.Now(),
  }
}

func (d *Dog) ID() string {
  return hash(d.Name)
}

type Human struct {
  Name string `firestore:"name"`
}

func (h *Human) ID() string {
  return hash(h.Name)
}

func main() {
  ctx := context.Background()
  project := os.Getenv("PROJECT")

  client, err := firestore.NewClient(ctx, project)

  if value := os.Getenv("FIRESTORE_EMULATOR_HOST"); value != "" {
    log.Printf("Using Firestore Emulator: %s", value)
  }

  if err != nil {
    log.Fatal(err)
  }
  defer client.Close()

  me := Human{
    Name: "me",
  }
  meDocRef := client.Collection("humans").Doc(me.ID())
  if _, err := meDocRef.Set(ctx, me); err != nil {
    log.Fatal(err)
  }

  freddie := NewDog("Freddie", 2, meDocRef)
  freddieDocRef := client.Collection("dogs").Doc(freddie.ID())
  if _, err := freddieDocRef.Set(ctx, freddie); err != nil {
    log.Fatal(err)
  }
}

Then you can interact instead with the Firestore Emulator.