Struggling with Golang structs

May 27, 2021 - 5 minutes read - 884 words

Julia’s post Blog about what you’ve struggled with resonates because I’ve been struggling with Golang structs in a project. Not the definitions of structs but seemingly needing to reproduce them across the project. I realize that each instance of these resources differs from the others but I’m particularly concerned by having to duplicate method implementations on them.

I’m kinda hoping that I see the solution to my problem by writing it out. If you’re reading this, I didn’t :-(

The original Customer type that I wrote for the Kubernetes Operator is definitive:

type CustomerSpec struct {
	Shard   uint8       `json:"shard" description:"used to partition customers across namespaces"`
	Name    string      `json:"name" description:"uniquely identifying name"`
	Domain  string      `json:"domain" description:"primary domain"`
	Enabled bool        `json:"enabled" description:"Whether Customer is enabled for use"`
	Billing bool        `json:"billing" description:"Whether Customer is enabled for billing"`
	Created metav1.Time `json:"created" description:"created time"`
	Updated metav1.Time `json:"updated,omitempty" description:"updated time"`
}

// ID is a method
func (s *CustomerSpec) ID() string {
	x := sha256.Sum224([]byte(s.Domain))
	return fmt.Sprintf("%x", x[:])
}

NOTE A minor point that Created and Updated must be metav1.Time but they’re time.Time with Firestore.

I need to be able to represent this object in Firestore too:

type Customer struct {
	Shard   uint8                  `firestore:"shard"`
	Name    string                 `firestore:"name" description:"Customer name"`
	Domain  *firestore.DocumentRef `firestore:"domain" description:"Customer primary domain name"`
	Enabled bool                   `firestore:"enabled" description:"Is Customer enabled?"`
	Billing bool                   `firestore:"billing" description:"Is Customer's billing enabled?"`
	Created time.Time              `firestore:"created,serverTimestamp" description:"Datetime when the Customer was created"`
	Updated time.Time              `firestore:"updated,omitempty" description:"Datetime when the Customer was updated"`
}

// ID is a function
func (c *Customer) ID(domain string) string {
	x := sha256.Sum224([]byte(domain))
	return fmt.Sprintf("%x", x[:])
}

NOTE In this case, ID is a function not a method and requires the domain (name) string.

The Domain field is a Firestore DocumentRef this time because I want to capture the fact that a domain name is unique and owned uniquely. In Firestore, it is represented by a reference to another Firestore document (record) and this record’s unique identifier is the SHA224 hash of the name.

When a Customer is added to Firestore, I need to be able to trigger the creation of the Kubernetes object as defined in the Operator. But, the Cloud Functions trigger needs a very distinct type:

type CustomerFields struct {
	Shard struct {
		IntegerValue string `json:"integerValue"`
	} `json:"shard"`
	Name struct {
		StringValue string `json:"stringValue"`
	} `json:"name"`
	Domain struct {
		ReferenceValue string `json:"referenceValue"`
	} `json:"domain"`
	Enabled struct {
		BooleanValue bool `json:"booleanValue"`
	} `json:"enabled"`
	Billing struct {
		BooleanValue bool `json:"booleanValue"`
	} `json:"billing"`
	Created struct {
		TimestampValue string `json:"timestampValue"`
	} `json:"created"`
	Updated struct {
		TimestampValue string `json:"timestampValue,omitempty"`
	} `json:"updated"`
}

In the trigger code, once I’ve created a variable of type CustomerFields, I’ve a method func (f *CustomerFields) Customer(*operator.Customer) to convert it back into the (definitive) Operator type.

Communication between the customer (CLI) and the services uses gRPC:

message Customer {
    string Name = 1;
    string DomainName = 2;
    bool Enabled = 3;
    bool Billing = 4;
}

NOTE The Protobuf encouraged me to think more clearly about names and, here DomainName is used to disambiguate from Domain which is a message representing domains. I should have used DomainName consistently.

Generating:

type Customer struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	Name       string `protobuf:"bytes,1,opt,name=Name,proto3" json:"Name,omitempty"`
	DomainName string `protobuf:"bytes,2,opt,name=DomainName,proto3" json:"DomainName,omitempty"`
	Enabled    bool   `protobuf:"varint,3,opt,name=Enabled,proto3" json:"Enabled,omitempty"`
	Billing    bool   `protobuf:"varint,4,opt,name=Billing,proto3" json:"Billing,omitempty"`
}

NOTE The Protobuf is not generating snake_case names. There is possibly an flag that I can set on protoc to change this. I’m not currently using the generated code for JSON objects. Protobuf: JSON Mapping

In the CLI, I want to be able to limit the customer’s exposure to the underlying type:

type Customer struct {
	Name       string `json:"name" yaml:"name" description:"Customer name"`
	DomainName string `json:"domain_name" yaml:"domain_name" description:"Customer primary domain name"`
	Enabled    bool   `json:"enabled" yaml:"enabled" description:"Is Customer enabled?"`
}

func (c *Customer) ID() string {
	x := sha256.Sum224([]byte(c.DomainName))
	return fmt.Sprintf("%x", x[:])
}

So, the flow becomes:

Customer issues a command-line representation of the Customer
A CLI Customer type is created
Converted to Protobuf Customer and shipped to the server
The server converts it into a Firestore Customer to operate on it
This probably includes updating Firestore using it

When a Customer object is requested, the above process is inverted starting with a Firestore Customer being created.

Any changes to Customer documents in Firestore, trigger Cloud Functions

A CustomerFields type is created
It is converted into a Kubernetes Customer
It is applied to a Kubernetes cluster

It is possible, as is shown in the generated protobuf code, to have multiple attributes (e.g. json, protobuf, firestore) in a field’s tag

So I’m wondering whether what I should do is start with the minimal Customer (CLI) and embed my way back up?

But, will each of the technologies I’m using accommodate this?

type ExternalCustomer struct {
	Name       string `firestore:"name" json:"name" yaml:"name" description:"Customer name"`
	DomainName string `firestore:"domain_name" json:"domain_name" yaml:"domain_name" description:"Customer primary domain name"`
	Enabled    bool   `firestore: "enabled" json:"enabled" yaml:"enabled" description:"Is Customer enabled?"`
}

type InternalCustomer struct {
    ExternalCustomer

	Shard   uint8 `firestore:"shard" json:"shard"`
	Billing bool  `firestore:"billing" json:"billing" description:"Is Customer's billing enabled?"`
}

type FirestoreCustomer struct {
    InternalCustomer

	DomainRef  *firestore.DocumentRef `firestore:"domain" json:"domain" description:"Customer primary domain name"`
	Created time.Time                 `firestore:"created,serverTimestamp" json:"created,omitempty" description:"Datetime when the Customer was created"`
	Updated time.Time                 `firestore:"updated,omitempty" json:"updated,omitempty" description:"Datetime when the Customer was updated"`
}

type CustomerSpec struct {
    InternalCustomer 

	Created metav1.Time `json:"created" description:"created time"`
	Updated metav1.Time `json:"updated,omitempty" description:"updated time"`
}

Are there best practices for this?

Golang