Remotely invoking WASM functions using gRPC and waPC

Following on from waPC & Protobufs, I can now remotely invoke (arbitrary) WASM functions:

Client:

The logging isn’t perfectly clear but, the client gets (a previously added) WASM binary from the server (using the SHA-256 of the WASM binary as a unique identifier). The result includes metadata that includes a protobuf descriptor of the WASM binary’s functions. The descriptor defines gRPC services (that represent the WASM functions) with input (parameters) and output (results) messages.

The complex.proto protobuf was described in the previous post but should be self-explanatory. For the mul (multiplication) method, we need 2 complex numbers (protobuf ComplexRequest includes fields a and b which are both Complex). The client generates random messages corresponding to ComplexRequest (itself comprising Complex). This explains why the log shows random:New for 2 messages each of which contains 2 float32s.

The client marshals the generated dynamicpb.Message (corresponding to the ComplexRequest) into anypb.Any (effectively bytes) as a way to envelope arbitrary message types, and ships this off to the server.

I should add better logging to output the randomly generated message but, the result of the multiplication is a ComplexResponse ({real:-0.22... imag:0.26..}).

[main:loop:dynamic-invoke] Method: mul
[random:New] Message
[random:New] Float32
[random:New] Float32
[random:New] Message
[random:New] Float32
[random:New] Float32
[Client:Invoke] Metadata: complex.wasm
[main:loop:dynamic-invoke] Success: result:{real:-0.22081086  imag:0.26051167}

NOTE a partial implementation for generating random protobuf message is in go-protobuf-extras

Server:

A client must have previously added complex.wasm to the server along with a protobuf descriptor defining the input (parameters) and output (result). When the server receives the invoke request, it determines which WASM binary (in this case complex.wasm) is required and it loads it from storage if it has not already cached an instance of it. It then uses waPC to invoke the mul function on the binary passing it the request bytes returning a result. The server ships the bytes back to the client.

Because the server associates protobuf descriptors with the WASM binaries, it could deal with the request|response bytes as concrete messages but this would require the client to have concrete implementations of these types too. This may be useful in practice but, for testing, a generic client is able to invoke arbitrary WASM functions because it can use dynamicpb to create arbitrary messages using the descriptors.

[Server:Invoke] Metadata: complex.wasm
[File:Open] Module: complex.wasm
[File:Filename] /wasm-cache/complex.wasm
[hash:Tee] Entered
[hash:Tee:go] 608c40aee5bda76aa6f43e55ea237a5b69e72536bdd33fb3e99b4f41fba63983
[Server:Invoke] Cacheing Instance: complex.wasm
[wapc:Function:Invoke] Entered
[Server:Invoke] Metadata: complex.wasm
[wapc:callback] binding: b namespace: complex::any operation: mul payload: 0.13414463+0.58088475i, 0.3424287+0.45920613i
[wapc:callback] binding: b namespace: complex::any operation: mul payload: 0.13414463+0.58088475i, 0.3424287+0.45920613i
[wapc:callback] binding: b namespace: complex::any operation: mul payload: -0.22081086+0.26051167i
[wapc:callback] binding: b namespace: complex::any operation: mul payload: -0.22081086+0.26051167i

Motivation

I’m exploring a solution in which arbitrary (gRPC-based) clients can ship WASM binaries and then remotely invoke functions in the binaries. The concept is similar to a simple chaincode. In order to define the input (parameters) and output (results) of the WASM functions, these are defined using protobufs.

Implementation

I’m using Rust and waPC to implement the WASM functions. waPC provides a mechanism by which arbitrary binary blobs may be transferred between the host runtime and the guest (WASM) runtime. When testing these implementations, I’ve been using Rust and waPC for the host runtime too. But, for this implementation, I’ve been using Go(lang) as the waPC host runtime. This is partly because of the well-documented Golang Protobuf APIv2 that I wrote about recently. The waPC Host for Go repo is here. The API is straightforward and, I’ve implemented this as an implementation of 3 interfaces in my solution:

type Runtime interface {
	Create([]byte) (Instance, error)
	Delete() error
}

type Instance interface {
	Delete() error
	Function(name string) (Function, error)
}
type Function interface {
	Invoke(*anypb.Any) (*anypb.Any, error)
}

For example, for complex.wasm, we must Create an Instance from the WASM binary bytes. With an Instance, we may then grab a reference to a specific Function (matching the WASM binary’s exported functions which also correspond to the protobuf Service names) and then, we can Invoke the function with the bytes corresponding to the marshaled protobuf Input message which results in bytes corresponding to the marshaled protobuf Output message.

Currently (!) the server uses the file system to persist the WASM binaries that have been Add(ed) to it.

The plan is to use Google Trillian as an alternative storage mechanism. I will describe this in my next post. Trillian will provide tamperproof storage for the WASM binaries.

So, for a Server that has the following type:

type Server struct {
	host wasm.Runtime
	repo wasm.Repository

	// Instances cache
	instance map[string]wasm.Instance
}

The Invoke method is simply:

func (c *Server) Invoke(ctx context.Context, r *pb.InvokeRequest) (*pb.InvokeResponse, error) {
	meta := r.GetMetadata()
	id := meta.GetId()
	log.Printf("[Server:Invoke] Metadata: %s", id)

	rdr, err := c.repo.Open(wasm.ToMetadata(meta))
	if err != nil {
		return &pb.InvokeResponse{}, err
	}

	// Cache WASM instances
	if c.instance[id] == nil {
		b, err := ioutil.ReadAll(rdr)
		if err != nil {
			return &pb.InvokeResponse{}, nil
		}

		log.Printf("[Server:Invoke] Cacheing Instance: %s", id)
		c.instance[id], err = c.host.Create(b)
		if err != nil {
			return &pb.InvokeResponse{}, err
		}
	}

	name := r.GetRequest().GetName()
	if name == "" {
		err := fmt.Errorf("WASM function name cannot be null")
		return &pb.InvokeResponse{}, err
	}

	// Invoke Function on cached Instance
	f, err := c.instance[id].Function(name)
	if err != nil {
		return &pb.InvokeResponse{}, err
	}

	params := r.GetRequest().GetParameters()
	result, err := f.Invoke(params)
	if err != nil {
		return &pb.InvokeResponse{}, err
	}

	return &pb.InvokeResponse{
		Response: &pb.FunctionResponse{
			Result: result,
		},
	}, nil
}

And, the waPC runtime is implemented around a type:

type Runtime struct {
	m *wapc.Module
}

That has an Invoke method:

func (f Function) Invoke(any *anypb.Any) (*anypb.Any, error) {
	log.Printf("[wapc:Function:Invoke] Entered")
	ctx := context.Background()
	b, err := f.i.Invoke(ctx, f.op, any.GetValue())
	if err != nil {
		return &anypb.Any{}, err
	}
	return &anypb.Any{
		TypeUrl: "",
		Value:   b,
	}, nil
}

And this, of course, calls the waPC Invoke function that does the heavy-lifting in invoking the call on the guest’s WASM function and returning the result.

• Wasm
• Protobufs
• Wapc
• GRPC