WASM Transparency

I’ve been playing around with a proof-of-concept combining WASM and Trillian. The hypothesis was to explore using WASM as a form of chaincode with Trillian. The project works but it’s far from being a chaincode-like solution.

Let’s start with a couple of (trivial) examples and then I’ll explain what’s going on and how it’s implemented.

2020/08/14 18:42:17 [main:loop:dynamic-invoke] Method: mul
2020/08/14 18:42:17 [random:New] Message
2020/08/14 18:42:17 [random:New] Float32
2020/08/14 18:42:17 [random:New] Float32
2020/08/14 18:42:17 [random:New] Message
2020/08/14 18:42:17 [random:New] Float32
2020/08/14 18:42:17 [random:New] Float32
2020/08/14 18:42:17 [Client:Invoke] Metadata: complex.wasm
2020/08/14 18:42:17 [main:loop:dynamic-invoke] Success: result:{real:0.036980484 imag:0.3898267}

After shipping a Rust-sourced WASM solution (complex.wasm) to the WASM transparency server, the client invokes a method mul that’s exposed by it using a dynamically generated request message and outputs the response. Woo hoo! Yes, an expensive way to multiple complex numbers.

Also:

2020/08/14 18:42:28 [main:loop:dynamic-invoke] Method: create
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [random:New] Message
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [Client:Invoke] Metadata: fabcarx.wasm
2020/08/14 18:42:28 [main:loop:dynamic-invoke] Success: car:{manufacturer:"X"  model:"X"  color:"X"  owner:"X"}
2020/08/14 18:42:28 [main:loop:dynamic-invoke] Method: query
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [Client:Invoke] Metadata: fabcarx.wasm
2020/08/14 18:42:28 [main:loop:dynamic-invoke] Success: car:{manufacturer:"X"  model:"X"  color:"X"  owner:"X"}
2020/08/14 18:42:28 [main:loop:dynamic-invoke] Method: change_owner
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [random:New] String
2020/08/14 18:42:28 [Client:Invoke] Metadata: fabcarx.wasm
2020/08/14 18:42:28 [main:loop:dynamic-invoke] Success: car:{manufacturer:"X"  model:"X"  color:"X"  owner:"X"}
2020/08/14 18:42:28 [main:loop:dynamic-invoke] Complete

NOTE I need to improve the randon message generator for strings so that the value is not always X; change_owner does work (issue)

Way back when I first began exploring blockchain technologies, I played around with Hyperledger. It has a prototypical example called fabcar and the above shows a WASM Transparency version of this app. After the fabcarx.wasm binary is shipped to the WAM Transparency server, the client invokes create, query and then change_owner methods exposed by it.

So, why is any of this complexity of any value?

The WASM binaries use waPC to transfer arbitrary data between the Transparency server’s Golang WASM host and the Rust WASM guests. In order to provide structure to this data, the solution uses protocol buffers. Specifically, starting with a protobuf definition that includes one of more service definitions, the Rust-based WASM code implements the protobuf services. A protobuf Descriptor file is generated from the protobuf sources too. The Descriptor (as metadata) is associated with the WASM binary. Descriptors are machine-readable variants of protobuf files and permit the Golang host to dynamically create clients.

A client uploads the WASM binary and its descriptor file to the Transparency server. The client (and server) computes RFC6962 hashes of the binary. The client must provide the server with this hash to get or invoke the binary. The client uses the returned Descriptor to enumerate the services exposed by the WASM binary and create request messages for these services. The client invokes services using the request message and uses the Descriptor to unmarshal the results.

The client envelopes these arbitrary requests and responses into protobuf’s well-known type Any. This permits a specific RPC to represent arbitrary WASM function request|response types. Conveniently Any is the combination of a unique type URL and a slice of bytes. waPC uses slices of bytes as its encoding for data between WASM host and WASM guests.

The full flow to invoke a function is thus:

1. Client initiates connection with the Transparency server.
2. Client gets metadata for a (previously added) WASM binary.
3. Client parses Descriptor, identifies a service and determines the service’s input and output message descriptors.
4. Client dynamically constructs a request message (using a random message generator in this case)
5. Client marshals the message into Any ([]byte) and sends, with the function name and WASM binary name to the server.
6. Server determines whether it has the WASM binary (using its hash)
7. Server unmarshals the Any into the service’s input type using the associated Descriptor.
8. Server uses waPC to invoke the designated WASM function with the request parameters and receives the result
9. Server envelopes the Any result into a response message that it returns to the Client
10. Client uses Descriptor to unmarshal the Any into a dynamically constructed message and outputs it

Let’s walk through these components with some code snippets.

Protobuf Descriptor (Sets)

There’s not (!?) much by way of documentation explaining protobuf descriptor (set) files. See the protoc command below which shows how to generate a descriptor file for the protobuf examples used in this solution. Descriptor files are binary equivalents of the human-readable *.proto (protobuf) sources. Google’s Golang protobuf APIv2 includes various packages that manipulate Descriptors and can create dynamic messages using these, e.g. protoreflect and dynamicpb

MODULE="github.com/DazWilkin/go-wasm-transparency"
protoc \
--proto_path=./examples \
--descriptor_set_out=./examples/descriptor.pb \
--go_out=plugins=grpc,module=${MODULE}:. \
./examples/*.proto

The descriptor file (as in this case) represents multiple *.proto files. Each protobuf file represents multiple services and can contain multiple message types etc. One service though binds an input message type with an output message type. As a result, the following uniquely (and sufficiently) define a WASM function for it be invoked:

NOTE in this prototype, the input message is randomly-generated

There are 2 levels of Protobuf serivces|messages here. The primary level is the WASM Transparency Service’s protobufs, e.g.:

package wasm_transparency.v1;

service WASMTransparency {
    ...
    rpc Get(GetRequest) returns (GetResponse) {};
    ...
}
message GetRequest {
    Metadata metadata = 1;
}
message GetResponse {
    bytes content = 1;
    string error = 2;
}

This level defines the messages used by the client to invoke methods on the server.

However, because the server must be capable of invoking arbitrary WASM functions and because these WASM functions are defined by protobufs and because protobufs does not support generic message types, the solution envelopes this second level of protobufs using Any as described by the previous reference.

NOTE Protobuf Any combine a type and a value ([]byte) and conveniently waPC uses []byte as the data type between hosts and guests.

So, for example complex.proto defines the 4 functions exported by complex.wasm:

package examples;

service ComplexService {
    rpc add(ComplexRequest) returns (ComplexResponse);
    rpc sub(ComplexRequest) returns (ComplexResponse);
    rpc mul(ComplexRequest) returns (ComplexResponse);
    rpc div(ComplexRequest) returns (ComplexResponse);
}

message ComplexRequest {
    Complex a = 1;
    Complex b = 2;
}

message ComplexResponse {
    Complex result = 1;
    string error = 2;
}

message Complex {
    float real = 1;
    float imag = 2;
}

Messages sent by the Client would be e.g. of type GetRequest containing the metadata described in the table above that’s sufficient to invoke a function. In practice, this would include a Descriptor representing complex.proto and an instance of ComplexRequest (as Any) that provides the request parameters.

Randomly-generated Messages

By way of explaining how the Client is able to generate messages to be used to invoke the WASM functions, let’s detour through go-protobuf-extras because it generates random protobuf messages from Descriptors.

It does this use a single, straightforward function with an extensive switch statement:

func Message(md protoreflect.MessageDescriptor) *dynamicpb.Message {
	m := dynamicpb.NewMessage(md)
	fds := md.Fields()
	// Iterate over *all* fields
	for k := 0; k < fds.Len(); k++ {
		fd := fds.Get(k)
		v := func() protoreflect.Value {
			var result protoreflect.Value
			switch fd.Kind() {
			case protoreflect.Int32Kind:
				log.Println("[random:New] Int32")
				result = protoreflect.ValueOfInt32(r.Int31())
			case protoreflect.FloatKind:
				log.Println("[random:New] Float32")
				result = protoreflect.ValueOfFloat32(r.Float32())
			case protoreflect.MessageKind:
				log.Println("[random:New] Message")
				md := fd.Message()
				result = protoreflect.ValueOfMessage(Message(md))
			case protoreflect.StringKind:
				log.Println("[random:New] String")
				result = protoreflect.ValueOfString("X")
			default:
				log.Fatal("[random:New] unanticipated kind")
			}
			return result
		}()
		m.Set(fd, v)
	}
	return m
}

NOTE this function is incomplete and does not (yet) cover all possible kinds.

Client

The function uses the Golang APIv2 dynamicpb package to create an empty protobuf message of the desired type (protoreflect.MessageDescriptor). For each of the Fields in the MessageDescriptor the function simply determines the type and creates a random value of that type. As you’ll see, MessageDescriptor is determined by parsing the protobuf Descriptor file.

The client creates a FileDescriptorSet and unmarshals the Descriptor Set file’s contents into it. It then creates a protodesc files registry that enables FindDescriptorByName. In this case, it uses a specific service descriptor i.e. examples.ComplexServer (the full name is scoped by the package name). It then iterates over all the methods (add, sub, mul and div) that are defined within the service. For a specific method e.g. mul defined to be rpc mul(ComplexRequest) returns (ComplexResponse), it grabs the input (ComplexRequest) message descriptor and creates a new dynamic, random message using it.

In order to invoke the WASM function on the server, the e.g. ComplexRequest is marshaled into an Any and this is used as the Parameters value in the InvokeRequest. Not shown here is the reverse process given the InvokeResponse.

// Load and parse the descriptor file
egDescriptorSet := &descriptorpb.FileDescriptorSet{}
if err := proto.Unmarshal(e.Metadata().DescriptorFile.Content, egDescriptorSet); err != nil {
	log.Fatal(err)
}

// Create a files registry
files, _ := protodesc.NewFiles(egDescriptorSet)

// Lookup the service
fullname := m.DescriptorFile.FullServiceName()
d, _ := files.FindDescriptorByName(protoreflect.FullName(fullname))

// Assert it into a ServiceDescriptor
s, ok := d.(protoreflect.ServiceDescriptor)

// Use the ServiceDescriptor to lookup the methods
mds := s.Methods()
for j := 0; j < mds.Len(); j++ {
	// Get this MethodDescriptor
	md := mds.Get(j)

	// Get the MessageDescriptor for the service's method's input
	i := md.Input()

	// New-up an random params message
	params := random.Message(i)

	b, err := proto.Marshal(params)
	if err == nil {
		any := &anypb.Any{
			TypeUrl: "",
			Value:   b,
		}
		rqst := &pb.InvokeRequest{
			Metadata: wasm.FromMetadata(m),
			Request: &pb.FunctionRequest{
				Name:       string(md.Name()),
				Parameters: any,
			},
		}
	}
}

Server

The server comprises 3 components:

1. gRPC server
2. Trillian personality
3. waPC host and guests

The gRPC server implementation is straightforward. It implements Add, Get and Invoke methods as defined in wasm_transparency.proto. These respectively, add a WASM binary and metadata to the server, get an existing WASM binary by its metadata and, most interestingly, permit clients to remotely invoke WASM functions.

I’ve discussed using Trillian elsewhere. Suffice to say that, this personality is very similar to the others with the exception that the client uses RFC6962 hashing (rather than pure SHA-256) in order to identify a specific WASM binary. See the spec’s section 2.1 for more details on hashing leaves. Add adds the WASM binary to Trillian, Get and Invoke look up existing binaries.

Golang waPC Host

When a client seeks to invoke a WASM binary’s function, the server retrieves the binary from Trillian and then creates a waPC host instance of it. In order to expedite invocations, if the server’s waPC host has previously invoked a function on the binary, there’s no need to retrieve the binary from Trillian again. Instead the server reuses an existing copy of the instance.

The client submitted the request parameters enveloped within the InvokeRequest as part of the FunctionRequest. This parameters are already marshaled into Any ([]byte).

syntax = "proto3";

package wasm_transparency.v1;

message FunctionRequest {
    string name = 1;
    google.protobuf.Any parameters = 3;
}
message FunctionResponse {
    google.protobuf.Any result = 1;
    string error = 2;
}

And so the actual Invoke function is straightforward, given the Any parameters, invoke the specific operation (e.g. mul) exported by the WASM function and return the result to the client:

func (f Function) Invoke(any *anypb.Any) (*anypb.Any, error) {
	log.Printf("[wapc:Function:Invoke] Entered")
	ctx := context.Background()
	b, err := f.i.Invoke(ctx, f.op, any.GetValue())
	if err != nil {
		return &anypb.Any{}, err
	}
	return &anypb.Any{
		TypeUrl: "",
		Value:   b,
	}, nil
}

NOTE even though Any requires TypeURL to uniquely identify these, they’re unused in this solution because the Client has the protobuf Descriptor and knows what to expect.

waPC uses a string to identify functions exported by a WASM binary. The convention in this solution is that the protobuf method name (e.g. mul) matches the waPC operation name that corresponds to the exported function. Here’s where this matching of mul to an actual (Rust) function is defined in the source of the WASM binary:

pub fn handle_wapc(operation: &str, msg: &[u8]) -> CallResult {
    match operation {
        "add" => complex::export::add(msg),
        "sub" => complex::export::sub(msg),
        "mul" => complex::export::mul(msg),
        "div" => complex::export::div(msg),
        e => Err((String::from("dispatch error: ") + &String::from(e)).into()),
    }
}

Rust waPC Guest

The waPC SDK provides example host and guest code. The implementation is straightforward. There is expected to be a handle_wapc function that, given an operation and a msg (a reference to a slice of bytes which should sound familiar!), matches on the operation name e.g. mul and invokes e.g. complex::export::mul(msg). This structure is user-defined but I wanted to show how these functions reflect the marshaling to protobuf.

So, export::mul takes the borrowed slice of bytes and creates a protobuf::well_known_types::Any from it:

pub fn mul(msg: &[u8]) -> CallResult {
    // From bytes --> Any
    // We must know the Type because it's lost in the transfer
    let rqst = Any {
        type_url: "type.googleapis.com/examples.ComplexRequest".to_string(),
        value: msg.iter().cloned().collect(),
        ..Default::default()
    };
    // Call the RPC
    let resp = super::any::mul(rqst)?;
    // Any --> bytes
    let b = resp.get_value();
    let v = b.iter().cloned().collect();
    Ok(v)
}

And then invokes any::mul which unpacks the Any into ComplexRequest as defined by the protobuf. This function then calls rpc::mul which does the multiplication and returns a ComplexResponse and this bubbles back up|down the functions to be returned to the waPC host.

pub fn mul(any: Any) -> Result<Any, ProtobufError> {
    // From Any --> ComplexRequest
    let mut rqst = unpack(any);
    // Call the RPC
    let resp: ComplexResponse = super::rpc::mul(&mut rqst);
    // From ComplexResponse --> Any
    Any::pack(&resp)
}

Lastly we implement the multiplication using Rust’s num_complex:

pub fn mul(rqst: &ComplexRequest) -> ComplexResponse {
    let (a, b) = complex(rqst);
    let result = a * b;
    ComplexResponse {
        result: protobuf::SingularPtrField::some(crate::protos::complex::Complex {
            real: result.re,
            imag: result.im,
            ..Default::default()
        }),
        ..Default::default()
    }
}

We must build the Rust WASM binary using:

cargo build --release --target wasm32-unknown-unknown

And this will generate e.g. complex.wasm in ./target/wasm32-unknown-unknown/release. This binary complex.wasm is what must be Add by the Client (along with the metadata) to the solution.

• Wasm
• Wapc
• Rust
• Golang
• Trillian
• Protobufs
• Gprc