FauxRPC using gRPCurl, Golang and rust
- 12 minutes read - 2402 wordsRead FauxRPC + Testcontainers on Hacker News and was intrigued. I spent a little more time “evaluating” this than I’d planned because I’m forcing myself to use rust as much as possible and my ignorance (see below) caused me some challenges.
The technology is interesting and works well. The experience helped me explore Testcontainers too which I’d heard about but not explored until this week.
For my future self:
| What | What? |
|---|---|
| FauxRPC | A general-purpose tool (built using Buf’s Connect) that includes registry and stub (gRPC) services that can be (programmatically) configured (using a Protobuf descriptor) and stubs (example method responses) to help test gRPC implementations. |
| Testcontainers | Write code (e.g. rust) to create and interact (test)services (running in [Docker] containers). |
| Connect | (More than a) gRPC implementation used by FauxRPC |
| gRPCurl | A command-line gRPC tool. |
I started following along with FauxRPC’s Testcontainers example but, being unfamiliar with Connect, I wasn’t familiar with its Eliza service. The service is available on demo.connectrpc.com:443 and is described using eliza.proto as part of examples-go. Had I realized this sooner, I would have used this example rather than the Health Checking protocol.
FileDescriptorSet
The gRPC Health Checking Protocol is my go-to service and the one I used instead.
We’re going to need a Protobuf Descriptor for the Health Checking protocol:
protoc \
--descriptor_set_out=health.bin \
health.proto
NOTE The Descriptor generated by default by
protocdiffers from the one generated by rust tonic. The latter includesSourceCodeInfoby default. The difference is not material to what follows.
To use gRPCurl with FauxRPC registry service, we’re going to need to create an AddDescriptorRequest and to format the message as JSON. It’s straightforward to do this using any of the Protocol Buffer languages.
We need to use protoc to compile the registry_service.proto:
PROTOS="/path/to/protos" # Including registry/v1/registry_service.proto
protoc \
--proto-path=${PROTOS} \
--python_out=${PWD} \
--pyi_out=${PWD} \
${PROTOS}/registry/v1/registry_server.proto
This will generate registry/v1/registry_service_pb2.py which includes AddDescriptorsRequest.
Here’s the Python code using protobuf:
from google.protobuf import json_format
from google.protobuf import descriptor_pb2
from registry.v1 import registry_service_pb2
with open("health.bin", "rb") as f:
d = descriptor_pb2.FileDescriptorSet()
with open("descriptor.json", "w") as f:
r = registry_service_pb2.AddDescriptorsRequest(
descriptors=fds
)
j = json_format.MessageToJson(r)
f.write(j)
The result is:
{
"descriptors": {
"file": [
{
"name": "health.proto",
"package": "grpc.health.v1",
"messageType": [
{
"name": "HealthCheckRequest",
"field": [
{
"name": "service",
"number": 1,
"label": "LABEL_OPTIONAL",
"type": "TYPE_STRING",
"jsonName": "service"
}
]
},
{
"name": "HealthCheckResponse",
"field": [
{
"name": "status",
"number": 1,
"label": "LABEL_OPTIONAL",
"type": "TYPE_ENUM",
"typeName": ".grpc.health.v1.HealthCheckResponse.ServingStatus",
"jsonName": "status"
}
],
"enumType": [
{
"name": "ServingStatus",
"value": [
{
"name": "UNKNOWN",
"number": 0
},
{
"name": "SERVING",
"number": 1
},
{
"name": "NOT_SERVING",
"number": 2
},
{
"name": "SERVICE_UNKNOWN",
"number": 3
}
]
}
]
}
],
"service": [
{
"name": "Health",
"method": [
{
"name": "Check",
"inputType": ".grpc.health.v1.HealthCheckRequest",
"outputType": ".grpc.health.v1.HealthCheckResponse"
},
{
"name": "Watch",
"inputType": ".grpc.health.v1.HealthCheckRequest",
"outputType": ".grpc.health.v1.HealthCheckResponse",
"serverStreaming": true
}
]
}
],
"options": {
"javaPackage": "io.grpc.health.v1",
"javaOuterClassname": "HealthProto",
"javaMultipleFiles": true,
"goPackage": "google.golang.org/grpc/health/grpc_health_v1",
"csharpNamespace": "Grpc.Health.V1"
},
"syntax": "proto3"
}
]
}
}
gRPCurl and buf curl
Even though I began by writing a Golang test for the Health Checking service under FauxRPC running in a Testcontainers (container!?), I’m going to reorder the work to show a more logical evolution. So, let’s start by installing the FauxRPC (Golang) binary and the Buf CLI:
go install github.com/sudorandom/fauxrpc/cmd/fauxrpc@latest
go install github.com/bufbuild/buf/cmd/buf@latest
NOTE You can
fauxrpc run --schema=health.prototo register the Health Checking Protocol directly:fauxrpc run --schema=health.protoAnd then interact with the service:
grpcurl \ -plaintext \ -proto health.proto \ localhost:6660 grpc.health.v1.Health/CheckReturning:
{ "status": "NOT_SERVING" }
For consistency with what follows, we’ll:
PORT="6660"
fauxrpc run --empty --addr=":${PORT}"
NOTE If you’d prefer you can run the FauxRPC container (
docker.io/sudorandom/fauxrpc)
FauxRPC (dev (none) @ unknown; go1.23.2) - 0 services loaded
Listening on http://:6660
OpenAPI documentation: http://:6660/fauxrpc/openapi.html
Example Commands:
$ buf curl --http2-prior-knowledge http://:6660 --list-methods
$ buf curl --http2-prior-knowledge http://:6660/[METHOD_NAME]
Server started.
And, per the instruction, you can:
buf curl \
--http2-prior-knowledge \
http://:6660 \
--list-methods
Which should be empty (per the --empty flag).
FauxRPC includes a registry service (registry_service.proto) and a stubs service (stubs_service.proto).
I’m going to use gRPCurl to interact with FauxRPC(’s gRPC services).
We will use the registry service to register the Health Checking Protocol using its descriptor (health.bin):
NOTE We saw this implicitly with the Python code but
registry_service.protoimports Google’sdescriptor.proto. The easiest way to avoid recursive dependencies is to clone Google’sgoogleapis
cat descriptor.json \
| grpcurl \
-plaintext \
--import-path=protos \
--proto=protos/registry/v1/registry_service.proto \
-d @ \
localhost:6660 \
registry.v1.RegistryService.AddDescriptors
{}
NOTE
AddDescriptorsreturnsAddDescriptorsResponsewhich is{}.
But, we can rerun the buf curl command to see that we know have the Health Checking Protocol’s Check and Watch methods in the registry:
buf curl --http2-prior-knowledge http://:6660 --list-methods
grpc.health.v1.Health/Check
grpc.health.v1.Health/Watch
Equivalently, we can use gRPCurl:
grpcurl -plaintext localhost:6660 describe grpc.health.v1.Health
grpc.health.v1.Health is a service:
service Health {
rpc Check ( .grpc.health.v1.HealthCheckRequest ) returns ( .grpc.health.v1.HealthCheckResponse );
rpc Watch ( .grpc.health.v1.HealthCheckRequest ) returns ( stream .grpc.health.v1.HealthCheckResponse );
}
And, these can be invoked:
grpcurl -plaintext localhost:6660 grpc.health.v1.Health/Check
{
"status": "NOT_SERVING"
}
Hang on, why are we getting NOT_SERVING from a service that is only a mock without any implementation? The reason is that FauxRPC is generating random values for the value of the enum used to represent status. If we repeat the command, we should get random results.
Using FauxRPC’s Stubs service, we can define a specific result: 2 corresponds to NOT_SERVING
D='{
"stubs":[
{
"ref":{
"id":"11",
"target":"grpc.health.v1.Health/Check"
},
"json":"{\"status\":2}"
}
]
}'
grpcurl \
-plaintext \
--import-path=protos \
-proto=protos/stubs/v1/stubs_service.proto \
-d "${D}" \
localhost:6660 \
stubs.v1.StubsService/AddStubs
{
"stubs": [
{
"ref": {
"id": "11",
"target": "grpc.health.v1.HealthCheckResponse"
},
"json": "{\"status\":2}"
}
]
}
Now, when we invoke the Check method, we’ll get NOT_SERVING as the result.
Golang
I wrote several Golang tests and implementations in order to help me build the rust clients.
The first was the result of following the FauxRPC tutorial to test running fauxrpctestcontainers. So, although it’s duplicative, it uses the Health Checking protocol instead of Eliza and includes some of the (possibly obvious steps) omitted from the tutorial.
But first, an explanation. testcontainers provides an interface for running arbitrary containers (often servers|services) to facilitate testing. Developers can also provide implementations that wrap some of this functionality to provide a more streamlined solution (see Modules). There is a fauxrpctestcontainers Golang package that provides an implementation for FauxRPC and makes it easier to use.
So here are two solutions in Golang. The first using the FauxRPC binary (e.g. fauxrpc run --empty --addr=":${PORT}" or using the container image) which makes the gRPC calls AddDescriptors and AddStubs explained previously:
package main
import (
"context"
"fmt"
"log/slog"
"os"
"testing"
healthpb "{MODULE}/protos/grpc/health/v1"
registrypb "{MODULE}/protos/registry/v1"
stubspb "{MODULE}/protos/stubs/v1"
"github.com/google/uuid"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/types/descriptorpb"
)
const (
health string = "/path/to/health.bin"
)
const (
endpoint string = ":6660"
)
func TestMain(t *testing.T) {
opts := []grpc.DialOption{
grpc.WithTransportCredentials(insecure.NewCredentials()),
}
conn, err := grpc.NewClient(endpoint, opts...)
if err != nil {
t.Error(err)
}
defer conn.Close()
ctx := context.Background()
{
clientRegistryService := registrypb.NewRegistryServiceClient(conn)
b, err := os.ReadFile(health)
if err != nil {
t.Error(err)
}
fds := &descriptorpb.FileDescriptorSet{}
if err := proto.Unmarshal(b, fds); err != nil {
t.Error(err)
}
rqst := ®istrypb.AddDescriptorsRequest{
Descriptors: fds,
}
resp, err := clientRegistryService.AddDescriptors(ctx, rqst)
if err != nil {
t.Error(err)
}
slog.Info("Response", "resp", resp)
}
{
clientStubsService := stubspb.NewStubsServiceClient(conn)
id := uuid.New().String()
method := "grpc.health.v1.Health/Check"
ref := &stubspb.StubRef{
Id: id,
Target: method,
}
b, err := proto.Marshal(&healthpb.HealthCheckResponse{
Status: healthpb.HealthCheckResponse_SERVING,
})
if err != nil {
t.Error(err)
}
content := stubspb.Stub_Proto{
Proto: b,
}
rqst := &stubspb.AddStubsRequest{
Stubs: []*stubspb.Stub{
{
Ref: ref,
Content: &content,
},
},
}
x, err := proto.Marshal(rqst)
if err != nil {
t.Error(err)
}
slog.Info("Request", "x", fmt.Sprintf("%x", x))
resp, err := clientStubsService.AddStubs(ctx, rqst)
if err != nil {
t.Error(err)
}
slog.Info("Response", "resp", resp)
}
{
clientHealthService := healthpb.NewHealthClient(conn)
rqst := &healthpb.HealthCheckRequest{
Service: "",
}
resp, err := clientHealthService.Check(ctx, rqst)
if err != nil {
t.Error(err)
}
slog.Info("Response", "resp", resp)
}
}
You’ll need to replace {MODULE} with your go mod init ${MODULE} value and, you’ll want to compile the protocol buffers sources that you can find in the FauxRPC repo proto folder:
PROTOS="/path/to/protos"
protoc \
--proto_path=${PROTOS} \
--go_out=${PWD} \
--go_opt=Mregistry/v1/registry_service.proto=protos/registry/v1 \
--go_opt=Mstubs/v1/stubs_service.proto=protos/stubs/v1 \
--go_opt=Mstubs/v1/stubs.proto=protos/stubs/v1 \
--go-grpc_out=${PWD} \
--go-grpc_opt=Mregistry/v1/registry_service.proto=protos/registry/v1 \
--go-grpc_opt=Mstubs/v1/stubs_service.proto=protos/stubs/v1 \
--go-grpc_opt=Mstubs/v1/stubs.proto=protos/stubs/v1 \
${PROTOS}/registry/v1/registry_service.proto \
${PROTOS}/stubs/v1/stubs_service.proto \
${PROTOS}/stubs/v1/stubs.proto
The second approach uses fauxrpctestcontainers and is more straightforward. It will run (and terminate on success) a docker.io/sudorandom/fauxrpc container for you so you don’t need to run fauxrpc manually:
package main
import (
"context"
"os"
"strings"
"testing"
fauxrpctestcontainers "github.com/sudorandom/fauxrpc/testcontainers"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
"google.golang.org/grpc/health/grpc_health_v1"
"google.golang.org/protobuf/proto"
"google.golang.org/protobuf/reflect/protodesc"
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/types/descriptorpb"
)
const (
health string = "/path/to/health.bin"
)
const (
img string = "docker.io/sudorandom/fauxrpc:v0.0.23"
method string = "grpc.health.v1.Health/Check"
status = grpc_health_v1.HealthCheckResponse_SERVING
)
func getHealthFileDescriptor(name string) (protoreflect.FileDescriptor, error) {
data, err := os.ReadFile(name)
if err != nil {
return nil, err
}
fds := &descriptorpb.FileDescriptorSet{}
if err := proto.Unmarshal(data, fds); err != nil {
return nil, err
}
fd, err := protodesc.NewFile(fds.File[0], nil)
return fd, err
}
func TestMain(t *testing.T) {
ctx := context.Background()
container, err := fauxrpctestcontainers.Run(ctx, img)
if err != nil {
t.Error(err)
}
t.Cleanup(func() {
container.Terminate(ctx)
})
fd, err := getHealthFileDescriptor(health)
if err != nil {
t.Error(err)
}
container.MustAddFileDescriptor(ctx, fd)
// Define method's response
want := status
container.MustAddStub(ctx, method, &grpc_health_v1.HealthCheckResponse{
Status: want,
})
// BaseURL includes "http://" which is incompatible with grpc.NewClient
baseURL := container.MustBaseURL(ctx)
baseURL = strings.TrimPrefix(baseURL, "http://")
conn, err := grpc.NewClient(
baseURL,
grpc.WithTransportCredentials(insecure.NewCredentials()),
)
if err != nil {
t.Error(err)
}
defer conn.Close()
healthClient := grpc_health_v1.NewHealthClient(conn)
rqst := &grpc_health_v1.HealthCheckRequest{}
resp, err := healthClient.Check(ctx, rqst)
if err != nil {
t.Error(err)
}
got := resp.Status
if got != want {
t.Errorf("got: %s, want: %s", got, want)
}
}
rust
These dependencies and build-dependencies are required by the code that follows:
[dependencies]
prost = "0.13.3"
prost-types = "0.13.3"
redis = "0.27.4"
testcontainers = "0.23.1"
tokio = "1.40.0"
tonic = { version = "0.12.3", features=["tls","tls-roots"] }
uuid = { version = "1.11.0", features=["v4"] }
[build-dependencies]
tonic-build = "0.12.3"
Rust’s tonic comes with a companion tonic-build that enables us to write a build.rs to compile protobufs:
build.rs:
use std::{env, path::PathBuf};
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Required by prost-reflection to create descriptors.bin in correct OUT_DIR
let out_dir = PathBuf::from(env::var("OUT_DIR").expect("unable to determine `OUT_DIR`"));
// `file_descriptor_set_path` appends `.bin` to the file that's created
tonic_build::configure()
.build_client(true)
.build_server(false)
.file_descriptor_set_path(out_dir.join("health.bin"))
.compile_protos(&["/path/to/protos/grpc/health/v1/health.proto"], &["/path/to/protos"])?;
tonic_build::configure()
.build_client(true)
.build_server(false)
.file_descriptor_set_path(out_dir.join("registry_service.bin"))
.compile_protos(
&["/path/to/protos/registry/v1/registry_service.proto"],
&["/path/to/protos"],
)?;
let googleapis = "/path/to/googleapis";
tonic_build::configure()
.build_client(true)
.build_server(false)
.file_descriptor_set_path(out_dir.join("stubs_service.bin"))
.compile_protos(
&["/path/to/protos/stubs/v1/stubs_service.proto"],
&["/path/to/protos", googleapis],
)?;
Ok(())
}
There are 3 variants of the rust implementation. The first also assumes fauxrpc --empty --addr=":${PORT} or that you’re running the FauxRPC container:
static FILE_DESCRIPTOR_SET: &[u8] = include_file_descriptor_set!("health");
#[tokio::test]
async fn test_grpc() -> Result<(), Box<dyn std::error::Error + 'static>> {
let host = "localhost";
let port = 6660;
let url = format!("http://{host}:{port}");
let uri: tonic::transport::Uri = url
.parse()
.expect("invalid gRPC endpoint");
let endpoint = Channel::builder(uri);
let channel = endpoint.connect()
.await
.expect("unable to connect");
let file_descriptor_set =
FileDescriptorSet::decode(FILE_DESCRIPTOR_SET)
.expect("invalid file descriptor set");
let mut registry_service_client =
RegistryServiceClient::new(channel.clone());
let rqst = tonic::Request::new(AddDescriptorsRequest {
descriptors: Some(file_descriptor_set),
});
let resp = registry_service_client
.add_descriptors(rqst)
.await
.expect("invalid response");
println!("{:?}", resp);
let mut stubs_service_client =
StubsServiceClient::new(channel.clone());
let id = Uuid::new_v4().as_hyphenated().to_string();
let method = "grpc.health.v1.Health/Check".to_string();
let r#ref = Some(stubs::v1::StubRef {
id: id,
target: method,
});
let b = HealthCheckResponse { status: 1 }.encode_to_vec();
let content = Some(stubs::v1::stub::Content::Proto(b));
let rqst = tonic::Request::new(AddStubsRequest {
stubs: vec![stubs::v1::Stub {
r#ref: r#ref,
content: content,
}],
});
let resp = stubs_service_client
.add_stubs(rqst)
.await
.expect("invalid response");
println!("{:?}", resp);
let mut health_client = HealthClient::new(channel.clone());
let rqst = tonic::Request::new(HealthCheckRequest {
service: "".to_owned(),
});
let resp = health_client.check(rqst).await?;
let msg = resp.into_inner();
assert_eq!(msg.status, 1);
Ok(())
}
Replacing the static references to a local fauxrpc --empty --addr=":${PORT} (or container):
let host = "localhost";
let port = 6660;
We can create a testcontainer:
let image = "docker.io/sudorandom/fauxrpc";
let tag = "v0.0.23";
let port: u16 = 6660;
let container = GenericImage::new(image, tag)
.with_exposed_port(port.tcp())
.with_wait_for(WaitFor::message_on_stdout("Server started."))
.with_cmd(vec![
"run",
"--log-level=debug",
"--empty",
format!("--addr=:{port}").as_str(),
])
.start()
.await?;
let host = container.get_host().await?;
let port = container.get_host_port_ipv4(port).await?;
I’m a rust noob but I’ve begun to implement something similar to fauxrpctestcontainers in rust and using testcontainers-modules as a guide.
It is very much a work-in-progress:
mod.rs:
use testcontainers::{
core::{ContainerPort, WaitFor},
Image, ImageExt,
};
use crate::{
registry::v1::{
registry_service_client::RegistryServiceClient, AddDescriptorsRequest,
AddDescriptorsResponse,
},
stubs::v1::{
stub::Content, stubs_service_client::StubsServiceClient, AddStubsRequest, AddStubsResponse,
Stub, StubRef,
},
};
use tonic::{transport::Channel, Status};
use uuid::Uuid;
const DEFAULT_IMAGE_NAME: &str = "docker.io/sudorandom/fauxrpc";
const DEFAULT_IMAGE_TAG: &str = "v0.0.23";
const FAUXRPC_PORT: ContainerPort = ContainerPort::Tcp(6660);
#[derive(Debug, Default, Clone)]
pub struct FauxRPC {}
impl FauxRPC {
pub async fn add_file_descriptor(
&self,
channel: Channel,
file_descriptor_set: prost_types::FileDescriptorSet,
) -> Result<AddDescriptorsResponse, Status> {
let mut client: RegistryServiceClient<Channel> =
RegistryServiceClient::new(channel);
let rqst = tonic::Request::new(AddDescriptorsRequest {
descriptors: Some(file_descriptor_set),
});
let resp = client.add_descriptors(rqst).await;
match resp {
Ok(r) => Ok(r.into_inner()),
Err(e) => Err(e),
}
}
pub async fn add_stub(
&self,
channel: Channel,
bytes: Vec<u8>,
) -> Result<AddStubsResponse, Status> {
let mut client: StubsServiceClient<Channel> =
StubsServiceClient::new(channel);
let id = Uuid::new_v4().as_hyphenated().to_string();
let method = "grpc.health.v1.Health/Check".to_string();
let rqst = tonic::Request::new(AddStubsRequest {
stubs: vec![Stub {
r#ref: Some(StubRef {
id: id,
target: method,
}),
content: Some(Content::Proto(bytes)),
}],
});
let resp = client.add_stubs(rqst).await;
match resp {
Ok(r) => Ok(r.into_inner()),
Err(e) => Err(e),
}
}
}
impl Image for FauxRPC {
fn name(&self) -> &str {
DEFAULT_IMAGE_NAME
}
fn tag(&self) -> &str {
DEFAULT_IMAGE_TAG
}
fn ready_conditions(&self) -> Vec<WaitFor> {
vec![WaitFor::message_on_stdout("Server started")]
}
fn expose_ports(&self) -> &[ContainerPort] {
&[FAUXRPC_PORT]
}
}
// impl ImageExt<I> for FauxRPC {}
#[cfg(test)]
mod tests {
use crate::{
fauxrpctestcontainers::FauxRPC,
testcontainers::runners::AsyncRunner, ImageExt,
};
#[tokio::test]
async fn test_fauxrpc() -> Result<(), Box<dyn std::error::Error + 'static>> {
let container = FauxRPC::default()
// Would be good for this to be the default
// But it requires impl ImageExt which has a bunch of functions to be implemented
.with_cmd(vec![
"run",
"--empty",
"--addr=:6660",
])
.start()
.await?;
let host = container.get_host().await?;
let port = container.get_host_port_ipv4(6660).await?;
let url = format!("{host}:{port}");
// assert_eq!(url, "127.0.0.1:6660");
Ok(())
}
}
And:
main.rs:
#[tokio::test]
async fn test_fauxrpc() -> Result<(), Box<dyn std::error::Error + 'static>> {
let port: u16 = 6660;
let container = FauxRPC::default()
.with_cmd(vec![
"run",
"--empty",
format!("--addr=:{port}").as_str(),
])
.start()
.await?;
let host = container.get_host().await?;
let port = container.get_host_port_ipv4(port).await?;
let url = format!("http://{host}:{port}");
let uri: tonic::transport::Uri = url
.parse()
.expect("invalid gRPC endpoint");
let endpoint = Channel::builder(uri);
let channel = endpoint.connect()
.await
.expect("unable to connect");
let file_descriptor_set =
FileDescriptorSet::decode(FILE_DESCRIPTOR_SET)
.expect("invalid file descriptor set");
let _r = container
.image()
.add_file_descriptor(channel.clone(), file_descriptor_set)
.await;
println!("{:?}", _r);
let b = HealthCheckResponse {
status: health_check_response::ServingStatus::Serving as i32,
}
.encode_to_vec();
let _r = container.image().add_stub(channel.clone(), b).await;
println!("{:?}", _r);
let mut health_client = HealthClient::new(channel.clone());
let rqst = tonic::Request::new(HealthCheckRequest {
service: "".to_owned(),
});
let resp = health_client.check(rqst).await?;
let msg = resp.into_inner();
assert_eq!(
msg.status,
health_check_response::ServingStatus::Serving as i32
);
Ok(())
}
That’s all!