Rust Crate Transparency && Rust SDK for Google Trillian

- 3 minutes read - 563 words

I’m noodling the utility of a Transparency solution for Rust Crates. When developers push crates to Cargo, a bunch of metadata is associated with the crate. E.g. protobuf. As with Golang Modules, Python packages on PyPi etc., there appears to be utility in making tamperproof recordings of these publications. Then, other developers may confirm that a crate pulled from cates.io is highly unlikely to have been changed.

On Linux, Cargo stores downloaded crates under ${HOME}/.crates/registry. In the case of the latest version (2.12.0) of protobuf, on my machine, I have:

ls -ld ${HOME}/.cargo/registry/*/*/protobuf*
~/.cargo/registry/cache/github.com-1ecc6299db9ec823/protobuf-1.7.5.crate
~/.cargo/registry/cache/github.com-1ecc6299db9ec823/protobuf-2.12.0.crate
~/.cargo/registry/cache/github.com-1ecc6299db9ec823/protobuf-2.8.0.crate
~/.cargo/registry/cache/github.com-1ecc6299db9ec823/protobuf-2.8.1.crate
~/.cargo/registry/cache/github.com-1ecc6299db9ec823/protobuf-2.8.2.crate

~/.cargo/registry/src/github.com-1ecc6299db9ec823/protobuf-1.7.5
~/.cargo/registry/src/github.com-1ecc6299db9ec823/protobuf-2.12.0
~/.cargo/registry/src/github.com-1ecc6299db9ec823/protobuf-2.8.0
~/.cargo/registry/src/github.com-1ecc6299db9ec823/protobuf-2.8.1
~/.cargo/registry/src/github.com-1ecc6299db9ec823/protobuf-2.8.2

NB The significance of 1ecc6299db9ec823 postfixing the github.com- directory name is unclear to me.

I can hash these crates, yielding:

NAME="protobuf-[0-9]*" # NB not a regex but a filename pattern limits results to `protobuf` excludes `protobuf-codegen`
for FILE in $(ls ${HOME}/.cargo/registry/cache/*/${NAME})
do
  HASH=$(cat ${FILE} | sha256sum | head --bytes 64)
  printf "crate: %s\thash: %s\n" ${FILE} ${HASH}
done

yields:

crate: ../protobuf-1.7.5.crate	hash: e14ccd6b79ec748412d4f2dfde1a80fa363a67def4062969f8aed3d790a30f28
crate: ../protobuf-2.12.0.crate	hash: 71964f34fd51cf04882d7ae3325fa0794d4cad66a03d0003f38d8ae4f63ba126
crate: ../protobuf-2.8.0.crate	hash: 8aefcec9f142b524d98fc81d07827743be89dd6586a1ba6ab21fa66a500b3fa5
crate: ../protobuf-2.8.1.crate	hash: 40361836defdd5871ff7e84096c6f6444af7fc157f8ef1789f54f147687caa20
crate: ../protobuf-2.8.2.crate	hash: 70731852eec72c56d11226c8a5f96ad5058a3dab73647ca5f7ee351e464f2571

Cargo.lock files include hashes for crates too, e.g.:

"checksum protobuf 2.12.0 (registry+https://github.com/rust-lang/crates.io-index)" = "71964f34fd51cf04882d7ae3325fa0794d4cad66a03d0003f38d8ae4f63ba126"

NB Fortunately, the hash produced by the bash script corroborates with the hash from the Cargo.lock file for protobuf=2.12.0

So, if we could publish unique details of the crate (developer, name, version, published date, hash etc.) to a Merkle tree, this would permit subsequent downloaders to validate the hash (remains unchanged).

I plan to use Google Trillian for this project. But first, it would be useful to have a Rust SDK for Google Trillian.

Rust SDK for Google Trillian

This is a work-in-progress and my first attempt at producing a crate.

For giggles, this should possibly be the first crate added to the transparency project.

The project will need to some fineigling on structure and naming.

In the Crate Transparency project, I’m referencing it locally as:

rust-trillian = { path = "../rust-trillian", version = "0.0.1" }

And then:

use rust_trillian::protos::{trillian_log_api, trillian_log_api_grpc};

let client = trillian_log_api_grpc::TrillianLogClient::new_plain(
    host,
    port,
    ClientConf::new(),
).expect("client");

In Go, Google provides a client SDK and the equivalent code is:

import (
    "github.com/google/trillian"
)

client := trillian.NewTrillianLogClient(conn)

So it’s not too far away particularly as I’m somewhat constrained by what protoc and the rust plugin (protoc-rust-grpc) generate.

IIUC, the idiomatic rust-way is to generate the rust sources using a build.rs. I had some challenges translating a working protoc into a working build.rs.

The protoc command would be:

protoc \
--proto_path=/path/to/trillian \
--proto_path=/path/to/googleapis \
--rust_out=./src/protos \
path/to/trillian/*.proto

To get protoc-rust-grpc to generate consistent sources, I ended up with:

fn main() {
    protoc_rust_grpc::run(protoc_rust_grpc::Args {
        out_dir: "src/protos",
        includes: &[
            "/path/to/googleapis",
            "/path/to/trillian",
            "/path/to/trillian/crypto/keyspb",
            "/path/to/trillian/crypto/sigpb",
        ],
        input: &[
            "/path/to/googleapis/google/api/annotations.proto",
            "/path/to/googleapis/google/api/http.proto",
            "/path/to/googleapis/google/api/httpbody.proto",
            "/path/to/googleapis/google/rpc/code.proto",
            "/path/to/googleapis/google/rpc/error_details.proto",
            "/path/to/googleapis/google/rpc/status.proto",
            "/path/to/trillian/trillian_admin_api.proto",
            "/path/to/trillian/trillian_log_api.proto",
            "/path/to/trillian/trillian_log_sequencer_api.proto",
            "/path/to/trillian/trillian_map_api.proto",
            "/path/to/trillian/trillian.proto",
            "/path/to/trillian/crypto/keyspb/keyspb.proto",
            "/path/to/trillian/crypto/sigpb/sigpb.proto",
            ],
        rust_protobuf: true,
        ..Default::default()
    })
    .expect("protoc");
}

NB I had to enumerate the absolute paths each time and I had to explicitly reference dependencies in googleapis.

Then I created a mod.rs to publish the modules that were generated:

pub mod annotations;
pub mod code;
pub mod error_details;
pub mod http;
pub mod status;

pub mod keyspb;
pub mod sigpb;

pub mod trillian;

pub mod trillian_admin_api;
pub mod trillian_admin_api_grpc;

pub mod trillian_log_api;
pub mod trillian_log_api_grpc;

pub mod trillian_log_sequencer_api;
pub mod trillian_log_sequencer_api_grpc;

pub mod trillian_map_api;
pub mod trillian_map_api_grpc;

I think this is probably where there’s room for structural improvement.

That’s all for today!