Rube Goldberg Cloud Build machine for solving Quadratic equations

September 28, 2020 - 10 minutes read - 2120 words

Google Cloud Build is described by Google as a “CI/CD platform” but it’s fundamentally a service that permits a series of containers to be chained together in a pipeline, optionally leveraging shared data.

As a CI/CD platform, it can be used to lint, test, compile and build software but, if you were looking for a way to explain its basic awesomeness, you could… I don’t know… build a Rube Goldberg style machine that solves Quadratic equations using it 😄

As many schoolchildren know, quadratic equations can always be solved using the Quadratic formula. This solves equations of the form ax²+bx+c=0 and provides at most 2 roots.

NOTE what follows should not be used as a production solution for solving Equdratic equations

NOTE if you’re allergic to Quadratic equations, don’t use this solution.

NOTE This solutoin contains no (!) error handling

We’re going to use bash and bc.

This calculation could be performed very simply but there would be no fun in that. We’re not trying to find the simplest solution, remember.

For convenience – I’ll explain more about this below – we will use 3 files (a,b,c) to hold the values from the formula.

E.g. for 8x²-10x+3=0, we’ll use:

echo "8" > a
echo "-10" > b
echo "3" > c

We can output the equation using:

echo "Quadratic: $(cat a)x²+$(cat b)x+$(cat c)s=0"

NOTE bonus point if you wish to replace the static + with - for negative values of b and c

There are other ways to achieve this result in bash that may be more elegant or efficient but, for simplicity and consistency, we’ll use echo throughout

Then, rather than calculate the roots in a single step, we’ll perform each constituent calculation in its own step, outputting these, intermediate results to a file:

#!/usr/bin/env bash

echo "Quadratic: $(cat a)x²+$(cat b)x+$(cat c)s=0"

echo "$(cat b) * $(cat b)" | bc -l > b2
echo "4 * $(cat a) * $(cat c)" | bc -l > 4ac
echo "$(cat b2) - $(cat 4ac)" | bc -l > b2-4ac

rm b2 4ac

echo "sqrt($(cat b2-4ac))" | bc -l > sqrt

rm b2-4ac

echo "-($(cat b))+$(cat sqrt)" | bc -l > add
echo "-($(cat b))-$(cat sqrt)" | bc -l > sub

rm sqrt

echo "2 * $(cat a)" | bc -l > 2a

echo "$(cat add)/$(cat 2a)" | bc -l > root1
echo "$(cat sub)/$(cat 2a)" | bc -l > root2

rm 2a add sub

# Complete
echo "Roots are: $(cat root1); $(cat root2)"

rm root1 root2

After creating the files a,b and c, you could simply paste the script above into your shell to produce the result:

Roots are: .75000000000000000000; .50000000000000000000

OK, so we have our calculator, how do we transform this into a Cloud Build configuration file?

Using Cloud Build, each step (echo ..., rm ...) in our calculation becomes a step in the Cloud Build process. Each step corresponds to the use of some container image. Any container image may be used by Cloud Build (as long the image is accessible to the service) and, when run, the container may be configured with arguments, working directory etc.

Behind the scenes, Cloud Build creates a VM for our job and it runs the containers (per the configuration file) in this VM. But, from where does it get data that it requires? The data may come from anywhere that a container image can acquire the data. A container could do an HTTP get, it could query a SQL database, another cloud service …. anything.

But, there’s a unique location for data in Cloud Build that corresponds to the VM’s filing system and a unique location in that filing system called /workspace. The /workspace directory receives source files that you designate when you trigger a build with gcloud builds submit .. (or equivalent) and by default, each step mounts /workspace as the container’s working directory.

The latter is similar to assuming that every step does something of the form:

docker run \
--volume=/workspace:/workspace some-image \
--workdir=/workspace \
...

This is important because the filing system is the only location within the build process where you can persist data.

You may read data from environment variables but environment variable updates will not persist across Cloud Build steps.

You may read and write data to arbitrary out-of-process locations e.g. via an API, SQL database etc.

What does this mean for our Rube Goldberg calculator? It means that we will recreate our inputs (a,b and c) automatically in Cloud Build when we perform the gcloud builds submit ... by including ${PWD} or (more succinctly) ., i.e. gcloud builds submit ${PWD} ... and it means we can create files corresponding to the intermediate steps as a way to persist these values through the steps of the calculation.

Let’s create a local (not Cloud Build) bash script that uses Docker images for each of our steps:

#!/usr/bin/env bash

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "Quadratic: $(cat a)x²+$(cat b)x+$(cat c)s=0"'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "$(cat b) * $(cat b)" | bc -l > b2'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "4 * $(cat a) * $(cat c)" | bc -l > 4ac'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "$(cat b2) - $(cat 4ac)" | bc -l > b2-4ac'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'rm b2 4ac'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "sqrt($(cat b2-4ac))" | bc -l > sqrt'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'rm b2-4ac'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "-($(cat b))+$(cat sqrt)" | bc -l > add'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "-($(cat b))-$(cat sqrt)" | bc -l > sub'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'rm sqrt'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "2 * $(cat a)" | bc -l > 2a'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "$(cat add)/$(cat 2a)" | bc -l > root1'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "$(cat sub)/$(cat 2a)" | bc -l > root2'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'rm 2a add sub'

# Complete
docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'echo "Roots are: $(cat root1); $(cat root2)"'

docker run \
--rm \
--volume=${PWD}:/workspace \
--workdir=/workspace \
busybox ash -c 'rm root1 root2'

As before, you should be able to paste these commands into your shell to calculate the roots.

We’re using busybox here as a small container image that includes many useful Linux utilities including bc.

You should see the same bc commands as before.

The only other addition is the use of --workdir=/workspace and --volume=${PWD}:/workspace. The volume flag mounts the current directory ({PWD}) into each container’s /workspace directory. The workdir flag makes /workspace the working directory for the containers. Effectively, these 2 flags resemble the behavior of steps in the Cloud Build service.

OK, we’re now ready to try our calculator on Cloud Build. You should (!) be able to run the following commands without incurring a bill along as you use less than 120-minutes of Cloud Build per day (link). This is because Cloud Build is included as part of Google’s Free Tier.

First we’ll create a project, enable billing (required), and enable Cloud Build:

PROJECT=[[YOUR-PROJECT]]
BILLING=[[YOUR-BILLING]]

gcloud projects create ${PROJECT}

gcloud beta billing projects link ${PROJECT} \
--billing-account=${BILLING}

gcloud services enable cloudbuild.googleapis.com \
--project=${PROJECT}

NOTE To get your billing account, you can BILLING=$(gcloud alpha billing accounts list --format="value(name)")

Then we can create our cloudbuild.yaml file:

steps:
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "Quadratic: $(cat a)x²+$(cat b)x+$(cat c)s=0"'
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "$(cat b) * $(cat b)" | bc -l > b2'
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "4 * $(cat a) * $(cat c)" | bc -l > 4ac'
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "$(cat b2) - $(cat 4ac)" | bc -l > b2-4ac'
  - name: busybox
    args:
      - ash
      - -c
      - "rm b2 4ac"
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "sqrt($(cat b2-4ac))" | bc -l > sqrt'
  - name: busybox
    args:
      - ash
      - -c
      - "rm b2-4ac"
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "-($(cat b)) + $(cat sqrt)" | bc -l > add'
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "-($(cat b)) - $(cat sqrt)" | bc -l > sub'
  - name: busybox
    args:
      - ash
      - -c
      - "rm sqrt"
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "2 * $(cat a)" | bc -l > 2a'
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "$(cat add)/$(cat 2a)" | bc -l > root1'
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "$(cat sub)/$(cat 2a)" | bc -l > root2'
  - name: busybox
    args:
      - ash
      - -c
      - "rm 2a add sub"
  - name: busybox
    args:
      - ash
      - -c
      - 'echo "Roots are: $(cat root1); $(cat root2)"'
  - name: busybox
    args:
      - ash
      - -c
      - "rm root1 root2"

And then we can submit the calculation.

Don’t forget to create a, b and c in your hosts current working directory:

gcloud builds submit ${PWD} \
--config=./cloudbuild.yaml \
--project=${PROJECT}

For me, this yields:

BUILD
Starting Step #0
Step #0: Pulling image: busybox
Step #0: Using default tag: latest
Step #0: latest: Pulling from library/busybox
Step #0: Digest: sha256:2ca5e69e...
Step #0: Status: Downloaded newer image for busybox:latest
Step #0: docker.io/library/busybox:latest
Step #0: Quadratic: 8x²+-10x+3s=0
Finished Step #0
Starting Step #1
Step #1: Already have image: busybox
Finished Step #1
Starting Step #2
Step #2: Already have image: busybox
Finished Step #2
Starting Step #3
Step #3: Already have image: busybox
Finished Step #3
Starting Step #4
Step #4: Already have image: busybox
Finished Step #4
Starting Step #5
Step #5: Already have image: busybox
Finished Step #5
Starting Step #6
Step #6: Already have image: busybox
Finished Step #6
Starting Step #7
Step #7: Already have image: busybox
Finished Step #7
Starting Step #8
Step #8: Already have image: busybox
Finished Step #8
Starting Step #9
Step #9: Already have image: busybox
Finished Step #9
Starting Step #10
Step #10: Already have image: busybox
Finished Step #10
Starting Step #11
Step #11: Already have image: busybox
Finished Step #11
Starting Step #12
Step #12: Already have image: busybox
Finished Step #12
Starting Step #13
Step #13: Already have image: busybox
Finished Step #13
Starting Step #14
Step #14: Already have image: busybox
Step #14: Roots are: .75000000000000000000; .50000000000000000000
Finished Step #14
Starting Step #15
Step #15: Already have image: busybox
Finished Step #15
PUSH
DONE

The key here is the output from step #14, the 15th (starting from 0):

Step #14: Roots are: .75000000000000000000; .50000000000000000000

NOTE since we’re using an ephemeral service and the storage is ~free, we could remote the steps that remove the intermediate files

If, for some crazy reason, you wish to persist the results, then we can add artifacts to the configuration.

Create a Cloud Storage bucket:

BUCKET=[YOUR-BUCKET]
gsutil mb -p ${PROJECT} gs://${BUCKET}

And then revise the Cloud Build configuration, removing the step that deletes root1 and root2, and replacing it with an artifacts configuration that references the files and the GCS bucket:

  # - name: busybox
  #   args:
  #     - ash
  #     - -c
  #     - "rm root1 root2"
artifacts:
  objects:
    location: "gs://[[YOUR-BUCKET]]/"
    paths:
      - "root1"
      - "root2"

and then:

gsutil ls gs://${BUCKET}
gs://[YOUR-PROJECT]/artifacts-ef3b...f98e.json
gs://[YOUR-PROJECT]/root1
gs://[YOUR-PROJECT]/root2

As you expect, the root? files contain the roots.

The artifacts-*.json file contains (in this case):

{
    "location":"gs://[[YOUR-BUCKET]]/root1#1601321178905090",
    "file_hash":[{
        "file_hash":[{
            "type":2,
            "value":"dVVtNERLMStyalZXZGprSVF4ajhxUT09"
        }]
    }]
}
{
    "location":"gs://[[YOUR-BUCKET]]/root2#1601321186774984",
    "file_hash":[{
        "file_hash":[{
            "type":2,
            "value":"VWtsODlSS1hmRjI4RFA4Q3JSdjhSdz09"
        }]
    }]
}

NOTE the structure of this artifacts file is explained here

We can have gsutil (re)compute files’ hashes:

gsutil gs://[[YOUR-BUCKET]]/root1
Hashes [base64] for root1:
    Hash (md5):        uUm4DK1+rjVWdjkIQxj8qQ==
    Hash (crc32c):     oR1wYQ==

NOTE the md5 hash value

And:

gsutil cat gs://[[YOUR-BUCKET]]/root1 \
| md5sum \
| head --bytes=32 \
| xxd -revert -plain \
| base64
uUm4DK1+rjVWdjkIQxj8qQ==

NOTE importantly, when using bash’s md5sum utility we get a hexdump value that must be (using xxd) reverted to binary

Hopefully, the above overly-complex Cloud Build example has helped explain the very general-purpose utility of Cloud Build.

To tidy up, you may simple delete everything:

gcloud projects delete ${PROJECT} --quiet

Cloud Build