Google Fit

- 6 minutes read - 1138 words

I’ve spent a few days exploring [Google Fit SDK] as I try to wean myself from my obsession with metrics (of all forms). A quick Googling got me to Robert’s Exporter Google Fit Daily Steps, Weight and Distance to a Google Sheet. This works and is probably where I should have stopped… avoiding the rabbit hole that I’ve been down…

I threw together a simple Golang implementation of the SDK using Google’s Golang API Client Library. Thanks to Robert’s example, I was able to infer some of the complexity this API particularly in its use of data types, data sources and datasets. Having used Stackdriver in my previous life, Google Fit’s structure bears more than a passing resemblance to Stackdriver’s data model and its use of resource types and metric types.

In essence, there are abstract types that can be measured e.g. Height, Steps, Calories. There are specific sources of these measurements e.g. your Android phone (not any Android phone) and, finally we have the measurements that are made which are the time-series (datasets) of e.g. my Android phone measuring my steps for January 2020. I think Google’s documentation would be improved by better explaining this.

Here are some ways that I’ve been trying to make sense of this API:

OAuth 2 w/ User data

Because Google Fit largely consumes user data, you must use the three-legged OAuth2 flow that prompts the user to accept scopes (permissions). Corrollary: you cannot use Application Default Credentials and a service account for what follows.

Google doesn’t provide an API for OAuth2 credentials provisioning so there’s some UI work in what follows.

PROJECT=[[YOUR-PROEJCT-ID]] # e.g. gfit-metrics
gcloud projects create ${PROJECT}
gcloud services enable fitness.googleapis.com --project=${PROJECT} # optionally: --async

NB The service is called fitness even though it’s commonly referred to as Google Fit.

Then browse to:

https://console.cloud.google.com/apis/credentials?project=${PROJECT}

And click “Create credentials”, “OAuth client ID”, “Other”, choose a Name (perhaps metrics) and then click “Create”.

All being well, you’ll be presented with a dialog “OAuth client” that provides a client ID and a client secret. You should record these but, after clicking “OK”, you should see the client listed on the screen and, on the right-hand side, there’s a button to download the client (and secret) as a JSON file (commonly client_secret.json). Do so:

SECRET=/path/to/your/client_secret.json

NB Google provides examples in each language for implementing the OAuth flow, e.g. Golang using Gmail. You may reuse the OAuth2 code from one of these as it’s general-purpose.

oauth2l

Google provides a command-line tool to authenticate and retrieve OAuth2 access tokens (essentially implementing the flow noted for Gmail above). This tool can be used to generate an access token using the OAuth client that was generated above to authenticate us to the Google Fit API. I go Docker but you do you:

docker run \
--rm --interactive --tty \
--volume=${SECRET}:/secret/client_secret.json \
gcr.io/oauth2l/oauth2l@sha256:166a23963f8e58c51039876e5471a6a471c6dfb6895c5e4085987c3a4e822e87 \
fetch \
  --credentials=/secret/client_secret.json \
  --scope=fitness.activity.read,fitness.body.read,fitness.location.read

NB ${SECRET} is the path to the client_secret.json that we created previously.

This command fetches the token using the OAuth client and scopes it to read-only requests. Exactly what we’ll need below. It will prompt you to open a URL, accept permission to use the API and then provide you with a code that you must copy-paste back to the oauth2l command. All being well, you’ll get an access token.

TOKEN=[[THE-ACCESS-TOKEN-PRODUCED]]

Curl

Let’s ensure alles ist in Ordnung:

curl \
--header "Authorization: Bearer ${TOKEN}" \
"https://www.googleapis.com/fitness/v1/users/me/dataSources"

Should return your list of Google Fit data sources. There may be many. If you’ve not already, I recommend installing jq

NB I learned this week that’s there a YAML-based sibling called yq that I must explore!

Let’s refine those results:

curl
--silent
--header "Authorization: Bearer ${TOKEN}" \
"https://www.googleapis.com/fitness/v1/users/me/dataSources" |\
jq '.dataSource | length'

NB I’ve added --silent to curl to minimize its output

The above should count your data sources. I have 220!!

What data types do we have?

curl
--silent
--header "Authorization: Bearer ${TOKEN}" \
"https://www.googleapis.com/fitness/v1/users/me/dataSources" |\
 jq '.dataSource | map(.dataType.name) | unique'
[
  "com.google.active_minutes",
  "com.google.activity.segment",
  "com.google.calories.bmr",
  "com.google.calories.expended",
  "com.google.distance.delta",
  "com.google.heart_minutes",
  "com.google.height",
  "com.google.internal.goal",
  "com.google.speed",
  "com.google.step_count.cadence",
  "com.google.step_count.cumulative",
  "com.google.step_count.delta",
  "com.google.stride_model",
  "com.google.weight"
]

One of the prevalent data types is com.google.step_count.delta. We can determine whether we have data streams for this type with:

curl
--silent
--header "Authorization: Bearer ${TOKEN}" \
"https://www.googleapis.com/fitness/v1/users/me/dataSources" |\
 jq -r '.dataSource|map(select(.dataType.name=="com.google.step_count.delta"))|.[].dataStreamId'
...
derived:com.google.step_count.delta:com.google.android.gms:merge_step_deltas
...

NB You may have many results here but, the one we’re interested in is the above. This appears to be (!?) a catch-all data stream for this data type

This permits us to query our (times-series) data for a period:

LIMIT="500"
DATASOURCE="derived%3Acom.google.distance.delta%3Acom.google.android.gms%3Amerge_distance_delta"
START=$(date --date='TZ="America/Los_Angeles" 1 January 2020' +"%s%N")
END=$(date --date='TZ="America/Los_Angeles" 10 January 2020' +"%s%N")
DATASET="${START}-${END}"
curl \
--silent \
--header "Authorization: Bearer ${TOKEN}" \
"https://www.googleapis.com/fitness/v1/users/me/dataSources/${DATASOURCE}/datasets/${DATASET}?limit=${LIMIT}"

Let’s unpack that:

  • We’ll limit the number of results to the first 500
  • ${DATASOURCE} is the url-encoded version of derived:com.google.step_count.delta:com.google.android.gms:merge_step_deltas
  • ${DATASET} is a string containing the start date in (UNIX epoch) nanosconds, a hyphen, then the end date in nanoseconds
  • ${START} and ${END} are rather convoluted bash that converts 01-January-2020 and 10-January-2020 (time-zone Los Angeles) to nanoseconds

And, to pull the values out for all the returned points:

curl \
--silent \
--header "Authorization: Bearer ${TOKEN}" \
"https://www.googleapis.com/fitness/v1/users/me/dataSources/${DATASOURCE}/datasets/${DATASET}?limit=${LIMIT}" |\
jq .point[].value[0].fpVal

APIs Explorer

All of the above are available using Google’s excellent APIs Explorer.

Specifically, for the Google Fit APIs

NB You’ll recall while “Google Fit”, the underlying service is called fitness.

Let’s use Users.dataSources:list

APIs Explorer

If you plug the following values in the form, you should be able to “EXECUTE” it and get some data back:
userId me
dataTypeName com.google.step_count.delta

Optionally, under “Credentials”, “Google OAuth 2.0”, click “Show scopes” and choose only:

  • fitness.activity.read
  • fitness.body.read
  • fitness.location.read

Important uncheck API key.

JavaScript

Lastly, I’d not noticed this before but APIs Explorer provides templated JavaScript to permit you to run APIs.

JavaScript

You will need to generate a new OAuth client for this web application. Proceed as before but instead of choosing “Other”, select “Web application”. You will also need to ensure that the value of “Authorized JavaScript origins” reflects that you’ll be running (see below) the web server locally. So use http://localhost.

You’ll need to tweak the JavaScript before use by replacing "YOUR_CLIENT_ID" with the value of client_id from the newly-generated OAuth client:

    "installed": {
        "client_id": "[[REDACTED]].apps.googleusercontent.com",
        "project_id": "[[REDACTED]]",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_secret": "[[REDACTED]]",
        "redirect_uris": [
            "urn:ietf:wg:oauth:2.0:oob",
            "http://localhost"
        ]
    }
}

If present, remove the line:

gapi.client.setApiKey("YOUR_API_KEY");

I recommend saving the JavaScript in a file called index.html in a directory called html. Then, you can use Nginx to host this content easily:

PORT="80"
docker run \
--name nginx \
--rm --interactive --tty \
--publish=${PORT}:80 \
--volume=${PWD}/html:/usr/share/nginx/html:ro \
nginx

Then browse to http://localhost:${PORT} and you should be able to click the button to authorize and load and then execute to run the command.

Because output is to the console, you’ll need to open your browser’s developer tools to observe the results.

That’s all!