Golang Xiaomi Bluetooth Temperature|Humidity (LYWSD03MMC) 2nd Gen

- 6 minutes read - 1144 words

Well, this became more of an adventure that I’d originally wanted but, after learning some BLE and, with the help of others (Thanks Jonatha, JsBergbau), I’ve sample code that connects to 4 Xiaomi 2nd gen. Thermometers, subscribes to readings and publishes the data to MQTT. From there, I’m scraping it using Inuits MQTTGateway into Prometheus.

Repo: https://github.com/DazWilkin/gomijia2

Prometheus

Thanks|Credit:

Background

I’ve been playing around with ESPHome and blogged around my very positive experience ESPHome, MQTT, Prometheus and almost Cloud IoT. I’ve ordered a couple of ESP32-DevKitC and hope this will enable me to connect to Google Cloud IoT.

Meantime I bought 4x Xiaomi 2nd gen. Thermometers (LYWSD03MMC). These are Bluetooth Low-Energy (BLE) enabled and are supported by ESPHome. Awaiting the ESP32s, I decided to try to connect to the devices using Golang. The Mi Home app does not find these devices.

I discovered Jonathan’s gomijia but this doesn’t work with the 2nd gen. devices. IIUC it’s because the 2nd gen. devices don’t advertize temperature|humidity values (the orignal device clearly does).

After much Googling, I discovered JsBergbau’s Python solution and this works. Critically (!) this code uses a device handle (0x0038 more later) seemingly (!) to tell the device to publish temperature|humidity data on (a different) handle (0x0035).

With these samples as a guide and after a bunch more Googling about BLE, services, characteristics etc., I was able to get the code to work

Services|Characteristics|Handles

I won’t duplicate an overview of BLE here (see Useful) but, understanding a BLE device’s GATT is critical. The Linux gatttool is very useful. I found PayPay’s Golang-based gatt to be most useful. Here’s the GATT per the PayPal tool for a LYWSD03MMC device:

State: PoweredOn
Scanning...

Peripheral ID:A4:C1:38:7B:B0:1A, NAME:(LYWSD03MMC)
  Local Name        = LYWSD03MMC
  TX Power Level    = 0
  Manufacturer Data = []
  Service Data      = []

Connected
Service: 1800 (Generic Access)
Service: 1801 (Generic Attribute)
Service: 180a (Device Information)
Service: 180f (Battery Service)
Service: 000102030405060708090a0b0c0d1912

Service: ebe0ccb07a0a4b0c8a1a6ff2997da3a6
  Characteristic  ebe0ccc17a0a4b0c8a1a6ff2997da3a6
    properties    read notify 
    value         000000 | "\x00\x00\x00"
  Descriptor      2901 (Characteristic User Description)
    value         54656d706572617475726520616e642048756d696469 | "Temperature and Humidi"
  Descriptor      2902 (Client Characteristic Configuration)
    value         0000 | "\x00\x00"

Service: fe95
Service: 0000010000656c622e746f696d2e696d

Disconnected
Done

It’s also possible to enumerate a list of handles. The following (0x0036 and 0x0038) were the ones of interest to me. 0x0038 is the handle that JsBergbau’s writes to in order to trigger being able to subscribe to handle 0x0036 to read temperature|humidity data.

Reviewing the device’s GATT, I remain unsure which service characteristic corresponds to handle 0x0038. I know it has a UUID of 00002902-0000-1000-8000-00805f9b34fb but the suffix -0000-1000-8000-00805f9b34fb denotes that this is a standard BLE characteristic and the 00002902 is not unique in the device’s GATT.

Fortunately, the characteristic corresponding to 0x0036 does appear in the GATT as shown above (ebe0ccc1-7a0a-4b0c-8a1a-6ff2997da3a6) has a descriptor value of Temperature and Humidi[ty]

Handle Characteristic
0x0036 ebe0ccc1-7a0a-4b0c-8a1a-6ff2997da3a6
0x0038 00002902-0000-1000-8000-00805f9b34fb

Golang

The code uses currantlabs ble. Jonathan’s code uses a fork of this repo. I do not know what currantlabs is but the repo works. I found it to be challenging to understand the code particularly in enumerating devices, services, characteristics etc. I will focus the rest of this discussion on my (!?) understanding of how the code works. I’d value feedback on my misunderstandings.

First step is to create a Linux (BLE) Device (host) and then use this to connect to BLE devices:

host,err := linux.NewDevice()
ctx := ble.WithSigHandler(context.WithTimeout(context.Background(), 1*time.Minute))
device, err = host.Dial(ctx, ble.NewAddr("a4:c1:38:00:00:00"))

TODO: I’ve been meaning to run each device in its own Goroutine. When I tried this previously, I think I experienced concurrency problems. Will retry.

The next step is the magical step that writes a magical value 0x1000 to the magical handler 0x0038.

c:=ble.MustParse("00002902-0000-1000-8000-00805f9b34fb")
if p, err := bleClient.DiscoverProfile(true); err != nil {
	if u := p.Find(ble.NewCharacteristic(c)); u != nil {
		if err := bleClient.WriteCharacteristic(u.(*ble.Characteristic), []byte{0x10,0x00}, false); err != nil {
			log.Print(err)
		}
	}
}

NB I don’t know why this works!

The Golang library requires characteristics to be used instead of handles. Using one of the command-line tools, I was able to enumerate all the handles and their associated characteristics. From other code, I learned that 0x0038 is critical and 00002902-0000-1000-8000-00805f9b34fb is the characteristic that corresponds to this handle.

I learned (!?) that the library requires DiscoverProfile(true) to be called against the device before other commands will work correctly. After discovering the profile, the code then searches it for the characteristic we’re interested in and then it writes 0x1000 to it.

TODO: I’ve been meaning to duplicate this approach of Find(...) in the next section to avoid iterating over services and characteristics.

Now that the value is written to 0x0038, we can look for 0x0036 and subscribe to notifications from it.

This involves (currently but see TODO above) iterating over services that match the filter for the service (ebe0ccb07a0a4b0c8a1a6ff2997da3a6) shown in the GATT above that includes the characteristic (ebe0ccc17a0a4b0c8a1a6ff2997da3a6) that corresponds to "Temperature and Humidi[ty]".

Again, It appears a requirement to iterate over all the characteristics Descriptors even though I don’t reference these!

The key section is:

if (c.Property & ble.CharNotify) != 0 {
	if c.CCCD == nil {
		continue
	}
	if c.UUID.Equal(ble.MustParse("ebe0ccc1-7a0a-4b0c-8a1a-6ff2997da3a6")) {
		if err := d.Client.Subscribe(c, false, handlerPublisher(...)); err != nil {
			log.Print(err)
		}
    }
}

NB CCCD means Client Characteristic Configuration Descriptor and this must be non-nil to be able tos subscribe to the characteristic.

The initial if checks whether this characteristic supports notifications (ble.CharNotify)

If this characteristic is the one corresponding to 0x0036 (i.e. ebe0ccc1-7a0a-4b0c-8a1a-6ff2997da3a6) then we add a handler to its notifications.

The handler is trivial:

func(req []byte) {
    s := hex.EncodeToString(req)
    r, err := Unmarshall(req)
    if err != nil {
			log.Printf("Unable to unmarshal data (%s)", s)
		}
		...
	}
}

The Unmarshall step is also a black box to me. I don’t know where this is duplicated but I used Fabien’s bash script as a guide:

type Reading struct {
	Temperature float64
	Humidity    float64
}
func Unmarshall(req []byte) (*Reading, error) {
	// 00 01 02 03 04
	// T2 T1 HX ?? ??
	l := len(req)
	if l != 5 {
		log.Printf("[X] Expecting 5 bytes; got %d", l)
		return &Reading{}, fmt.Errorf("Expecting 5 bytes got %d", l)
	}
	// Temperature is stored little endian
	t := float64(int(binary.LittleEndian.Uint16(req[0:2]))) / 100.0
	h := float64(req[2]) / 100.0
	return &Reading{
		Temperature: t,
		Humidity:    h,
	}, nil
}

NB I don’t know what the other 2 bytes represent :-(

MQTT | MQTTGateway

The MQTT publishing is straightforward. I’m using Inuits MQTTGateway as described in the other post on ESPHome. The MQTTGateway subscribes to MQTT topics and converts these into Prometheus metrics. It’s then trivial to scrape these using Prometheus to produce the graph shown at the top of the post.

Useful