Real Talk on Matter Protocol: Software Evaluation

By: Valerii Haidarzhy, 24 May 2024

15 min read

Reading Time: 15 minutes

Explore Our Series Matter Technology

Dive deeper into Matter with our comprehensive series of articles. Each piece offers unique insights and in-depth analysis, guiding you through different facets of the topic. Whether you’re a beginner or an expert, this series aims to provide valuable information that will enhance your understanding and spark further interest. Explore the articles listed below:

At Sirin Software, our approach to exploring the Matter protocol is straightforward and detailed. We dig deep to understand every aspect. As we already discussed in “Evaluating the Matter Protocol: First Steps,” our interest in Matter comes from seeing that it often doesn’t meet the hype. Many are excited to update their devices to Matter, only to face issues that leave them frustrated. The problem is, that there’s a lack of detailed sharing about what goes wrong, how it happens, and how to fix it. Here, Our goal is to get a real sense of what Matter can and can’t do. We’re looking for real answers that can help businesses make informed decisions. This means not just playing with technology but analyzing it across various platforms to truly understand its potential and limitations.

Approach

Our journey into Matter wasn’t a leap in the dark. We meticulously examined documentation from ESP and official Matter sources and combed through community forums, aiming to grasp the full scope and boundaries of Matter’s functionality. Forums proved especially valuable, offering insights into practical challenges and solutions encountered by real users.

Hardware

Our choice of ESP32C3-M1 and ESP32H2 boards was aimed at efficiently prototyping and testing the Matter protocol across different network types. The ESP32C3-M1, with its BLE/Wi-Fi capabilities, was ideal for Wi-Fi trials, providing a reliable platform for initial testing phases.

The ESP32H2, equipped with BLE/Thread capabilities, gave us the necessary tools to delve into the Thread network, enabling a complete evaluation of Matter’s performance in varied environments.

The decision to focus on these ESP boards for public discussion is not merely a case of convenience. Our extensive research initially contained larger players like NXP, Synaptics, Murata, and innovative startups like Alif Semiconductor, diving into their sophisticated ecosystems. However, much of this work is covered under non-disclosure agreements (NDAs), limiting our ability to share detailed findings publicly.

Therefore, we pivoted to ESP solutions as a case study to transparently demonstrate our capabilities and insights into the Matter protocol. Using ESP as a focal point allows us to present a simplified yet effective overview of our in-depth research with the big names. We also recommend starting with ESP to reckon the integration and performance of Matter before scaling up to more complex solutions offered by the larger vendors.

OS

We picked Ubuntu 22.04 because it’s stable, familiar, and well-supported. It’s a solid choice for Linux development, especially for Matter, and its widespread use means getting help is easy. We value stability, and this OS minimizes system issues, so we can focus on development. With its long-term support, we won’t need frequent updates or system changes, allowing us to stay focused on our work.

Google Nest

We chose Google Nest for testing because it’s a common system that many people use in their homes. It was important for us to see if Matter could actually work in the real world, not just in theory. Google Nest helped us test Matter’s integration with other devices and see where it might fail or struggle. We needed to know if Matter could handle the demands of a popular smart home platform and where the rough edges were. This wasn’t about making Matter look good in ideal conditions, but more about finding its weak spots in a typical home setup.

Google Home Developer Console

Dealing with the bureaucracy of development and certification is tedious. Devices must be certified to integrate with Google Home, which requires engaging with the Google Home Developer Console during development. This process can seem like endless hoop-jumping, but it’s vital for ensuring our devices meet industry standards and work seamlessly with established platforms.

SDK Support

Dealing with SDK support was a real test of patience. It’s not comprehensive; for example, support for the ESP32H2 was restricted to certain applications, like the Lighting App. However, we saw this as a chance to get creative, using other examples like Light Switch and Light from Espressif’s SDK for Matter. This situation pushed us to flex our technical muscles and adapt to the tools at hand.

Evaluating Matter Over WiFi

For our WiFi testing of the Matter protocol, we chose ESP32C3 boards as our hardware platform. These boards stood out for their dual capability of handling both Bluetooth Low Energy (BLE) and WiFi, crucial for Matter’s communication requirements. The ESP32C3 isn’t just affordable; it’s also powerful enough to run the complex processes involved in Matter communication.

Our testing aimed to mimic real-life conditions as closely as possible. We set up scenarios where two Matter-enabled devices, using the ESP32C3 boards, could communicate. This setup wasn’t just about checking if they could send and receive messages. We wanted to see how these devices would perform in an environment similar to what they would encounter in actual use, such as a smart home or office space.

This involved creating a network where these devices could interact with each other and other network elements, just as they would in a real-world smart environment. We looked at how these interactions happen over WiFi, scrutinizing the connection stability, response time, and overall reliability.

Setting Up the Software Environment and Commissioning with the Chip Tool

Establishing the correct software environment was our initial task. We sourced the esp-idf and esp-matter SDKs directly from Espressif’s documentation, ensuring we had the official, most up-to-date tools for our development. In more straightforward terms, these SDKs are like specialized toolkits that helped us to build and test Matter functionalities specifically for the ESP32C3’s capabilities. The esp-idf SDK lays the groundwork, providing the basic operating environment and programming functionalities for the ESP32C3. The esp-matter SDK brings in the Matter-specific elements, allowing us to develop applications that can communicate using the Matter protocol.

Initialization and Pairing

– Initiating Commissioning: We started with the ‘chip-tool interactive start‘ command, a step to begin the commissioning process. This command prepares the Chip Tool to interact with our devices.

– Pairing Devices: The pairing command syntax was as follows:

pairing ble-wifi 12345 <ssid> <passphrase> 20202021 3840

where:

12345 – served as the node ID, which could be any arbitrary value.

20202021 – was our chosen pin code for secure pairing.

3840 – was the discriminator, needed in distinguishing multiple similar devices during the commissioning process. The discriminator is mandatory in Matter. It helps in provisioning the correct device, especially when multiple identical devices are connected simultaneously. Its default value is 3840, but it can be modified through menuconfig or directly in the device application console.

Commissioning Two Devices:

We commissioned two ESP32C3 boards – one with the Lighting Switch application and the other with the Lighting Example. To ensure unique identification during commissioning, we assigned different discriminators to each board:

For the Switch:

chip-tool interactive start
pairing ble-wifi 0x7283 <ssid> <passphrase> 20202021 3840

For the Light:

chip-tool interactive start
pairing ble-wifi 0x5164 <ssid> <passphrase> 20202021 3841

Configuring Discriminator on Device Console: If commissioning multiple devices, they need distinct discriminators. This can be set via the device console using commands like:

matter config discriminator f00
matter config discriminator 3841

The console confirms the change with the message ‘Setup discriminator set to: 3841.’\

Binding Devices and Controlling Light on ESP32C3

With both ESP32C3 boards commissioned, the next step was to establish a binding between them, enabling the Light Switch to control the Lighting Example. We used the chip-tool to write Access Control List (ACL) and binding entries. This is to define which devices in our network could control others:

accesscontrol write acl ‘[{“privilege”: 5, “authMode”: 2, “subjects”: [112233, 29315], “targets”: null}]’ 0x5164 0x0
binding write binding ‘[{“node”:20836, “endpoint”:1, “cluster”:6}]’ 0x7283 0x1

This command line setup allowed the Light Switch (node 20836) to control the Lighting Example (node 5164).

With the binding in place, we were now able to control the Lighting Example from the Light Switch using the onoff cluster command over the Chip Tool.

– Toggling the Light via Chip Tool: We issued the following command to toggle the light’s state:

onoff toggle 0x5164 0x1

This command triggered an action in the Lighting Example to change its state (on or off).

For a more hands-on experience, we also used the command directly on the Light Switch’s console:

matter esp client invoke 0x1 0x5164 0x1 0x6 0x2

This allowed us to manually trigger the onoff cluster command, offering insight into Matter’s internal command structure and device response behavior.

Integration with Google Home and Advanced Control Mechanisms

Expanding the scope of our Matter protocol exploration, we integrated the ESP32C3 devices with Google Home, specifically focusing on commissioning to the Google Nest Router Wi-Fi network. We needed this step in assessing Matter’s compatibility with prevalent smart home platforms.

As we started the application on the ESP32C3 boards, they began advertising over BLE. It makes the devices discoverable to the Google Home app. Using the Google Home app, we added a new device, which then scanned for the Matter device’s advertising packets. Upon detection, the app prompted us to scan a QR code. The QR code, needed for pairing, was generated by the device and appeared in the start logs:

I (1367) chip[SVR]: SetupQRCode: [MT:Y.K9042C00KA0648G00]
I (1377) chip[SVR]: https://project-chip.github.io/connectedhomeip/qrcode.html?data=MT%3AY.K9042C00KA0648G00

When we scanned the QR code with the Google Home app’s camera, it quickly paired our device with the Google Home system.

After connecting with Google Home, we could control the Lighting Example using its on-off feature through the Google Home app.

Using Google Home to turn the lights on and off gave us a real sense of how people will use Matter devices in their homes. This testing phase showed that Matter could easily be part of larger smart home setups. Being able to manage Matter devices with a well-known app like Google Home shows that the Matter protocol is designed with users in mind and can be adapted to fit into various smart home environments.

AWS IoT-Matter Cluster Architecture and MQTT Client Setup

In connecting the Matter protocol with AWS IoT services, we selected the ESP SDK, partly because it works well with the aws-iot-sdk, among other considerations. Our selection was motivated by the desire for a system that was both manageable and scalable, capable of integrating smoothly with coreJSON and AWS IoT Device Shadow libraries. Although the aws-iot-sdk offers a full suite of features, including an MQTT client, it also includes extra components that we didn’t need for our specific project. ESP SDK strikes the right balance. It’s lightweight enough not to overload our system yet powerful enough to handle our requirements for AWS IoT integration effectively. This approach allowed us to access the features necessary for our work without being weighed down by additional libraries.

Configuring MQTT Client

Our approach to setting up the MQTT client was focused on establishing a secure and efficient connection to AWS IoT. The esp_mqtt_client_config_t structure in mqtt_app.c was configured to fulfill our specific communication needs with AWS IoT. Here’s a closer look at the configuration:

esp_mqtt_client_config_t mqtt_cfg = {
.broker = {
.address = {
.uri = MQTT_URL,
.transport = MQTT_TRANSPORT_UNKNOWN,
},
.verification = {
.certificate = MQTT_CA_CERT,
},
},
.credentials = {
.client_id = THING_NAME,
.authentication = {
.certificate = MQTT_CERT,
.key = MQTT_KEY,
},
},
.session = {
.keepalive = 60,
.protocol_ver = MQTT_PROTOCOL_V_3_1_1,
},
.buffer = {
.size = 1538,
},
// Other configuration…
};

This configuration included details like the MQTT broker address, credentials, session settings, buffer sizes, and network parameters:

Broker Configuration: Ensured a secure connection to the MQTT broker with the proper URI and transport type.

Credential Management: Specified client authentication details, including certificates and keys, for secure communication.

Session Settings: Defined parameters like keepalive intervals and MQTT protocol versions for maintaining a stable connection.

MQTT Client Event Handling

We implemented a function mqtt_event_handler to manage various MQTT events such as connection, subscription, data reception, and disconnection. This function was responsible for handling the MQTT lifecycle and responding to different MQTT-related activities. The core of this function looked like this:

static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data) {
// Event handling logic
switch ((esp_mqtt_event_id_t)event_id) {
case MQTT_EVENT_CONNECTED:
// Subscribe to the AWS shadow topic upon connection
break;
case MQTT_EVENT_DATA:
// Handle incoming data, possibly parse Shadow updates
break;
// Handle other events like DISCONNECTED, SUBSCRIBED, PUBLISHED…
}
}

This event handler is needed to maintain a stable and responsive MQTT connection with AWS IoT:

Handling Connection Events: Managed actions upon successful connection, like subscribing to the desired AWS IoT topic.

Processing Incoming Data: Dealt with data received from AWS IoT, which could include Shadow updates or other MQTT messages.

Once the MQTT client was set up and event handling was in place, the next steps were to connect the MQTT client to AWS and subscribe to a specific MQTT topic to communicate with the AWS IoT Device Shadow. To synchronize our device’s state with AWS IoT, we created and sent shadow updates to a specific AWS topic:

bool update_device_shadow(bool led_on) {
const char *valve_shadow = // Shadow JSON structure
// Creating the shadow update JSON string
char *shadow_update = malloc(shadow_length);
// Populate the shadow JSON with the LED state
sprintf(shadow_update, valve_shadow, led_on ? “ON” : “OFF”);
// Send the shadow update to AWS IoT
bool ret = shadow_send_json(THING_NAME, shadow_update);
free(shadow_update);
return ret;
}

This function:

Generated Shadow JSON: Created a JSON structure representing the current state of our device (e.g., LED state).

Sent Shadow Update: Published the JSON to the appropriate AWS IoT Shadow topic for synchronization.

LED State Control via AWS IoT Shadow and Google Home Integration

To manage the LED state of our ESP32C3 devices through AWS IoT, we utilized the ‘desired’ shadow mechanism. This method allowed us to send state commands from AWS IoT, which were then executed on the device.

We created a function update_device_shadow that constructs and sends a shadow update JSON string to AWS IoT. This string included the desired LED state (“ON” or “OFF”). When the LED state changed from the Google Home dashboard, this function was triggered to synchronize the state with AWS IoT.

Our shadow_send_json function handled the actual sending of the shadow update to AWS IoT. It composed the MQTT topic for the shadow update and published the JSON message:

bool shadow_send_json(const *serial, char *shadow) {
// Compose and publish MQTT message
}

Upon receiving a ‘desired’ state update from AWS IoT, we parsed the JSON payload to extract the new LED state. This was done in the shadow_handle_data function within the MQTT event handler for MQTT_EVENT_DATA.

bool shadow_handle_data(esp_mqtt_event_handle_t event) {
// Parse ‘desired’ state and call callback to update LED state
}

After that, we implemented a mechanism to trigger callbacks and handle MQTT events efficiently. In our main application file (AppTask.c), we registered a callback function to handle MQTT events related to the LED state:

mqtt_register_event_callback(AppTask::MQTTEventHandler);

The MQTTEventHandler function in AppTask.cpp was designed to handle these events. It processed the parsed ‘desired’ state and triggered a new event to update the on/off cluster in the LightingActionEventHandler:

void AppTask::MQTTEventHandler(uint8_t action, uint8_t data) {
// Trigger new event for LED state update
}

The LightingActionEventHandler then updated the actual LED state on the ESP32C3 board and reflected this change in the Google Home dashboard. This was needed to maintain consistency between the physical device state and its representation in the Google Home interface.

The integration with Google Home allowed us to control the LED state directly from the dashboard. Any change in the LED state, whether triggered from Google Home or AWS IoT, was synchronized across platforms.

Evaluating Matter Over Thread

In our quest for a reliable and versatile platform to evaluate the Matter protocol over a Thread network, we zeroed in on the ESP32H2. This decision was driven by its robust wireless capabilities, compatibility with Thread, and an optimal balance of performance and cost. If anyone runs into problems while setting up the Thread End device, it’s a good idea to look for help online. The GitHub issue link http://github.com/espressif/esp-matter/issues/465 is a valuable resource for solving specific problems related to this device.

We continued using the Chip-tool from the setup phase to the control phase. This tool is excellent for executing precise commands and offers in-depth control, making it invaluable for managing complex settings and adjustments. Additionally, integrating the ESP32H2 with Google Home simplifies its control, making the chip’s functionalities easy to use for a broader audience, not just tech experts.

Initially, we tried the official lighting example from ‘connectedhomeip’, but we faced problems, especially with reading the destination MAC address. So, we switched to a different lighting example from the esp-matter/example folder. Our setup included both the ESP32C3 and ESP32H2. We configured the ESP32C3 with the ot_br example app and the ESP32H2 with the ot_rcp example app.

We had an issue with the ot_br board rebooting repeatedly and fixed it by switching to GPIO5/GPIO6. This change was needed for the Thread Border Router (TBR) to work correctly. Connecting the TBR to the Wi-Fi network using

wifi connect -s <ssid> -p <password>

and preparing a dataset with the network key and PAN ID for the Thread End device, following the ot_cli example, were our next steps in creating a reliable network.

Extending the Setup and Network Configuration

We added the ESP32H2 to the TBR thread network using the ot_cli example app. This included running a TCP server-client connection, which was needed to check how well the network worked. We set the network’s parameters, including the Network Name, PAN ID, Extended PAN ID, Network Channel, Master key, and Pre-Shared Key. We followed the default settings listed in the ‘sdkconfig’ for both the ‘ot_br’ and ‘lighting’ examples.

Steps for Commissioning and Observing Logs

We followed a specific process for commissioning the ESP32H2. This involved retrieving the active dataset from the TBR console using

$ ot-ctl> dataset active -x

Then, we used a series of commands in the Chip-tool for commissioning. An example of such a command is

$ ./out/debug/chip-tool pairing ble-thread 12345 hex:<operational-dataset> 20202021 3841.

Here, <operational-dataset> is the active dataset from the TBR, 12345 is the node ID, 20202021 is the pincode, and 3841 is the discriminator. It’s important to make sure that the device running Chip-tool is on the same Wi-Fi network as the TBR.

Using Chip-tool in interactive mode is helpful, particularly if the first commissioning attempt fails but succeeds on a subsequent try, as it allows the Chip-tool to connect to the commissioned devices. The TBR logs are very informative about the commissioning process. For instance, a log entry like

I (7026401) OPENTHREAD: Multicast listener add: FF05::2

signals a successful addition of a multicast listener. Initially, we observed some commissioning failures, evident from log entries like

I (757963) chip[NP]: Reverting Thread operational dataset.

However, the process was successful after the second attempt, as indicated by entries like

I(758093) OPENTHREAD:[I] Notifier——: StateChanged.

Commissioning the Lighting Example with Google Home

We took on the challenge of integrating the Light example with the Google Nest Router, turning it into our Thread Border Router (TBR) and managing everything through Google Home. This task was similar to how we worked with the ESP32C3, requiring us to register our Thread End device on the Google Home Developer Console. We found out that we didn’t need to tweak the network settings this time because we had already registered the device in the Google project earlier. It’s important, though, to turn on Developer Options on your Android device for this to work. Plus, if you’ve already got a Matter device set up in Google Home, you’ll need to use a new discriminator to successfully add another one.

A key part of this process was using the getPreferredCredentials function from the Google Home SDK. This meant we had to make our own mobile app to add Matter devices into a Thread Network.

Exploring AWS IoT Integration with Matter over Thread

The integration of AWS IoT with Matter over Thread presents a more complex scenario. Directly connecting a Thread end device to AWS IoT via MQTT without Wi-Fi is not typically viable due to MQTT’s reliance on IP-based networks like Wi-Fi, Ethernet, or cellular data. However, employing a Thread Border Router (TBR) that bridges the Thread network with an IP-based network offers a feasible solution. This TBR can translate messages between Thread and MQTT, enabling communication with AWS IoT Core.

A potential approach involves using a TBR with both Thread and IP network connectivity, configuring it to translate between Thread and MQTT over IP. The TBR can then relay messages from the Thread network to AWS IoT using MQTT over IP.

An alternative method could be the use of MQTT-SN (MQTT for Sensor Networks), designed for sensor networks with constraints like low bandwidth and intermittent connectivity. This would involve setting up an MQTT-SN broker, implementing an MQTT-SN client on Thread end devices, and establishing an MQTT-SN Gateway to act as a bridge. This gateway would provide communication between MQTT-SN devices and the MQTT broker, translating messages for devices in the Thread network.

While we are yet to implement communication with AWS IoT over MQTT and the commissioning of a custom TBR (ESP32C3+ESP32H2) with AWS IoT integration, the exploration into these advanced integrations holds promising prospects for controlling LED both from AWS and Google Home.

Hands-On Review of Matter with ESP Boards

Working with ESP boards and the Matter protocol was a mixed bag. Initially, the journey was smooth; the ESP32C3 board, in particular, had plenty of examples in the SDK, making the start relatively straightforward. However, when we shifted to the ESP32H2 board, the resources thinned out, forcing us to revert to an older software version (5.1.1) due to compatibility issues. Another hiccup was with the Lighting application from the Matter SDK; it simply wouldn’t cooperate with any Thread network, pushing us to find an alternative example to proceed.

A notable point for developers: if you’re used to Windows, you might find a slight mismatch here, as the ESP and Matter ecosystem leans more towards Linux and Mac systems, offering a smoother operational experience.

Community Engagement and Support

The Matter community is vibrant and supportive, with developers actively participating in discussions. However, given the novelty of the Matter protocol, we stumbled upon uncharted territories where clear answers were scarce, likely due to the lack of extensive real-world testing.

Dealing with Example Applications

The provided example applications were user-friendly and flexible, mostly written in C++, though working with C was also feasible. A word of caution: downloading the esp-matter and esp-idf SDKs is time-consuming and storage-intensive. Additionally, these applications are resource-heavy, occasionally slowing down our development environment, like VS Code, during the build process.

Integration into the Matter Ecosystem

Integrating our devices with the Matter ecosystem, particularly with Google Nest, was straightforward, aided by comprehensive guides. Setting up a new project in Google’s Developer Console and registering our device was all it took to connect with Google Nest’s Wi-Fi and Thread networks. While our integration with Google Home was seamless using a Google Pixel, the experience might differ with other devices.

Initially, we anticipated needing specific network configurations for the ESP32H2 Thread End device with Google Nest. However, it turned out that the setup process mirrored that of a Matter-over-Wi-Fi board, a realization dawned upon consulting Google’s documentation. It’s important to note that fully integrating a product requires device certification, a step we haven’t delved into yet. Furthermore, for custom Thread Border Router integration with the Matter network, developing a proprietary mobile app might be necessary to obtain the correct credentials.

Google’s resources, including their codelabs, provide substantial guidance for developing mobile apps to connect and test Matter devices. The CHIP-tool from Matter has also been invaluable, significantly helping in our testing, setup, and management of Matter devices.

Sirin Software: Simplifying Advanced Technology

At Sirin Software, we tackle new technologies, focusing on real-world applications. With experience across various platforms, we understand the technical nuances and provide clear, actionable insights. Our involvement in tech communities enhances our knowledge and keeps us updated with industry trends. We’re not just about solving today’s problems – we’re also looking ahead, preparing for future technological shifts. One such forward-looking project has been our successful development of a Matter-enabled smart lighting solution that seamlessly integrates with any Smart Home ecosystem. Partnering with us means getting a team that’s committed to clear communication, practical solutions, and forward-thinking strategies, helping you stay ahead in the evolving tech landscape.

Latest articles