Thibault Maekelbergh

🎛 Building an ESP32 MQTT button box

How this project started out

I can't quite remember when, but a couple of months after buying my 3D printer I was exploring Thingiverse like crazy. In a crazy crusade across Thingiverse I randomly discovered Adafruit's Mini UNTZtrument.

The gist of the project already sounded fun: making a MIDI instrument with an Arduino. But I saw other potential. At this time I was already becoming more accustomed with advanced electronics and advanced ESPHome and building my own devices using ESP8266 and 3D printing. To me this project looked more like a great fit to build my own ESP based button controller (or "macropad") much like you would see in futuristic space ship consoles: one controller to trigger a lot of actions, acting like an extension of my hand, toggling home automation flows / triggers / states like a true spaceship operator.

So I set out and started this rather long print for the enclosure...

The journey towards finding the right hardware

OK. So there were a couple of things I needed here:

  • A wifi connected microcontroller to interface with Home Assistant or other devices on my LAN network

  • 16 push buttons for the bottom part of the controller, to trigger different actions

  • 4 components with a round shaft for the top part of the controller, options here were still open.

At that time I always hastily defaulted to an ESP8266 (Wemos D1 Mini) for anything smart home that I made myself. So with my soldering iron in hand, I set out to my desk to wire up some components on a protoboard. But the first obstacle already found its way on my path here.

How can I wire up 16 buttons to an ESP8266 that has only 11 GPIO inputs?

It took a lot of research because it was a new concept to me but there is a concept called button matrix that allows you to wire up a lot of buttons using a limited amount of pins. Curious to try this out, I started soldering buttons on protoboard with the best understanding of button matrices in the back of my head:

A 3 by 4 grid of buttons, lined up on protoboard to start prototyping the build

And I failed. It didn't work as it should, the concept baffled me and as to this day, I still kind of don't know how it works but I believe it identifies the buttons in the matrix by measuring a variable resistance that is mapped to each button.

So there it was: at this point I gave up. I could not find the joy/strenght/time anymore to continue with my controller, so the enclosure, a flaky button protoboard and ESP8266 landed in my "half-assed projects I gave up on" box.

Until, over half a year later my friends got me excited again to finish this. But this time I had much more newly acquired knowledge.

ESP32 to the rescue

My friend Toon told me about ESP32 and in an impulse shopping spree I ordered a few a couple of months back. The one thing I had remembered him saying was that ESP32 had a lot of more GPIO pins. This was my solution! All that was required was to swap my trusty ESP8266 with a greenfield ESP32. Additional benefit: I had multiple ADC pins if I wanted more analog inputs and multiple GNDs (Wemos D1 Mini only has one).

Suddenly it all made sense to me. I had long prototyping sessions, soldering nights and diagram sketching headaches but eventually landed on this bill-of-materials:

  • Microcontroller: NodeMCU ESP32S

  • Top left: Momentary latching push button

  • Middle left: 10K Potentiometer

  • Right left: Single WS2812 LED cut from a strip

  • Top right: Rotary encoder

  • 16 large square push buttons

Wiring up the hardware

Connecting the inputs to the ESP32 is straight forward: there are 17 buttons to connect to digital GPIO inputs, 1 potentiometer to connect to an ADC input, 1 LED to connect to a digital GPIO input and then the rotary encoder which connects to 1 GPIO input for it's rotary movement and 1 for the press which acts as a button.

In order to keep this project somewhat modular so I could swap out the ESP32 later, I decided to add headers to the protoboard and then solder screw terminals for the VCC/GND lines required for the inputs.

ESP32 soldered to protoboard with some male headers to connect inputs to.

Then I started to wire up GND wires for each input:

GND wires wired up to each individual push button

Then it was ready for its first POC by wiring up some buttons to the screw terminals and inputs to validate the button presses are being recognized by the ESP32 and my wiring was correct.

First try, connecting push buttons to the ESP32 via dupont wire and screw terminals

Once I verified everything was OK, I wired the whole thing up in a somewhat tedious manner. My goal here is to not open the case up unless I really need to so this wiring works fine for now. A later revision would probably have some neater wiring or wire routing.

The WS2812 LED requires 5V to function. Luckily the ESP32 (and the ESP8266) have a 5V (VIN) pin which can be used to either supply 5V to the ESP32 and power it, or in my case: pass through the 5V received from the micro USB port to other components, like the LED.

The ESP32 controller all wired up, ready to be placed in its enclosure

And after this: ready for enclosing it in the box. I still had some final prints to do (keycaps and a LED diffuser) but it was ready for its first real use via ESPHome and Node-RED!

The first finished version of the controller in its enclosure

Installing the correct firmware

ESPHome makes writing/configuring firmware for ESP-based MCU's dead easy. There really isn't much to write here that the ESPHome docs don't already cover.

I decided to publish each button to a esphome/${device_name}/button topic with the number ID in the payload and then follow that topic format for the rotary encoder and potentiometer. For the LED, I am not exposing this to HA, MQTT or Node-RED since I just wanted it to act as a status LED for the controller's state.

You could just copy my config below, but I suggest looking into ESPHome docs for a bit. I'm using advanced concepts like lambdas & packages here to make automations easier.

yaml
substitutions:
device_name: esp32_mqtt_button_box
friendly_name: "ESP32 MQTT Button Box"
packages:
base_config: !include ../../common/base.yaml
wifi: !include ../../common/wifi.yaml
mqtt: !include ../../common/mqtt.yaml
esphome:
platform: ESP32
board: nodemcu-32s
on_boot:
then:
- sensor.template.publish:
id: mode
state: 0
- light.turn_on:
id: color_led
brightness: 0.5
effect: "Breathe"
red: 100%
green: 100%
blue: 100%
- light.turn_off:
id: color_led
- light.turn_on:
id: color_led
brightness: 0.5
red: 100%
green: 100%
blue: 100%
ota:
binary_sensor:
- platform: gpio
name: "${friendly_name} — Rotary Encoder Button"
internal: true
pin:
number: GPIO21
mode: INPUT_PULLUP
inverted: true
on_click:
min_length: 1000ms
max_length: 2000ms
then:
- sensor.template.publish:
id: mode
state: !lambda |-
auto call = id(color_led).turn_on();
call.set_transition_length(1000);
call.set_rgb(1.0, 1.0, 1.0);
call.perform();
return 0;
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "rotary_encoder"
- sensor.template.publish:
id: mode
state: !lambda |-
auto call = id(color_led).turn_on();
call.set_transition_length(1000);
if (id(mode).state == 0) {
call.set_rgb(1.0, 1.0, 0.0);
call.perform();
return 1;
}
if (id(mode).state == 1) {
call.set_rgb(1.0, 0.0, 1.0);
call.perform();
return 2;
}
if (id(mode).state == 2) {
call.set_rgb(0.0, 1.0, 1.0);
call.perform();
return 3;
}
if (id(mode).state == 3) {
call.set_rgb(0.5, 1.0, 0.8);
call.perform();
return 4;
}
if (id(mode).state == 4) {
call.set_rgb(1.0, 1.0, 1.0);
call.perform();
return 0;
}
- platform: gpio
name: "${friendly_name} — Top Button"
pin:
number: GPIO13
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "top"
- platform: gpio
name: "${friendly_name} — Button 1"
pin:
number: GPIO33
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "1"
- platform: gpio
name: "${friendly_name} — Button 2"
pin:
number: GPIO32
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "2"
- platform: gpio
name: "${friendly_name} — Button 3"
pin:
number: GPIO14
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "3"
- platform: gpio
name: "${friendly_name} — Button 4"
pin:
number: GPIO18
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "4"
- platform: gpio
name: "${friendly_name} — Button 5"
pin:
number: GPIO26
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "5"
- platform: gpio
name: "${friendly_name} — Button 6"
pin:
number: GPIO25
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "6"
- platform: gpio
name: "${friendly_name} — Button 7"
pin:
number: GPIO15
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "7"
- platform: gpio
name: "${friendly_name} — Button 8"
pin:
number: GPIO5
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "8"
- platform: gpio
name: "${friendly_name} — Button 9"
pin:
number: GPIO12
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "9"
- platform: gpio
name: "${friendly_name} — Button 10"
pin:
number: GPIO27
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "10"
- platform: gpio
name: "${friendly_name} — Button 11"
pin:
number: GPIO16
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "11"
- platform: gpio
name: "${friendly_name} — Button 12"
pin:
number: GPIO17
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "12"
- platform: gpio
name: "${friendly_name} — Button 13"
pin:
number: GPIO4
mode: INPUT_PULLUP
inverted: true
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "13"
light:
- platform: fastled_clockless
id: color_led
name: "${friendly_name} — Color LED"
internal: true
pin: GPIO19
chipset: WS2811
num_leds: 1
rgb_order: GRB
effects:
- automation:
name: "Breathe"
sequence:
- delay: 1s
- light.dim_relative:
id: color_led
relative_brightness: 50%
transition_length: 4s
- delay: 1s
- light.dim_relative:
id: color_led
relative_brightness: -50%
transition_length: 4s
sensor:
- platform: template
name: "${friendly_name} - Mode"
id: mode
icon: mdi:select-all
- platform: adc
name: "${friendly_name} - Potentiometer"
pin: GPIO34
update_interval: 10s
attenuation: 11db
- platform: rotary_encoder
id: rot_enc
name: "${friendly_name} — Rotary Encoder"
pin_a: GPIO23
pin_b: GPIO22
on_clockwise:
- mqtt.publish:
topic: "esphome/${device_name}/rotary_encoder/cw"
payload: !lambda |-
return to_string(id(rot_enc).state);
on_anticlockwise:
- mqtt.publish:
topic: "esphome/${device_name}/rotary_encoder/ccw"
payload: !lambda |-
return to_string(id(rot_enc).state);

Integrating MQTT, Node-RED and the controller

Previous projects all used the native Home Assistant integration for ESPHome and that was fine for simple sensors that just exposed a value. This controller however performs a lot of interaction, with all these inputs.

I already had Node-RED running for some other automations. To make all the different button mappings clear to me I decided the best way would be to post MQTT messages via the controller and the visualise them in Node-RED. This also makes it a lot easier to remap button functionality and keep the controller agnostic of what 'frontend' is using it.

Node-RED flows listening to MQTT topics and triggering actions like controlling lights, playing music or activating scenes

Adding advanced features

Even 17 buttons couldn't limit me! I wanted a mode selector that would remap the buttons when under a certain mode. Pressing the first button under mode 1 (the default) for example would turn all the lights off, but then pressing that same button in a different mode could perform a different action.

The rotary encoder's button press was the most logical choice for this. Pressing it would then just iterate over the modes and keep a global reference on the ESP32 as to what mode was currently selected. Then all the buttons could use that mode. I mapped a long press on the button to reset the mode to 0.

yaml
esphome:
on_boot:
then:
- sensor.template.publish:
id: mode
state: 0
binary_sensor:
- platform: gpio
name: "${friendly_name} — Rotary Encoder Button"
internal: true
pin:
number: GPIO21
mode: INPUT_PULLUP
inverted: true
on_click:
min_length: 1000ms
max_length: 2000ms
then:
- sensor.template.publish:
id: mode
state: !lambda |-
return 0;
on_press:
- mqtt.publish:
topic: "esphome/${device_name}/button"
payload: "rotary_encoder"
- sensor.template.publish:
id: mode
state: !lambda |-
if (id(mode).state == 0) {
return 1;
}
if (id(mode).state == 1) {
return 2;
}
if (id(mode).state == 2) {
return 3;
}
if (id(mode).state == 3) {
return 4;
}
if (id(mode).state == 4) {
return 0;
}

And then in the lambda's I could integrate this to set a different color on the LED (see the full config).

Persisting the mode was not necessary for me. Technically you could write this to the limited amount of flash memory on the ESP32, but I'd rather reset the mode to 0 when the ESP boots.

Conclusion

Tackling this project in my freetime has learned me a lot. I would do a lot of stuff differently next time I start on something like this:

  • Instead of wiring individual GND wires to each button, just connect all grounds of the buttons and have 1 wire for the bottom part connected to the ESP

  • Don't do hasty prints for something like this. Take the time to print at a higher resolution, because now removing the hot glue to move to a newly printed case is just not worth the effort

  • ESP32 is still limited in GPIO input, I would need to look into some sort of GPIO expander IC / addon, but for now what I have will do.

  • Potentiometers are really not that great of a component for projects like this, instead I would've just used 2 rotary encoders.

  • I would probably go for Cherry MX keys and design my own case, since keycaps for true Cherry MX macropads are more easily available and my own printed keycaps for push buttons are a bit flaky. Or buy the Adafruit Trellis.

Nevertheless I had soooo much fun creating this. I learned even more about electronics, Node-RED and home automation with this project and surely see it evolving even more in the future, maybe even for a v2 of the controller!

But for now this looks great on my desk:

Finished controller with 3d printed keycaps and LED diffuser