Bluetooth: keys, an HMI link and a speaker
A touchscreen that types, one that tells a host what is happening on it, a speaker a phone plays through - or a radio that plays to a Bluetooth speaker.
Four roles, and a board has one of them
A keyboard (HID). A button sends a shortcut, types a piece of text or presses a media key. To a PC or a phone the board is a Bluetooth keyboard: nothing is installed, and whatever is in front on the host gets the key. This is the macro pad, the stream deck and the media remote.
An HMI link. A widget tells a host what happened to it - pressed, clicked, moved, and to what value - as a line of text, and the host sets the design's variables by sending lines back. It is how a serial HMI panel has always worked; here the wire is Bluetooth's serial service.
A speaker (A2DP). A phone or a PC pairs with the board and plays through its audio output. The track's title and artist come with the sound, the phone's volume buttons work, and the board can ask the phone to pause or skip. More below.
Plays to a Bluetooth speaker (A2DP). The other way round: the board finds a Bluetooth speaker and plays to it - a station, a card's files, a sound. An action chooses, while it runs, between the speaker and the board's own output. More below.
A board is one of them, never two. To a host they are different devices: a keyboard is paired with the operating system, which then owns the connection, an HMI is opened by a program, and the two audio roles are a different kind of Bluetooth altogether. The choice is four cards under Hardware > Bluetooth, and it can be changed at any time; what the design holds for another role is kept, named in the Problems list, and does nothing.
Bluetooth is written for the Arduino export, and needs a chip that has the radio: an ESP32, S3, C3, C5 or C6. An S2 has none. A P4 has no radio of its own and reaches Bluetooth LE through its ESP32-C6 Wi-Fi co-processor: see the ESP32-P4. A speaker, and a board that plays to one, need the original ESP32: both are Bluetooth Classic, and every later chip has Bluetooth LE only. Playing to a Bluetooth speaker is free. The other three roles are each part of Pro: free to design and preview with, and generating code for it needs Pro.
Switching it on
Hardware > Bluetooth. Tick This board uses Bluetooth, choose A keyboard (HID), An HMI link, A speaker (A2DP) or Plays to a Bluetooth speaker (A2DP), and give it a name - what a host lists it as, in plain letters, twenty at most.
When the board starts says whether it is on the air from the first moment. Switched off, it cannot be found or connected to until an event runs the action Bluetooth on or off - a button that makes the board discoverable when you want it to be. The same action takes it off the air again, and lets go of whoever was connected. It is in every role.
The Bluetooth icons
The Bluetooth LE icon, in the palette's Ready-made group: the Bluetooth mark, in the colour of what Bluetooth is doing. It follows the board by itself - no event and no binding - and it works in every role. Dropping one into a design with Bluetooth off switches it on. It is part of Pro.
For sound there is the Bluetooth audio icon beside it: the same mark with two sound waves, the same colours and the same states, for a board that plays to a Bluetooth speaker or is one. It is free, offered on the original ESP32 (the one chip with Bluetooth Classic), and dropped into a design with Bluetooth off it switches it on as a player for a Bluetooth speaker.
| Bluetooth is | The icon | Colour to start with |
|---|---|---|
| Switched off | Lit, in the colour for off. | Grey |
| Waiting to be paired | Flashes: lit and dark in turn. | Blue |
| Connected | Lit. | Blue |
| Sound is playing | Lit. A speaker, while the phone sends sound; a board that plays to a speaker, while the sound is going to it. | Blue, the same as connected |
| Trouble | Lit. Pairing failed, or a host keeps connecting and dropping - it has to forget the board and pair again. | Red |
Its settings are those five colours, its Size, and Flashes every - how long it is lit and how long dark while it waits, in milliseconds; 0 keeps it lit. Nothing else: it has no text colour or font of its own, because the colour is the state's. To act on a press, give it an event like any widget - Clicked running Bluetooth on or off makes the icon its own switch.
The same answer is a value, What Bluetooth is doing, as a number (0 off, 1 waiting, 2 connected, 3 playing, 4 trouble), for anything else that should follow it.
Sending is an action, so anything can send
A key or a line is sent by an action, and an action runs wherever an event fires. On a widget that is its Events: Clicked, Pressed, Released, Checked, Value changed and the rest. Off the screen it is an automation on the Data tab, which runs on hardware rather than on a touch:
| What fires it | For instance |
|---|---|
| Encoder turned, Encoder pressed | A rotary encoder as a volume knob: clockwise sends Volume up, anticlockwise Volume down, a press sends Mute. Once for every click of the knob. |
| Digital input changes, held | A foot switch or an arcade button on a pin, sending a shortcut. |
| Analog crosses threshold | A reading going over a limit, telling the host so. |
| Variable changes | A state the design keeps - "live" becoming 1 - sending the key that goes with it. |
| Every interval, On startup | A reading sent to the host every few seconds. |
A rotary encoder is added under Hardware > Inputs. With Navigate the screen with it switched off it is for events only, so turning the volume does not also walk a focus ring round the keys. On an ESP32 both of its phases interrupt, so no click is lost while the screen is drawing.
The board as a keyboard
Three actions, in the Bluetooth group:
| Action | Fields | What it does |
|---|---|---|
| Send a key or shortcut | Key; Ctrl, Shift, Alt, Win / Cmd; Do | One key with whatever is held with it: Ctrl + C, Alt + F4, F13. Press and let go is the usual one. Hold down on Pressed with Let go of everything on Released makes a button a key that is down for as long as the finger is. |
| Type text | Text; Press Enter after it | Types it a key at a time: a sentence, a command, an address. However long. |
| Press a media key | Key | Play / pause, next, previous, stop, volume up and down, mute, and a few for the browser and the desktop. |
F13 to F24 are in the list and on no real keyboard. A shortcut the host gives one of them - a scene in OBS, a macro in a game, an AutoHotkey script - can only ever be this panel, which is what a stream deck's keys are.
The host's keyboard layout
A keyboard sends keys, not characters. The host turns each into a character by the layout it
is set to, so Shift and 3 is a # to a host set to US and a pound sign to one set to UK.
Type text is therefore typed for a layout, chosen on the Bluetooth tab under Host's keyboard
layout: US or UK. Pick the one the host is set to and the text arrives as it was written. A
character that layout has no key for is left out, and the Problems list names it.
A shortcut is a key position and needs no layout: Ctrl + C is the same key on both.
A stream deck for OBS Studio
The gallery's OBS Stream Deck is fifteen keys, an ON AIR lamp and two timers on a 3.5-inch ESP32-S3 touchscreen, and it is all of the above in one design: open a copy, and every key's shortcut is an event you can read and change.
- Flash the board. On the PC add a Bluetooth device and choose OBS Deck. There is no number to type.
- In OBS open Settings > Hotkeys. Click the box beside an action and tap its key on the deck: OBS writes the shortcut in for you. A key with two actions sends the second on its second tap - tap REC for Start Recording, click the Stop Recording box, tap REC again.
- Or type them in from this table. Ctrl is on, Alt is off and Shift is a single press, on every key.
| Key on the deck | It sends | In OBS, Settings > Hotkeys, give that to |
|---|---|---|
| INTRO | Ctrl + F13 | Switch to scene, on your opening scene |
| CAMERA | Ctrl + F14 | Switch to scene, on your camera scene |
| SCREEN | Ctrl + F15 | Switch to scene, on your screen or game scene |
| BRB | Ctrl + F16 | Switch to scene, on your "be right back" scene |
| OUTRO | Ctrl + F17 | Switch to scene, on your closing scene |
| MUTE MIC | Ctrl + F18 on, Alt + F18 off | Mic/Aux > Mute, Mic/Aux > Unmute |
| TALK | Shift + F18, held for as long as the key is | Mic/Aux > Push-to-talk |
| MUTE PC | Ctrl + F19 on, Alt + F19 off | Desktop Audio > Mute, Desktop Audio > Unmute |
| HIDE CAM | Ctrl + F20 on, Alt + F20 off | Hide (your camera source), Show (your camera source) |
| SNAP | Shift + F21 | Screenshot Output |
| REC | Ctrl + F23 on, Alt + F23 off | Start Recording, Stop Recording |
| PAUSE | Ctrl + F22 on, Alt + F22 off | Pause Recording, Unpause Recording |
| REPLAY | Shift + F22 | Save Replay |
| V-CAM | Ctrl + F21 on, Alt + F21 off | Start Virtual Camera, Stop Virtual Camera |
| GO LIVE | Ctrl + F24 to start, Alt + F24 to stop. Hold the key. | Start Streaming, Stop Streaming |
The scene keys are named for a common set: bind them to whichever five scenes you use and change the captions to suit. Push-to-talk also needs Enable Push-to-talk ticked for the microphone under Settings > Audio. Hide and Show are listed under the scene a source is in, once for each source.
Three things about the scheme are worth keeping if you change it. No key is used bare beside a modified one and none takes two modifiers, because a program that matches loosely takes Ctrl + Shift + F23 for a press of Ctrl + F23 as well. Start and Stop are two hotkeys, never one that toggles: a keyboard only talks one way, so the deck cannot ask OBS whether it is recording, and with two the lamp means "the last thing I sent was Start" and the next press always sends Stop. And GO LIVE takes a hold, both ways.
The board as an HMI
Report to a host
Select a widget and open Report to a host in its properties. Each tick box is a line the host is sent, and the box shows the line with the widget's own name in it. No event is needed.
| Tick box | Sent when | The line |
|---|---|---|
| Pressed | The finger goes down. | Button1 pressed |
| Released | The finger comes up. | Button1 released |
| Clicked | Pressed and released on the widget. | Button1 clicked |
| Changed | Its value moved. Offered on a slider, arc, spinbox, dropdown, roller, switch, checkbox, text area and a checkable button. | Slider1 changed 42, Switch1 changed 1 |
Name sent is what the host knows the widget by; left empty it is the widget's own name. A slider being dragged says where it is: a line still waiting to go is replaced by the newer one, so the host is never working through a backlog.
For a line that says something else, the action Send a line to the host sends any text, with the value or the text of a widget on the end if one is chosen - from a widget's event or from an automation, so a physical button or a click of an encoder reports the same way.
What the host can say
| The host sends | The board |
|---|---|
set count 42 | Sets the variable count, and whatever is bound to it follows. Text takes the rest of the line: set title Kitchen lights. |
get count | Answers count is 42. |
screen Settings | Goes to the screen of that name. |
ping | Answers pong. |
Each on a line of its own. A line that is not understood comes back with a question mark in front of it. The host can set a variable that has no source; one that is read from a sensor or a formula is read again on the next pass, so the board says so instead of pretending. The Bluetooth tab lists the ones the host can set.
The board as a speaker
Choose A speaker (A2DP) and the board appears in a phone's or a PC's Bluetooth settings as a loudspeaker. There is no number to type - like any speaker, it accepts whoever asks while it is on the air - and it plays for one phone at a time. What the phone plays comes out of the board's audio output, which has to be switched on: the tab offers to do it.
The sound goes through the same output as an internet station does, and that is the point of how it is built:
| You get | How |
|---|---|
| What is playing | The track's title, artist and album, the phone's name, and whether it is playing, as values. Put one in a variable and bind a label to it. Many players send all three; some send nothing. |
| Level meters | Audio > Level, left and right follow the phone's sound, as they follow a station. Bind a bar to each and it is a VU meter. |
| Play, pause and skip | The action Ask the phone to play, pause or skip, on a button or anything else that fires an event - a rotary encoder's press, say. The phone's own player is asked, as a headset's buttons ask it; a phone with nothing open has nothing to play. |
| Volume | The board's own: the Set volume action, or a slider bound to Audio > Volume. The phone's volume buttons move the same volume, and the slider follows. |
Whichever started last has the speaker. A phone that starts playing stops a station; a station that is started asks the phone to pause, and the phone is not heard again until it starts afresh. Stop audio stops either.
At power-up it looks for the phone it last played for and connects to it, as a speaker does; switch Go back to the last phone off to have it wait to be chosen. While a phone is connected the board is off the air to everybody else.
How it sounds is the output's doing. On a board whose output is the chip's own converter - a CYD, an E32R28T - that is eight bits and one channel: clear for speech and background music, and not hi-fi. An I2S amplifier or a codec plays the phone's sixteen bits in stereo. The codec is SBC, which every phone speaks; a phone that prefers AAC or aptX falls back to it.
It is not a hands-free set unless you ask for one: see Phone calls. And it cannot be built on an S3, C3 or C6 - their radios are Bluetooth LE, whose own audio is not in the Arduino core.
Phone calls (hands-free)
Under the speaker's settings, Phone calls has a Hands-free calls option. Switch it on and the board is the phone's headset as well (HFP): a call comes out of the board's audio output and the caller hears the board's microphone. A phone connected for sound only is asked for calls too, a few seconds after it connects. It is part of Pro, like the speaker, and it is Arduino only on the original ESP32. Compiled, not yet run on a board.
- The design needs an audio output (Hardware > Audio) and a microphone. Without either, the Problems list says which.
- The microphone needs a port of its own, such as an I2S microphone. A microphone that shares the speaker's port does not work, because the call takes the output over while it plays.
- Calls are mono at 8 or 16 kHz, the rate the phone picks. The phone's volume keys move the player's volume.
The Hands-free call action (group Bluetooth) has one choice, Do: answer the call, hang up or turn an incoming call away, call the last number again, mute the microphone, un-mute it, or mute or un-mute whichever it is not. Three values report on it: The phone call, as a number (0 no call, 1 coming in, 2 calling out, 3 in a call), Who is calling (the number as the phone gives it, or empty when it does not say) and The microphone is muted for the call.
Bluetooth LE on an ESP32-P4
A P4 has no radio. Its Bluetooth LE goes through the ESP32-C6 Wi-Fi co-processor on the same lines as its Wi-Fi, so the co-processor has to be in the design (Hardware > Devices; most P4 boards carry one). That works for A keyboard (HID) and An HMI link, on the Arduino export. A speaker, and a board that plays to one, are refused on a P4: the C6 has no Bluetooth Classic.
Bluetooth starts on a task of its own, so it never holds up start-up. The C6 must run ESP-Hosted 2.6 or later. With an older firmware Bluetooth does not start, a sentence says so in What Bluetooth is doing, and the Bluetooth icon shows trouble. ESP-IDF refuses a P4 with Bluetooth. Compiled, not yet run on a board.
The board playing to a Bluetooth speaker
Choose Plays to a Bluetooth speaker (A2DP) and everything the board's player plays - an internet station, a card's MP3 files, a sound of the design, the other end of a two-way radio - can go to an ordinary Bluetooth speaker instead of the board's own output. The volume and the level meters are the board's own either way, so a VU meter keeps moving whichever way the sound goes. It needs the audio output switched on: that is the output the speaker stands in for. It is free on every plan.
| You get | How |
|---|---|
| Which output | The action Choose the audio output: the Bluetooth speaker, the board's own output, or the other one. The choice is kept through a restart. While no speaker is connected the sound comes out of the board, and it goes back to the speaker when one is. |
| Finding a speaker | The action Find or forget a Bluetooth speaker: look round (about ten seconds) and connect, connect to the one remembered, let it go, or forget it. Switch the speaker on and into pairing mode first. |
| Which speaker | Speaker to look for on the Bluetooth tab: part of its name, in any case - "JBL" finds a JBL Flip 5. Left empty, the board takes whichever audio device answers loudest. |
| What it is doing | The speaker's name, whether the sound is going to it, the output chosen and a sentence, as values. The Bluetooth icon shows it at a glance: flashing while it looks or connects, lit while connected. |
| The speaker's own buttons | Play, pause, next, previous and volume on the speaker work the board's player. Switch it off on the tab if they should not. |
At power-up it goes back to the speaker it last played to, and, while the speaker is the output chosen and it is not there, asks again every half minute - a speaker is as often switched on after the board as before it. With no speaker remembered, it looks round once by itself unless Look for one at power-up is off. Let the speaker go stops it going back until something asks again; Forget lets it go for good, so that the next look round can choose another.
The stream to the speaker runs only while the speaker is the output chosen. Choosing the board's own output suspends it and choosing the speaker starts it again, which takes a moment. That is because Wi-Fi and Bluetooth share the one radio: a stream of silence would take air time from a station for nothing. With both busy a station has less margin, and a bigger stream buffer under Hardware > Audio is the cure for one that stutters.
The sound goes to the speaker as the board decodes it, in stereo at 44.1 kHz - a station at another rate is brought to it on the way - and the ESP32's own Bluetooth stack encodes it (SBC, which every speaker takes). On a board whose own output is the chip's 8-bit converter, a CYD or an E32R28T, the speaker gets the full sixteen bits that converter cannot.
With Wi-Fi beside it, playing to a speaker needs a classic ESP32 with PSRAM, a WROVER module. On a Cheap Yellow Display, which has none, an internet radio that played to a speaker ran out of memory: with LVGL and Wi-Fi running there was too little left to start Bluetooth Classic, and the board restarted. That is why the blocky radio tutorial plays through the board's own speaker connector instead.
It cannot be built on an S3, C3 or C6: their radios are Bluetooth LE, whose own audio is not in the Arduino core.
What a design can read
| Value | Type | Notes |
|---|---|---|
| A host is connected | Boolean | The PC or phone it is a keyboard to, the host on the link, or the phone a speaker plays for. Bind a key's Disabled to it and a key cannot be pressed with nobody listening. |
| What Bluetooth is doing, as a number | Number | 0 switched off, 1 waiting to be paired, 2 connected, 3 sound is playing, 4 trouble. What the Bluetooth icon shows. |
| Bluetooth is switched on | Boolean | False once the Bluetooth on or off action has taken it off the air. |
| The phone is playing | Boolean | A speaker only: sound is arriving. |
| The phone's name | Text | A speaker only: what the connected phone or PC calls itself. |
| Track title, artist, album | Text | A speaker only: what the phone says it is playing. Empty until it says. A variable holds as much of it as its length allows, so give a title's variable room. |
| What Bluetooth is doing | Text | A sentence: waiting, connected, paired - or that the host keeps dropping the connection and has to forget the board. Worth a label of its own. |
| Bluetooth name | Text | The name it goes by on the air. |
| Last line from the host | Text | Whatever the host last sent, as it came. An HMI link only. |
| Sound goes to the Bluetooth speaker | Boolean | A board that plays to a speaker: true while the speaker is the output chosen and connected. |
| Audio output chosen | Number | 1 the Bluetooth speaker, 0 the board's own output: what Choose the audio output last chose. Bind a key's Checked to it to light the one chosen. |
| The Bluetooth speaker's name | Text | The speaker connected, or the one remembered. Empty when it knows none. |
| Looking for a Bluetooth speaker | Boolean | True for the ten seconds or so a look round takes. |
| How the stream to the speaker is going | Text | A line of figures for finding a fault: "short" going up while it plays is sound breaking up. |
Connecting
As a keyboard: add a Bluetooth device in the host's settings and choose the board's name. There is no number to type. The host remembers it and reconnects by itself.
As a speaker: the same, from a phone or a PC: it is listed with a loudspeaker beside its
name. A phone from before pairing needed no number is answered with 0000.
After the board's flash has been erased it has forgotten the host, and the host has not forgotten it: it connects, fails, drops and tries again. Remove the board from the host's Bluetooth devices and pair once more. A label bound to What Bluetooth is doing says this on the panel when it is happening.
The HMI link is the Nordic UART service, which Bluetooth terminal apps speak: Serial
Bluetooth Terminal and nRF Connect on Android, Bluefruit Connect on a phone of either kind. For a
program of your own, the service is 6E400001-B5A3-F393-E0A9-E50E24DCCA9E; write to
6E400002 and take notifications from 6E400003 (the rest of each address is
the same).
What it costs
Measured by building the same sketch with and without it:
| Chip | Added to the firmware |
|---|---|
| ESP32-S3, C3, C5, C6 | About 0.3 MB. |
| Classic ESP32 | About 0.8 MB, and memory at run time that a board with no PSRAM is short of. It uses a different, heavier Bluetooth stack. |
| A speaker (classic ESP32 only) | About 0.8 MB as well. A whole design - screen, player and speaker - was 1.7 MB of the 3 MB a 4 MB board gives its firmware. |
| Playing to a speaker (classic ESP32 only) | About the same. An internet radio with Wi-Fi set up from a phone and the speaker was 2.2 MB of the 3 MB. |
WiFi and Bluetooth share one radio. They run together; a design that streams audio over WiFi while Bluetooth is busy is asking a lot of it.
In preview
The drawer has a Bluetooth section while the design uses it: choose what Bluetooth is doing - switched off, waiting, connected, playing, trouble - and the icon and every value that reads it follow. The icon flashes while it waits, at its own rate. Bluetooth on or off and Ask the phone move the same state, so a play button can be tried, and a speaker's track values read a sample title and artist to lay a screen out around.
Nothing is on the air in a browser. A host is taken to be connected, and the Console says what would have been sent: the shortcut, the text, the media key, and the line each tick box makes - from a widget's event, and from an automation that runs on a timer or on a variable changing. An action that is for the role the board is not in says so there instead.