Audio and stations

Playing a stream, a station list, and meters driven by the audio itself.

Hardware › Audio turns the board into a player. It decodes an MP3 stream off the network, drives the chip's own DAC, an external I2S one or an ES8311 codec, and hands the design a volume, a track title and two levels it can put on screen. A board preset with a speaker fills in the wiring; switching it on is one tick box, and nothing is added to the firmware until it is. The same tab has the microphone: its level, a recording, and sound sent to a web address that can answer out loud.

Where the sound comes out

On a board with an ES8311

The Waveshare ESP32-S3-Touch-LCD-3.5B preset brings the ES8311 already wired: I2S on GPIO 44, 13, 15 and 16, the codec at 0x18 on the board's I2C bus, and the amplifier's enable on the TCA9554 expander. The codec's master clock runs at 256 times the sample rate, which holds for every rate a stream arrives at, so it is set up once and the I2C bus is left to touch afterwards. The board's microphone goes into the same codec: see The microphone.

Speaker gain on an ES8311

A small speaker on an ES8311 can be quiet at full volume. Hardware › Audio › Speaker gain, shown when the output is an ES8311, is a slider from 0 to +24 dB: the codec’s own amplification, on top of the volume. Too high and loud sounds distort. A preset may set it for you: the Waveshare ESP32-C6-Touch-AMOLED-2.16 starts at +12 dB and the ESP32-C6-Touch-LCD-2.8 at +15 dB, both chosen by ear.

To change it while the board runs, there is the value Speaker gain (0 to 100 % of the 24 dB, two-way, so a slider bound to it sets it) and the action Set speaker gain, with the same choices as the microphone’s: a widget’s value, a fixed level, or louder or quieter by a step. A change made at run time lasts until the board restarts. Arduino and ESP-IDF; MicroPython and ESPHome refuse the action.

On a Cheap Yellow Display

There is no choice to make, and it is worth knowing why. The board brings out exactly one free pair of general-purpose pins - CN1 - and an external I2S DAC needs three. So audio is the internal DAC on GPIO 26, which is where the onboard SC8002B amplifier and the 2-pin JST speaker connector already are. The preset fills it in for you.

That also settles two other things. The ceiling is an 8-bit mono DAC, so a stream above about 128 kbps costs CPU and heap without sounding better; and the codec should be MP3, because the AAC decoder wants around 45 KB of working memory that a board with no PSRAM does not have spare.

The pin that is not free

The internal DAC is driven through I2S, and I2S maps its left channel to DAC2 (GPIO 26) and its right to DAC1 (GPIO 25). The usual Arduino audio library enables both channels when you ask for internal-DAC output.

On a CYD, GPIO 25 is the touch controller's SPI clock. Left at the default, pressing Play puts a DMA-driven analog waveform on the touch clock line. The generated lw_audio.cpp enables one channel only, and says so in a comment where somebody editing it would otherwise undo it.

Settings

Stations, and building a radio

What a radio plays is a list of objects on the Data tab, with a field each for the name, the address and the genre. The Internet Radio widget says which list, and which field is which - that is set in the Inspector. Dropping one onto a blank project does the rest: audio is switched on and a Stations list is made with one station in it, so it plays on the first build.

The first Internet Radio’s list is the player’s station table. A Play audio action set to a station picks an item from it, Next or previous station steps through it, and Play audio set to toggle with no address of its own starts its first station. Stations your own code adds with var_Stations_add() are part of it too.

A radio built from a blank project is usually: an Internet Radio, a Play/Stop button (toggle), Next and Previous buttons, a slider bound both ways to the player’s volume, and labels bound to the station, the genre and the track title - see Binding to the player. The radio also sends events a design can act on:

The last four follow the player rather than a button, so a design reacts to what actually happened: a stream that dies is Stopped and Stream failed even though nobody pressed anything.

http and https stations

A station at an https:// address plays, and so does one whose http:// address sends the radio on to an https one, which most now do: the radio follows up to five such hops, whatever each is written as. A playlist (.pls, .m3u) at an https address is fetched the same way.

A secure connection costs memory: roughly 40 KB while it is being made. A board with PSRAM has it. On a board without, the decoder, the buffer, the WiFi stack and the LVGL draw buffer may not leave that much, the station does not start, and the radio's status says No secure connection to the station - so the designer warns about an https station on such a board, rather than refusing it, and http is the address to use where a station offers both.

The certificate of an https station is not checked: the board carries no list of authorities to check one against. What comes down the connection is a public broadcast and carries no credentials, which is a fact about this payload, not a general position. It is not a reason to relax anything that does carry them.

How far it has been tested: https stations and an https playlist have played on an ESP32-S3 board with PSRAM. No board without PSRAM has tried one.

Playlist links

Half the URLs people call "the stream" are really .pls or .m3u playlists. The firmware follows one to the first stream inside it, which is the difference between a station working and "it just does not play".

Stations served by a web server

Classic radio servers (Icecast, SHOUTcast) send a stream as one long answer. A station behind an ordinary web server - Nightride, for one - may send it in chunks instead, each with a small header of its own, and the audio library reads the raw connection with no idea those headers are there. They land in the MP3 data, the station's song-title text is found in the wrong place, and the result is a stream that arrives at full speed and still stutters and crackles.

The firmware avoids the question by asking for streams the older HTTP/1.0 way, which a server is not allowed to answer in chunks. So both kinds play; if you edit lw_audio.cpp, keep the useHTTP10() call where it is.

A radio of your own: the Radio icon

The Internet Radio widget is a station list. To lay a radio out yourself, drop a Radio icon instead: it holds the same station list and fields, shows one small mark in the colour of what the player is doing, and leaves the screen to you. Then:

A bound Station or Playing is acted on when it changes, not held against the player: a station that cannot be reached is asked for once, not ten times a second, and nothing plays for the value a design starts with unless Playing starts true.

Files from an SD card

The same player plays files. Give the Internet Radio (or the Radio icon) a folder under Also plays the MP3 files in - /music, or / for the whole card - and when the board starts, every .mp3 in that folder joins the station list after the stations the design has. From then on a file is a station: it is in the list, a touch plays it, Next or previous station steps to it, the volume and the level meters are the same ones. A design with no stations at all is an MP3 player; one with both is a radio that also plays what is on its card. The board needs an SD card among its devices (Hardware › Devices) and audio set to decode MP3. It is free on every plan, as the Internet Radio is.

What a file is called

In the list, and as the player's Station name, a file is its name without the ending, with spaces where the name has underlines. Files are in the order of their addresses, capitals aside, so an album stays together and 01, 02, 03 play in order. Hidden files are left out, which is also the ._ twin a Mac leaves beside every file it copies.

What an MP3 file says of itself is the player's Track title: its artist and title, read from the file's ID3 tag (the older kind at the end of a file and versions 2.2, 2.3 and 2.4 at its front, in any of the four ways a tag writes text) and shown as Artist - Title, however long that is. A file with no tag has no title, so put the station name somewhere too. The tag's picture is stepped over, not shown. Only an MP3's tag is read: a WAV or FLAC file shows no title.

Playing, stopping and carrying on

Play audio set to toggle is play and pause for a file: stopped part-way, the next press carries on from where it stopped. Choosing the file in the list, or stepping to it with Next or Previous, begins it at its beginning. Two more values say where a file has got to - see Binding to the player - so a bar under the title is one binding.

Shuffle, repeat all and seek

Switch on Shuffle, repeat all and seek on the Internet Radio (or the Radio icon) and the player gains:

• Shuffle the files (action, group Audio): Switch it, On or Off. The next file is picked at random from those not yet played. It applies when a file ends into Play the next in the list, and to Next and Previous.
• Repeat all the files (action): the same three choices. Off, the player stops after the last file, or, shuffled, after every file once. Next on a button always carries on.
• Seek in the file (action), Go to The value of a widget (a slider, arc or bar: its whole range maps onto the file, so put this on the slider's Released event), A fixed place (a percentage), or Forward by a step or Back by a step. The seek happens a moment after the request, and a later request replaces it.
• The values Shuffle and Repeat all, two-way: a switch bound to either is its button.
• The value Position in the file (set it to seek), a percentage and two-way: a slider bound to it both ways is a seek bar. The seek happens a moment after the slider stops moving.

Seeking works in MP3 and AAC files. A WAV or FLAC file plays from its beginning only: Play after a stop, and a seek, start it again from the start. A design that reads these while the setting is off fails validation, with a message that says to switch it on. Arduino, compiled, not yet run on a board.

An SD card swapped while the board runs

In a design that uses the card while it runs (the player, a Stream to card log on the card, or a variable that reads the value below), the board checks the card about once a second. Pull the card and a file that is playing stops, the list empties of its files and the player says “The SD card was taken out”. Put a card back, the same or another, and its files are found and listed again, without a restart. Pulling an SPI card costs the screen one stall of about half a second, once.

The SD card present (1) or out (0) value (group SD card on the Data tab) says which, for a label or an icon. It is on wherever the design has a card, and a design that uses none of this is exported exactly as before. Arduino on an ESP32 only; the other exports refuse it with a sentence. Ran on an ES3C35P with a four-line SD card slot; an SPI card has been compiled but not run on a board.

How many, and what is not read

• A list holds 256 items at most, and the files join the list the Internet Radio shows, so that is the most tracks a board lists. Set the list's Room for on the Data tab to as many as the card will hold. When there are more files than room, the board lists what fits and says so: the player's Last problem reads “The list has room for 200 of the 340 files on the card”. It says so too when there is no card, no such folder, or nothing in it to play.
• MP3 only, unless File types says more: then WAV, AAC (raw .aac, not .m4a) and FLAC too. A file whose name ends .mp3 and is something else is tried, and Last problem says it is not MP3.
• The card is read when the board starts. A card put in or taken out while the board runs is handled too: see An SD card swapped while the board runs.
• The built-in fonts have the letters of English, so a name or a title with other letters shows a box for each. Import a font with the characters you need (Western European, Cyrillic...) and use it for those labels.
• Arduino export, on an ESP32 with an SD card. In Preview there is no card: the list shows the design's own stations.

With hundreds of files in it the list still costs what a short one does: it makes only the rows its box can show and moves them under the names as you scroll.

Sounds of the design

A click, a chime, a pop: a short sound kept in the design and played from the board's own flash, with no SD card and no network. It is free, on the Arduino export, for an ESP32 with audio switched on.

1. Open the Assets tab at the bottom and press Add sound. Choose a WAV or an MP3 of up to 256 KB. It joins the list under the images, where its name can be changed and the arrow beside it plays it.
2. On a button, a screen or an automation, add the action Play audio, set Play to A sound of the design and choose it under Sound.
3. Generate the code and upload it, and upload the filesystem: the sounds are files in the export's data/snd/ folder, and they go onto the board with the filesystem image, as images kept on the filesystem do. The export's README says how for PlatformIO and for the Arduino IDE. Skip it and each sound is silent, the serial port names the missing file, and the player's Last problem value says "A sound is not on the board".

WAV or MP3. A WAV starts the moment it is asked for and costs almost nothing to play, so it is the choice for anything short; one second of 16-bit sound at 22,050 Hz is 44 KB. An MP3 is about a tenth of that and starts a decoder (29 KB of memory) each time. Either may be mono or stereo.

What a sound does to what is playing. There is one player. A sound cuts off whatever was playing, another sound or a station, and a station stays stopped afterwards: play it again from the event that follows if it should come back. A sound asked for while another is still waiting to start replaces it.

The call returns at once. The file is opened and played by the audio task (or, with On its own task switched off, by the next pass of the loop), so a button does not wait for the filesystem. In your own code it is audio_play_sound(AUDIO_SOUND_<NAME>), with the names listed in lw_audio.h.

The sounds are on the same filesystem as the images when those are on flash, and on LittleFS otherwise. The Problems tab counts them against the filesystem partition and says when they do not fit. In Preview the browser plays the file itself. A design published to the gallery carries its sounds, as it carries its pictures, so only add sounds you have the right to hand on.

Stations from a playlist

The stations do not have to be typed in. Give the Internet Radio (or the Radio icon) a playlist under Takes more stations from, and the board fetches it once it is on a network and puts every station in it into the list, after the ones the design has. A playlist is a text file of names and addresses - .m3u, .m3u8 or .pls - and there are a great many of them: a radio directory's download, a list somebody keeps on GitHub, one you write yourself and put on any web server. Change the file and the board has the new stations at its next start, with nothing exported again. It is free on every plan, as the Internet Radio is.

A list of genres

1. On the Data tab add a list of objects - call it Genres - with two text fields, Name and Playlist, and an item for each playlist: what to call it, and its address.
2. On the Internet Radio set Takes more stations from to The playlist chosen in a list and choose Genres under List of playlists. The address field is found by its name.
3. Give people a way to choose. A Data list that shows Genres chooses an item when it is tapped (the Data list is part of Pro). On any plan, two buttons with the action Choose list item - The next one and The one before - step through them, and a label on a variable that reads the chosen item's Name says which it is.
4. Nothing is chosen when the board starts, so nothing is fetched until something is. For a radio that starts with stations in it, add an automation: On startup, Choose list item, The first.

A station that is playing goes on playing while another genre is looked through, and its name stays on the screen. Go back to its genre and its row is lit again.

Two settings worth changing for a radio like this. The station list's own stations stay at the top of the list whichever playlist is chosen - a dropped Internet Radio comes with one - so for a radio that lists only the chosen genre's, delete them on the Data tab. And a playlist's names are as long as whoever wrote them made them: set the radio's Row height to 0 and each row is as tall as what it says, where a row of a set height ends a long name in dots.

The address

An http:// or https:// address of the file itself. For a file on GitHub, the address of its page - github.com/…/blob/main/…, which is what the browser shows and what people copy - is understood, and so is the one its Raw button leads to. A server that answers “go elsewhere” is followed.

What is read from it

• A station's name, from the line before its address (#EXTINF:-1,Name) or from Title3= in a .pls. A station with no name is listed by its server's.
• Its group, which is what the list says under the name: group-title="…", a line #EXTGRP:, or a remark # Group: Jazz for every station after it. Where the playlist gives none, the setting Listed as is used.
• Any other remark is passed over, one with an address in it included.

What is left out, and how many fit

• The player decodes one kind of sound, the one chosen under Hardware › Audio - MP3 unless you changed it. A playlist's AAC and Ogg stations are listed all the same, and one that is chosen says so in the player's Last problem: “That stream is not MP3”.
• A station whose address ends .m3u8 sends its sound as a list of short files (HLS), which this player cannot play. Those are left out, and the serial port says how many.
• A list holds 256 items at most. Choosing a playlist gives the station list that much room, unless you have set its Room for on the Data tab yourself. A playlist with more stations than there is room for is listed as far as it fits, and Last problem says so: “The list has room for 200 of the playlist's 340 stations”.
• A playlist that cannot be fetched - no network, a wrong address - is said in Last problem and tried again every half minute.
• An https address is reached without checking its certificate, as a station's is. On a board without PSRAM the 40 KB a secure connection wants may not be to spare while a station plays.
• Two values say what it is doing - Fetching a playlist and Stations from the playlist: see Binding to the player.
• Arduino export, on an ESP32. In Preview nothing is fetched: the list shows the design's own stations and a line where a playlist's will go.

How far it has been tested: a list of sixteen genres, each a playlist kept on GitHub, has run on an ESP32-S3 board with PSRAM: the playlists were fetched and their stations listed and played, in rows as tall as what they say. The part that reads a playlist is held to some 1,400 of them on a PC. No board without PSRAM, and no classic ESP32, has fetched one.

Actions

Binding to the player

A binding can read the player directly, with no variable in between - pick From the player as the binding's source. A level is a reading rather than something anything computes with, so making you declare a variable to show one would be ceremony.

Volume is the only one a control can drive back; the rest are measurements, so the direction choice only appears where it means something.

Why the meters are real

The audio library hands its output one frame at a time and keeps no record of it, so the export subclasses that output and keeps a decaying peak per channel on the way past. It costs about twenty lines and it is the difference between a VU meter and an animation: bind a bar to Level, left and it moves with the music, not with a timer.

The preview does the same thing from the other end. It never fetches the station - it synthesises its own music in the browser and taps the output with an analyser - so the meters in the designer are also driven by real samples, and each station sounds different because its name seeds the tune.

A phone, over Bluetooth

On the original ESP32 the board can be a Bluetooth speaker: a phone pairs with it and plays through this same output. Nothing on this page changes - the volume is the same volume, and Level, left and right follow the phone's sound as they follow a station, so a VU meter built for the radio is one for the phone. Whichever started last has the speaker: a phone that starts playing stops a station, and a station that is started asks the phone to pause.

The other way round, the board can play to a Bluetooth speaker. Everything on this page then goes to the speaker or to this output, whichever the action Choose the audio output last chose - a station, a card's files, a sound - and the choice is kept through a restart. While no speaker is connected the sound comes out of the board as before. The volume and the meters are this output's, so they work either way. A CYD's own 8-bit converter is then the fallback, and the speaker gets the full sixteen bits.

The microphone

Hardware › Audio › Microphone has a switch of its own, because a sound-level meter has a microphone and no speaker. Switched on, the export adds lw_mic.h and lw_mic.cpp: a task that reads the microphone all the time, so there is always a level to show, and that sends what it hears one of three ways when an action asks.

The presets for the Waveshare ESP32-S3-Touch-LCD-3.5B and the LCDWIKI ES3C35P bring the data pin with them (GPIO 14 and GPIO 16), so on those boards it is one tick box. So does the Waveshare ESP32-S3-Touch-LCD-3.49, whose two microphones are on an ES7210 (GPIO 6).

Where the sound goes

A slider bound both ways to Microphone sensitivity does the same with no event to wire, exactly as one bound to the player's volume is the volume control.

One thing at a time: starting any of them finishes whichever was running, and a stream that is playing is stopped first, so what is recorded or sent is the room and not the radio.

What a design can read

These are offered wherever the player's values are - as a variable's source, or straight from a binding - whenever the microphone is switched on.

Sending to a service, and a spoken answer

The board sends one request: a POST whose body is a WAV file (Content-Type: audio/wav, 16 kHz, 16-bit mono) of exactly the length you chose. The length is fixed up front so the samples can go as they are heard, without the recording ever being held in memory - which is what lets a board with no PSRAM send ten seconds of sound. Stop early sends what there is at once and fills the rest with silence.

What comes back decides what happens next:

• Text - any Content-Type that is not sound - becomes Reply from the service, for a label to show.
• Sound - audio/mpeg or audio/wav - is played through the speaker as it arrives. This is how an assistant answers out loud. To show the words as well, send them in an X-Reply-Text header, percent-encoded, and they become the reply text.

The address is one you run: the board posts a WAV and reads an answer, and what does the listening and the talking is up to whatever is behind it. Speech services want a key and a request shape of their own, and neither belongs in firmware. A relay is a page of code:

// node relay.js - then point the action at http://<this computer>:8080/hear
const http = require('http');
http.createServer((req, res) => {
  const parts = [];
  req.on('data', (d) => parts.push(d));
  req.on('end', () => {
    const wav = Buffer.concat(parts);              // 16 kHz, 16-bit mono
    // ...hand wav to whatever listens, and get its answer...
    const text = 'I heard ' + ((wav.length - 44) / 32000).toFixed(1) + ' seconds.';
    res.writeHead(200, { 'Content-Type': 'text/plain; charset=utf-8' });
    res.end(text);                                 // shown by "Reply from the service"
    // A spoken answer instead:
    //   res.writeHead(200, { 'Content-Type': 'audio/mpeg',
    //                        'X-Reply-Text': encodeURIComponent(text) });
    //   res.end(mp3);
  });
}).listen(8080);

Both http:// and https:// addresses work. Over https the connection is encrypted but the server's certificate is not checked - the firmware carries no list of authorities to check one against - so it is privacy on the wire and not proof of who answered. It also costs about 40 KB of heap for the handshake, which a board with no PSRAM may not have to spare. The design needs a WiFi Setup widget, or it has no network to send over.

Recordings

A second of sound is 32 KB. A 4 MB board's filesystem holds under a megabyte, which is about twenty-five seconds; an SD card holds hours, and Keep it on chooses between them. The Check tab says how many seconds the flash has room for, and a recording stops with "the filesystem is full" in Microphone problem rather than overrunning.

The length in a WAV file's header is written first, for the whole time asked for. A recording stopped early has it corrected on a card, on FFat and on SPIFFS. LittleFS cannot change the first bytes of a file without writing the rest again, so there the header is left saying the length asked for: players read to the end of the file and stop, but the length they show beforehand is the one you set, not the one you recorded.

One port, two directions

A codec has one set of clock pins, so its microphone and its speaker are one I2S port. With the ES8311 microphone on - or an ES7210 beside it, which runs on the same clocks - the port is opened once at boot and stays open - a microphone that only worked while music played would be no microphone - and the player writes to it rather than opening its own. While a stream plays, the port runs at the stream's rate: the level still moves, but a recording or a send would be at the wrong speed, so starting one stops the stream, and starting a stream ends one.

Wake words and spoken commands

The board can listen for a wake word and then for a command, on the chip, with Espressif’s ESP-SR. Under Hardware › Audio › Microphone, tick Listen for a wake word and commands (the field is called Wake words and spoken commands). The microphone has to be switched on first.

What it needs, all at once:

• An ESP32-S3, with PSRAM switched on and 16 MB of flash, chosen under Hardware › Board. Another chip, no PSRAM or another flash size gets a sentence saying so in the Errors list.
• A microphone, switched on under Hardware › Audio.
• Images kept on LittleFS, SPIFFS or a card, not FFat.
• The Arduino export. ESP-IDF, MicroPython and ESPHome refuse it.

Say “Hi ESP”, then a command. It is one command for each wake word: after a command, or after a few seconds of silence, the board waits for the wake word again. Two automation triggers react: Wake word heard, and Command heard with the Command to wait for. Each can run any action, so a command can change a screen, set a variable or switch an output.

Flash layout. The speech model, 3.3 MB, sits in a partition of its own, so the design uses the Arduino core’s own ESP SR 16M partition scheme and the app slot is 3 MB. In the Arduino IDE choose ESP SR 16M under Tools › Partition Scheme (the README the export writes says so); with PlatformIO the export flashes the model for you. Arduino only, compiled; nothing has been heard on a board yet, and Preview does not run the triggers.

In the preview

The preview listens to nothing and sends nothing - a design under test should not post to anybody's server. The level is a slider in the drawer, under Simulated inputs; a recording runs its time and stops at its limit; and a send takes the seconds you gave it, then reports status 200 and a stand-in line of text, so the widgets bound to each of those can be seen working before there is a board.

Why the files start with lw_

The generated support files are lw_wifi, lw_audio and lw_config rather than the obvious names. The sketch folder is on the compiler’s include path, and Windows and macOS have case-insensitive filesystems - so a file called wifi.h sitting beside the sketch is what #include <WiFi.h> finds, before the Arduino core’s. Its include guard has already fired by then, so the real header is never read and the compiler says 'WiFi' was not declared in this scope about a line that looks perfectly correct.

What it costs

On a board with no PSRAM everything comes out of the same 320 KB: the MP3 decoder wants about 29 KB, the ring buffer whatever you set, the WiFi stack around 40 KB, and the LVGL draw buffer its own share. It fits, with room to be careless in one place only. The designer adds up the first three and warns when they get tight, and the free heap is worth putting on a settings screen - it is the number you want when it starts misbehaving an hour in.