Translated from the original Japanese article
Tech11 min read

M5Stack CoreS3 filler audio: chunked HTTP, SD cache, LCD draw breaking SD.begin

M5StackESP32MicrocontrollerElectronicsHardwareExperiment
IkesanContents

Update (2026-07-21): Stage 2 got the lip-syncing face working from flash-embedded sprites, plus tap/hold/flick gesture controls → M5Stack CoreS3 lip sync: 87ms sprite decode and why BtnA/B/C never fire

In the previous spec check I concluded that the three pieces — filler playback, recording, and lip sync — are all feasible on this hardware. Now for the implementation. Instead of building everything at once I split the work into stages: this first one covers fetching the filler audio, saving it to SD, playing it back, plus the volume UI. The face (lip sync) comes next, and the actual talk-to-it part after that.

The route goes through a VPS relay

An ESP32 can’t run a Tailscale client, so it can’t join the Tailscale network. A packet aimed at a tailnet address (100.x.x.x) has nowhere to go. I knew this from the start, and the original plan was to put a Raspberry Pi in between as a relay. But with not much budget to spare, I instead dropped a single PHP relay script on a public VPS, and the VPS forwards to the home server over Tailscale.

flowchart LR
    A[CoreS3] -->|HTTP + Bearer auth| B[VPS<br/>voice-relay.php]
    B -->|Tailscale| C[Home server<br/>STT+LLM+TTS]

From the device’s side, the only thing visible is the VPS URL. The audio URLs inside the response JSON are rewritten by the relay to point at the VPS too, so the device-side logic stays “GET whatever URL comes back.” Since this endpoint sits on the public internet, every request must carry a Bearer token. Anything without one gets a 401 — a simple mechanism, but it keeps third parties who learn the URL from spinning up the GPU on my home server.

Fillers get cached on SD

When I built the API, the assumption was “fetch from the server at boot and hold it in PSRAM.” But on a cold start the home server takes 45 seconds to load the STT model, and the filler list comes back empty until the TTS warmup finishes. That wait would repeat on every single power-on of the device. So the device saves the files to SD on the first fetch, and from the second boot on it reads from SD. Even with no network, the fillers can still speak.

As worked out earlier, my SD card only accepts writes at 10MHz, and after a soft reset it won’t remount until the power is cut. Because of that, fetch → save → read-back all completes within a single boot, and the bus opens at 10MHz from the start.

static constexpr uint32_t SD_FREQ_HZ = 10000000;  // 手持ちSDは書き込み10MHz縛り
g_sdOk = SD.begin(4, SPI, SD_FREQ_HZ);

Each filler is 350–440KB and there are three. Downloads stream to SD in 4KB chunks, then get read back into PSRAM to stay resident. playWav plays asynchronously and the buffer must not be freed until playback finishes, so I keep them resident. If the SD write fails, it switches to going straight to PSRAM and carries on. The cache stops working, but filler playback itself keeps running.

For SD-write testing I also added a serial command: send refetch and it deletes the fillers on SD and re-fetches them from the server.

Keeping the power button alive

I looked into how the power button works when I restored this unit as a CO2 monitor. The button is wired to the AXP2101, and hardware alone never powers the unit off. The program has to read M5.BtnPWR and call M5.Power.powerOff() — only then does it turn off.

Which means if any single wait loop forgets to pump M5.update(), the power button stops responding for that stretch. This code is full of waits — WiFi connect, filler-list retries, streaming downloads — so I wrote one function that bundles the power button and the power-status display, and made it the rule that every loop pumps it.

// 待ちループ用: これを回し続ければ電源ボタンと電源表示が維持される
static void power_tick() {
  M5.update();
  handle_power_button();          // BtnPWRで powerOff()
  static uint32_t last = 0;
  if (millis() - last >= 1000) {
    last = millis();
    update_power_indicator();     // USB給電/バッテリー駆動の表示
  }
}

The one thing this can’t cover is HTTPClient’s blocking calls, so there I cap the timeouts at 5–20 seconds, which bounds how long the power button can be unresponsive.

That said, I think this pump-it-in-every-loop approach gets too bloated to be a reasonable design once the loop handling grows. Depending on what features go in, I’m loosely thinking about switching to state management.

Since this round is mainly about audio, the CO2 monitor logic will be folded in separately later and is excluded from this program. Each stage is written as its own program, to be merged once the hardware tests are done. The sensor stays plugged into Port A; this program just doesn’t touch it.

Why the volume UI comes first

The CoreS3 has no physical volume keys. If you build the sound features first and leave the adjustment for later, the device can suddenly start talking at a very loud volume, which is a real problem. So the volume UI went into the same stage as playback.

Two touch buttons [-] [+] sit at the bottom of the screen, with 11 steps from 0 to 10. M5.Speaker.setVolume() takes 0–255, so each step maps to ×25.5, rounded. The value is stored in NVS (Preferences) so it survives reboots, and the default is a modest 3/10.

static void apply_volume(int level, bool save) {
  if (level < 0) level = 0;
  if (level > VOL_STEPS) level = VOL_STEPS;
  g_volLevel = level;
  M5.Speaker.setVolume((uint8_t)(level * 255 / VOL_STEPS));
  if (save) g_prefs.putUChar("vol", (uint8_t)level);
}

That’s the whole interface: tap the upper half to cycle through the fillers (tap again to stop), [-] [+] at the bottom for volume, power button to power off.

Testing on the device

No Content-Length in the response

On the first boot after flashing, WiFi connect and the filler-list fetch both worked, but all three WAV downloads failed. A look at the VPS response headers explained why.

HTTP/1.1 200 OK
Content-Type: audio/wav
Transfer-Encoding: chunked

The PHP relay streams its output, so there is no Content-Length. My first implementation called getSize() to size the PSRAM allocation, got -1 back, and never got started. On top of that, reading through the raw getStreamPtr() mixes the chunk framing (size lines) into the audio data, so writing that straight to SD corrupts the WAV.

HTTPClient has writeToStream(), which decodes the chunking and writes to a Stream, so I switched to that. For SD you just hand it a File. The PSRAM side needs a write target, so I wrapped a variable-length PSRAM buffer in a Stream interface.

// 可変長のPSRAMバッファに書き込むStream(チャンク転送でサイズ不明でも受けられる)
class PsramBufferStream : public Stream {
 public:
  uint8_t* buf = nullptr;
  size_t len = 0, cap = 0;
  size_t write(const uint8_t* data, size_t size) override {
    if (len + size > cap) { /* heap_caps_reallocで倍々に拡張 */ }
    memcpy(buf + len, data, size);
    len += size;
    return size;
  }
  // read系は空実装
};

int written = http.writeToStream(&ps);  // チャンクデコード込み

After the fix, all three fillers (353–403KB) landed in PSRAM. Fetched data is verified by checking the leading “RIFF” bytes before use. If the relay ever returns an error page and you play the non-audio data anyway, it either fails to play or comes out as garbage noise.

Playback and volume

Cycling through the fillers by tap. The first three plays are at volume 3; for the last one I dropped to volume 1 with [-].

A 48kHz/16bit/mono WAV passed straight to playWav speaks correctly. Looking at the video’s audio track as a waveform, playback at volume 3 swings the RMS to 5–12x the silent floor, while volume 1 barely rises above the background noise. The power status display stayed at “USB powered, 100%” throughout the test.

SD not recognized at any clock speed

The biggest fight was the SD card. SD.begin() failed on the first boot, and from there I sank into a swamp.

First, I re-made a mistake documented in my own past post. The CoreS3’s SD won’t mount unless the pins are set explicitly with SPI.begin(36, 35, 37, 4) — and I had forgotten that. But fixing it changed nothing. Dropping the clock 25MHz→10MHz→4MHz→400kHz didn’t help, a full power cut didn’t help, it simply wouldn’t mount.

I flashed the minimal program from the earlier isolation (it just retries every 2 seconds) and let it run — the moment I reseated the card it flipped to MOUNT OK. It looked like a contact problem.

Draw to the screen before mount and the SD stops mounting

That seemed to settle it, until this remained: with the same card and the same power-cut routine, the minimal program mounted on the first try while the real code went 0 for 5. Comparing the two and eliminating the differences, the cause was drawing to the screen before mounting the SD.

On the CoreS3 the LCD and the microSD share a single SPI bus (SCK=36/MISO=35/MOSI=37), with only the CS lines separate. The screen is driven by M5GFX, the SD by Arduino’s SPI class — two separate drivers touching the same bus. When display clocks get mixed into the delicate init sequence before the card enters SPI mode, initialization apparently never succeeds again. The same holds after mounting: writes issued after drawing or WiFi work failed.

The fix: bundle all SD work into the moment right after boot, and draw nothing at all during it.

// 起動フロー: 描画禁止フェーズにSD作業を集約する
delay(2000);
SPI.begin(36, 35, 37, 4);          // ピン明示は必須
// SD.begin(25MHz)を2秒間隔でリトライ(この間、画面は真っ黒のまま)
// → キャッシュがあれば読み込み / なければ取得してSDへ保存
SD.end();                           // 以後この起動ではSDに触らない
// ここから初めて画面描画・UIを解禁する

With this structure, mounting and writing both went through at 25MHz without a fuss. The “writes only work at 10MHz” result from the earlier isolation, and today’s total failure, most likely both trace back to card contact and bus interference during init.

I also learned something about how to cut the power. A card that got stuck once did not come back with a quick OFF→ON. Switch off, wait 10 seconds, then switch back on. With that, the next cold boot mounted with no trouble. My guess is that residual charge on the DIN BASE side had not fully drained.

Here is the final boot log. From the second boot on, it never touches WiFi or the server — it comes up from the SD cache alone.

SD.begin OK @25MHz (attempt 2): size=30000 MB
SDから読み込み: /fillers/filler_0.wav (433964 bytes)
SDから読み込み: /fillers/filler_1.wav (353324 bytes)
SDから読み込み: /fillers/filler_2.wav (403244 bytes)
volume: 1/10 (NVS)
SDキャッシュから起動

Progress this session

  • WiFi connect → filler list fetched through the VPS relay (with Bearer auth)
  • 48kHz/16bit/mono WAV plays as-is through playWav
  • Volume changes with [-] [+]
  • Power status display (USB powered / on battery) shows correctly
  • Filler WAVs (350–440KB each) write to SD (succeeded at 25MHz)
  • Second and later boots load from the SD cache (no WiFi, no server)
  • Volume setting survives reboots (NVS; volume 1 restored across a power cut)
  • Power-off via the power button keeps working

Planned for next session

  • Sprites from the 3 flash-embedded PNGs
  • How RGBA transparency renders
  • pushSprite switching: flicker and speed
  • How fixed-interval lip sync looks
  • Filler playback and lip-sync drawing at the same time
  • Volume overlay coexisting with the lip sync
  • SD-cache boot still working