Translated from the original Japanese article
Tech9 min read

M5Stack CoreS3 voice recorder: mic/speaker I2S switching and a DTR/RTS freeze

M5StackESP32MicrocontrollerElectronicsHardwareExperiment
IkesanContents

Update (2026-07-22): The follow-up takes this recording all the way to a spoken reply: hand-written RIFF header, multipart POST from PSRAM, and 12.4–13.8s sync round trips → M5Stack CoreS3 voice chat: multipart WAV POST from PSRAM, 12s sync round trip

In the previous part the face, the lip sync and the gesture controls were all working.
What’s left is the actual “talk to it” part, and it has three hard bits: recording, networking, and the async plumbing. Doing them all at once would make any failure impossible to untangle, so I split it into one hard bit per stage.

This time it’s the first one: just the recording.

A voice recorder with no server and no SD

No WiFi and no SD this time. All the filler-fetch and SD-cache code from stages 1 and 2 is gone.
The point is to never be in the situation where a broken recording could be the network, the SD card or the mic — in this build, if something goes wrong, it’s the mic path.

In short: a voice recorder that records a fixed number of seconds on tap and plays your own voice right back.

The face, lip sync, gestures and power handling carry over from stage 2 unchanged. The only difference is that it plays back my own voice instead of a filler, so the lip sync should work with the same code.

Recording format: 48kHz/16bit/mono

As settled in the spec check, the voice-chat server accepts 48kHz, 16bit, mono WAV. So I record in that format from the start.

That’s 94KB per second, 937KB for the 10-second maximum. It doesn’t fit in internal RAM, so it’s allocated from PSRAM (8MB) at boot.

static constexpr uint32_t REC_RATE = 48000;
static constexpr int REC_SEC_MAX = 10;

g_recBuf = (int16_t*)heap_caps_malloc(
    (size_t)REC_RATE * REC_SEC_MAX * sizeof(int16_t), MALLOC_CAP_SPIRAM);

Mic and speaker fight over the same I2S bus

The CoreS3’s mic (ES7210) and speaker (AW88298) hang off the same I2S bus (I2S_NUM_1, BCK=GPIO34/WS=GPIO33), so they can’t run at the same time.
With M5Unified you switch explicitly with end() and begin().

flowchart TD
    A[待機<br/>Speaker on] -->|タップ| B[Speaker.end<br/>200ms待機]
    B --> C[Mic.begin<br/>録音 5秒]
    C --> D[Mic.end<br/>200ms待機]
    D --> E[Speaker.begin<br/>録音した声を再生]
    E -->|再生完了| A

The switch looks like this. Example implementations wait 200–500ms around the swap, so I’m starting with 200ms.

// --- Speaker → Mic ---
M5.Speaker.stop();
M5.Speaker.end();
delay(200);
M5.Mic.begin();

// (録音)

// --- Mic → Speaker ---
M5.Mic.end();
delay(200);
M5.Speaker.begin();
M5.Speaker.setVolume(vol);  // begin後に音量を再適用

This switching behavior is exactly what I want to check on the real device: does the swap add a pop, does the head of the recording or the playback get clipped? At the spec-check stage, all I knew was that waiting 200–500ms is supposedly enough.

Keeping the power button alive inside the recording loop

M5Unified’s record() is asynchronous — a background task fills the buffer you hand it. When the queue is full it blocks until the recording catches up, so handing it 5 seconds at once stalls the loop for 5 seconds.

The power button on this device is handled in software (read M5.BtnPWR, call powerOff()), so a stalled loop means the device can no longer be switched off. Under the rule in place since stage 1, the power handling has to keep running in the wait loop during recording too.

So I queue 100ms (4,800 samples) at a time and run the power handling and the remaining-seconds display in between.

size_t off = 0;
while (off < total) {
  size_t n = min(REC_CHUNK, total - off);   // 100ms分ずつ
  M5.Mic.record(g_recBuf + off, n, REC_RATE);
  off += n;
  power_tick();                             // 電源ボタン+電源表示の維持
  show_rec_countdown();                     // 残り秒数のオーバーレイ
}
while (M5.Mic.isRecording()) { power_tick(); delay(10); }

Mic gain is adjustable over serial

M5Unified’s mic config has a software gain called magnification (default 16). There’s no way to know the right value without actually recording, so I added a serial command gain <n> that applies from the next recording.

Recording length (rec <sec>), replay (play) and volume (vol <n>) can be changed over serial the same way. The point is to loop record → listen → compare at different gains without reflashing.

Controls

The gesture scheme settled in stage 2 carries over. The tap’s role changed from “fake response” to “record”.

GestureAction
Tap (anywhere)Start recording (default 5s); when it ends, your voice plays back automatically (with lip sync)
Tap during playbackStop
Long press (0.5s)Toggle face screen ⇔ UI screen
Vertical flickVolume ± (overlay shown for 1 second)
Power buttonPower off (as before)

While recording, a red “recording n” overlay at the bottom edge counts down the remaining seconds.

With no SD, neither the “no-drawing phase” right after boot nor the full power-off after flashing — both mandatory in stages 1 and 2 — is needed this time.

On-device test

Boot log right after flashing. A soft reset was enough and the face appeared instantly.

VoiceChat Step3a: voice recorder
rec buffer: OK (937 KB PSRAM)
face sprites: OK (decode 86 ms)
volume: 1/10 (NVS)

The 937KB PSRAM allocation, the face sprite decode at 86ms (same level as stage 2) and the NVS volume carry-over were all fine.

The screen freeze the serial monitor causes

On the first test the screen froze. Taps did nothing, and even after unplugging USB the power indicator kept saying “USB powered”.

I suspected a hang somewhere in the I2S switch, added logs to each step of the swap, added a recnow serial command to trigger recording from the host, and reflashed. But over serial, recording → switch → playback all went through. It never reproduced, no matter how many times I tried.

The cause wasn’t the firmware — it was how the serial port I was monitoring with got handled.
The CoreS3’s USB is the ESP32-S3’s built-in USB-Serial-JTAG, and the host’s DTR/RTS signals are wired straight into the chip’s reset control. It’s the same path the flashing tool uses for auto-reset, and when the port I had opened for monitoring got closed, the signal was left on the reset-hold side. The chip stops, but the LCD controller keeps showing the last frame out of its own RAM, so it looks like the device froze on the last screen.

The device and the firmware were both fine; after a fresh reset it just worked. From then on I kept the monitor port open for the entire test session.

There was one more “it should be playing but I hear nothing” — that one was the volume 1/10 left in NVS, carried over from a previous-stage test where I had turned it down.

Record → playback

The test utterance is the one used to check the voice-chat server: “What is the highest mountain in Japan?” (in Japanese). This recording goes straight to the server in the next stage, so the test data matches too.

The numbers and the sound came out like this.

A 5-second recording (240,000 samples) measured 5007–5010ms — stacking 100ms chunks still lands almost exactly on 5 seconds.
Mic.begin() succeeded first try, every time. Repeated record → playback cycles never failed.
Gain stayed at the default magnification=16, and at desk distance the recording came out loud enough to hear clearly.

The moment of the switch does add a pop, and the 200ms wait doesn’t remove it.
On the other hand the heads of the recording and the playback weren’t noticeably clipped, and the end of playback was uneventful.
The remaining-seconds countdown and the power indicator kept updating throughout the recording.

There’s a usability problem too: it’s hard to tell when recording actually starts after the tap, and if you start talking late, the tail of what you said gets cut off.
That comes with fixed-length recording. A start cue (a sound or a change in the face) and whether 5 seconds is even enough are things to sort out before the real build.

What to do about the pop

My read on the pop: it’s the DC offset step at the moment Speaker.begin() reinitializes the amp (AW88298) and I2S.
The output jumps to the silent level, and that step is the click — so stretching the wait doesn’t remove it in principle. Which also explains why the 200ms wait didn’t help.

There are a few candidate fixes.

FixTake
Play a few hundred ms of silence right after begin() before the real audioThe pop still happens, but at the head of the silence, separated from the speech. A few lines of code, the lightest option
Volume ramp (volume 0 right after begin, restore during the silence)Doesn’t address amp-side pops, but a backup to combine with the silence
Poke the AW88298’s soft-mute register directlyReaches outside M5Unified’s management, so last resort
Do nothingThe final device plays a filler right after recording ends, so the pop may hide under the filler’s head

Whichever one wins, the next stage (sending the recording to the server) does the same switch every time. That’s where the fix gets tested.

Progress this time

  • 937KB recording buffer allocated in PSRAM
  • Boot with no SD / no WiFi (no no-drawing phase, boots instantly)
  • Tap → 5s recording → auto playback, repeatedly, without failures
  • Noise and clipped heads around the mic/speaker swap → pop on switch (fix later); no clipped heads
  • Mic gain fine at the default magnification=16
  • Remaining-seconds display and power indicator keep updating while recording
  • Volume flick and screen toggle still work

Next up

Recording works, so next is sending it to the server and getting a reply back (synchronous, minimal round trip).

  • Write a RIFF header onto the recording to make it a WAV
  • multipart POST to action=sync (field name audio)
  • Show the transcript and the reply on screen
  • Download and play the reply WAV (single concatenated file)