StackChan CoreS3 with SCD41: Feeding Room CO2 into LLM Voice Context
Contents
In the previous post, StackChan’s overhead touch pad kept firing phantom triggers when left untouched, so I reverted voice input back to the physical Key unit.
On that same day, the voice chat server API was updated.
A new endpoint allows pushing arbitrary sensor readings separately from voice audio so the LLM can reference them during responses. I hooked up an SCD41 CO2 sensor to test whether StackChan could answer questions about the room’s indoor climate.
Test Environment
| Item | Environment Used |
|---|---|
| Device | M5Stack CoreS3 (ESP32-S3), StackChan Body |
| Voice input trigger | Key Unit (U144), overhead Port C (G18) |
| CO2 sensor | SCD41 (CO2, temperature, humidity), Port A on side (G2/G1, I2C) |
| Firmware stack | ESP-IDF 5.5.4, arduino-esp32 3.3.10, M5Unified 0.2.17, StackChan-BSP 1.1.0 |
| Network route | Tailscale direct connection (no proxy/relay) |
| Voice server | Dedicated PC. Qwen3-ASR-0.6B for ASR, Qwen3.8-Max via ModelScope for LLM, local TTS |
| Baseline comparison | SwitchBot CO2 & Temp/Humidity Sensor |
Server-Side Additions
The voice server returns its own API specification in Markdown via GET /help.
The primary changes affecting the client in the September 27, 2026 update are listed below.
| Item | Details | Client-side handling |
|---|---|---|
POST /context | Push arbitrary location or sensor JSON whenever needed. The server injects the body directly into the LLM system prompt without parsing | Newly implemented |
| Value lifespan | JSON keys overwrite on receipt. Cleared from system prompt after 30 minutes | Resend every 5 minutes |
memories in /job | Long-term memories recalled for that answer (0–3 entries). Debug info | Logged to serial console |
| LLM model | Switched to Qwen3.8-Max. Response format remains identical | No changes needed |
| Direct route | Client bypasses any proxy and calls server directly | Retained direct Tailscale link |
The server’s system clock injects the current timestamp into the prompt on each request, so the client does not need to send the time.
The system prompt instructs the LLM to use context values only when relevant to the conversation and not to volunteer raw numbers unprompted.
Resolving Initial Input Inaction
When I booted up StackChan, pressing the Key unit produced no reaction.
At the end of the previous session, I had flashed a standalone diagnostic firmware that ran only the touch processing loop to record overnight logs.
That firmware was still on the device, and because it was purely for touch logging, it had no voice interaction logic.
I re-flashed the voice chat firmware and restored the non-volatile storage partition (containing Tailscale keys and volume settings) backed up prior to flashing.
Boot-up to direct Tailscale connection took 22 to 30 seconds.
Reading SCD41 and Sending via POST /context
For the SCD41 driver, I reused code from an earlier standalone CO2 monitor project.
It sends direct I2C commands over Port A, running on a separate I2C bus from the internal peripherals (overhead touch controller and speaker amp).
| Action | Specification |
|---|---|
| Reading | Sample every 5 seconds. If disconnected, retry probe every 30 seconds |
| When to send | Initial boot, every 5 minutes, or when CO2 changes ≥100 ppm or temperature changes ≥1°C |
| When not to send | During recording, response waiting, or audio playback (to prevent blocking HTTP calls) |
| Payload body | {"room":{"co2_ppm":982,"temp":"27.4 C","humidity_pct":71}} |
The initial HTTP POST returned HTTP 200 in 109 ms. A subsequent GET /context query from a PC confirmed the payload was stored correctly on the server:
{"values":{"room":{"value":{"co2_ppm":982,"temp":"27.4 C","humidity_pct":71},"age_s":24}},"max_age_s":1800}Testing Dialogue with Context Data
Pressing the Key unit turns on the red LED; speaking and then pausing for 1.5 seconds automatically triggers transmission.
Latency is measured from end of transmission to the start of speaker output.
| Prompt spoken | Context value prior to query | Response | First sentence playback |
|---|---|---|---|
| It’s hot today, isn’t it? | 1079 ppm, 26.0°C, 87% | Yeah, it really is muggy. The air feels heavy and uncomfortable indoors. Make sure to stay hydrated and stay cool! | 13.7 s |
| What’s the room temperature? | 1354 ppm, 26.0°C, 88% | Looks like it’s right around 26 degrees right now. But with high humidity, it definitely feels warmer than that. | 7.6 s |
| What’s the CO2 level in here? | 1454 ppm, 26.0°C, 88% | It’s around 1454 ppm right now, which is a bit on the higher side. Opening a window for fresh air would probably help clear things up. | 10.9 s |
| Hello. | ~1210 ppm | Hello! Good to see you today. | 7.0 s |
The LLM accurately reported the transmitted temperature and CO2 values.
For the CO2 question, it cited 1454 ppm, which had been posted just 3 seconds prior to pressing the button.
For “Hello”, it made no mention of sensor readings. For “It’s hot today”, it refrained from quoting raw figures and instead referenced perceived humidity (“muggy”, “feels heavy”).
The first query took 9.9 seconds for speech recognition, while subsequent queries took between 3.9 and 7.3 seconds.
The server specification notes that the first inference after an extended idle period may suffer cold-start latency, which matches this behavior.
No long-term memories were stored after these interactions. The four exchanges contained no personal facts formatted as “You are…”, so there was nothing suitable for extraction.
Investigating Overhead Touch Phantom Triggers
I initially intended to keep only the petting gesture, disabling tap triggers entirely and displaying a 2.5-second happy face on a recognized stroke without playing sound or moving servos.
The stroke algorithm ported from official firmware calculates a weighted pad position (-100 to 100) across three capacitive pads; if touch displacement exceeds 40 from touch onset, it registers as petting.
For the first 8 minutes after boot, contact remained at 0 while the device sat untouched on the desk.
However, around 560 seconds in, the touch log erupted with erratic triggers, and StackChan started displaying the happy face repeatedly on its own.
| Time window | Contact events | Petting triggers |
|---|---|---|
| 17:17 | 28 | 10 |
| 17:18 | 0 | 0 |
| 17:19 | 44 | 12 |
During these episodes, raw touch intensities (0 to 3) peaked at 1 across pads in brief, sporadic bursts.
When the front and rear pads toggled to 1 alternately, the weighted position bounced between -100 and 100, fulfilling the petting distance threshold.
I added a filter requiring at least one pad to reach intensity ≥2 during contact before classifying a stroke as petting.
In the first 6 minutes after re-flashing, this filter suppressed phantom petting completely across 184 raw contact events.
Yet the false triggers returned.
As run time increased, the intensity of phantom touches escalated.
| Time window | Intensity 1 | Intensity 2 | Intensity 3 | Petting triggers |
|---|---|---|---|---|
| 17:20 | 417 | 5 | 1 | 0 |
| 17:30 | 547 | 76 | 1 | 19 |
| 17:40 | 703 | 366 | 7 | 92 |
| 17:50 | 743 | 711 | 55 | 182 |
Even intensity 2 phantoms manifested as 1 1 1 → 2 1 1 → 2 1 2, showing simultaneous noise across all three pads rather than a localized finger touch.
Raising the threshold to intensity 3 still allowed ~50 phantom triggers in a 10-minute window.
Filtering for directional travel—requiring an active outer pad transitioning to the opposite outer pad with intermediate zero readings—cut false detections by about 90% in logs, but failed to reach zero and risked dropping deliberate strokes.
Since algorithmic discrimination proved unreliable, I disabled touch-driven facial expressions entirely and left the detector purely in diagnostic logging mode.
Sending pet 1 over serial console still triggers the happy face manually.
While these tests ran with the SCD41 connected, the identical phantom ramp occurred in earlier runs without the sensor, indicating the sensor bus is not the root cause.
Driving Neck Motion from Response Emotion
With overhead touch disabled, the StackChan Body’s only physical actuators are the two-axis neck servos.
Because the server outputs an emotion tag with each response, I tied servo movement to the start of speech playback alongside facial emotion updates.
| Axis | Range | Zero position |
|---|---|---|
| Yaw (horizontal) | ±128° | Center |
| Pitch (vertical) | 0° to 90° | Bottom limit; cannot tilt below horizontal |
StackChan-BSP calculates neck movement using spring physics.
Servos run in a dedicated task, so motion commands return immediately without stalling audio playback or dialogue processing.
Because pitch cannot angle downward, sorrow was mapped to a slow sideways turn and pause rather than looking down.
| emotion | Neck Gesture |
|---|---|
| smile | Single subtle nod |
| joy | Tilt upward with two quick bounces |
| anger | Rapid left-right shake |
| sorrow | Slow turn to the side and hold |
| fun | Slight upward tilt with gentle side-to-side sway |
| (thinking) | Slight upward tilt with a tilted head during filler audio |
When speech ends, the head recenters. Recording does not begin until head motion has completely stopped.
Servo motor whine is noticeable, but easily masked by increasing speaker volume.
Initially, gestures ran sequentially by waiting for each step to fully settle, which made nodding sluggish.
Even with speed maxed out at 1000, spring stiffness limits acceleration; as the servo approaches target angle, motion decelerates, delaying the settlement detector.
Transitioning to the next step once the head arrived within a few degrees of the target brought single nod duration down to 0.2–0.4 seconds.
However, setting this cutoff margin to 2° caused a 2.6-second stall during the second oscillation of fun.
The servo occasionally stopped 1° to 2° short of target (e.g., stopping at -22.8° against a -25° command), never tripping the 2° threshold and timing out.
Widening the margin to 4° eliminated timeouts, completing joy in ~1.1 seconds and fun in ~1.6 seconds.
During testing, “I’m starving and have no energy” triggered sorrow. Together with the first sentence “It’s so rough when you’re hungry…”, StackChan took 0.9 seconds to slowly glance away, conveying a genuinely downtrodden presence.
SCD41 Readings Diverging from SwitchBot
During spoken tests, I noticed SCD41 readings drifted considerably from an adjacent SwitchBot monitor.
| SCD41 | SwitchBot | |
|---|---|---|
| CO2 | 1216 ppm | 733 ppm |
| Temperature | 26.6°C | 27.8°C |
| Humidity | 84.9% | 65% |
The SCD41 had been in storage for several months.
Its automatic baseline calibration requires days of continuous operation with periodic exposure to fresh outdoor air (~400 ppm).
Forced calibration likewise requires exposure to an atmosphere of known CO2 concentration.
Operating fresh out of the drawer without either routine explained the large discrepancy.
Temperature readings depend on the sensor’s internal offset parameter, which compensates for self-heating.
Querying the device returned an offset of 6.02°C, well above the factory default of 4.00°C.
This was residual tuning from July, when the sensor had been housed inside a compact custom CO2 monitor case.
| Parameter | July (Inside monitor case) | Current (Mounted on StackChan) |
|---|---|---|
| Temperature offset | 4.00°C → 6.02°C | 6.02°C → 4.34°C → 2.41°C |
| Humidity factor | Scaled by 0.803 in software | Scaled by 0.880 in software |
| Forced CO2 calibration | +313 ppm against 1172 ppm baseline | -382 ppm against 763 ppm baseline |
On each boot, SCD41 temperature readings started around 31°C and took approximately 10 minutes to finish descending.
Recalculating offset against an actual room temperature of 27.8°C yielded 4.34°C, after which readings stabilized between 28.0°C and 28.1°C.
Adjusting temperature offset left relative humidity unchanged at 79%, far from SwitchBot’s 65%.
Writing calibration data directly to sensor EEPROM would disrupt calibration if the unit were ever moved back to the dedicated monitor enclosure. Instead, I saved offsets in the voice firmware’s NVS partition to push during boot initialization.
During the first 10 minutes post-boot, the firmware suppresses temperature and humidity reporting, sending only CO2.
However, rebooting with this 4.34°C offset caused temperature to settle at 25.9°C before dropping further to 25.3°C.
At 19:22, SwitchBot showed 27.3°C, 67%, and 763 ppm, while SCD41 reported 25.4°C, 85.4%, and 1147 ppm.
Outdoor temperature was 22°C; because the SCD41 protrudes externally from Port A, it may catch incoming draft earlier.
The 1.9°C delta held consistent for over 30 minutes, leaving the earlier alignment at 4.34°C unexplained.
After 20 minutes of continuous operation, I performed forced CO2 calibration using SwitchBot’s 763 ppm reading as baseline.
Applying a -382 ppm correction brought post-calibration CO2 to 739 ppm.
With temperature delta holding steady, I re-adjusted offset against 27.3°C, arriving at 2.41°C.
After waiting 10 minutes for temperature to stabilize at 27.4–27.5°C, I applied a software scaling multiplier of 0.880 to map raw humidity (76.2%) to SwitchBot’s 67%.
| SCD41 (19:45, post-calibration) | SwitchBot | |
|---|---|---|
| CO2 | 766 ppm | 762 ppm |
| Temperature | 27.5°C | 27.3°C |
| Humidity | 67.0% | 67% |
This procedure is an interim alignment against an indoor commercial sensor rather than a formal calibration protocol.
Without outdoor forced recalibration or several days of automatic baseline tracking, CO2 drift will inevitably reoccur.
The temperature offset delta between 4.34°C and 2.41°C within the same day also indicates that thermal tracking requires further tuning under varying ambient conditions.