Objective and use case

What you’ll build: A real-time audio spectrogram on an ESP32 that captures audio from an INMP441 I2S microphone, computes an FFT-based spectrogram, streams it over WebSocket, and drives a WS2812B LED ring for visualization.

Why it matters / Use cases

• Real-time audio analysis for music visualizations at live events.
• Educational tool for teaching audio signal processing and visualization techniques.
• Integration with IoT devices for smart home audio applications.
• Prototype for audio-based interactive art installations.

Expected outcome

• Real-time audio streaming with less than 50 ms latency.
• FFT computation with a frequency resolution of 100 Hz.
• WebSocket data transmission rate of 30 packets/s.
• LED ring visualization updates in sync with audio input at 60 FPS.

Audience: Advanced DIY enthusiasts; Level: Advanced

Architecture/flow: ESP32 captures audio -> INMP441 I2S microphone -> FFT processing -> WebSocket streaming -> WS2812B LED visualization.

Advanced Hands‑On Practical: I2S Spectrogram + WebSocket + NeoPixel Visualizer on ESP32

Objective: i2s-spectrogram-websocket-neopixel

This tutorial walks you through building a real‑time audio spectrogram pipeline on an ESP32 that:

• Captures audio from an INMP441 I2S microphone
• Computes an FFT-based spectrogram (log‑scaled frequency bins)
• Streams spectra over a WebSocket to a browser (canvas display)
• Drives a WS2812B LED ring with a live spectrum visualizer

Target device family and exact device model: ESP32-DevKitC V4 + INMP441 I2S mic + WS2812B LED ring

This is an advanced project designed to be precise, reproducible, and portable via PlatformIO.

Prerequisites

• A PC with PlatformIO Core 6.x (via the VS Code extension or CLI) and Git installed.
• Serial driver for the ESP32-DevKitC V4 (CP210x family). Most DevKitC V4 boards use the Silicon Labs CP2102/CP210x USB‑UART bridge.
• Windows: Install “CP210x Universal Windows Driver”
• macOS: Generally works out of the box; if needed, install Silicon Labs CP210x VCP driver.

• Linux: Usually no driver needed; confirm with dmesg after plugging in.

• A stable Wi‑Fi AP (2.4 GHz), with SSID and password you can hardcode for development.

• Power:
• ESP32 via USB (5 V).
• WS2812B ring powered by a dedicated 5 V supply sized for the number of LEDs (e.g., 24 LEDs × 60 mA worst case ≈ 1.44 A). Start with 5 V 2 A.
• Basic soldering/jumper wires and a breadboard or headers.
• This guide assumes PlatformIO CLI and Arduino framework for ESP32 (espressif32 platform) with AsyncWebServer.

Materials (exact models)

• 1x ESP32-DevKitC V4 (with CP2102/CP210x USB‑UART)
• 1x INMP441 I2S digital microphone module (pins: VDD, GND, SCK/BCLK, WS/LRCL, SD, L/R)
• 1x WS2812B LED ring (e.g., 24 LEDs, 5 V)
• Optional but strongly recommended: 74AHCT125 or SN74HCT245 for 3.3 V → 5 V level shifting on the LED data line
• Wires, breadboard, 1000 µF electrolytic capacitor across the LED ring 5 V and GND, and a 330–470 Ω series resistor in the LED data line

Setup / Connection

The INMP441 is an I2S microphone that sends 24‑bit PCM in a 32‑bit I2S slot. The ESP32 I2S peripheral will be configured as master receiver. You do not need MCLK for the INMP441.

The WS2812B expects 5 V power and ~800 kHz 1‑wire data. It often works with 3.3 V logic from ESP32 if the ring is powered ~4.5–5 V and wiring is short, but the recommended approach is a proper 3.3 V→5 V level shifter.

Important: Common ground is mandatory for ESP32, mic, and LED ring.

Pin mapping and wiring

• ESP32-DevKitC V4 default I2S pins chosen:
• BCLK (SCK): GPIO26
• LRCL (WS): GPIO25
• DOUT (mic SD): GPIO33
• INMP441 L/R: tie to GND to output “left” channel (we’ll configure ESP32 to read only left)
• WS2812B LED ring data: GPIO18 (via 330–470 Ω resistor and level shifter if available)
• LED power: external 5 V supply; add a 1000 µF cap across +5 V and GND near the ring; connect grounds

Table: exact connections

Driver notes:
– If you don’t see a new serial port when plugging in the DevKitC V4, install the Silicon Labs CP210x driver and reconnect. Then check:
– Windows: Device Manager → Ports (COM & LPT) → “Silicon Labs CP210x USB to UART Bridge” (note the COM port)
– macOS: ls /dev/cu. (look for /dev/cu.SLAB_USBtoUART)
– Linux: dmesg | tail and ls /dev/ttyUSB

Full Code

We will use PlatformIO with Arduino framework and the following libraries:
– ESPAsyncWebServer (Async WebSocket)
– AsyncTCP
– arduinoFFT
– Adafruit NeoPixel

We’ll serve a minimal HTML/JS page from PROGMEM so no filesystem is required. The browser will show a scrolling spectrogram canvas and live dB readout.

platformio.ini

Use the generic “esp32dev” PlatformIO board profile, which matches ESP32-DevKitC V4.

; platformio.ini (tested with PlatformIO Core 6.x)
[env:esp32dev]
platform = espressif32 @ 6.5.0
board = esp32dev
framework = arduino

; Serial and upload
monitor_speed = 115200
upload_speed = 921600

; If you want to force the serial port, uncomment and set accordingly:
; upload_port = /dev/ttyUSB0
; monitor_port = /dev/ttyUSB0

build_flags =
  -DASYNCWEBSERVER_REGEX=1
  -DCORE_DEBUG_LEVEL=0

lib_deps =
  ; Async WebServer fork maintained for PlatformIO compatibility
  ottowinter/ESPAsyncWebServer-esphome @ ^3.1.0
  me-no-dev/AsyncTCP @ ^1.1.1
  kosme/arduinoFFT @ ^1.6.2
  adafruit/Adafruit NeoPixel @ ^1.12.3

src/main.cpp

#include <Arduino.h>
#include <WiFi.h>
#include "driver/i2s.h"
#include <AsyncTCP.h>
#include <ESPAsyncWebServer.h>
#include <Adafruit_NeoPixel.h>
#include <arduinoFFT.h>

// ================== User configuration ==================
static const char* WIFI_SSID     = "YOUR_WIFI_SSID";
static const char* WIFI_PASSWORD = "YOUR_WIFI_PASSWORD";

// I2S pins for INMP441
#define I2S_PORT          I2S_NUM_0
#define I2S_PIN_BCLK      26  // SCK
#define I2S_PIN_WS        25  // LRCL/WS
#define I2S_PIN_SD        33  // DOUT from mic to ESP32 data_in

// LED ring
#define LED_PIN           18
#define LED_COUNT         24

// Signal processing
#define SAMPLE_RATE       16000
#define FFT_SIZE          1024
#define SPEC_BINS         64
#define MIN_FREQ_HZ       60
#define MAX_FREQ_HZ       8000
#define FRAME_HOP         FFT_SIZE    // 100% overlap=FFT_SIZE, lower for overlap-add

// WebSocket broadcasting
#define WS_PATH           "/ws"
#define HTTP_PORT         80
#define FRAME_INTERVAL_MS 0  // 0 = stream each frame as computed

// ========================================================

Adafruit_NeoPixel strip(LED_COUNT, LED_PIN, NEO_GRB + NEO_KHZ800);
AsyncWebServer server(HTTP_PORT);
AsyncWebSocket ws(WS_PATH);

arduinoFFT FFT = arduinoFFT(); // double-precision routines on floats
static float vReal[FFT_SIZE];
static float vImag[FFT_SIZE];
static float windowHann[FFT_SIZE];

static int binEdges[SPEC_BINS + 1];     // FFT bin indices per log bin (inclusive start, exclusive end)
static float binPower[SPEC_BINS];       // linear power per bin
static float binDb[SPEC_BINS];          // dBFS per bin (negative values)
static float noiseFloorDb = -70.0f;     // adaptive floor for LED mapping
static uint32_t lastFrameMs = 0;

// Serve a tiny HTML/JS page from PROGMEM
static const char INDEX_HTML[] PROGMEM = R"HTML(
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>ESP32 I2S Spectrogram</title>
<style>
  body { font-family: sans-serif; margin: 0; padding: 0; background: #111; color: #ddd; }
  #top { padding: 12px; background: #222; position: sticky; top: 0; }
  #stats { margin-left: 10px; }
  canvas { display: block; width: 100vw; height: 60vh; image-rendering: pixelated; }
  #legend { padding: 8px; }
</style>
</head>
<body>
<div id="top">
  <span>ESP32 I2S Spectrogram</span>
  <span id="stats"></span>
</div>
<canvas id="spec"></canvas>
<div id="legend">WebSocket stream of log-spaced frequency bins (JSON). Resize window to adjust canvas.</div>
<script>
const canvas = document.getElementById('spec');
const ctx = canvas.getContext('2d');
let width = 512, height = 256;
function resize() {
  width = Math.floor(window.innerWidth);
  height = Math.floor(window.innerHeight * 0.6);
  canvas.width = width; canvas.height = height;
}
window.addEventListener('resize', resize); resize();

const nBins = 64;
const specBuffer = [];
const maxCols = 1024;

const statsEl = document.getElementById('stats');
let fpsCnt = 0, lastT = performance.now();

function updateStats() {
  const now = performance.now();
  if (now - lastT > 1000) {
    statsEl.textContent = ` | FPS: ${fpsCnt}`;
    fpsCnt = 0; lastT = now;
  }
}

function palette(v) {
  // v in [0,1]; return [r,g,b]
  const c = Math.max(0, Math.min(1, v));
  const r = Math.floor(255 * Math.pow(c, 1.8));
  const g = Math.floor(255 * Math.sqrt(c));
  const b = Math.floor(255 * (1 - c));
  return [r, g, b];
}

function draw() {
  const cols = specBuffer.length;
  if (cols === 0) return;
  const img = ctx.createImageData(width, height);
  const yBins = nBins;
  // Map columns to width by sampling
  for (let x = 0; x < width; x++) {
    const colIdx = Math.floor((cols - 1) * x / (width - 1));
    const col = specBuffer[colIdx];
    for (let y = 0; y < height; y++) {
      const bin = yBins - 1 - Math.floor(y * yBins / height);
      const v = (col[bin] + 90) / 90;  // -90..0 dB -> 0..1
      const [r,g,b] = palette(v);
      const p = (y * width + x) * 4;
      img.data[p+0] = r; img.data[p+1] = g; img.data[p+2] = b; img.data[p+3] = 255;
    }
  }
  ctx.putImageData(img, 0, 0);
}

let ws;
function connectWS() {
  const loc = window.location;
  const proto = loc.protocol === 'https:' ? 'wss://' : 'ws://';
  ws = new WebSocket(proto + loc.host + '/ws');
  ws.onopen = () => console.log('WS open');
  ws.onmessage = (ev) => {
    try {
      const obj = JSON.parse(ev.data);
      if (!obj || !obj.db) return;
      specBuffer.push(obj.db);
      if (specBuffer.length > maxCols) specBuffer.shift();
      fpsCnt++; updateStats();
      draw();
    } catch(e) {}
  };
  ws.onclose = () => { console.log('WS closed, retrying...'); setTimeout(connectWS, 1000); };
}
connectWS();
</script>
</body>
</html>
)HTML";

// Helper: safe log10f
static inline float log10f_safe(float x) {
  return log10f((x <= 1e-20f) ? 1e-20f : x);
}

// Design Hann window
void makeHann() {
  for (int n = 0; n < FFT_SIZE; n++) {
    windowHann[n] = 0.5f * (1.0f - cosf(2.0f * PI * n / (FFT_SIZE - 1)));
  }
}

// Compute log-spaced bin edges over FFT bins
void makeLogBins() {
  const float fMin = MIN_FREQ_HZ;
  const float fMax = min((float)MAX_FREQ_HZ, SAMPLE_RATE * 0.5f);
  const float logMin = logf(fMin);
  const float logMax = logf(fMax);

  for (int i = 0; i <= SPEC_BINS; i++) {
    const float alpha = (float)i / SPEC_BINS;
    const float f = expf(logMin + alpha * (logMax - logMin));
    int k = (int)roundf(f * FFT_SIZE / SAMPLE_RATE);
    if (k < 1) k = 1;                       // ignore DC bin
    if (k > FFT_SIZE / 2) k = FFT_SIZE / 2; // Nyquist
    binEdges[i] = k;
  }
  // Ensure strictly increasing
  for (int i = 1; i <= SPEC_BINS; i++) {
    if (binEdges[i] <= binEdges[i - 1]) binEdges[i] = binEdges[i - 1] + 1;
    if (binEdges[i] > FFT_SIZE / 2) binEdges[i] = FFT_SIZE / 2;
  }
}

// Simple DC removal filter
float dcMean = 0.0f;
const float dcAlpha = 0.995f;

// LED color mapping
uint32_t colorForDb(float dB) {
  // Normalize from [-90, 0] to [0,1]
  float v = (dB + 90.0f) / 90.0f;
  if (v < 0) v = 0; if (v > 1) v = 1;
  // HSV-like mapping: blue (low) -> green -> yellow -> red (high)
  uint8_t r = (uint8_t)(255.0f * powf(v, 1.8f));
  uint8_t g = (uint8_t)(255.0f * sqrtf(v));
  uint8_t b = (uint8_t)(255.0f * (1.0f - v));
  return strip.Color(r, g, b);
}

// Acquire FFT_SIZE samples from I2S
bool readSamples() {
  size_t bytesRead = 0;
  const size_t wantBytes = FFT_SIZE * sizeof(int32_t); // reading 32-bit frames
  static int32_t i2sBuf[FFT_SIZE];

  esp_err_t err = i2s_read(I2S_PORT, (void*)i2sBuf, wantBytes, &bytesRead, portMAX_DELAY);
  if (err != ESP_OK || bytesRead != wantBytes) return false;

  for (int i = 0; i < FFT_SIZE; i++) {
    // INMP441 provides 24-bit in 32-bit frame, MSB aligned. Shift down to 24-bit signed.
    int32_t s = (int32_t)(i2sBuf[i] >> 8);
    float x = (float)s / 8388608.0f; // 2^23
    // DC high-pass
    dcMean = dcAlpha * dcMean + (1.0f - dcAlpha) * x;
    x -= dcMean;
    // Window
    vReal[i] = x * windowHann[i];
    vImag[i] = 0.0f;
  }
  return true;
}

// Compute FFT and log-binned spectrum in dBFS
void computeSpectrum() {
  FFT.Windowing(vReal, FFT_SIZE, FFT_WIN_TYP_RECTANGLE, FFT_FORWARD); // already windowed
  FFT.Compute(vReal, vImag, FFT_SIZE, FFT_FORWARD);
  FFT.ComplexToMagnitude(vReal, vImag, FFT_SIZE);

  // vReal now holds magnitudes: index k => bin frequency k*Fs/N
  // Accumulate power into log bins
  for (int b = 0; b < SPEC_BINS; b++) binPower[b] = 0.0f;

  for (int b = 0; b < SPEC_BINS; b++) {
    int k0 = binEdges[b];
    int k1 = binEdges[b + 1];
    if (k1 <= k0) k1 = k0 + 1;
    float p = 0.0f;
    for (int k = k0; k < k1 && k < FFT_SIZE / 2; k++) {
      const float m = vReal[k];
      p += m * m; // power
    }
    binPower[b] = p / (float)(k1 - k0);
  }

  // Convert to dBFS; estimate maxRef using a running peak or fixed reference
  static float ref = 1e-3f; // avoid too large dB
  for (int b = 0; b < SPEC_BINS; b++) {
    if (binPower[b] > ref) ref = 0.999f * ref + 0.001f * binPower[b];
  }

  for (int b = 0; b < SPEC_BINS; b++) {
    float dB = 10.0f * log10f_safe(binPower[b] / (ref + 1e-20f));
    if (dB < -120.0f) dB = -120.0f;
    if (dB > 0.0f) dB = 0.0f;
    // Smooth a little
    binDb[b] = 0.6f * binDb[b] + 0.4f * dB;
  }

  // Update adaptive noise floor for LED mapping
  float avg = 0.0f;
  for (int b = 0; b < SPEC_BINS; b++) avg += binDb[b];
  avg /= SPEC_BINS;
  noiseFloorDb = 0.99f * noiseFloorDb + 0.01f * avg;
}

// Map spectrum to LED ring
void updateLeds() {
  // Reduce bins to LED_COUNT by grouping
  for (int i = 0; i < LED_COUNT; i++) {
    int b0 = (i * SPEC_BINS) / LED_COUNT;
    int b1 = ((i + 1) * SPEC_BINS) / LED_COUNT;
    if (b1 <= b0) b1 = b0 + 1;
    float acc = 0.0f;
    for (int b = b0; b < b1; b++) acc += binDb[b];
    float dB = acc / (float)(b1 - b0);

    // Boost relative to noise floor
    float rel = dB - noiseFloorDb; // e.g., if floor -70 dB and dB -40 dB => +30 dB
    float norm = (rel + 60.0f) / 60.0f; // map [-60..+?] dB above floor to [0..1]
    if (norm < 0) norm = 0;
    if (norm > 1) norm = 1;

    // Brightness scaled by norm; color derived from dB itself
    uint32_t c = colorForDb(-90.0f + 90.0f * norm);
    strip.setPixelColor(i, c);
  }
  strip.show();
}

// Broadcast spectrum over WebSocket as compact JSON
void wsBroadcast() {
  // JSON: {"n":64,"db":[-XX,...]}
  // Keep it small
  static char buf[2048];
  char* p = buf;
  p += sprintf(p, "{\"n\":%d,\"db\":[", SPEC_BINS);
  for (int b = 0; b < SPEC_BINS; b++) {
    float d = binDb[b];
    if (d < -90.0f) d = -90.0f;
    if (d > 0.0f) d = 0.0f;
    p += sprintf(p, "%.1f%s", d, (b == SPEC_BINS - 1) ? "" : ",");
  }
  p += sprintf(p, "]}");
  ws.textAll(buf);
}

// I2S configuration
void setupI2S() {
  i2s_config_t i2s_config = {
    .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
    .sample_rate = SAMPLE_RATE,
    .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT,
    .channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
#if SOC_I2S_SUPPORTS_PDM
    .communication_format = I2S_COMM_FORMAT_STAND_MSB,
#else
    .communication_format = I2S_COMM_FORMAT_STAND_MSB,
#endif
    .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
    .dma_buf_count = 8,
    .dma_buf_len = 256,
    .use_apll = false,
    .tx_desc_auto_clear = false,
    .fixed_mclk = 0
  };

  i2s_pin_config_t pin_config = {
    .bck_io_num = I2S_PIN_BCLK,
    .ws_io_num = I2S_PIN_WS,
    .data_out_num = I2S_PIN_NO_CHANGE, // RX only
    .data_in_num = I2S_PIN_SD
  };

  ESP_ERROR_CHECK(i2s_driver_install(I2S_PORT, &i2s_config, 0, NULL));
  ESP_ERROR_CHECK(i2s_set_pin(I2S_PORT, &pin_config));
  ESP_ERROR_CHECK(i2s_set_clk(I2S_PORT, SAMPLE_RATE, I2S_BITS_PER_SAMPLE_32BIT, I2S_CHANNEL_MONO));
}

void setupWiFi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  Serial.printf("Connecting to WiFi SSID '%s'...\n", WIFI_SSID);
  int tries = 0;
  while (WiFi.status() != WL_CONNECTED) {
    delay(300);
    Serial.print(".");
    if (++tries > 100) {
      Serial.println("\nWiFi connect timeout, restarting...");
      ESP.restart();
    }
  }
  Serial.printf("\nWiFi connected. IP: %s\n", WiFi.localIP().toString().c_str());
}

void setupWeb() {
  server.on("/", HTTP_GET, [](AsyncWebServerRequest* req) {
    req->send_P(200, "text/html", INDEX_HTML);
  });
  ws.onEvent([](AsyncWebSocket* s, AsyncWebSocketClient* c, AwsEventType t, void* arg, uint8_t* data, size_t len) {
    if (t == WS_EVT_CONNECT) {
      Serial.printf("WS client %u connected\n", c->id());
    } else if (t == WS_EVT_DISCONNECT) {
      Serial.printf("WS client %u disconnected\n", c->id());
    }
  });
  server.addHandler(&ws);
  server.begin();
}

void setup() {
  Serial.begin(115200);
  delay(100);
  Serial.println("\nESP32 I2S Spectrogram + WebSocket + NeoPixel starting...");

  makeHann();
  makeLogBins();
  setupI2S();

  strip.begin();
  strip.clear();
  strip.show();

  setupWiFi();
  setupWeb();

  Serial.printf("I2S: SR=%d, FFT=%d, Bins=%d; WS at ws://%s%s\n",
                SAMPLE_RATE, FFT_SIZE, SPEC_BINS, WiFi.localIP().toString().c_str(), WS_PATH);
}

void loop() {
  if (!readSamples()) return;
  computeSpectrum();
  updateLeds();

  uint32_t now = millis();
  if (FRAME_INTERVAL_MS == 0 || (now - lastFrameMs) >= FRAME_INTERVAL_MS) {
    wsBroadcast();
    lastFrameMs = now;
  }

  // Give AsyncTCP time
  ws.cleanupClients();
}

Notes on the code:
– FFT size 1024 at 16 kHz → ~15.6 Hz bin spacing in raw FFT; after log binning we get 64 bins covering 60–8000 Hz.
– We read INMP441 as 32‑bit samples and right‑shift by 8 to get the 24‑bit signed value, normalize to float, apply a Hann window, run FFT, and aggregate power into log-spaced bins.
– A simple adaptive reference and noise floor are used for stable LEDs and spectrogram scaling.
– The HTML page connects to ws:///ws and draws a scrolling spectrogram.
– NeoPixel map groups the 64 bins down to 24 LEDs for a compact visual display.

Build / Flash / Run commands

You can use either VS Code + PlatformIO extension or pure CLI.

1) Initialize (if starting from an empty folder):

pio project init --board esp32dev

2) Put the above platformio.ini and src/main.cpp in your project.

3) Install/resolve libraries and check the environment:

pio pkg update
pio run

4) Connect the ESP32-DevKitC V4 via USB. Identify serial port:
– Windows: check Device Manager for the COM port (CP210x).
– macOS: ls /dev/cu. (likely /dev/cu.SLAB_USBtoUART).
– Linux: ls /dev/ttyUSB or dmesg | tail.

Optionally set upload_port in platformio.ini.

5) Build and upload:

pio run -t upload

6) Open the serial monitor to watch logs:

pio device monitor --baud 115200

7) Find the printed IP address. In your browser on the same LAN, open:
– http:// (serves the spectrogram UI)
– The WebSocket endpoint is at ws:///ws (handled by the page automatically)

Step‑by‑step Validation

Follow this sequence to validate each subsystem before evaluating the full integrated pipeline.

1) USB/UART enumeration (driver)

• Plug in your ESP32-DevKitC V4.
• Verify the CP210x serial device appears.
• If not, install the Silicon Labs CP210x driver and try a different USB cable/port.

2) Firmware boots and Wi‑Fi connects

• Open serial monitor:
• pio device monitor –baud 115200
• Expected output:
• “ESP32 I2S Spectrogram + WebSocket + NeoPixel starting…”
• “Connecting to WiFi SSID ‘…’ ….”
• “WiFi connected. IP: x.x.x.x”
• “I2S: SR=16000, FFT=1024, Bins=64; WS at ws://x.x.x.x/ws”
• If it reboots or brownouts, see “Troubleshooting” (power/USB issues).

3) INMP441 hardware sanity

• With the device running and serial monitor open, speak or clap near the mic.
• If you added a temporary debug print inside computeSpectrum (optional) for avg dB, you should see values rising on sound.
• If it is all zeros:
• Check the I2S pins 26/25/33 match your wiring.
• Ensure INMP441 L/R is tied to GND (left channel).
• Confirm 3.3 V power (not 5 V!) to the mic and solid ground.

4) WebSocket and spectrogram UI

• In a desktop browser on the same network, go to http://.
• The page should load and connect to the WebSocket (check browser console).
• You should see a colored spectrogram that scrolls as frames arrive.
• Speak or play a test tone and watch a narrow band appear:
• 1 kHz test tone should appear in the lower quarter of the vertical axis (log‑scaled).
• You can verify frame rate via the FPS indicator (top bar).

Optional: generate test tones from your computer speakers:
– macOS:
– brew install sox
– play -n synth 30 sin 1000
– Linux with sox:
– play -n synth 30 sin 1000
– Windows (PowerShell, if sox installed and on PATH):
– play -n synth 30 sin 1000

5) LED ring spectrum

• Observe the ring while producing sounds:
• Low frequencies will excite certain LEDs (depending on the grouping), color shifting from blue/green to yellow/red with higher energy.
• If no LEDs light:
• Verify LED 5 V power, common ground, data on GPIO18, series resistor present.
• Try adding a level shifter (74AHCT125).
• Check strip.begin() and strip.show() are called; try a quick self-test: set all to red in setup() to validate hardware.

6) Frequency sanity check

• With SAMPLE_RATE=16000 and FFT_SIZE=1024, the raw FFT bin spacing is Fs/N ≈ 15.625 Hz.
• Our log bins are defined from 60–8000 Hz (clamped to Nyquist 8 kHz).
• Playing 1 kHz should show a bright band near 1 kHz; try 2 kHz, 4 kHz and observe vertical position changes correspondingly.

7) Performance/latency check

• Expected end‑to‑end latency (mic → FFT → WebSocket → draw) is roughly one frame duration (≈64 ms) plus Wi‑Fi/JS overhead.
• CPU usage on ESP32 is moderate with FFT_SIZE=1024 and AsyncWebSocket; if frames drop, consider reducing SPEC_BINS or switching to binary frames (see Improvements).

Troubleshooting

• No serial port / COM port missing:
• Use a data‑capable USB cable.
• Install CP210x driver, reconnect, try another USB port.
• Reboots or “Brownout detector was triggered”:
• Provide stable 5 V supply, avoid powering a large LED ring from the ESP32 USB.
• Add a 1000 µF capacitor on the LED ring 5 V/GND near the strip.
• INMP441 reads all zeros:
• Verify pin mapping (26=BCLK, 25=WS, 33=SD).
• Confirm I2S mode: master RX, 32‑bit slots, only left channel.
• L/R must be tied to GND on INMP441 to output left; or change I2S_CHANNEL_FMT_ONLY_RIGHT if tied to VDD.
• No MCLK is required for INMP441; do not connect it.
• Distorted or very low amplitude:
• Check the right shift by 8 for 24‑bit data from 32‑bit frame.
• Try reducing ambient noise and verify mic orientation (sound port facing the source).
• Wi‑Fi connects but page does not load:
• Ensure the PC is on the same subnet.
• Check firewall rules.
• Try http:/// in an incognito window.
• WebSocket connects but no frames:
• Ensure ws.cleanupClients() is called in loop().
• Check that wsBroadcast() is called regularly.
• Watch serial logs for memory errors.
• LED ring flickers or random colors:
• Common ground between ESP32 and LED supply is required.
• Add 330–470 Ω series resistor in DIN.
• Prefer a proper 3.3→5 V level shifter (74AHCT125).
• Keep data wire short; avoid running alongside power lines.
• High CPU or dropped frames:
• Use smaller FFT (e.g., 512) or fewer SPEC_BINS (e.g., 48).
• Set FRAME_INTERVAL_MS to throttle WebSocket sending.
• Consider switching to binary WebSocket frames to cut JSON overhead.
• Spectrogram skew or odd banding:
• Verify SAMPLE_RATE and FFT_SIZE constants on both acquisition and binning stages.
• For musical content, mel-scaling can improve perceptual distribution (see Improvements).

Improvements

• Binary WebSocket frames:
• Replace JSON with a compact binary blob (e.g., int16 dB × 64) to dramatically reduce bandwidth and GC on the ESP.
• On the browser side, use WebSocket binaryType=’arraybuffer’ and DataView for parsing.
• Accurate clocking:
• Enable APLL in I2S to minimize sampling rate drift for better frequency accuracy.
• Overlap and windowing:
• Implement 50% overlap (hop size FFT_SIZE/2) and use an additional window to stabilize time resolution.
• Mel filter bank:
• Convert FFT magnitude to mel bins for a perceptually meaningful spectrogram; fewer bins (e.g., 40) are sufficient.
• LED mapping:
• Gamma correction and a perceptual color palette (e.g., viridis) improve visibility.
• Map low/mid/high bands to distinct LED groups with different colors.
• Dynamic range control:
• Implement AGC on the magnitude spectrum for consistent visuals across environments.
• OTA updates:
• Add ArduinoOTA or ESP32 HTTP OTA for faster iteration without USB.
• Credentials and provisioning:
• Move Wi‑Fi SSID/password to a separate header or use WiFiManager for runtime provisioning.
• Multi‑client streaming:
• Throttle sending (e.g., 10–20 FPS) and add a ring buffer for late joiners.
• Power safety:
• If you scale up LEDs, add a fuse on the 5 V line and ensure adequate PSU headroom.

Final Checklist

• Hardware
• ESP32-DevKitC V4 connected via CP210x USB‑UART
• INMP441 wired: 3.3 V, GND, SCK→GPIO26, WS→GPIO25, SD→GPIO33, L/R→GND
• WS2812B ring: +5 V, GND common, data from GPIO18 via series resistor and ideally a 74AHCT125 level shifter

• 1000 µF cap across LED 5 V and GND

• Software

• PlatformIO Core 6.x installed
• platform = espressif32 @ 6.5.0, framework = arduino
• lib_deps: ESPAsyncWebServer-esphome, AsyncTCP, arduinoFFT, Adafruit NeoPixel

• monitor_speed = 115200, upload_speed = 921600

• Build/Flash

• pio run -t upload succeeds without errors
• Serial monitor shows Wi‑Fi IP and I2S params

• Browser loads http:// and spectrogram updates in real time

• Validation

• Claps/voice produce visible bands on the canvas
• 1 kHz test tone yields a stable horizontal band around 1 kHz

• LED ring responds to audio with color/brightness changes

• Stability

• No brownouts or random resets under typical use
• LED supply sized appropriately; grounds are common
• WebSocket stable with at least one browser client

With the ESP32-DevKitC V4 + INMP441 I2S mic + WS2812B LED ring configured as above, you now have a robust i2s-spectrogram-websocket-neopixel pipeline suitable for further DSP experimentation, UI enhancements, and embedded audiovisual projects.

Objective and use case

What you’ll build: This project implements real-time acoustic anomaly detection on an ESP32 using an I2S MEMS microphone (INMP441). Anomaly alerts are routed over MQTT, with visual status via a WS2812B LED and audible feedback through an I2S DAC/amplifier (MAX98357A).

Why it matters / Use cases

• Real-time monitoring of industrial equipment for anomalies, reducing downtime and maintenance costs.
• Smart home applications where audio feedback can alert users to unusual sounds, enhancing security.
• Environmental monitoring systems that detect specific sound patterns indicative of wildlife or machinery.
• Healthcare applications for monitoring patient environments, alerting staff to unusual sounds.

Expected outcome

• Detection of anomalies with a latency of less than 100 ms.
• MQTT alert messages sent within 200 ms of detection.
• Visual feedback via WS2812B LED indicating status changes with a response time of 50 ms.
• Audio feedback generated through MAX98357A with a response time of 100 ms.

Audience: Intermediate developers; Level: Advanced

Architecture/flow: ESP32 processes audio input from INMP441, analyzes it for anomalies, and communicates results via MQTT while providing visual and audio feedback.

ESP32 Advanced Practical: I2S Anomaly Detection with MQTT, Audio Beep, and LED Feedback

Objective: i2s-anomaly-detection-mqtt
Device family and exact model used: Espressif ESP32-DevKitC V4 (ESP32-WROOM-32) + INMP441 + MAX98357A + WS2812B

This advanced hands-on guide implements real-time acoustic anomaly detection on an ESP32 using an I2S MEMS microphone (INMP441), with anomaly alerts routed over MQTT, visual status via a WS2812B LED, and an audible beep generated through an I2S DAC/amplifier (MAX98357A). The project is reproducible with PlatformIO, includes full source, pin mapping, build/flash commands, and a validation workflow.

---

Prerequisites

• Comfortable with:
• ESP32 and PlatformIO on VS Code or CLI.
• Basic DSP concepts (FFT, feature extraction, thresholds).
• MQTT brokers and topics.
• Soldering and breadboarding.
• OS with serial driver support:
• Windows: Silicon Labs CP210x USB-to-UART driver (for DevKitC’s CP2102/CP2104).
• macOS: Typically driverless for CP210x, but install from Silicon Labs if needed.
• Linux: Usually driverless; confirm device appears as /dev/ttyUSBx or /dev/tty.SLAB_USBtoUART.
• Installed software:
• PlatformIO Core (CLI) 6.x (tested with 6.1.11) or VS Code + PlatformIO extension.
• Mosquitto clients for validation (mosquitto_pub, mosquitto_sub).
• A working MQTT broker (e.g., Mosquitto on localhost or a network host).
• Solid 5V power if you plan to drive a speaker from MAX98357A (ESP32’s 5V pin can power small loads via USB, but consider a separate supply if needed).

---

Materials (Exact Models)

• Microcontroller: Espressif ESP32-DevKitC V4 (ESP32-WROOM-32)
• Digital MEMS Microphone (I2S): INMP441
• I2S DAC/Amplifier: MAX98357A (Adafruit MAX98357 I2S Class-D Mono Amp or equivalent breakout)
• Addressable RGB LED: WS2812B (single LED or short strip; you can cut just one pixel from a strip)
• Speaker: 4–8 Ω small speaker for MAX98357A output
• Breadboard + jumpers
• USB cable for ESP32 DevKitC
• Optional level shifting for WS2812B data signal (often works at 3.3 V for a single LED, but best practice is to buffer)

Driver note:
– If your Espressif ESP32-DevKitC V4 enumerates as an unknown device on Windows, install the Silicon Labs CP210x driver from https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers.

---

Setup/Connection

We will use two I2S peripherals in the ESP32:

• I2S0 (RX) for INMP441 microphone capture.
• I2S1 (TX) for audio output to MAX98357A.

And a separate GPIO for WS2812B data.

Recommended pin mapping (tested, doesn’t conflict with flash/boot pins):

• INMP441 (I2S RX, 3.3V device):
• VDD → 3V3
• GND → GND
• L/R → GND (selects Left channel)
• SCK/BCLK → GPIO32
• WS/LRCL → GPIO33
• SD (DOUT) → GPIO34 (input-only pin)
• MAX98357A (I2S TX, powered from 5V):
• VIN → 5V
• GND → GND
• DIN → GPIO26
• BCLK → GPIO27
• LRC → GPIO25
• SD/GAIN pin per board variant (tie high to enable or leave floating as per module; see your board’s datasheet)
• Speaker → + and – outputs
• WS2812B:
• 5V → 5V
• GND → GND (must share common ground with ESP32)
• Data In → GPIO4
• Place a 330–470 Ω resistor in series with the data line if possible; add a 1000 µF capacitor across 5V/GND near the LED strip if driving more than one LED (for a single LED, optional).

Power notes:
– The ESP32 DevKitC can supply limited current from 5V via USB. The MAX98357A plus a small speaker at moderate volume is typically fine. If you experience brownouts, provide an external 5V supply and connect grounds.

I2S details:
– Microphone (INMP441) outputs 24-bit samples, left-justified in I2S frames. We’ll capture as 32-bit and shift to 16-bit for DSP.
– MAX98357A reads standard I2S stereo (we’ll feed identical samples for L/R or generate a mono stream in both channels).

---

Table: Wiring Summary

---

Full Code

The project is structured for PlatformIO (Arduino framework). It uses:

• I2S driver from ESP-IDF headers (via Arduino core) for both RX and TX.
• FFT-based features using arduinoFFT.
• MQTT via PubSubClient.
• LED status via Adafruit NeoPixel.

Create a new PlatformIO project and replace the files as shown below.

platformio.ini

Place this in the project root.

; File: platformio.ini
[env:esp32dev]
platform = espressif32 @ 6.5.0
board = esp32dev
framework = arduino
board_build.flash_mode = dio
; optional: larger app partition if you add more features
; board_build.partitions = huge_app.csv

monitor_speed = 115200
monitor_filters = time, esp32_exception_decoder, default

lib_deps =
  arduinoFFT@^1.6.0
  knolleary/PubSubClient@^2.8
  adafruit/Adafruit NeoPixel@^1.12.0

build_flags =
  -DCORE_DEBUG_LEVEL=3
  -DARDUINO_USB_MODE=1
  -DWS2812_PIN=4
  -DMIC_BCLK=32
  -DMIC_LRCL=33
  -DMIC_DOUT=34
  -DSPK_BCLK=27
  -DSPK_LRCL=25
  -DSPK_DIN=26
  -DMQTT_TOPIC_PREFIX=\"device/esp32-anom\"
  -DMQTT_QOS=1

src/main.cpp

// File: src/main.cpp
#include <Arduino.h>
#include <WiFi.h>
#include <PubSubClient.h>
#include <Adafruit_NeoPixel.h>
#include <driver/i2s.h>
#include <arduinoFFT.h>

// ---------------------- User Configuration ----------------------
static const char* WIFI_SSID = "YOUR_WIFI_SSID";
static const char* WIFI_PASS = "YOUR_WIFI_PASSWORD";

// MQTT broker settings
static const char* MQTT_HOST = "192.168.1.100";
static const uint16_t MQTT_PORT = 1883;
static const char* MQTT_USER = "";  // optional
static const char* MQTT_PASS = "";  // optional

// Topics
#ifndef MQTT_TOPIC_PREFIX
#define MQTT_TOPIC_PREFIX "device/esp32-anom"
#endif

// ---------------------- I2S Config ------------------------------
// Microphone (I2S0 RX)
#ifndef MIC_BCLK
#define MIC_BCLK 32
#endif
#ifndef MIC_LRCL
#define MIC_LRCL 33
#endif
#ifndef MIC_DOUT
#define MIC_DOUT 34
#endif

// Speaker (I2S1 TX)
#ifndef SPK_BCLK
#define SPK_BCLK 27
#endif
#ifndef SPK_LRCL
#define SPK_LRCL 25
#endif
#ifndef SPK_DIN
#define SPK_DIN 26
#endif

// ---------------------- LED Config ------------------------------
#ifndef WS2812_PIN
#define WS2812_PIN 4
#endif
#define WS2812_COUNT 1

// ---------------------- DSP/Anomaly Config ----------------------
static const uint32_t SAMPLE_RATE = 16000;    // 16 kHz
static const size_t FFT_N = 512;              // 32 ms frame
static const size_t HOP = FFT_N;              // no overlap for simplicity
static const uint8_t NUM_BANDS = 16;          // coarse spectral bands
static const size_t BASELINE_FRAMES = 300;    // ~9.6 s baseline (300 * 32ms)
static const float ANOMALY_THRESHOLD = 25.0f; // sum |z| across bands

// ---------------------- Globals --------------------------------
WiFiClient espClient;
PubSubClient mqtt(espClient);

Adafruit_NeoPixel pixel(WS2812_COUNT, WS2812_PIN, NEO_GRB + NEO_KHZ800);

// FFT arrays
arduinoFFT FFT = arduinoFFT();
double vReal[FFT_N];
double vImag[FFT_N];

float bandMeans[NUM_BANDS];
float bandStd[NUM_BANDS];
bool baselineDone = false;
size_t baselineCount = 0;

char deviceId[32];

// ---------------------- Utilities -------------------------------
void setLED(uint8_t r, uint8_t g, uint8_t b) {
  pixel.setPixelColor(0, pixel.Color(r, g, b));
  pixel.show();
}

void ledPulseRed() {
  static uint8_t phase = 0;
  uint8_t level = (uint8_t)(127 + 127 * sin(phase * 0.1));
  setLED(level, 0, 0);
  phase++;
}

void ledGreenSolid() {
  setLED(0, 64, 0);
}

void ledBlueBlink() {
  static bool on = false;
  setLED(0, 0, on ? 64 : 0);
  on = !on;
}

void beepAlert(uint16_t freq = 1000, uint16_t ms = 200, uint16_t volume = 2000) {
  // Simple sine wave beep on I2S1 TX
  i2s_config_t i2s_config = {
    .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_TX),
    .sample_rate = (int)SAMPLE_RATE,
    .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
    .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,
    .communication_format = I2S_COMM_FORMAT_I2S_MSB,
    .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
    .dma_buf_count = 8,
    .dma_buf_len = 256,
    .use_apll = false,
    .tx_desc_auto_clear = true,
    .fixed_mclk = 0
  };

  i2s_pin_config_t pin_config = {
    .bck_io_num = SPK_BCLK,
    .ws_io_num = SPK_LRCL,
    .data_out_num = SPK_DIN,
    .data_in_num = I2S_PIN_NO_CHANGE
  };

  i2s_driver_install(I2S_NUM_1, &i2s_config, 0, NULL);
  i2s_set_pin(I2S_NUM_1, &pin_config);
  i2s_set_clk(I2S_NUM_1, SAMPLE_RATE, I2S_BITS_PER_SAMPLE_16BIT, I2S_CHANNEL_STEREO);

  const uint32_t total_samples = (SAMPLE_RATE * ms) / 1000;
  const float omega = 2.0f * PI * freq / SAMPLE_RATE;

  for (uint32_t n = 0; n < total_samples; n++) {
    int16_t s = (int16_t)(volume * sinf(omega * n));
    uint32_t stereo = ((uint16_t)s << 16) | ((uint16_t)s);
    size_t written;
    i2s_write(I2S_NUM_1, &stereo, sizeof(stereo), &written, portMAX_DELAY);
  }

  i2s_driver_uninstall(I2S_NUM_1);
}

// Compute 16 log-energy bands from FFT magnitude
void computeBands(const double* mag, float* bands, size_t nfft, uint32_t fs) {
  // Frequency resolution
  const double df = (double)fs / (double)nfft;
  // Use pseudo-mel spacing over 0..8kHz (simple linear-to-log mapping)
  // Band edges exponentially spaced
  double fmin = 100.0, fmax = fs / 2.0;
  double logmin = log10(fmin);
  double logmax = log10(fmax);
  for (int b = 0; b < NUM_BANDS; b++) {
    double e0 = pow(10.0, logmin + (logmax - logmin) * b / NUM_BANDS);
    double e1 = pow(10.0, logmin + (logmax - logmin) * (b + 1) / NUM_BANDS);
    int k0 = (int)floor(e0 / df);
    int k1 = (int)ceil(e1 / df);
    if (k0 < 1) k0 = 1; // skip DC
    if (k1 >= (int)(nfft / 2)) k1 = (nfft / 2) - 1;

    double sum = 0.0;
    for (int k = k0; k <= k1; k++) {
      sum += mag[k] * mag[k]; // power
    }
    bands[b] = (sum > 1e-12) ? (float)log10(sum) : -12.0f;
  }
}

float updateBaselineAndScore(const float* bands) {
  if (!baselineDone) {
    // Welford-like online mean/std (keep running stats; std computed post)
    static double m[NUM_BANDS] = {0};
    static double s[NUM_BANDS] = {0};
    baselineCount++;
    for (int b = 0; b < NUM_BANDS; b++) {
      double x = bands[b];
      double delta = x - m[b];
      m[b] += delta / baselineCount;
      s[b] += delta * (x - m[b]);
    }
    if (baselineCount >= BASELINE_FRAMES) {
      for (int b = 0; b < NUM_BANDS; b++) {
        bandMeans[b] = (float)m[b];
        double var = (baselineCount > 1) ? s[b] / (baselineCount - 1) : 1e-3;
        bandStd[b] = (float)sqrt(var + 1e-6);
      }
      baselineDone = true;
    }
    return 0.0f; // no scoring during baseline
  } else {
    float score = 0.0f;
    for (int b = 0; b < NUM_BANDS; b++) {
      float z = (bands[b] - bandMeans[b]) / (bandStd[b] + 1e-6f);
      score += fabsf(z);
    }
    return score;
  }
}

void publishMQTT(const String& topic, const String& payload, bool retain = false) {
  String full = String(MQTT_TOPIC_PREFIX) + "/" + topic;
  mqtt.publish(full.c_str(), payload.c_str(), retain);
}

void connectWiFi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(WIFI_SSID, WIFI_PASS);
  Serial.printf("[WiFi] Connecting to %s\n", WIFI_SSID);
  uint32_t start = millis();
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
    ledBlueBlink();
    if (millis() - start > 30000) {
      Serial.println("\n[WiFi] Timeout, restarting...");
      ESP.restart();
    }
  }
  Serial.printf("\n[WiFi] Connected, IP: %s\n", WiFi.localIP().toString().c_str());
}

void connectMQTT() {
  mqtt.setServer(MQTT_HOST, MQTT_PORT);
  mqtt.setKeepAlive(30);
  while (!mqtt.connected()) {
    String clientId = String("esp32-anom-") + String(deviceId);
    Serial.printf("[MQTT] Connecting to %s as %s ...\n", MQTT_HOST, clientId.c_str());
    if (mqtt.connect(clientId.c_str(), MQTT_USER, MQTT_PASS)) {
      Serial.println("[MQTT] Connected");
      publishMQTT("status", "{\"state\":\"online\"}", true);
      publishMQTT("info", String("{\"device\":\"") + deviceId + "\",\"sr\":" + SAMPLE_RATE + ",\"fft\":" + FFT_N + "}");
    } else {
      Serial.printf("[MQTT] failed rc=%d, retrying in 3s\n", mqtt.state());
      delay(3000);
    }
  }
}

void initI2S_RX() {
  i2s_config_t i2s_config = {
    .mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
    .sample_rate = (int)SAMPLE_RATE,
    .bits_per_sample = I2S_BITS_PER_SAMPLE_32BIT,
    .channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
    .communication_format = I2S_COMM_FORMAT_I2S_MSB,
    .intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,
    .dma_buf_count = 8,
    .dma_buf_len = 256,
    .use_apll = false,
    .tx_desc_auto_clear = false,
    .fixed_mclk = 0
  };
  i2s_pin_config_t pin_config = {
    .bck_io_num = MIC_BCLK,
    .ws_io_num = MIC_LRCL,
    .data_out_num = I2S_PIN_NO_CHANGE,
    .data_in_num = MIC_DOUT
  };
  i2s_driver_install(I2S_NUM_0, &i2s_config, 0, NULL);
  i2s_set_pin(I2S_NUM_0, &pin_config);
  i2s_set_clk(I2S_NUM_0, SAMPLE_RATE, I2S_BITS_PER_SAMPLE_32BIT, I2S_CHANNEL_MONO);
  Serial.println("[I2S] RX initialized");
}

void setup() {
  Serial.begin(115200);
  delay(200);

  // Unique device id
  uint64_t mac = ESP.getEfuseMac();
  snprintf(deviceId, sizeof(deviceId), "%04X%04X", (uint16_t)(mac >> 32), (uint32_t)(mac & 0xFFFF));

  pixel.begin();
  pixel.clear();
  pixel.show();
  setLED(0, 0, 32); // Blue: boot

  connectWiFi();
  connectMQTT();

  initI2S_RX();

  setLED(32, 32, 0); // Yellow: baseline learning
}

void loop() {
  mqtt.loop();

  // Buffer to read one frame
  static int32_t raw32[FFT_N];
  size_t bytes_read = 0;
  size_t want = sizeof(raw32);
  uint8_t* ptr = (uint8_t*)raw32;
  while (bytes_read < want) {
    size_t n = 0;
    esp_err_t err = i2s_read(I2S_NUM_0, ptr + bytes_read, want - bytes_read, &n, portMAX_DELAY);
    if (err != ESP_OK) {
      Serial.printf("[I2S] read error: %d\n", (int)err);
      break;
    }
    bytes_read += n;
  }

  // Convert to float/double for FFT, with 16-bit scaling
  // INMP441: 24-bit valid left-justified in 32-bit word
  // Shift right 8 to get 24->16 bits, then apply window
  double dc = 0.0;
  for (size_t i = 0; i < FFT_N; i++) {
    int32_t s32 = raw32[i] >> 8; // 24b -> 16b
    int16_t s16 = (int16_t)(s32 & 0xFFFF);
    double x = (double)s16 / 32768.0;
    vReal[i] = x;
    dc += x;
    vImag[i] = 0.0;
  }
  // Remove DC offset
  dc /= FFT_N;
  for (size_t i = 0; i < FFT_N; i++) {
    vReal[i] -= dc;
  }

  // Hann window
  for (size_t i = 0; i < FFT_N; i++) {
    vReal[i] *= 0.5 * (1.0 - cos((2.0 * PI * i) / (FFT_N - 1)));
  }

  // FFT
  FFT.Windowing(vReal, FFT_N, FFT_WIN_TYP_RECTANGLE, FFT_FORWARD); // already windowed manually
  FFT.Compute(vReal, vImag, FFT_N, FFT_FORWARD);
  FFT.ComplexToMagnitude(vReal, vImag, FFT_N);

  // Compute bands
  float bands[NUM_BANDS];
  computeBands(vReal, bands, FFT_N, SAMPLE_RATE);

  // Update baseline or compute score
  float score = updateBaselineAndScore(bands);

  static uint32_t lastPub = 0;
  uint32_t now = millis();

  if (!baselineDone) {
    if (now - lastPub > 1000) {
      String payload = String("{\"state\":\"baseline\",\"frames\":") + baselineCount + "}";
      publishMQTT("status", payload);
      lastPub = now;
    }
    // LED: yellow during baseline
    setLED(32, 32, 0);
  } else {
    // LED and beeper behavior
    if (score > ANOMALY_THRESHOLD) {
      ledPulseRed();
      beepAlert(1000, 150, 2500);
      // Publish anomaly event with summary
      String payload = String("{\"anomaly\":true,\"score\":") + String(score, 2) +
                       ",\"sr\":" + SAMPLE_RATE + ",\"fft\":" + FFT_N + "}";
      publishMQTT("anomaly", payload);
    } else {
      ledGreenSolid();
    }

    // Periodic metrics
    if (now - lastPub > 2000) {
      // publish a compressed band snapshot (first 5 for brevity) and score
      String payload = String("{\"score\":") + String(score, 2) + ",\"bands\":[";
      for (int i = 0; i < NUM_BANDS; i++) {
        payload += String(bands[i], 3);
        if (i < NUM_BANDS - 1) payload += ",";
      }
      payload += "]}";
      publishMQTT("metrics", payload);
      lastPub = now;
    }
  }
}

Notes on code:

• I2S RX: Configured for 32-bit samples; the INMP441’s 24-bit payload is right-shifted to get a 16-bit normalized signal for processing.
• Feature extraction: 16 log-power bands from the magnitude spectrum; baseline mean/std built online; the anomaly score is the sum of absolute z-scores.
• MQTT: Periodically publishes metrics; sends an anomaly message when threshold is exceeded.
• LED: Blue during boot; Yellow during baseline; Green when normal; pulsing Red on anomaly.
• Audio: On anomaly, a short sine beep is generated and sent to MAX98357A via I2S1.

---

Build/Flash/Run Commands

Use PlatformIO Core (CLI). Ensure your USB serial port is recognized (e.g., COMx on Windows, /dev/ttyUSBx or /dev/tty.SLAB_USBtoUART on Linux/macOS).

Initialize (if creating from scratch):

pio project init --board esp32dev

Build:

pio run

Upload (auto-detects port):

pio run --target upload

Open serial monitor at 115200 baud:

pio device monitor -b 115200

If you have multiple serial devices, specify port:

pio device monitor -b 115200 -p /dev/ttyUSB0

Upgrade PlatformIO platforms if needed:

pio platform update espressif32

---

Step-by-step Validation

Follow these steps in order. Keep the serial monitor open to observe logs.

1) Power and Basic Bring-up
– Connect the ESP32-DevKitC V4 via USB.
– Verify the CP210x driver is installed if the serial port is missing.
– On reset, the LED should show blue, then yellow while baseline is building.

2) Verify Wi-Fi and MQTT Connectivity
– Ensure WIFI_SSID and WIFI_PASS are correct in main.cpp.
– Ensure MQTT_HOST and MQTT_PORT point to your broker.
– Monitor serial output:
– [WiFi] Connected, IP: …
– [MQTT] Connected
– Use mosquitto_sub to watch topics:

mosquitto_sub -h 192.168.1.100 -t "device/esp32-anom/#" -v

You should see “status”, “info”, and periodic “metrics” messages once baseline completes.

3) Verify Microphone Capture
– During baseline, keep the environment at normal noise levels. Speak or clap and observe that baseline count increases to 300 (~10 seconds).
– After baselineDone is true (shown implicitly by metrics messages), the LED should turn green when quiet.

4) Validate Anomaly Detection
– Create a distinct sound (e.g., jingling keys, whistle, or banging a cup).
– Observe:
– The LED should pulse red briefly during the event.
– You should hear a short beep from the speaker (MAX98357A).
– In mosquitto_sub, you should see an “anomaly” message like:
device/esp32-anom/anomaly {«anomaly»:true,»score»:38.42,»sr»:16000,»fft»:512}

5) Validate Metrics Payloads
– Monitor metrics payload content:
– Confirm “score” keeps low steady values when quiet and higher spikes during events.
– Confirm “bands” contains 16 values (log-power per band).

6) Test MQTT Retained Status
– Restart the ESP32; check that device/esp32-anom/status retains {«state»:»online»} after reconnection.
– Confirm the “info” message with device id, sample rate, FFT size is sent on connect.

7) Validate Speaker Output
– If no beep is heard, verify wiring to MAX98357A:
– DIN → GPIO26, BCLK → GPIO27, LRC → GPIO25, VIN → 5V, GND → GND.
– Try increasing beep volume parameter in beepAlert (e.g., 3000–6000). Beware of clipping and speaker limits.

8) Validate LED Behavior
– During baseline: Yellow.
– Normal: Green solid.
– Anomaly: Pulsing Red during the event.

---

Troubleshooting

• Serial port missing (Windows):
• Install Silicon Labs CP210x VCP driver.
• Try a different USB cable or port.
• Boot loops or random resets:
• Power supply marginal. Lower speaker volume, use a powered USB hub, or provide a separate 5V supply for MAX98357A and LED. Always share grounds.
• No MQTT messages:
• Confirm broker IP and port.
• Test with mosquitto_pub to ensure broker is reachable:
  mosquitto_pub -h 192.168.1.100 -t test -m hello
• Ensure no firewall blocks port 1883.
• No “anomaly” events:
• The environment may be too consistent; try louder or more distinct acoustic events.
• Lower ANOMALY_THRESHOLD (e.g., 18.0f).
• Extend BASELINE_FRAMES (more stable baseline; also increases baseline time).
• Microphone reads all zeros or garbage:
• Verify INMP441 wiring. L/R must be tied to GND for Left. Ensure SD to GPIO34 (input-only).
• Swap MIC_LRCL and MIC_BCLK if miswired.
• Check that sample rate is compatible (16 kHz generally works; you can try 32 kHz or 48 kHz).
• Distorted or missing beep:
• Confirm MAX98357A SD/EN pin is not held low (some boards have SD pin; tie high).
• Ensure I2S1 sample rate (16 kHz) matches beep generation and that speaker is connected properly.
• WS2812B flicker or no light:
• Ensure a common ground.
• Add a 330–470 Ω resistor in series with the data line.
• Try lowering data line length; ensure 5V power is stable.

---

Improvements

• Use TLS for MQTT:
• Switch to WiFiClientSecure and configure CA certs; use port 8883.
• Smarter anomaly detection:
• Replace band z-score with Mahalanobis distance using covariance matrix.
• Compute MFCCs (requires DCT and mel filters) and train a baseline GMM or One-Class SVM offline, then run inference on-device.
• Adaptive baseline:
• Slowly adapt means/variance to non-anomalous drift to reduce false positives.
• Overlap and windowing:
• Use 50% overlap to improve time resolution; use Hanning or Blackman windows directly via FFT library for consistency.
• On-device logging:
• Send ring buffers of raw audio around anomaly to MQTT or to an SD card for offline analysis.
• Multi-topic segregation:
• Separate “metrics” and “events” into different QoS and retention policies.
• Power optimization:
• Use light sleep between frames; reduce sample rate when idle, increase upon suspected activity.
• LED UI:
• Map anomaly score to LED color (green to red gradient).
• Calibrated thresholds:
• Implement a calibration mode via MQTT command (e.g., device/esp32-anom/cmd) to restart baseline remotely.

---

Final Checklist

• Hardware
• Espressif ESP32-DevKitC V4 (ESP32-WROOM-32) connected via USB.
• INMP441 wired: VDD→3V3, GND→GND, L/R→GND, SCK→GPIO32, WS→GPIO33, SD→GPIO34.
• MAX98357A wired: VIN→5V, GND→GND, DIN→GPIO26, BCLK→GPIO27, LRC→GPIO25, speaker attached.
• WS2812B wired: 5V→5V, GND→GND, Data→GPIO4 (optional 330 Ω resistor).
• Software
• PlatformIO installed; project contains platformio.ini and src/main.cpp as provided.
• Wi-Fi and MQTT credentials configured in src/main.cpp.
• Build succeeds via pio run.
• Flash via pio run –target upload; serial monitor at 115200.
• Validation
• Device connects to Wi-Fi and MQTT; status/info messages published.
• Baseline completes (yellow LED transitions to green).
• Making a distinct sound triggers red LED pulse, beep, and MQTT anomaly message.
• Metrics publish periodically and scores make sense.
• Troubleshooting
• If no anomaly triggers, adjust ANOMALY_THRESHOLD and verify sensor wiring.
• If audio issues occur, recheck MAX98357A pin mapping and supply voltage.
• If LED doesn’t light, verify WS2812 power, ground, and data pin.

With this setup, your Espressif ESP32-DevKitC V4 (ESP32-WROOM-32) + INMP441 + MAX98357A + WS2812B operates as a robust, real-time i2s-anomaly-detection-mqtt node, streaming health metrics, announcing anomalies with visual and audible cues, and providing a solid base for more advanced on-device audio analytics.

Objective and use case

What you’ll build: A robust TWAI/CAN-to-MQTT gateway using ESP32-WROOM-32, SN65HVD230, and SSD1306 OLED, enabling seamless communication between CAN networks and MQTT brokers.

Why it matters / Use cases

• Integrate legacy CAN devices into modern IoT ecosystems, allowing for remote monitoring and control.
• Utilize the SSD1306 OLED for real-time status updates, enhancing user experience and system observability.
• Implement a reliable data pipeline for automotive applications, ensuring timely data transmission and reception.
• Facilitate smart agriculture solutions by connecting CAN-based sensors to cloud platforms via MQTT.
• Enable remote diagnostics and troubleshooting of CAN networks through MQTT messaging.

Expected outcome

• Achieve a minimum of 95% message delivery success rate between CAN and MQTT.
• Maintain latencies under 100ms for message transmission across the gateway.
• Process up to 100 packets per second in high-load scenarios without data loss.
• Display real-time CAN bus status on the SSD1306 OLED with less than 1 second update intervals.
• Ensure system uptime of 99.9% in production environments through robust error handling and reconnection logic.

Audience: Advanced users; Level: Intermediate to Advanced

Architecture/flow: ESP32-WROOM-32 interfaces with SN65HVD230 for CAN communication, bridging data to MQTT topics for cloud integration.

ESP32 TWAI–CAN to MQTT Gateway (Advanced) on ESP32-WROOM-32 DevKitC + SN65HVD230 CAN Transceiver + SSD1306 OLED

This hands-on practical case builds a robust TWAI/CAN-to-MQTT gateway using the exact hardware combination: ESP32-WROOM-32 DevKitC + SN65HVD230 CAN Transceiver + SSD1306 OLED. The gateway bridges CAN frames to MQTT topics and vice versa, with live status on a 128×64 OLED and serial logs. We’ll use PlatformIO for reproducible builds and target advanced users who want a production-style implementation with queues, reconnection logic, and clean topic design.

The implementation prioritizes:
– Reliable TWAI (CAN) RX/TX via the ESP-IDF TWAI driver.
– MQTT publish/subscribe bridging with sane JSON payloads.
– Non-blocking architecture with FreeRTOS tasks and queues.
– On-device observability via SSD1306.
– Clear setup, precise commands, and repeatable validation with mosquitto tools.

---

Prerequisites

• Host OS: Windows 10/11, macOS 12+, or Ubuntu 22.04+.
• PlatformIO Core (CLI) 6.1+ and PlatformIO IDE (VS Code) recommended.
• Python 3.10+ (installed automatically by PlatformIO on first run).
• USB drivers for the ESP32-WROOM-32 DevKitC:
• CP210x (Silicon Labs) or CH34x (WCH), depending on your DevKitC’s USB-UART bridge.
• Windows: Install CP210x driver from Silicon Labs website or CH340 driver from WCH if needed.
• macOS: Usually plug-and-play for CP210x; CH340 may require driver.
• Linux: Uses kernel driver; ensure user is in dialout/uucp group (udev rules).
• A working MQTT broker (e.g., Mosquitto 2.0+ on your LAN).
• A CAN bus to connect to, or a second CAN node for test. For standalone local tests, we also include a NO_ACK self-test mode.

---

Materials (Exact Models)

• ESP32-WROOM-32 DevKitC (original Espressif DevKitC; USB-C or micro-USB variant is fine)
• SN65HVD230 CAN Transceiver module (3.3V; RS pin exposed)
• SSD1306 OLED 0.96″ I2C (128×64, address 0x3C typical)
• CAN bus wiring (twisted pair), with proper termination (120 Ω at each end)
• Jumper wires (female-female)
• USB cable for ESP32 (ensure data-capable)

Optional:
– Second CAN node (e.g., USB-CAN adapter) to inject/observe frames
– 120 Ω resistor if your transceiver board has no onboard termination or it’s disabled

---

Setup/Connection

Carefully wire the modules before powering up. The SN65HVD230 is 3.3V compatible—do not connect to 5V.

Pin Assignments

• TWAI (ESP32) pins used:
• TWAI_TX: GPIO 5
• TWAI_RX: GPIO 4
• I2C (ESP32) pins used for OLED:
• SDA: GPIO 21

• SCL: GPIO 22

• SN65HVD230 typical pins (module names vary slightly): VCC, GND, TXD, RXD, RS, CANH, CANL.

• SSD1306 I2C pins: VCC, GND, SDA, SCL (address typically 0x3C).

Connection Table

Additional CAN wiring notes:
– Use a twisted pair for CANH/CANL.
– Ensure 120 Ω termination at both ends of the bus (do not over-terminate).
– If testing with only the ESP32 node and a second CAN node, ensure the two ends are terminated.
– RS to GND puts SN65HVD230 in normal high-speed mode; you can also control RS via GPIO for standby/slope control, but we keep it grounded for simplicity.

---

Full Code

We use PlatformIO (Arduino framework) but call the ESP-IDF TWAI driver directly. MQTT is via PubSubClient; OLED via Adafruit SSD1306. We handle CAN RX via a FreeRTOS task and queue; CAN TX events come from MQTT and are queued to a TX task. OLED shows Wi-Fi/MQTT state and last CAN ID.

Create this project structure:
– platformio.ini
– src/main.cpp

platformio.ini

[env:esp32dev]
platform = espressif32@6.5.0
board = esp32dev
framework = arduino
monitor_speed = 115200
board_build.partitions = default.csv
build_flags =
  -DCORE_DEBUG_LEVEL=3

lib_deps =
  knolleary/PubSubClient@^2.8
  adafruit/Adafruit SSD1306@^2.5.9
  adafruit/Adafruit GFX Library@^1.11.11
  bblanchon/ArduinoJson@^7.0.4

src/main.cpp

#include <Arduino.h>
#include <WiFi.h>
#include <PubSubClient.h>
#include <Wire.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>
#include <ArduinoJson.h>
#include "driver/twai.h"

// -------- User Configuration --------
#define WIFI_SSID       "YOUR_WIFI_SSID"
#define WIFI_PASS       "YOUR_WIFI_PASSWORD"
#define MQTT_HOST       "192.168.1.10"   // Your broker IP or hostname
#define MQTT_PORT       1883
#define MQTT_USER       ""               // or set if broker requires
#define MQTT_PASS       ""               // or set if broker requires
#define MQTT_CLIENT_ID  "esp32-can-gw-01"

// Topics
static const char* TOPIC_RX      = "gw/can0/rx";     // publish CAN->MQTT
static const char* TOPIC_TX      = "gw/can0/tx";     // subscribe MQTT->CAN
static const char* TOPIC_STATUS  = "gw/can0/status"; // LWT + status
static const char* TOPIC_STATS   = "gw/can0/stats";  // periodic metrics

// OLED
#define OLED_ADDR       0x3C
#define OLED_WIDTH      128
#define OLED_HEIGHT     64
#define OLED_RESET_PIN  -1 // not used, set to -1 for I2C-only

// I2C pins for ESP32 DevKitC
#define I2C_SDA         21
#define I2C_SCL         22

// TWAI pins (SN65HVD230)
#define TWAI_TX_PIN     GPIO_NUM_5
#define TWAI_RX_PIN     GPIO_NUM_4

// CAN bitrate (pick a config macro)
#define USE_CAN_500K    1

// Test mode: if you want to allow transmit without ACK (single-node lab test)
// Set to 1 to start in TWAI_MODE_NO_ACK; set 0 for normal (bus expected)
#define TWAI_NO_ACK_TEST_MODE  0

// Queue sizes
#define RX_QUEUE_LEN    32
#define TX_QUEUE_LEN    32

// -------- Globals --------
WiFiClient wifiClient;
PubSubClient mqtt(wifiClient);
Adafruit_SSD1306 display(OLED_WIDTH, OLED_HEIGHT, &Wire, OLED_RESET_PIN);

typedef struct {
  uint32_t id;
  bool ext;
  bool rtr;
  uint8_t dlc;
  uint8_t data[8];
  uint64_t ts_us;
} can_frame_t;

QueueHandle_t rxQueue = nullptr;
QueueHandle_t txQueue = nullptr;

volatile uint32_t can_rx_count = 0;
volatile uint32_t can_tx_count = 0;
volatile uint32_t can_tx_fail  = 0;
volatile uint32_t can_err_warn = 0;

static uint32_t last_id_display = 0;
static bool last_id_ext = false;

void oledPrint(const String& l1, const String& l2, const String& l3, const String& l4) {
  display.clearDisplay();
  display.setTextSize(1);
  display.setTextColor(SSD1306_WHITE);
  display.setCursor(0, 0);  display.print(l1);
  display.setCursor(0, 16); display.print(l2);
  display.setCursor(0, 32); display.print(l3);
  display.setCursor(0, 48); display.print(l4);
  display.display();
}

void connectWiFi() {
  WiFi.mode(WIFI_STA);
  WiFi.begin(WIFI_SSID, WIFI_PASS);
  uint32_t start = millis();
  while (WiFi.status() != WL_CONNECTED) {
    delay(300);
    if (millis() - start > 20000) break;
  }
}

void mqttReconnect() {
  if (mqtt.connected()) return;
  mqtt.setServer(MQTT_HOST, MQTT_PORT);
  mqtt.setSocketTimeout(5);
  mqtt.setKeepAlive(15);
  mqtt.setBufferSize(1024);
  String cid = String(MQTT_CLIENT_ID) + "-" + String((uint32_t)ESP.getEfuseMac(), HEX);

  if (mqtt.connect(cid.c_str(), MQTT_USER, MQTT_PASS, TOPIC_STATUS, 1, true, "offline")) {
    mqtt.publish(TOPIC_STATUS, "online", true);
    mqtt.subscribe(TOPIC_TX);
  }
}

static inline String hexByte(uint8_t b) {
  char buf[3];
  snprintf(buf, sizeof(buf), "%02X", b);
  return String(buf);
}

void publishCanFrame(const can_frame_t& f) {
  StaticJsonDocument<256> doc;
  doc["bus"] = "can0";
  doc["ts_us"] = f.ts_us;
  char idbuf[11];
  if (f.ext) snprintf(idbuf, sizeof(idbuf), "0x%08lX", (unsigned long)f.id);
  else       snprintf(idbuf, sizeof(idbuf), "0x%03lX",  (unsigned long)f.id);
  doc["id"] = idbuf;
  doc["ext"] = f.ext;
  doc["rtr"] = f.rtr;
  doc["dlc"] = f.dlc;

  String dataHex;
  if (!f.rtr) {
    for (uint8_t i = 0; i < f.dlc && i < 8; i++) {
      dataHex += hexByte(f.data[i]);
    }
  } else {
    dataHex = "";
  }
  doc["data"] = dataHex;

  char out[320];
  size_t len = serializeJson(doc, out, sizeof(out));
  if (mqtt.connected()) {
    mqtt.publish(TOPIC_RX, out, len);
  }
}

uint32_t parseHexOrDecId(const String& s, bool* ok) {
  char* endp = nullptr;
  uint32_t val = 0;
  if (s.startsWith("0x") || s.startsWith("0X")) {
    val = strtoul(s.c_str() + 2, &endp, 16);
  } else {
    val = strtoul(s.c_str(), &endp, 10);
  }
  *ok = (endp != nullptr) && (*endp == '\0');
  return val;
}

bool parseHexPayload(const String& hex, uint8_t* out, uint8_t* dlc) {
  size_t n = hex.length();
  if (n == 0) { *dlc = 0; return true; }
  if (n % 2 != 0 || n/2 > 8) return false;
  for (size_t i = 0; i < n/2; i++) {
    char h[3] = { hex[2*i], hex[2*i+1], 0 };
    out[i] = (uint8_t)strtoul(h, nullptr, 16);
  }
  *dlc = n/2;
  return true;
}

void mqttCallback(char* topic, uint8_t* payload, unsigned int length) {
  String t = String(topic);
  String pl;
  pl.reserve(length + 1);
  for (unsigned int i = 0; i < length; i++) pl += (char)payload[i];

  if (t == TOPIC_TX) {
    StaticJsonDocument<256> doc;
    DeserializationError err = deserializeJson(doc, pl);
    if (err) return;

    bool okId = false;
    String idStr = doc["id"] | "";
    uint32_t id = parseHexOrDecId(idStr, &okId);
    bool ext = doc["ext"] | false;
    bool rtr = doc["rtr"] | false;
    String dataHex = doc["data"] | "";
    if (!okId) return;

    can_frame_t f = {};
    f.id = id & (ext ? 0x1FFFFFFF : 0x7FF);
    f.ext = ext;
    f.rtr = rtr;

    if (!rtr) {
      uint8_t dlc = 0;
      if (!parseHexPayload(dataHex, f.data, &dlc)) return;
      f.dlc = dlc;
    } else {
      uint8_t dlc = doc["dlc"] | 0; // allow explicit dlc for RTR
      if (dlc > 8) dlc = 8;
      f.dlc = dlc;
    }
    f.ts_us = (uint64_t)esp_timer_get_time();

    if (txQueue) {
      xQueueSend(txQueue, &f, 0);
    }
  }
}

void setupMQTT() {
  mqtt.setCallback(mqttCallback);
  mqttReconnect();
}

// TWAI setup
bool startTWAI() {
  twai_general_config_t g_config = TWAI_GENERAL_CONFIG_DEFAULT(TWAI_TX_PIN, TWAI_RX_PIN, TWAI_MODE_NORMAL);
#if TWAI_NO_ACK_TEST_MODE
  g_config.mode = TWAI_MODE_NO_ACK;
#endif
#if USE_CAN_500K
  twai_timing_config_t t_config = TWAI_TIMING_CONFIG_500KBITS();
#else
  twai_timing_config_t t_config = TWAI_TIMING_CONFIG_250KBITS();
#endif
  twai_filter_config_t f_config = TWAI_FILTER_CONFIG_ACCEPT_ALL();

  if (twai_driver_install(&g_config, &t_config, &f_config) != ESP_OK) {
    Serial.println("[TWAI] driver install failed");
    return false;
  }
  if (twai_start() != ESP_OK) {
    Serial.println("[TWAI] start failed");
    return false;
  }
  return true;
}

void TwaiRxTask(void* pv) {
  twai_status_info_t st;
  for (;;) {
    twai_message_t msg;
    esp_err_t r = twai_receive(&msg, pdMS_TO_TICKS(100));
    if (r == ESP_OK) {
      can_frame_t f = {};
      f.id = msg.identifier;
      f.ext = msg.extd;
      f.rtr = msg.rtr;
      f.dlc = msg.data_length_code;
      if (!msg.rtr) memcpy(f.data, msg.data, f.dlc);
      f.ts_us = (uint64_t)esp_timer_get_time();
      can_rx_count++;
      if (rxQueue) xQueueSend(rxQueue, &f, 0);

      // Track for OLED
      last_id_display = f.id;
      last_id_ext = f.ext;
    }

    // Periodically check error counters
    if (twai_get_status_info(&st) == ESP_OK) {
      if (st.tx_error_counter > 0 || st.rx_error_counter > 0 || st.bus_state != TWAI_BUS_STATE_RUNNING) {
        can_err_warn++;
      }
    }
  }
}

void TwaiTxTask(void* pv) {
  for (;;) {
    can_frame_t f;
    if (xQueueReceive(txQueue, &f, pdMS_TO_TICKS(100)) == pdTRUE) {
      twai_message_t msg = {};
      msg.identifier = f.id;
      msg.extd = f.ext;
      msg.rtr = f.rtr;
      msg.data_length_code = f.dlc;
      if (!f.rtr && f.dlc > 0) memcpy(msg.data, f.data, f.dlc);
      if (twai_transmit(&msg, pdMS_TO_TICKS(50)) == ESP_OK) {
        can_tx_count++;
      } else {
        can_tx_fail++;
      }
    }
  }
}

unsigned long lastStatsMs = 0;

void publishStats() {
  StaticJsonDocument<192> doc;
  doc["ip"] = WiFi.localIP().toString();
  doc["rx"] = can_rx_count;
  doc["tx"] = can_tx_count;
  doc["tx_fail"] = can_tx_fail;
  doc["warns"] = can_err_warn;
  char out[256];
  size_t len = serializeJson(doc, out, sizeof(out));
  if (mqtt.connected()) mqtt.publish(TOPIC_STATS, out, len, false);
}

// Setup & loop
void setup() {
  Serial.begin(115200);
  delay(100);

  Wire.begin(I2C_SDA, I2C_SCL);
  if (!display.begin(SSD1306_SWITCHCAPVCC, OLED_ADDR)) {
    Serial.println("[OLED] init failed");
  } else {
    oledPrint("ESP32 CAN<->MQTT", "Booting...", "", "");
  }

  connectWiFi();
  oledPrint("WiFi:", WiFi.isConnected() ? WiFi.localIP().toString() : "Not connected",
            "MQTT: connecting", "");

  setupMQTT();

  if (!startTWAI()) {
    oledPrint("TWAI init failed", "", "", "");
  } else {
    oledPrint("TWAI: OK", WiFi.localIP().toString(),
              "MQTT: " + String(mqtt.connected() ? "OK" : "Pending"), "");
  }

  rxQueue = xQueueCreate(RX_QUEUE_LEN, sizeof(can_frame_t));
  txQueue = xQueueCreate(TX_QUEUE_LEN, sizeof(can_frame_t));

  xTaskCreatePinnedToCore(TwaiRxTask, "twai_rx", 4096, nullptr, 10, nullptr, APP_CPU_NUM);
  xTaskCreatePinnedToCore(TwaiTxTask, "twai_tx", 4096, nullptr, 10, nullptr, APP_CPU_NUM);
}

void loop() {
  // Wi-Fi reconnect
  if (WiFi.status() != WL_CONNECTED) {
    connectWiFi();
  }

  // MQTT reconnect
  if (!mqtt.connected()) {
    mqttReconnect();
  }
  mqtt.loop();

  // Drain RX queue and publish to MQTT
  for (int i = 0; i < 8; i++) {
    can_frame_t f;
    if (rxQueue && xQueueReceive(rxQueue, &f, 0) == pdTRUE) {
      publishCanFrame(f);
      // Update OLED with last ID and counters
      char idbuf[16];
      if (f.ext) snprintf(idbuf, sizeof(idbuf), "0x%08lX", (unsigned long)f.id);
      else       snprintf(idbuf, sizeof(idbuf), "0x%03lX",  (unsigned long)f.id);

      String l1 = "IP: " + (WiFi.isConnected() ? WiFi.localIP().toString() : String("No WiFi"));
      String l2 = String("MQTT: ") + (mqtt.connected() ? "OK" : "NC") + " RX:" + String(can_rx_count);
      String l3 = String("TX:") + String(can_tx_count) + " Fail:" + String(can_tx_fail);
      String l4 = String("Last ID ") + idbuf;
      oledPrint(l1, l2, l3, l4);
    } else {
      break;
    }
  }

  // Periodic stats to MQTT
  unsigned long now = millis();
  if (now - lastStatsMs > 5000) {
    publishStats();
    lastStatsMs = now;
  }

  delay(5);
}

Payload conventions:
– CAN-to-MQTT publishes on gw/can0/rx: JSON with fields bus, ts_us, id (hex string), ext, rtr, dlc, data (hex string).
– MQTT-to-CAN expects JSON on gw/can0/tx: id (hex string or decimal), ext, rtr, data (hex string) and optional dlc for RTR.

Example TX message to publish to gw/can0/tx:

{"id":"0x123","ext":false,"rtr":false,"data":"112233AABBCCDD00"}

---

Build/Flash/Run Commands

Initialize a PlatformIO project (if not using IDE):

mkdir esp32-twai-can-mqtt-gateway
cd esp32-twai-can-mqtt-gateway

# Initialize PlatformIO project for ESP32 DevKitC
pio project init --board esp32dev

Place platformio.ini and src/main.cpp as shown above.

Build, upload, and monitor:

# Build
pio run

# Upload (auto-detects serial port, use --upload-port if needed)
pio run -t upload

# Serial monitor at 115200 baud
pio device monitor -b 115200

Common upload ports:
– Windows: COM3, COM4, …
– macOS: /dev/tty.SLAB_USBtoUART or /dev/tty.usbserial-xxxx
– Linux: /dev/ttyUSB0 or /dev/ttyACM0 (ensure user in dialout group)

Specify upload port explicitly if needed:

pio run -t upload --upload-port /dev/ttyUSB0

---

Step-by-step Validation

Follow this sequence to validate the gateway incrementally.

1) Power-on and basic bring-up

• With SN65HVD230 and SSD1306 connected as per the table, power the ESP32 via USB.
• Open the serial monitor. You should see boot logs.
• OLED should display:
• Your IP if Wi-Fi connected, or “Not connected”
• MQTT status (OK once connected)
• TWAI: OK if driver started successfully

If Wi-Fi doesn’t connect, verify SSID/password in code and signal strength.

2) Validate MQTT connectivity and topics

On a host machine with mosquitto-clients installed:

# Subscribe to CAN RX and stats
mosquitto_sub -h 192.168.1.10 -t gw/can0/rx -v

In another terminal:

# Subscribe to gateway status and stats
mosquitto_sub -h 192.168.1.10 -t gw/can0/status -v
mosquitto_sub -h 192.168.1.10 -t gw/can0/stats -v

• You should receive «online» on gw/can0/status after the ESP32 connects.
• Every 5 seconds you should see a gw/can0/stats JSON with rx/tx counters.

3) Loopback-style test (NO_ACK mode optional)

If you don’t have a second CAN node yet:
– Optionally set TWAI_NO_ACK_TEST_MODE to 1 in main.cpp and rebuild/flash. This allows transmit without a receiver acknowledging (for single-node lab tests). Note: You still won’t see frames “on the bus” without a second node, but transmit calls will succeed.
– Publish a TX frame from MQTT to the ESP32:

mosquitto_pub -h 192.168.1.10 -t gw/can0/tx -m '{"id":"0x123","ext":false,"rtr":false,"data":"11223344"}'

• Check the serial monitor: you should see TX and no errors (tx counter increments in stats).
• Since this is NO_ACK, you will not see the same frame on gw/can0/rx unless another node (or internal echo) is present. This step is just to ensure TX path + parsing works.

Switch NO_ACK back to 0 for real bus testing later.

4) Two-node CAN validation

Connect a real CAN bus with proper termination and a second node (e.g., USB-CAN dongle) at 500 kbit/s (matches our TWAI config).

• Ensure SN65HVD230 RS is tied to GND and the bus is 120 Ω terminated at both ends.
• On your second node, send a frame:

Example using a Linux system with can-utils (virtual example, your USB-CAN may use slcand or native SocketCAN):

# Example for SocketCAN interface can0 at 500k
# Send standard ID 0x321 with 8 bytes
cansend can0 321#DEADBEEF01020304

• On your MQTT subscriber (gw/can0/rx), you should now see a JSON like:

gw/can0/rx {"bus":"can0","ts_us":1234567,"id":"0x321","ext":false,"rtr":false,"dlc":8,"data":"DEADBEEF01020304"}

• OLED should update the Last ID to 0x321 and RX counter should increase.

5) End-to-end bridge test (MQTT -> CAN -> Other Node)

Now send from MQTT to the CAN bus via the gateway:

mosquitto_pub -h 192.168.1.10 -t gw/can0/tx -m '{"id":"0x18FF50E5","ext":true,"rtr":false,"data":"A1B2C3D4E5F60708"}'

• On your USB-CAN tool, listen for frames (again with can-utils):

candump can0

• You should see the extended frame with ID 0x18FF50E5 and the given data.
• Check that gw/can0/stats shows tx increment.

6) Remote frames (RTR) test

Publish an RTR from MQTT:

mosquitto_pub -h 192.168.1.10 -t gw/can0/tx -m '{"id":"0x123","ext":false,"rtr":true,"dlc":4}'

• On the other node, you should see a remote frame for 0x123 DLC 4.

7) Stress/soak test

• With your USB-CAN, generate a stream of frames and watch the gateway:

# e.g., transmit 10 frames/second for 1 minute (custom tooling or your CAN dongle GUI)

• Observe OLED counters and ensure gw/can0/rx receives JSON messages without drops. Serial logs should show no TWAI errors and stats should increment steadily.

---

Troubleshooting

• No serial port / upload fails:
• Check USB cable (data-capable).
• Install CP210x or CH340 drivers as needed.

• On Linux, add your user to the dialout group: sudo usermod -aG dialout $USER and re-login.

• Wi-Fi not connecting:

• Verify SSID/PASS in code exactly.
• Check 2.4 GHz band availability; ESP32 does not support 5 GHz.

• Scan RSSI; move closer to AP.

• MQTT not connecting:

• Confirm broker IP and port; test with mosquitto_sub from your PC.
• If broker requires authentication, fill MQTT_USER/MQTT_PASS.

• Firewalls may block port 1883.

• OLED stays blank:

• Double-check I2C pins (SDA=21, SCL=22) and address (0x3C).
• Confirm 3.3V power and ground.

• Try scanning I2C from code (a quick I2C scanner sketch) to verify address.

• No CAN frames received:

• Verify SN65HVD230 wiring: TXD->GPIO5, RXD->GPIO4, RS->GND, VCC 3.3V, GND common.
• Ensure the CAN bus is terminated with 120 Ω at both ends and that another active node is present.
• Confirm bus speed (500 kbit/s by default). Mismatched bitrate means no frames will be decoded.

• Check ground reference between all CAN nodes.

• CAN transmit fails (tx_fail increases):

• If normal mode (ACK required) and only single node exists: frames may fail due to missing ACK. Use TWAI_NO_ACK_TEST_MODE 1 for solo lab tests.
• Bus-off or error-passive: power issues or wiring/termination fault. Inspect st.tx_error_counter via code (we increment warns on anomalies).

• Try slower bitrate (250 kbit/s) by changing timing macro if your bus requires it.

• Random resets or instability:

• Power from a reliable USB source; avoid noisy hubs.

• Ensure no 5V is fed into SN65HVD230 VCC (must be 3.3V).

• JSON parsing errors on MQTT->CAN:

• Ensure payload matches schema: id, ext, rtr, data (hex). For RTR, specify dlc if you need >0 DLC.
• Data hex length must be even and ≤16 hex chars (≤8 bytes).

---

Improvements

• Security/TLS:
• Use MQTT over TLS (port 8883) with WiFiClientSecure and broker CA cert. Increase PubSubClient buffer accordingly.
• Dynamic configuration:
• Store Wi-Fi, broker, topics, and CAN bitrate in NVS or a config file. Add a captive portal for provisioning.
• Advanced filters:
• Replace TWAI_FILTER_CONFIG_ACCEPT_ALL with tight acceptance filters to reduce CPU and bandwidth.
• Retained device info:
• Publish retained metadata: firmware version, build time, CAN bitrate, pin mapping.
• OTA updates:
• Add ArduinoOTA or HTTPS OTA for remote firmware updates.
• Stats and health:
• Periodic twai_get_status_info details (error counters, bus state) in gw/can0/stats.
• OLED UI:
• Add paging and buttons to cycle views; show broker host, RSSI, and queue depths.
• Asynchronous MQTT:
• Switch to AsyncMqttClient + AsyncTCP for non-blocking operations if you expect high message rates.
• Multiple buses:
• If using ESP32 variants with two TWAI controllers (not on WROOM-32), expand to gw/can1.
• Protocol-specific:
• Add decoders for J1939, CANopen, or custom DBC, and publish human-friendly metrics.

---

Checklist (Before You Start the Bus Test)

• Materials:
• ESP32-WROOM-32 DevKitC ready and enumerates on USB
• SN65HVD230 wired to 3.3V, GND, TXD=GPIO5, RXD=GPIO4, RS=GND
• SSD1306 wired on I2C: SDA=21, SCL=22, address 0x3C
• CAN bus has 120 Ω at both ends; ground reference shared
• Software:
• PlatformIO installed; project builds with provided platformio.ini
• Drivers installed for CP210x/CH34x as needed
• MQTT broker reachable from ESP32 IP (ping broker from PC)
• Firmware:
• Wi-Fi SSID/PASS set in main.cpp
• MQTT host/port (and credentials if needed) set in main.cpp
• CAN bitrate matches your bus (default 500 kbit/s)
• Uploaded and serial monitor shows IP and “TWAI: OK”
• Validation:
• mosquitto_sub shows gw/can0/status “online”
• gw/can0/stats updates every ~5 s
• Injecting frames from second node appears on gw/can0/rx
• Publishing to gw/can0/tx produces frames on the bus

With the above, your ESP32-WROOM-32 DevKitC + SN65HVD230 + SSD1306 OLED is a functional twai-can-mqtt-gateway suitable for lab use and extensible to production with TLS, OTA, and richer filtering.

Objective and use case

What you’ll build: Implement a predictive vibration sensing system using the ICM-42688-P sensor, ESP32-S3, and ATECC608A for secure data transmission over LoRa.

Why it matters / Use cases

• Monitor industrial machinery for vibration anomalies to prevent costly downtimes.
• Utilize remote sensors in agricultural equipment to ensure optimal performance and maintenance scheduling.
• Deploy in smart cities to track the health of infrastructure like bridges and roads.
• Enhance predictive maintenance in automotive applications by analyzing vibration data from vehicles.

Expected outcome

• Real-time anomaly detection with a precision of 95% in identifying vibration irregularities.
• Data transmission latency of less than 2 seconds over LoRa.
• Uplink of feature data and anomaly scores with a packet success rate of 98%.
• Reduction in maintenance costs by 30% through proactive interventions based on vibration data.

Audience: Engineers and developers in IoT; Level: Advanced

Architecture/flow: ESP32-S3 processes sensor data from ICM-42688-P, signs it with ATECC608A, and transmits via LoRa using RFM95W.

ESP32-S3-DevKitC-1 + RFM95W (SX1276) + ICM-42688-P + ATECC608A: LoRa Predictive Vibration Sensing (Advanced)

Objective: lora-predictive-vibration-icm42688 — implement an on-device vibration anomaly detector using ICM‑42688‑P, sign results with ATECC608A, and uplink features/anomaly score over raw LoRa using an SX1276 (RFM95W) from an ESP32‑S3‑DevKitC‑1. This guide uses PlatformIO for build reproducibility and provides code, connections, commands, and validation.

---

Prerequisites

• Skills: C++ for embedded, basic signal processing (RMS, FFT), SPI/I2C wiring, CLI usage.
• OS: Windows 10/11, macOS 12+, or Ubuntu 22.04+.
• Drivers:
• ESP32‑S3‑DevKitC‑1 typically uses native USB Serial/JTAG (no extra driver needed on Win10+/macOS/Linux).
• If your USB/UART interface is CP210x (some carriers or external FTDI adapters), install “Silicon Labs CP210x” drivers.
• If your adapter is CH34x (less common here), install WCH CH34x drivers.
• Software:
• Python 3.10+ (optional for log parsing).
• PlatformIO Core (CLI): install via pipx install platformio or pip install -U platformio.
• Git (optional, for version control).

---

Materials (exact models)

• MCU: ESP32‑S3‑DevKitC‑1 (ESP32‑S3, 3.3 V logic).
• LoRa radio: RFM95W (Semtech SX1276, 868/915 MHz module).
• IMU: ICM‑42688‑P 6‑DoF accelerometer/gyro module (I2C breakout).
• Secure element: ATECC608A (I2C variant, address 0x60).
• Passives/wires: 3.3 V operation only, jumper wires, small breadboard.
• Optional: A second SX1276-based node for RF receive validation (or an SDR).

Regulatory note: Select the frequency (868 MHz in EU, 915 MHz in US/ANZ) according to your local regulations and duty-cycle limits.

---

Setup/Connection

All modules are 3.3 V logic. Do not apply 5 V to any signal pin.

We will use:
– I2C for ICM‑42688‑P and ATECC608A on the same bus.
– SPI for RFM95W (SX1276).
– Explicit GPIO assignments on ESP32‑S3‑DevKitC‑1; we configure pins in software.

Recommended wiring (you can adjust, but match the code/pins):

Pinout table

Notes:
– Keep I2C wiring short and tidy; use 4.7 kΩ pull-ups to 3.3 V if your breakouts do not include them.
– For the RFM95W, use a proper antenna tuned to your band. Never transmit without an antenna.
– Avoid ESP32‑S3 strapping pins (GPIO0, GPIO45, GPIO46) for critical peripheral signals.

---

Full Code

This PlatformIO project implements:
– ICM‑42688‑P acquisition at ~1 kHz with 256‑sample windows.
– Feature extraction: RMS, peak-to-peak, crest factor, kurtosis, two dominant spectral peaks.
– Baseline learning (first N windows), then anomaly scoring (sum of squared z-scores).
– SHA‑256 digest created in the ATECC608A and ECDSA P‑256 signature from private key in Slot 0.
– Raw LoRa uplink of a compact payload containing features, anomaly score, and signature.

platformio.ini

Create a new folder (e.g., lora-predictive-vibration-icm42688) and put this file at the project root:

; File: platformio.ini
[env:esp32-s3-devkitc-1]
platform = espressif32@6.5.0
board = esp32-s3-devkitc-1
framework = arduino
upload_speed = 921600
monitor_speed = 115200
monitor_filters = time, colorize
; Enable USB CDC for serial monitor on S3
build_flags =
  -D ARDUINO_USB_CDC_ON_BOOT=1
  -D LORA_FREQ_MHZ=915.0       ; set to 868.1 for EU, 915.0 for US/AU/NZ
  -D LORA_SYNC_WORD=0x12       ; 0x12 private, 0x34 public
  -D LORA_TX_POWER_DBM=17
  -D LORA_BW_KHZ=125
  -D LORA_SF=7
  -D LORA_CR=5                 ; coding rate 4/5
  -D I2C_SDA_PIN=5
  -D I2C_SCL_PIN=6
  -D RFM95_SCK=36
  -D RFM95_MISO=37
  -D RFM95_MOSI=35
  -D RFM95_CS=34
  -D RFM95_RST=33
  -D RFM95_DIO0=2
  -D RFM95_DIO1=7
lib_deps =
  jgromes/RadioLib@^6.5.0
  adafruit/Adafruit BusIO@^1.16.1
  adafruit/Adafruit ICM42688@^1.1.0
  ArduinoECCX08@^1.3.7
  kosme/arduinoFFT@^2.0.1

If you operate in EU868, change LORA_FREQ_MHZ to 868.1. Use a frequency and duty cycle legal in your region.

src/main.cpp

Create src/main.cpp:

#include <Arduino.h>
#include <Wire.h>
#include <SPI.h>
#include <RadioLib.h>
#include <Adafruit_ICM42688.h>
#include <ArduinoECCX08.h>
#include <arduinoFFT.h>

// ----------------- User-configurable via build_flags -----------------
#ifndef LORA_FREQ_MHZ
#define LORA_FREQ_MHZ 915.0
#endif
#ifndef LORA_SYNC_WORD
#define LORA_SYNC_WORD 0x12
#endif
#ifndef LORA_TX_POWER_DBM
#define LORA_TX_POWER_DBM 17
#endif
#ifndef LORA_BW_KHZ
#define LORA_BW_KHZ 125
#endif
#ifndef LORA_SF
#define LORA_SF 7
#endif
#ifndef LORA_CR
#define LORA_CR 5
#endif

#ifndef I2C_SDA_PIN
#define I2C_SDA_PIN 5
#endif
#ifndef I2C_SCL_PIN
#define I2C_SCL_PIN 6
#endif

#ifndef RFM95_SCK
#define RFM95_SCK 36
#endif
#ifndef RFM95_MISO
#define RFM95_MISO 37
#endif
#ifndef RFM95_MOSI
#define RFM95_MOSI 35
#endif
#ifndef RFM95_CS
#define RFM95_CS 34
#endif
#ifndef RFM95_RST
#define RFM95_RST 33
#endif
#ifndef RFM95_DIO0
#define RFM95_DIO0 2
#endif
#ifndef RFM95_DIO1
#define RFM95_DIO1 7
#endif

// ----------------- Globals -----------------
Adafruit_ICM42688 icm;
SPIClass spiLoRa;
Module* loraModule = nullptr;
SX1276* lora = nullptr;

static const size_t N_SAMPLES = 256;    // window length for FFT/features
static const float FS_HZ = 1000.0f;     // effective sample rate
static const float DT_MS = 1000.0f / FS_HZ;
double vReal[N_SAMPLES];
double vImag[N_SAMPLES];
arduinoFFT FFT(vReal, vImag, N_SAMPLES, FS_HZ);

// Baseline stats for features (mean/variance) - simple diagonal covariance
// Feature order: [rms, p2p, crest, kurt, f1, f2] -> 6 features
static const size_t N_FEATS = 6;
float feat_mean[N_FEATS] = {0};
float feat_var[N_FEATS]  = {1};
uint32_t baseline_count = 0;
const uint32_t BASELINE_WINDOWS = 30;

// Utility: constrain to bounds
template<typename T>
T clamp(T v, T lo, T hi) { return (v < lo) ? lo : (v > hi ? hi : v); }

// Compute features from time series 'mag' of length N
void compute_features(const float* mag, size_t n, float out_feats[6]) {
  // Time-domain
  double sum = 0.0, sumsq = 0.0, sum4 = 0.0;
  float maxv = -1e9, minv = 1e9;
  for (size_t i = 0; i < n; i++) {
    float x = mag[i];
    sum += x;
    sumsq += (double)x * x;
    double x2 = (double)x * x;
    sum4 += x2 * x2;
    if (x > maxv) maxv = x;
    if (x < minv) minv = x;
  }
  double mean = sum / n;
  double rms = sqrt(sumsq / n);
  double p2p = (double)maxv - (double)minv;
  double crest = (rms > 1e-9) ? ((double)maxv / rms) : 0.0;
  // Excess kurtosis (normalized 4th central moment - 3)
  double m2 = sumsq / n - mean * mean;
  if (m2 < 1e-12) m2 = 1e-12;
  double m4 = (sum4 / n) - 4*mean*(sumsq/n) + 6*mean*mean*(sum/n) - 3*pow(mean,4); // approximate; acceptable here
  double kurtosis = (m4 / (m2 * m2));

  // FFT for spectral peaks (magnitude spectrum)
  for (size_t i = 0; i < n; i++) {
    vReal[i] = (double)mag[i];
    vImag[i] = 0.0;
  }
  FFT.Windowing(FFT_WIN_TYP_HAMMING, FFT_FORWARD);
  FFT.Compute(FFT_FORWARD);
  FFT.ComplexToMagnitude();

  // Find top two peaks excluding DC bin 0
  int peak1 = 1, peak2 = 2;
  double p1 = 0.0, p2 = 0.0;
  for (size_t i = 1; i < n/2; i++) { // only positive freqs
    double a = vReal[i];
    if (a > p1) {
      p2 = p1; peak2 = peak1;
      p1 = a; peak1 = (int)i;
    } else if (a > p2) {
      p2 = a; peak2 = (int)i;
    }
  }
  double f1 = (peak1 * FS_HZ) / n;
  double f2 = (peak2 * FS_HZ) / n;

  out_feats[0] = (float)rms;
  out_feats[1] = (float)p2p;
  out_feats[2] = (float)crest;
  out_feats[3] = (float)kurtosis;
  out_feats[4] = (float)f1;
  out_feats[5] = (float)f2;
}

float update_baseline_and_score(const float feats[6], bool& baseline_active) {
  if (baseline_count < BASELINE_WINDOWS) {
    // Online mean/var (Welford)
    baseline_active = true;
    for (size_t i = 0; i < N_FEATS; i++) {
      float x = feats[i];
      float delta = x - feat_mean[i];
      feat_mean[i] += delta / (float)(baseline_count + 1);
      float delta2 = x - feat_mean[i];
      // Unbiased variance update
      if (baseline_count == 0) {
        feat_var[i] = 0.0f;
      } else {
        feat_var[i] = ((float)baseline_count * feat_var[i] + delta * delta2) / (float)(baseline_count + 1);
      }
      feat_var[i] = max(feat_var[i], 1e-6f);
    }
    baseline_count++;
    return 0.0f;
  }
  baseline_active = false;
  // Anomaly score: sum of squared z-scores (diagonal Mahalanobis)
  float score = 0.0f;
  for (size_t i = 0; i < N_FEATS; i++) {
    float z = (feats[i] - feat_mean[i]) / sqrtf(feat_var[i]);
    score += z * z;
  }
  return score;
}

// Build minimal payload and sign with ATECC608A (slot 0)
int build_and_sign_payload(uint8_t* buf, size_t buf_len,
                           const float feats[6], float score,
                           size_t& out_len, uint8_t* sig64) {
  // Payload layout (little-endian):
  // [0]   : version = 1
  // [1]   : flags: bit0=baseline_active
  // [2..5]: uptime_ms (uint32)
  // [6..17]: features packed Qm.n -> here: 6x int16 (scale defined)
  // [18..19]: anomaly score * 100 (uint16, capped)
  // Note: signature not part of the hashed payload in this buffer; we sign the payload bytes below.
  if (buf_len < 32) return -1;

  uint8_t version = 1;
  bool baseline_active = (baseline_count < BASELINE_WINDOWS);
  uint8_t flags = baseline_active ? 0x01 : 0x00;

  buf[0] = version;
  buf[1] = flags;
  uint32_t t = millis();
  memcpy(&buf[2], &t, sizeof(uint32_t));

  // Pack features into int16 using per-feature scales
  // scales: rms, p2p in g*1000; crest*100; kurt*100; f1,f2 Hz*10
  int16_t fpack[6];
  fpack[0] = (int16_t)clamp((int32_t)lroundf(feats[0] * 1000.0f), -32768, 32767);
  fpack[1] = (int16_t)clamp((int32_t)lroundf(feats[1] * 1000.0f), -32768, 32767);
  fpack[2] = (int16_t)clamp((int32_t)lroundf(feats[2] * 100.0f), -32768, 32767);
  fpack[3] = (int16_t)clamp((int32_t)lroundf(feats[3] * 100.0f), -32768, 32767);
  fpack[4] = (int16_t)clamp((int32_t)lroundf(feats[4] * 10.0f), -32768, 32767);
  fpack[5] = (int16_t)clamp((int32_t)lroundf(feats[5] * 10.0f), -32768, 32767);
  memcpy(&buf[6], fpack, sizeof(fpack));

  uint16_t score_scaled = (uint16_t)clamp((int32_t)lroundf(score * 100.0f), 0, 65535);
  memcpy(&buf[18], &score_scaled, sizeof(uint16_t));

  size_t payload_len = 20; // header + feats + score
  out_len = payload_len;

  // Compute digest and sign via ATECC608A
  if (!ECCX08.begin()) {
    Serial.println(F("[ATECC] begin() failed"));
    return -2;
  }
  // Ensure a private key exists in slot 0; if not, generate it (one-time)
  if (!ECCX08.locked()) {
    Serial.println(F("[ATECC] Device not locked; generating key in slot 0 (dev mode)"));
    if (!ECCX08.generatePrivateKey(0)) {
      Serial.println(F("[ATECC] Key generation failed"));
      return -3;
    }
    // In production you must configure and lock the ATECC securely using Microchip provisioning.
  }

  if (!ECCX08.beginSHA256()) { Serial.println(F("[ATECC] SHA256 begin failed")); return -4; }
  ECCX08.updateSHA256(buf, payload_len);
  uint8_t digest[32];
  if (!ECCX08.endSHA256(digest)) { Serial.println(F("[ATECC] SHA256 end failed")); return -5; }

  if (!ECCX08.sign(0, digest, sig64)) {
    Serial.println(F("[ATECC] sign failed"));
    return -6;
  }
  return 0;
}

bool initICM() {
  Wire.begin(I2C_SDA_PIN, I2C_SCL_PIN, 400000);
  delay(10);
  if (!icm.begin_I2C(0x68, &Wire)) {
    Serial.println(F("[ICM42688] not found at 0x68, trying 0x69"));
    if (!icm.begin_I2C(0x69, &Wire)) {
      Serial.println(F("[ICM42688] begin failed"));
      return false;
    }
  }
  // Configure ranges/data rates
  icm.setAccelRange(ICM42688_ACCEL_RANGE_4_G);
  icm.setGyroRange(ICM42688_GYRO_RANGE_2000_DPS);
  icm.setAccelDataRate(ICM42688_ACCEL_RATE_1000_HZ);
  icm.setGyroDataRate(ICM42688_GYRO_RATE_1000_HZ);
  icm.setFilterBandwidth(ICM42688_ACCEL_BW_258_9HZ, ICM42688_GYRO_BW_258_9HZ);
  icm.setAccelPowerMode(ICM42688_ACCEL_LOW_NOISE);
  icm.setGyroPowerMode(ICM42688_GYRO_OFF); // we only use accelerometer for now
  delay(50);
  Serial.println(F("[ICM42688] initialized"));
  return true;
}

bool initLoRa() {
  // SPI for LoRa
  spiLoRa = SPIClass(FSPI);
  spiLoRa.begin(RFM95_SCK, RFM95_MISO, RFM95_MOSI, RFM95_CS);
  loraModule = new Module(RFM95_CS, RFM95_DIO0, RFM95_RST, RADIOLIB_NC, spiLoRa, SPISettings(8000000, MSBFIRST, SPI_MODE0));
  lora = new SX1276(loraModule);

  // Start radio
  int state = lora->begin(LORA_FREQ_MHZ, LORA_BW_KHZ, LORA_SF, LORA_CR, LORA_SYNC_WORD, LORA_TX_POWER_DBM, 8, 0);
  if (state != RADIOLIB_ERR_NONE) {
    Serial.print(F("[LoRa] begin failed, code ")); Serial.println(state);
    return false;
  }
  // Optional fine-tuning
  lora->setCRC(true);
  lora->setCurrentLimit(100);  // mA
  lora->setDio0Action([](){ /* TX done / RX done IRQ callback not used here */ });
  Serial.println(F("[LoRa] initialized"));
  return true;
}

void setup() {
  Serial.begin(115200);
  delay(200);

  Serial.println();
  Serial.println(F("ESP32-S3 lora-predictive-vibration-icm42688"));
  Serial.printf("Build: %s %s\n", __DATE__, __TIME__);

  if (!initICM()) {
    Serial.println(F("ICM init failed; halt"));
    while (true) delay(1000);
  }
  if (!initLoRa()) {
    Serial.println(F("LoRa init failed; halt"));
    while (true) delay(1000);
  }
  // Initialize ATECC (test presence early)
  if (!ECCX08.begin()) {
    Serial.println(F("[ATECC] not detected; signing will fail"));
  } else {
    Serial.printf("[ATECC] locked: %s\n", ECCX08.locked() ? "yes" : "no");
  }

  Serial.println(F("Baseline learning... keep device in normal state"));
}

void loop() {
  // Acquire a window of samples from accelerometer (m/s^2 or g? library returns m/s^2)
  // Adafruit ICM42688 returns accel in m/s^2; convert to g
  sensors_event_t accel, gyro, temp;
  static float mag[N_SAMPLES];

  for (size_t i = 0; i < N_SAMPLES; i++) {
    icm.getEvent(&accel, &gyro, &temp);
    float ax_g = accel.acceleration.x / 9.80665f;
    float ay_g = accel.acceleration.y / 9.80665f;
    float az_g = accel.acceleration.z / 9.80665f;
    // Remove gravity by high-pass: approximate via subtracting running mean; here subtract DC later by FFT windowing
    // We compute magnitude which includes gravity; to reduce DC, subtract mean later via windowing. For time-domain features, it’s acceptable.
    float m = sqrtf(ax_g*ax_g + ay_g*ay_g + az_g*az_g);
    mag[i] = m;
    delayMicroseconds((int)(DT_MS * 1000.0f)); // coarse pacing, not perfect
  }

  float feats[6];
  compute_features(mag, N_SAMPLES, feats);
  bool baseline_active = false;
  float score = update_baseline_and_score(feats, baseline_active);

  // Prepare payload and sign
  uint8_t payload[24];
  size_t payload_len = 0;
  uint8_t sig[64];
  int rc = build_and_sign_payload(payload, sizeof(payload), feats, score, payload_len, sig);

  // Assemble final packet: payload + signature (total ~84 bytes)
  uint8_t packet[128];
  if (rc == 0) {
    memcpy(packet, payload, payload_len);
    memcpy(packet + payload_len, sig, 64);
  } else {
    // If signing failed, send payload only with flags indicating failure
    payload[1] |= 0x80; // set 'signing failed' flag
    memcpy(packet, payload, payload_len);
  }
  size_t packet_len = (rc == 0) ? (payload_len + 64) : payload_len;

  // Transmit
  int state = lora->transmit(packet, packet_len);
  Serial.printf("[TX] len=%u state=%d baseline=%s score=%.2f feats=[%.3f,%.3f,%.2f,%.2f,%.1f,%.1f]\n",
                (unsigned)packet_len, state, baseline_active ? "yes" : "no", score,
                feats[0], feats[1], feats[2], feats[3], feats[4], feats[5]);

  // Duty-cycling: respect band limits; for lab demo, 2 s pause
  delay(2000);
}

Optional receiver sketch for a second node (SX1276) is provided in Validation.

---

Build/Flash/Run Commands

Use PlatformIO CLI for deterministic builds.

pio --version

# Create the project folder (if not already created)
mkdir -p ~/work/lora-predictive-vibration-icm42688 && cd ~/work/lora-predictive-vibration-icm42688

# Initialize for ESP32-S3-DevKitC-1 (already provided platformio.ini)
pio project init --board esp32-s3-devkitc-1

# Put platformio.ini and src/main.cpp as shown above, then build:
pio run

# Connect board via USB (ESP32-S3 native USB). Flash:
pio run -t upload

# Open serial monitor at 115200 baud:
pio device monitor -b 115200

# Optional: change LoRa frequency (EU868) via environment override:
pio run -e esp32-s3-devkitc-1 -t upload --project-option "build_flags=-D LORA_FREQ_MHZ=868.1"

Windows note: If your board enumerates as “USB JTAG/serial” on COMx, use that port. If your hardware variant uses CP210x, ensure the Silicon Labs driver is installed. On macOS, the device appears under /dev/tty.usbmodem* (CDC) or /dev/tty.SLAB_USBtoUART (CP210x).

---

Step-by-step Validation

Follow these steps to validate sensing, analytics, cryptography, and RF.

1) Power-on self-test and I2C presence

• Open the serial monitor:
• You should see:
  • “[ICM42688] initialized”
  • “[LoRa] initialized”
  • “[ATECC] locked: yes/no”
  • “Baseline learning… keep device in normal state”
• If the IMU is not detected:
• Verify SDA/SCL pins (GPIO5/6) and address pins (AD0 to GND = 0x68).
• Check for pull-ups (4.7 kΩ) if your breakouts do not include them.

2) IMU measurement sanity

• Keep the device stationary; observe the log every ≈2 s:
• “baseline=yes” for the first 30 windows (about 1 minute).
• Features should be small and consistent, e.g., RMS near the noise floor (a few tens of mg) depending on your setup.
• Gently tap or place the board on a running device (electric toothbrush, small fan, phone vibration):
• RMS and peak-to-peak should increase significantly.
• Spectral peaks (f1, f2) should show identifiable frequencies (e.g., around 100–300 Hz for small motors).
• After baseline completes (baseline=no), the anomaly score should rise when vibrations deviate from baseline.

Tip: For reproducible testing, record quiet baseline, then attach the board near a small DC motor with an eccentric load.

3) Baseline/anomaly behavior

• During the first 30 windows, the device learns mean and variance for the 6 features (simple diagonal covariance).
• After learning:
• At rest (normal), anomaly score should be low (near 0–3).
• Excitation (abnormal), anomaly score should rise (e.g., 10–80+ depending on severity).
• If baseline drifts undesirably, reboot or adapt the code to re-baseline periodically or on command.

4) ATECC608A signature

• The code computes a SHA‑256 digest of the payload on the ATECC608A and signs with ECDSA P‑256 in slot 0.
• In the serial log, if the secure element is present and locked, no “[ATECC] sign failed” should appear.
• To verify the signature off-device:
• Retrieve the generated public key from slot 0 in a provisioning build (e.g., add ECCX08.generatePublicKey(0, pubkey) and print it once), then verify signatures on a receiver/server.
• In production, the ATECC608A should be configured and locked with a proper provisioning flow and certificates.

5) LoRa RF validation (two options)

A. With a second SX1276 node (recommended)
– Flash a receiver sketch onto another SX1276-based board (same frequency, SF, BW, CR, sync word).
– Minimal RadioLib receiver code:

// Receiver minimal sketch for validation (SX1276 + any Arduino/ESP32)
#include <Arduino.h>
#include <SPI.h>
#include <RadioLib.h>

#define RFM95_SCK  18
#define RFM95_MISO 19
#define RFM95_MOSI 23
#define RFM95_CS   5
#define RFM95_RST  14
#define RFM95_DIO0 26

SPIClass spiLoRa(VSPI);
Module* mod;
SX1276* lora;

void setup() {
  Serial.begin(115200);
  spiLoRa.begin(RFM95_SCK, RFM95_MISO, RFM95_MOSI, RFM95_CS);
  mod = new Module(RFM95_CS, RFM95_DIO0, RFM95_RST, RADIOLIB_NC, spiLoRa, SPISettings(8000000, MSBFIRST, SPI_MODE0));
  lora = new SX1276(mod);
  int state = lora->begin(915.0, 125, 7, 5, 0x12, 17, 8, 0);
  if (state != RADIOLIB_ERR_NONE) { Serial.println("begin failed"); while(1){} }
  lora->setCRC(true);
  Serial.println("RX ready");
}

void loop() {
  String str;
  int state = lora->receive(str);
  if (state == RADIOLIB_ERR_NONE) {
    Serial.print("RX: "); Serial.println(str.length());
    // Print hex
    for (size_t i = 0; i < str.length(); i++) {
      uint8_t b = (uint8_t)str[i];
      Serial.printf("%02X ", b);
    }
    Serial.println();
  }
}

• Match frequency/SF/BW/CR/sync word with the transmitter (see platformio.ini).
• Confirm received packets and lengths ≈84 bytes.
• Optionally, parse the first 20 bytes as payload and the last 64 as ECDSA signature; verify on a host tool.

B. With SDR or spectrum analyzer
– Use an SDR (e.g., RTL‑SDR + inspectrum) tuned to your frequency to confirm chirp transmissions every ~2 seconds.
– You won’t decode content but can validate on-air activity and duty cycle.

6) End-to-end functional check

• Start with the device still, allow baseline to complete.
• Cause a controlled vibration (e.g., small motor), observe:
• Features change and anomaly score increases.
• LoRa packets continue to transmit at the expected interval.
• If you have a receiver:
• Log received payloads, confirm that feature fields and anomaly score correlate with observed vibration.
• Optionally verify the signature using the known public key.

---

Troubleshooting

• No serial output:
• Ensure monitor at 115200.
• For ESP32‑S3 CDC, try a different USB cable/port. On Windows, look for “USB JTAG/serial” COM device.
• ICM‑42688‑P not found:
• Check address pin AD0/SA0: GND=0x68, VCC=0x69. Try both in code.
• Verify 3.3 V supply and I2C pull-ups; ensure SDA=GPIO5, SCL=GPIO6 match wiring.
• Reduce I2C speed to 100 kHz for long wires: change Wire.begin(..., 100000).
• ATECC608A sign failed:
• Confirm it’s the I2C variant (0x60). Check wiring on the same I2C bus.
• If ECCX08.locked() is false, your device is unconfigured; the demo generates a key in slot 0 for testing. In production, follow Microchip’s configuration/locking procedure.
• LoRa begin failed (RadioLib error):
• Re-check SPI pins and CS/RST/DIO0 wiring.
• Ensure an antenna is connected and the module’s band matches your chosen frequency.
• Lower SPI speed: change SPISettings to 2 MHz if needed.
• No RF reception with second node:
• Ensure all PHY params match (freq, SF, BW, CR, sync word).
• Place nodes at least a few meters apart to avoid front-end desense.
• Anomaly score always high:
• Increase baseline windows (BASELINE_WINDOWS) or ensure true “normal” state during learning.
• Add high-pass filtering to remove gravity bias if your mounting orientation changes often.
• Features look noisy:
• Mechanically isolate the board. Consider rigidly mounting to the machine surface for consistent coupling.
• Increase window length (e.g., 512) and adjust sampling rate to keep spectral resolution acceptable.

---

Improvements

• LoRaWAN instead of raw LoRa:
• The RFM95W (SX1276) is fine for LoRaWAN using an appropriate stack, but this example uses raw LoRa for simplicity. For TTN/LoRaWAN, integrate an LMIC or a RadioLib-based LoRaWAN stack, observe payload size and duty-cycle regulations, and handle join/session keys securely in ATECC608A.
• Power optimization:
• Increase window interval, lower TX power, batch multiple windows per uplink, or use adaptive intervals based on anomaly score.
• Use light sleep between windows; wake on a hardware timer.
• Better features and models:
• Add band power in specific bands (1X, 2X, 3X shaft frequency), spectral kurtosis, or cepstral coefficients.
• Replace z-score anomaly with a compact on-device model (e.g., 1‑class SVM or an autoencoder via TensorFlow Lite Micro).
• Sensor configuration:
• Use the ICM‑42688‑P FIFO and hardware data-ready interrupts instead of polling for consistent sampling and reduced jitter.
• Apply a proper DC removal/high-pass filter before RMS and crest calculation.
• Cryptography and provisioning:
• Lock and provision ATECC608A in production with a secure configuration zone, store the public key with your backend, and attach a compressed signature (ASN.1 DER) if needed.
• Packet protocol:
• Add a message counter, device ID, and CRC at the application level; optionally compress features further or switch to CBOR/FlatBuffers.
• Edge commands:
• Implement downlink over LoRa (requires an RX loop and coordination) to adjust thresholds, request re-baseline, or change the TX interval.

---

Checklist

• Materials
• ESP32‑S3‑DevKitC‑1, RFM95W (SX1276), ICM‑42688‑P, ATECC608A, antenna, wires.
• Wiring
• RFM95W: SPI to GPIO36/37/35, CS=34, RST=33, DIO0=2, GND, 3.3 V.
• ICM‑42688‑P: I2C SDA=5, SCL=6, 3.3 V, GND, AD0=GND (0x68).
• ATECC608A: I2C SDA=5, SCL=6, 3.3 V, GND.
• Software
• PlatformIO Core installed; project created with platformio.ini and libraries.
• Build/flash with pio run and pio run -t upload.
• Serial monitor at 115200 with pio device monitor.
• Validation
• ICM initialized, LoRa initialized, ATECC found (and locked status noted).
• Baseline completes (first ~60 s), features reasonable at rest.
• Vibration test increases RMS/peak and anomaly score.
• LoRa packets observed (second node or SDR); optional signature verification.
• Troubleshooting done if any step failed.
• Regulatory
• Frequency plan and duty cycle are configured for your region (LORA_FREQ_MHZ, TX interval).

---

This advanced case provides a complete and reproducible reference for implementing predictive vibration analytics on ESP32‑S3 with ICM‑42688‑P, securing results via ATECC608A, and uplinking over raw LoRa with SX1276. Adapt the PHY parameters and features to your machine’s vibration profile and local radio regulations for robust, production-ready deployments.