Skip to content

Troubleshooting

Common failure modes, what causes them, and how to recover. For hardware-level diagnostics see the Hardware section.

Boot Issues

Symptom: No serial output at boot

  • Check baud rate — 230 400, 8N1. Not 115 200.
  • USB-C cable — must be data-capable (not power-only).
  • Try a different USB port — some hubs flake.
  • Unplug the main power cable — on V9, connect main power or USB-C, not both. See USB-C.

Symptom: Boot loops (repeated === SYSTEM STARTUP ===)

Most common causes:

Cause Indicator Fix
FATAL: ... ps_malloc failed Boot log shows the failed allocation PSRAM is missing or mis-detected. Verify the WROOM-1 part has the -N16R8 suffix (16 MB flash, 8 MB PSRAM). Rebuild with PSRAMMode=enabled
Watchdog reboot during setup Guru Meditation Error or rst:0xc reset reason A specific setup step is hanging. Comment out subsystems one at a time to bisect — start with initializeHardware()
Brownout rst:0xf (BROWNOUT_RST) Power supply can't deliver enough current during boot. Add bulk capacitance at the input
Stuck on Cannot continue without filesystem User-data storage failed to mount and the automatic reformat also failed The firmware already attempts a reformat on its own (formatOnFail), so reaching this halt means storage is not recoverable in place. Power-cycle once more; if it persists, contact joe@xengineering.net

Symptom: Booted but no WiFi network appears

  • First boot detection — the firmware enters CONFIG mode automatically on first boot. The AP should be at ALTERNATOR_WIFI / alternator123.
  • GPIO46 stuck low — also forces AP mode. Check the boot serial log; "GPIO46 LOW: OPERATIONAL AP MODE" means the pin is being held low.
  • GPIO45 stuck low — forces CONFIG mode regardless of saved credentials. Check for "GPIO45 LOW: FORCED CONFIGURATION MODE" in the boot log.
  • WiFi hardware fault — rare. Look for "WiFi connection failed" repeats in the serial log.

Forced recovery (any state)

Trick Effect
Hold GPIO45 LOW at boot Forces CONFIG mode with default AP credentials. Alternator disabled
Hold GPIO46 LOW at boot Forces AP mode with custom credentials. Full alternator operation
Hold GPIO41 LOW at boot Boots from factory partition (recovers from a bad OTA image)

Each pin has its own dedicated solder pad / pin header on the board — see the schematic for the exact location.

WiFi / Dashboard Issues

Symptom: Dashboard won't load

  • Wrong URL — Client mode uses http://alternator.local. The regulator's own hotspot uses http://192.168.4.1. Use HTTP, not HTTPS.
  • mDNS not resolving — some networks block mDNS, and it is unreliable over a phone hotspot. Find the IP directly: from your router's connected-devices list, or your phone's hotspot Connected Devices list, then browse to http://<that-IP>.
  • Over a phone hotspot the address changes — the regulator can come back at a different 172.20.10.X after the hotspot restarts. The iOS app re-finds it automatically; in a browser, re-check the address rather than trusting a bookmarked IP. See Reaching the dashboard over a hotspot.
  • Loading from the hosted site over https — a secure page can't reach a plain-http local device. Go directly to http://alternator.local or the regulator's IP.
  • Browser cache — hard-reload (Cmd-Shift-R / Ctrl-F5) once. The dashboard is five files served together (index.html, styles.css, script.js, and the two plotting-library files), and they are a matched set: every one is sent with Cache-Control: no-cache and an ETag, so the browser re-checks each on every load and normally gets a 304 with no re-download. A hard reload clears the case where one of the five was held back from an older bundle.
  • Capacitor app — see JavaScript Logic → Reconnect Logic.

Symptom: Regulator connects to a phone hotspot, then immediately drops

  • Budget prepaid carrier blocking it — a few low-cost prepaid plans (Visible is a confirmed one) restrict their Personal Hotspot and deauthenticate the regulator within a second or two of it associating, even when it is the only device connected. On the serial console you'll see a WiFi connect followed right away by WiFi STA disconnect reason: 2 (AUTH_EXPIRE), then CLIENT MODE FAILED. Nothing on the regulator can override this — the carrier is refusing the device, not the regulator failing to join (it associates fine; the network kicks it off). Use a standard (postpaid) plan, or put a travel router in client mode between the hotspot and the regulator. See WiFi Setup → Using your phone's hotspot.

Symptom: Dashboard loads but values don't update

  • Connection indicator (top right) — should be green. Yellow = reconnecting. Red = SSE stream lost. Manual reconnect button forces a retry.
  • CSV schema mismatch — open the browser console (F12). A warning like [CSV1] schema mismatch: ESP32=34, UI=33 means the firmware and dashboard versions are out of sync. Re-flash both web files (flashFactory or flashOTA) and firmware together.
  • 5+ minutes of disconnects, then auto-retry stops — Reconnect cap hit (MAX_SSE_RECONNECTS = 10). Press the manual reconnect button.

Symptom: Settings save fails

  • Settings unlocked? — press Unlock Settings in the header first. The unlock window on the device lasts 30 minutes, then the device relocks and silently-looking save failures start appearing (the Console tab logs each rejected write). Unlock again and re-submit.
  • WiFi password forgotten — the two WiFi passwords are recovered differently. See Forgotten passwords below.

Forgotten passwords

The regulator has two passwords, both WiFi-related. (Changing dashboard settings needs no password — just the Unlock Settings button.) Recovery:

Password What it protects If forgotten
Ship's WiFi password (Client mode) Joining your boat's network Boot with GPIO45 LOW → setup page appears → enter new ship-WiFi credentials
Hotspot password (AP mode, default alternator123) Joining the regulator's own WiFi Boot with GPIO45 LOW → the setup hotspot comes up with default credentials (ALTERNATOR_WIFI / alternator123) regardless of what you customized → set a new hotspot password

Note the live dashboard stays fully readable while settings are locked, and turning the alternator off never requires unlocking — only changing settings is gated.

Charging Issues

Symptom: Field never engages (duty stays at 0)

  • Ignition not detected — verify wiring to the ignition input (GPIO1, optocoupler). The dashboard shows IGN status in the header.
  • OnOff toggle off — the master alternator-enable toggle in the header.
  • BMS gate — if bmsLogic == 1, the BMS input (GPIO42) must be in the active state. Check bmsLogicLevelOff polarity setting.
  • In Idle stage (UseFloat=0 post-absorption) — by design. Will rebulk on voltage sag or discharge current.
  • In Lockout — a recent safety event triggered the cooldown (FIELD_COLLAPSE_DELAY, default 30 s). Wait it out; the dashboard shows remaining seconds.
  • In Limp Home — LimpHome == 1 is set. Console will show "LIMP HOME MODE" every 30 s.
  • RPM < MinRPMForField — engine not running, or RPM sensor not reading. Check Channel 2 raw voltage on the dashboard.
  • Weather mode disabled charging — currentWeatherMode == 1 means the forecast says solar is sufficient. Disable in Setup → Weather to override.

Symptom: Field engages but voltage doesn't reach bulk

  • uTargetAmps capped by RPM table — at low RPM the cap table is low by design. Increase the cap-table entry for your operating RPM if your alternator can sustain it.
  • Thermal derate — thermalPenaltyAmps > 0. The temperature loop is reducing current to protect the alternator. Look at AlternatorTemperatureF and projectedTempF.
  • MaxTableValue ceiling — sanity cap on the cap table. Increase only if you understand the consequences.

Symptom: Voltage overshoots target

  • Group 1/2/3 overvoltage supervisors not tuned — if the dashboard shows frequent fastOvClampCount increments and cv_I collapses, the CV loop is over-correcting. Run the Waveform Generator test to find better VoltageKp / VoltageKi values.
  • Load dump — g_loadDumpCount incrementing means loads are switching off and the current spike is being caught. Working as intended.
  • System voltage class mismatch — verify BulkVoltage is in the right range for your battery (e.g. 14.4 V on a 12 V system, not 14.4 V on a 24 V system).

Symptom: Repeated lockouts

  • Sensor disagreement — BatteryV (ADS) vs IBV (INA) differ by more than VoltageDisagreeThreshold for VoltageDisagreeTimeout. Check wiring, calibration, and noise on both inputs.
  • Voltage spike — AlternatorHardShutdownV exceeded. This is by design; it means the system saw a dangerous voltage. Look at the cv-log for what happened.
  • Temperature warning sustained — TempToUse > TemperatureLimitF + TempWarnExcess continuously for TempSustainedTimeout (default 2 min). Reduce alternator load, improve cooling, or raise TemperatureLimitF if your alternator is rated for higher.

Sensor Issues

Symptom: Temperature stale / 20 s cut

  • TempTask hung — Console will show "CRITICAL: TempTask hung up". Indicates Core 0 has stalled. Most often caused by I²C bus issues affecting other Core 0 work.
  • DS18B20 read failures — open Live Data → Stats, look at tempReadFailCount, tempCrcFailCount, tempCrcRecoveredCount. Climbing CRC failures = noisy bus, marginal pull-up, or corroded crimp.
  • Re-enumeration retry — after an enumeration failure the task keeps retrying: every 1 s while the engine is turning (RPM ≥ 200), where the 20 s staleness cut is armed and a fast recovery matters. With the engine stopped (RPM < 200) the retry is throttled to save standby power — every 5 s while a dashboard is connected, every 60 s when nothing is watching. A hot-unplugged sensor is picked back up on the first retry after it is reconnected.

Symptom: ADS1115 errors

  • adsI2CErrorCount climbing — bus contention or wiring fault. Inspect cabling.
  • adsSlowReadCount non-zero — I2C operation took > 5 ms. Indicates bus stall. Bench-test with a logic analyzer.
  • ADS1115Disconnected = 1 (auto) — 5 consecutive failures. Re-enabling requires reboot, or set to 0 via /get after fixing the wiring.

Symptom: INA228 errors

  • INA228 read failed in console — the I²C transaction threw an exception. Throttled to one per 10 s.
  • INADisconnected = 1 — manual disable (no auto-disable for INA228 right now). Voltage/current freeze at last value.

Symptom: IMU disabled

  • imuEnabled = false with "IMU disabled: N I2C errors in 60s" — 10+ I²C failures in any rolling minute. Check IMU wiring and I²C bus integrity. The bus already runs at 400 kHz (in-spec Fast-mode for every chip on it), so speed is not the cause — an 800 kHz experiment was tried and reverted after the ADS1115 dropped off, and Wire.setClock(400000) in initializeHardware() carries a comment saying not to raise it again.

Crash / Reset Investigation

Reading the reset reason

The dashboard's Stats panel shows LastResetReason — and, beside it, ancientResetReason, the reset before that one. Both are stored as small integer codes and rendered by the dashboard as plain-English labels. The ones that matter:

Label shown Meaning
Power-on (plugged in) Normal cold boot
Software reset (unscheduled) Firmware called ESP.restart() (factory reset, post-OTA)
Scheduled maintenance restart A restart the firmware planned — distinguished from the unscheduled case by a flag file written before rebooting
Task watchdog (loop blocked) / Interrupt watchdog / Other watchdog Watchdog timeout — the main loop (Core 1) hung
Panic/Exception (crash) Crash — see coredump partition; this is the case that prints a backtrace
CPU lockup (double exception) Fault while already handling a fault
Brownout (power issue) / Power glitch (supply transient) Supply voltage dipped or glitched
External reset (button) / USB reset / JTAG reset Reset asserted from outside the firmware

An unrecognized code displays as "Unknown reset". The full code-to-label map is resetReasonLookup in web_src/script.js; the firmware side is captureResetReason() in 5_functions.ino, which also records the raw ESP-level and per-CPU ROM reset codes for cases the plain-English label collapses.

Decoding a crash backtrace

When the device prints Guru Meditation Error followed by a backtrace, decode it with the ESP32 toolchain's address-to-source tool (xtensa-esp32s3-elf-addr2line, installed with the board package) against the .elf file from your build. Feed it the 0x42...... application addresses from the backtrace; it prints the source file and function for each.

Cloud / OTA Issues

Symptom: Cloud upload says "queued" but never completes

  • WiFi RSSI — uploads gate on RSSI ≥ −80 dBm. Weak signal blocks uploads. (OTA firmware downloads use a deliberately looser bar, −90 dBm / OTA_MIN_RSSI_DBM: signal strength was never the thing that made a download fail, so the gate was widened and the timeouts lengthened instead.)
  • fieldOffSettled(10 s) gate — the field must be off for 70 s before uploads run. Cycling the engine on/off resets the timer.
  • Backoff active — after 5 consecutive failures, uploads suspend for 30 s. Console shows "Cloud sync paused, too many failures."

Symptom: OTA update fails

  • Signature verification fail — wrong bundle for this device, or the public key in firmware doesn't match the signing private key. Check OTA_PUBLIC_KEY in Xregulator.ino against the build pipeline.
  • Heap integrity fail — system is in a degraded state. Boot from factory partition (GPIO41 LOW) to clear, then retry.
  • Mid-download disconnect — automatic rollback. Factory partition keeps running.

Recovery from a bad OTA image

  1. Power off the regulator.
  2. Ground GPIO41.
  3. Power on. Boot log will show "We are in factory partition."
  4. Connect to the dashboard normally and clear the bad OTA from the Cloud → Firmware tab.

Last-Resort Recovery

Factory reset (preserves only the cloud registration token)

  • Via dashboard — Setup → System → Security & Maintenance → Erase All Memory (the button reads "Restore Defaults"; settings must be unlocked, plus a confirm dialog). Reformats the LittleFS data partition (logs, history rings, buffered uploads), erases every NVS namespace — which is where user settings, the vessel description, and lifetime counters actually live — and restarts. Defaults are re-created on the next boot, not during the reset itself, so nothing carries the old values back into flash.
  • The one deliberate exception is the cloud registration token (authToken): performDeepFactoryReset() reads it out before the wipe and writes it back afterwards, so your cloud account, history, and leaderboard entries survive. It is held only in RAM for the two or three seconds of the wipe, so losing power during the reset loses the token and needs support contact to re-register. To delete the cloud account too, press Delete All Data first — see Cloud Account Management.
  • Via GPIO45 boot — enters CONFIG mode with default credentials so you can at least reach the device, but does NOT wipe settings. Use to recover from a lost WiFi password without losing data.

Re-flashing firmware over USB

If OTA is broken and GPIO41 boot doesn't help, fall back to USB-C:

  1. Connect USB-C.
  2. Hold GPIO0 LOW, press Reset, release GPIO0. The chip enters bootloader mode.
  3. Upload the sketch from the Arduino IDE — see Building the Firmware. (flashFactory / flashOTA do not re-flash firmware: they only rebuild the compressed web bundle and write it to the factory_fs / prod_fs filesystem partitions.)

An Arduino IDE upload writes the factory application slot only — it never touches user settings, because the LittleFS and NVS partitions are left alone. Note that a device that has already taken an over-the-air update boots from ota_0, so USB-flashed code does not run until you force the factory partition (GPIO41 LOW at boot).

Getting more help

  • GitHub Issues — github.com/markliquid1/Regulator2026-public/issues for bug reports and feature requests.
  • Console output — the dashboard's Console tab is the live serial-equivalent log. Save it before reporting an issue.
  • Stats panel — Live Data → Stats shows error counters and runtime diagnostics. Include them in bug reports.