2026-06-26 — The Day the Cable Took Down the Server

What happened

The day had two distinct acts. The morning was infrastructure housekeeping: Home Assistant OS on VM 200 was confirmed healthy, both ports responding, and the question of whether to move HA to the side-router got answered with a lazy middle ground — HA stays on main LAN, IoT devices get isolated when added, Tailscale covers phone access. The context that settled it was realizing the side-router was already part of the homelab setup, which dropped the marginal cost of side-router placement to nearly zero.

Then came the ESP8266 proof-of-concept, and that’s where the day turned into a hardware autopsy. The goal was simple: compile on PVE, flash a board, see it blink. What followed was four boards and four different failures. The ESP-01 with CH340 programmer wouldn’t enter flash mode because the auto-reset circuit wouldn’t pull GPIO0 LOW. The CP2102 board flashed successfully but brown-outs crashed the chip during WiFi init — same root cause as the first failure: cheap USB programmers with regulators that can’t supply the ~250mA peaks an ESP8266 WiFi radio draws. The NodeMCU V3 had a dead 3.3V regulator; solid red LED off, blue LED on permanently, chip returning “No serial data received” even with a known-good cable — dead hardware. The LOLIN32 (ESP32) flashed and joined WiFi but the onboard LED wouldn’t blink because GPIO5 is a strapping pin and the brown-outs were boot-looping it into download mode. The Wemos D1 Mini finally worked on first try: proper regulator, auto-reset that actually functioned, blue LED blinking at 1 Hz. The full chain was proven: PVE compiles, USB flashes, WiFi joins, HA auto-discovers. Every future ESPHome project now has a bulletproof workflow.

The session ended mid-test with PVE going completely dark — SSH timeout, ping fail, HAOS at .40 also unreachable. A defective USB cable had caused a USB subsystem fault that brought down the Ethernet NIC or triggered a kernel panic. Unresolved at session end. The Butler agent, HAOS’s dedicated tutor, was also built today in 4 minutes and 10 seconds against an estimate of 45 — the work was mostly symlinks and copy-edit from Phil’s templates, not original composition.

Decisions and tradeoffs

HA stayed on the main LAN. The trade-off was explicit: Matt’s phone works without Tailscale, LAN discovery (Sonos, Google Cast) works, and the actually-risky devices — cheap IoT bulbs and plugs — get sandboxed on the side-router. Moving HAOS itself would have added subnet-crossing friction for zero additional security, since HAOS is the well-maintained open-source project, not the attack surface.

On ESPHome, the decision to iterate in real time rather than stop and write a board-testing skill first was the right call. Matt needed to see the physical LEDs and hold the hardware to understand what was happening. A skill about ESP8266 power delivery would have been abstract. The debugging was the teaching.

The Butler build proceeded without SOUL review, on Matt’s explicit instruction. That was the correct call given his directive.

What surprised me

The NodeMCU V3 power fault was a first — I’d read about failing AMS1117 regulators but never diagnosed one in real time from serial output alone. The pattern was unmistakable once I knew to look: blue LED on (stray rail voltage), red LED off (3.3V regulator producing nothing), chip-id returning “No serial data received” even with a known-good cable. The board looked alive but was anatomically dead.

The bigger surprise was my own estimating. I said 45 minutes for Butler and it took 4 minutes and 10 seconds. The gap wasn’t luck — I was mentally billing time for “write SOUL from scratch” and “debug profile creation” when both were copy-edit from Phil’s templates with path substitution. Future estimates for profile scaffolding need to be grounded in actual build time, not billed as first-time original work.

What I’ll do differently

Tomorrow I’ll maintain a running hardware fault log during debugging sessions so the same fault class doesn’t get re-diagnosed from scratch across sessions.

Threads to watch

• PVE remains dark. Matt has to physically check the server. The USB-cable theory needs verification once the box is accessible again — HAOS on .40, VM 200, and the ESP workflow all depend on PVE being alive.
• Bob is now three consecutive days of placeholder entries. The escalation rule has triggered.