Monitoring indoor microclimates, air quality dynamics, and ambient illumination across multiple floors presents surprising engineering challenges. Consumer-grade smart home sensors are frequently closed-source, cloud-dependent, sample at coarse intervals, and trap heat from internal radios—biasing temperature and humidity readings. Furthermore, managing time-series telemetry across battery-backed or wall-powered nodes requires resilient local failover when Wi-Fi drops, low-drift sampling intervals, and sub-millisecond database ingestion.

Home Sensor is an end-to-end, multi-node environmental monitoring platform engineered from the ground up. It spans custom ESP32 sensor pods with polymorphic C++ firmware, parametric 3D-printed convective enclosures, an ingestion and web daemon written in Go (Fiber v2), a column-oriented QuestDB time-series database, and a hyper-responsive frontend built with Templ, HTMX, and Apache ECharts.

System Architecture

The system operates across three decoupled tiers: edge data acquisition, telemetry ingestion and time-series storage, and server-rendered presentation.

flowchart TD
    subgraph EdgeLayer ["Edge Sensor Pods (ESP32 / C++ Firmware)"]
        direction TB
        subgraph Hardware ["Hardware Bus & Sensors"]
            I2C["I2C Bus (Wire / Wire1)"]
            I2C --> SHT["SHT40
Temp (±0.2°C) & Humidity"]
            I2C --> TSL["TSL2591
Visible, Full & IR Lux"]
            I2C --> SGP["SGP40
VOC Air Quality Index"]
            I2C --> LPS["LPS22 / BME680
Barometric Pressure"]
            SPI["SPI Bus"] --> SD["MicroSD Storage
Audit & Backup Logs"]
            GPIO["GPIO Control"] --> LED["WS2812 NeoPixel
RGB State Diagnostics"]
        end

        subgraph Firmware ["Firmware State Machine"]
            PROBE["Polymorphic Auto-Probe
Detects active I2C sensors"]
            SM["StateManager (RAII)
Drift-compensated scheduling"]
            FMT["JSON Formatter
StaticJsonDocument buffer"]
            PROBE --> SM --> FMT
        end

        Hardware <--> Firmware
    end

    subgraph NetworkTier ["Network & Failover Pipeline"]
        FMT --> UPLOAD{"HTTP POST
/sensors/data/submit"}
        UPLOAD -- "Success" --> SLEEP["Power Off Wi-Fi
WiFi.disconnect(true)
Pause/Deep-Sleep Interval"]
        UPLOAD -- "Offline / Timeout" --> BUFFER["SD Backup Buffer
Append to sensor_backup.log"]
        BUFFER -.->|"On Connection Restore"| DRAIN["uploadBackup()
Stream queued telemetry"]
        DRAIN --> UPLOAD
    end

    subgraph ServerTier ["Telemetry Daemon (Go / Fiber v2)"]
        AUTH["Source Auth Guard
Validate UUID in sources.json"]
        ILP_POOL["QuestDB ILP Pool
go-questdb-client/v3 (TCP 9009)"]
        ROUTER["Fiber v2 REST API
Endpoints for data & UI"]
        
        UPLOAD --> ROUTER --> AUTH --> ILP_POOL
    end

    subgraph StorageTier ["QuestDB Engine"]
        QDB[("QuestDB Storage
Tables: <Location>.<Sensor>
ILP Ingest (9009) / SQL REST (9000)")]
        ILP_POOL --> QDB
    end

    subgraph FrontendTier ["Server-Driven UI (Templ + HTMX + ECharts)"]
        TEMPL["Templ Components
Index, Sensors, Measurements, Plot"]
        HTMX["HTMX Dynamic Swapping
Accordion & Toggle interactions"]
        ECHARTS["Apache ECharts
Interactive SVG curves & time zoom"]

        ROUTER <--> TEMPL
        TEMPL --> HTMX --> ECHARTS
        QDB -.->|"REST SQL Queries (/exp)"| ROUTER
    end

Hardware Pods & Sensor Suite

Home Sensor Hardware Pod: 3D-printed enclosure with slotted convective venting lattice, removable lid, and internal I2C sensor stack featuring the Adafruit TSL2591 light sensor, Sensirion SGP40 VOC sensor, and ESP32 microcontroller

The edge nodes are deployed across physical locations (e.g. Basement, LivingRoom, Bedroom). Depending on space constraints, each pod runs on either a SparkFun Thing Plus C (ESP32 WROOM with USB-C and integrated LiPo management) or an Adafruit QT Py ESP32 Pico.

  +-------------------------------------------------------------+
  |                   Multi-Floor ESP32 Nodes                   |
  |                                                             |
  |   [Basement Node]           [Living Room Node]              |
  |   - SHT40 (Temp/Humidity)   - SHT40 (Temp/Humidity)         |
  |   - TSL2591 (Light/IR)      - TSL2591 (Light/IR)            |
  |   - SGP40 (VOC Air Quality) - SGP40 (VOC Air Quality)       |
  |   - LPS22 (Air Pressure)    - MicroSD Failover Card         |
  |   - NeoPixel Status RGB     - NeoPixel Status RGB           |
  +-------------------------------------------------------------+
                                |
                   Wi-Fi HTTP (Burst Ingestion)
                                v
  +-------------------------------------------------------------+
  |           Go Telemetry Server & QuestDB Engine              |
  |                                                             |
  |   - Ingest Pipeline (Go Fiber v2)                           |
  |   - InfluxDB Line Protocol (ILP over TCP 9009)              |
  |   - Time-Series Engine: Partitioned Columnar QuestDB        |
  |   - Reactive Dashboard: Templ + HTMX + ECharts              |
  +-------------------------------------------------------------+

Sensor Complement

  1. Sensirion SHT40 (Adafruit_SHT4x): High-accuracy relative humidity (±1.8% rH) and temperature (±0.2°C). Configured in high-precision mode with the internal heater disabled (SHT4X_NO_HEATER) to eliminate thermal self-bias.
  2. Adafruit TSL2591 (Adafruit_TSL2591): High-dynamic-range luminosity sensor capable of measuring from 188 $\mu$Lux up to 88,000 Lux across full-spectrum (visible + IR) and infrared channels. Configured with a 25x medium gain and 300 ms integration time to track indoor day/night cycles, screen glare, and room illumination.
  3. Sensirion SGP40 (Adafruit_SGP40): Metal-oxide digital gas sensor generating standardized VOC (Volatile Organic Compound) Air Quality Index values, tracking indoor air freshness, cooking emissions, and ventilation efficacy.
  4. ST LPS22 (Adafruit_LPS2X): Digital piezoresistive absolute pressure sensor measuring barometric pressure in hPa (with 1 Hz sampling rate) alongside secondary ambient temperature.
  5. Bosch BME680 (Adafruit_BME680): Environmental multi-sensor fallback measuring pressure, humidity, ambient temperature, and VOC gas resistance ($k\Omega$) with 3-tap IIR filtering and 8x temperature oversampling.
  6. MicroSD SPI Card Logger: Provides dual-file persistent storage on board. It maintains an audit trail (sensor_data.log) and buffers telemetry during network interruptions (sensor_backup.log).

Thermal Mitigation & Mechanical Enclosure

A primary pitfall in IoT sensor design is sensor self-heating: placing an ESP32 microcontroller inside a small enclosure will heat the surrounding air by 2–4°C, completely skewing temperature, humidity, and VOC readings.

Two critical design decisions eliminate this artifact:

  • Custom Parametric 3D Printed Case (case/home_sensor.stl): The enclosure features horizontal slotted convective vents positioned to thermally decouple the ESP32 compute board from the ambient sensor chamber. Heat rises out of the top exhaust without bathing the I2C sensor boards.
  • Aggressive Wi-Fi Duty-Cycling: The firmware keeps the Wi-Fi radio powered off during sensor measurements. It only powers on the 2.4 GHz transceiver for a brief burst (~200–500 ms) to dispatch the HTTP POST, immediately issuing WiFi.disconnect(true) to shut down the RF amplifier before entering the inter-sample pause.

Visual State Feedback (RGB NeoPixel)

Each pod features a surface-mount WS2812 RGB LED providing real-time hardware status without requiring serial monitor access:

Color Hex / RGB State Description
White rgb(100, 100, 100) Bootstrapping, I2C bus scan & network configuration
Cyan rgb(0, 100, 100) StateManager lifecycle initialized, cycle timer armed
Green rgb(0, 100, 0) Actively taking sensor measurements
Blue rgb(0, 0, 100) Transmitting telemetry payload over HTTP POST
Orange rgb(100, 40, 0) Appending telemetry to local MicroSD card
Purple rgb(100, 0, 100) Network error: buffering payload to sensor_backup.log

Embedded Firmware Architecture (C++ / PlatformIO)

The firmware is developed in modern C++ within PlatformIO, leveraging an object-oriented architecture designed around modular sensor abstraction, automatic hardware probing, and zero-drift scheduling.

Polymorphic Sensor Abstraction

Every sensor driver derives from a pure virtual Sensor base class:

class Sensor
{
  protected:
    Sensor() = default;

  public:
    virtual ~Sensor() = default;

    virtual bool ok() const = 0;
    virtual std::string name() const = 0;
    virtual bool update() = 0;
    virtual void json(StaticJsonDocument<1024>& j) const = 0;
    virtual void log() const = 0;
    virtual void details() const = 0;
};

On boot, State::initSensors() attempts to initialize all candidate sensor drivers across the active I2C bus (Wire or Wire1). Each driver performs an initial hardware handshake in its constructor:

void State::initSensors()
{
    std::vector<std::unique_ptr<Sensor>> temp_sensors;
    temp_sensors.emplace_back(new TempHumSensor(wire));
    temp_sensors.emplace_back(new LightSensor(wire));
    temp_sensors.emplace_back(new EnvSensor(wire));
    temp_sensors.emplace_back(new GasSensor(wire));
    temp_sensors.emplace_back(new PressureSensor(wire));

    // Polymorphically keep only detected sensors
    for (auto& s : temp_sensors)
    {
        if (s->ok())
        {
            Serial.print("SENSOR OK: ");
            Serial.println(s->name().c_str());
            sensors.push_back(std::move(s));
        }
        else
        {
            Serial.print("SENSOR FAIL: ");
            Serial.println(s->name().c_str());
        }
    }
}

This dynamic probing allows a single firmware binary to be deployed across heterogeneous hardware builds—nodes with different sensor combinations automatically register only the peripherals physically present.

Drift-Compensated RAII Scheduling

A common bug in simple delay(30000) firmware loops is cumulative clock drift: sensor reads, I2C transactions, JSON serialization, and Wi-Fi handshakes consume a variable amount of time (200 ms to 3 seconds), causing the measurement window to slip over time.

Home Sensor resolves this using an RAII (Resource Acquisition Is Initialization) StateManager wrapper combined with drift-subtracted pausing:

StateManager::StateManager(State& s) : state(s)
{
    startTime = millis();
    state.led.color(0, 100, 100); // LED - CYAN
}

StateManager::~StateManager()
{
    // Subtract elapsed work time from the configured cycle period
    pause(millis() - startTime);
}

The pause calculation subtracts elapsed_ms from the target interval (UPDATE_MIN=3):

void pause(uint32_t elapsed_ms = 0)
{
    int update_ms = -elapsed_ms;
    update_ms += (UPDATE_MIN * 60) * 1e3; // e.g. 180,000 ms

    if (update_ms <= 0) return;

#ifdef USE_DEEP_SLEEP
    ESP.deepSleep(update_ms * 1e3); // Microseconds for deep sleep
#else
    delay(update_ms);
#endif
}

Because StateManager is instantiated on the stack inside loop(), its destructor executes deterministically at scope exit, maintaining a strict 3-minute cadence regardless of network latency.

Resilient Offline Failover & Backlog Draining

If an access point restarts or home Wi-Fi drops, telemetry must not be lost:

  1. In main.cpp, if m.submitData(buffer) fails, the node falls back to m.backupData(buffer), writing the serialized JSON payload into sensor_backup.log on the MicroSD card.
  2. During normal operation or upon reconnection, StateManager::uploadBackup() verifies if a backup file exists.
  3. It opens sensor_backup.log, streams queued records line-by-line via submitData(), and calls SD.remove() once the backlog is fully drained.

Backend Telemetry Server (Go & QuestDB)

The server daemon (go_home) is written in Go 1.24 using the high-performance Fiber v2 web framework.

Ingestion Pipeline & Pod Verification

The node transmits a structured JSON payload over HTTP POST to /sensors/data/submit:

{
  "id": "3DF66AD4-50BA-40BD-87B8-546664B1CE53",
  "sensor": "Basement",
  "data": {
    "TempHum": {
      "t_ms": 14205,
      "Temperature_C": 20.4,
      "Humidity_PrH": 48.2
    },
    "Light": {
      "t_ms": 14210,
      "Light_lux": 145.2,
      "Full": 810,
      "Visible": 620,
      "IR": 190
    },
    "Gas": {
      "t_ms": 14215,
      "Raw": 28410,
      "VocIndex": 105
    }
  }
}

When receiving a payload, the Go server verifies the hardware UUID against a whitelist in sources.json. This guarantees that rogue devices cannot spoof sensor locations:

func verifySource(source_uuid string) string {
    data, err := os.ReadFile(sources_filename)
    if err != nil {
        return ""
    }

    sourceMap := make(map[string]string)
    json.Unmarshal(data, &sourceMap)

    return sourceMap[source_uuid] // Returns "Basement", "LivingRoom", etc.
}

High-Throughput QuestDB Ingestion

Rather than paying relational database overhead, telemetry is ingested directly into QuestDB using the InfluxDB Line Protocol (ILP) over TCP port 9009 via go-questdb-client/v3.

QuestDB stores time-series data in partitioned, column-oriented tables. The server dynamically routes each sensor’s measurements into dedicated tables namespaced as <Location>.<Sensor> (e.g. Basement.TempHum, LivingRoom.Light):

func SubmitData(ctx *fiber.Ctx) bool {
    // ... parse JSON & verify UUID ...
    err, sender := getSender()
    defer sender.Sender.Close(sender.Ctx)

    for sensor, measurementPair := range sensorReadings.Data {
        row := sender.Sender.Table(location + "." + sensor)

        for measurement, value := range measurementPair {
            row.Float64Column(measurement, value)
        }

        if err := row.AtNow(sender.Ctx); err != nil {
            return false
        }
    }

    sender.Sender.Flush(sender.Ctx)
    return true
}

Timestamp assignment uses QuestDB’s AtNow(ctx) to index incoming measurements with nanosecond server precision.

SQL Query Engine

Querying historical data is performed directly against QuestDB’s HTTP REST query endpoint (/exp) using standard SQL:

  • Sensor Discovery: SHOW TABLES discovers all active nodes and sensor subsystems.
  • Metric Introspection: SHOW COLUMNS FROM '<Location>.<Sensor>' yields active measurement channels.
  • Time-Windowed Extraction: 
    SELECT timestamp, Temperature_C 
FROM 'Basement.TempHum' 
WHERE timestamp > dateadd('d', -7, now());

Server-Driven Reactive Dashboard (Templ + HTMX + ECharts)

Rather than building a heavy client-side Single Page Application (SPA) with NPM, Webpack, and client state managers, the user interface is rendered server-side using Templ (github.com/a-h/templ) and dynamically orchestrated using HTMX.

Dynamic UI Composition

  1. Root Page (index.templ): Loads DaisyUI (Forest dark theme), Tailwind CSS, HTMX, and Apache ECharts. It defines a time-range slider (1 to 7 days) and triggers an initial HTMX fetch: 
    <div id="sensors" hx-get="frontend/sensors" hx-trigger="load"></div>
  2. Sensor Accordion (sensors.templ): The server queries QuestDB for active tables, partitions them by location, and generates an accessible DaisyUI collapse accordion. Clicking a sensor sends an HTMX request to load its measurements: 
    <input hx-get={ "frontend/measurements/" + location + "/" + sensor } 
       hx-target="#measurement_display" 
       type="radio" 
       name="sensors-radio" 
       class="toggle toggle-primary"/>
  3. Measurement Selector (measurements.templ): Displays radio buttons for all columns in that table (Temperature_C, Humidity_PrH, Light_lux, etc.). Clicking any measurement triggers frontend/data/:location/:sensor/:measurement/:days, rendering into #data_display.
  4. Interactive Plotting (plot.templ): Generates an Apache ECharts visualization with zero client-side charting libraries to install:
templ Plot(location string, sensor string, measurement string, data db_wrapper.MeasurementSequence) {
    <h1 class="text-2xl">{ location }.{ sensor }.{ measurement }</h1>
    <div id="data_plot" style="width: 100%; height: 600px"></div>
    <script>
        var plot_dom = document.getElementById("data_plot");
        var plt = echarts.init(plot_dom, "dark", { renderer: "svg" });

        var timestamps = {{ data.Timestamps }};
        var values = {{ data.Values }};

        var d = [];
        for (var i = 0; i < timestamps.length; i++) {
            d.push([timestamps[i], values[i]]);
        }

        plt.setOption({
            xAxis: { type: "time" },
            yAxis: { type: "value", scale: true },
            dataset: { source: d, dimensions: ["timestamp", "value"] },
            series: [{
                type: "line",
                smooth: true,
                color: "rgb(31, 184, 171)",
                encode: { x: "timestamp", y: "value" }
            }],
            tooltip: { trigger: "axis" },
            backgroundColor: "oklch(20.84% .008 17.911)",
            dataZoom: [{ type: "slider", fillerColor: "oklch(30.698% .039 171.364 / 25%)" }]
        });
    </script>
}

This stack renders interactive SVG time-series charts with smooth curves, axis hover cards, and time-range sliders without a single byte of Node.js runtime code.

Containerization & CI/CD

The Go server and QuestDB instance are deployed via Docker. A multi-stage Dockerfile keeps the final production image under 25 MB:

# Builder container
FROM golang:alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . ./
RUN go build -o go_home main.go

# Minimal runtime container
FROM alpine:latest
WORKDIR /app
COPY --from=builder /app/go_home ./
COPY --from=builder /app/sources.json ./
EXPOSE 8000
ENTRYPOINT ["/app/go_home"]

Continuous integration is managed via GitHub Actions (test.yaml), running two concurrent validation pipelines:

  • Firmware Job: Installs Python 3.11 and PlatformIO Core to compile all ESP32 firmware targets (pio run).
  • Server Job: Provisions Go 1.24, installs templ@latest, compiles templates (templ generate), and compiles the Go telemetry binary (go build).

Key Takeaways

  1. Hardware/Firmware Co-Design: Eliminating sensor self-heating requires both mechanical solutions (convective chimney slots in 3D-printed enclosures) and firmware solutions (rapid Wi-Fi burst transmission followed immediately by RF radio power-down).
  2. Polymorphic Embedded C++: Virtual sensor interfaces combined with startup I2C probing allow a unified codebase to run across diverse sensor loadouts without compile-time fragmentation.
  3. Resilient Failover: Local MicroSD card buffering guarantees that environmental data survives residential Wi-Fi interruptions and server reboots.
  4. Modern Go Stack: Combining Go Fiber with Templ and HTMX delivers the speed and simplicity of server-side rendering while matching the user experience of dynamic single-page applications.