architecture docs
Data flow

From a device to AWS

Everything that happens between a real NS-C1 sending a message and that message being stored and acted on.

One message from a real device. The virtual device follows exactly the same path.

1. Identity and connection

  • Each device is a thing in AWS IoT, named by its 16-character device ID, with its own certificate.
  • It connects to the stage’s IoT Core endpoint over MQTT with TLS, using the device ID as its client ID.
  • The device policy only lets it connect as itself, publish to its own telemetry topic, and read and update its own shadow. It can’t see any other device.
  • AWS publishes a presence event on every connect and disconnect. That’s how the admin knows a device is online, and which certificate it used.
Not built yet

How real devices get their certificate at the factory (AWS IoT Fleet Provisioning) isn’t set up yet. Virtual devices already register themselves on dev and staging.

2. What a device sends, and how often

The format is the hardware team’s spec, draft v0.1 (firmware 0.3.0). All times are UTC.

Message When Stored as
Position report (GNSS, or cell tower when there’s no fix) every 300 s with a fix; a cell report at most every 600 s without one pings
Track (extra GNSS fixes) every 60 s while moving extra pings
Battery (soc) with reports on the ping and the device
Draw (draw_data) after each draw draws
Pod (pod_data) attach or remove pod_events
Charge events plugged in or out charges
Online, firmware update, log on boot and changes device_status_events
Presence (sent by AWS) connect and disconnect the device’s connected state

Indoors, devices often have no GNSS fix: about 40% of night reports are cell lookups. A cell report is turned into a position by AWS Device Location; one it can’t place is kept and marked unresolved.

3. Topic rules: from MQTT to a queue

Three IoT topic rules copy messages to one SQS queue. Each adds where the message came from and when the broker received it, so ingest never relies on the device’s clock alone.

Rule Listens to Carries
telemetry ns-c1/+/telemetry everything the device publishes
shadow documents $aws/things/+/shadow/update/documents the shadow before and after a change: how the device’s lock state comes back
presence $aws/events/presence/+/+ connects and disconnects

If a rule itself fails, the message goes straight to the dead-letter queue (DLQ).

4. The queue

  • The queue absorbs bursts, so a slow database never loses device data.
  • A message that fails 3 times moves to the DLQ, which keeps it for 14 days so someone can look at it.
  • No batching delay: a quiet queue hands messages over at once. With one, AWS could hold them for up to 20 s, and an owner waiting on the unlock screen would feel it.

5. Ingest: store, then react

  1. Validate Each message is checked against the firmware wire format. An invalid one goes back to the queue and, after its retries, to the DLQ.
  2. Normalize Firmware messages become plain events: positions, draws, charging, pods, battery, firmware status.
  3. Place cell reports Cell towers become positions through AWS Device Location.
  4. Store The events, and the device’s state: connected, battery, charging, and lock state as the device reports it.
  5. Run the realtime models Today school-zone/v1, for the devices that just sent something.
  6. Apply the trust ladder Any resulting lock, unlock or SMS goes into the outbox. See From AWS back to a device.
Safe to receive twice

SQS can deliver a message more than once. Duplicates are ignored, state only moves forward, and models resume from where they stopped. If models fail for one device, only that device’s messages are retried.

What happens later

Some things only show over days, or must happen even when a device is silent. A second Lambda, owner-check, runs every morning at 07:00 Montréal time. It runs the daily models (today owner-change/v1) and enforces deadlines: a missed re-verification becomes a hard lock, and a scheduled check that has come due starts one.