Electronics Guide

What an Update System Must Guarantee

Five properties motivate nearly every decision that follows, and three of them belong to the fleet and its operator rather than to the device. The device must never become unbootable, because a device that stops responding cannot be updated again, which makes the failure permanent and the repair a truck roll. The update must be atomic from the device's point of view: at every instant it runs the old software or the new, never a mixture, since a half-updated device usually boots and misbehaves subtly rather than failing loudly. Only authorized images may run, and that authorization must survive a compromised server, a compromised network, and — in demanding threat models — a stolen signing key. The fleet must be steerable: the operator chooses which devices update, in what order and how fast, and can stop, because a rollout that cannot be halted is an outage generator with a schedule. The outcome must be observable, which covers both how many devices took the update and the harder question of whether they still work.

The Update Client and Its State Machine

On the device, the update system is a small piece of software with an outsized responsibility. It is worth designing as an explicit state machine, written down and reviewed, because the interesting behavior lies in the transitions taken after an unexpected reset and a state machine makes those transitions enumerable.

The States

A representative client moves through Idle, the normal running condition; Checking, in which it presents its identity and current version and receives either nothing or a manifest; Downloading, which must be interruptible and resumable; Verifying, where it authenticates the manifest, checks the payload digest, and confirms the image is meant for this hardware; Installing; Pending, in which the new image is written and marked as a candidate but has not yet run; Testing, its first boot, during which it must prove itself; and the terminal outcomes Confirmed and Reverted, both of which must be reported.

The state must persist across a reset and must change only at points where a power loss leaves the device in a condition the boot code can interpret. In practice it lives in a small, separately erasable region — a dedicated flash sector, an EEPROM area, or a battery-backed register file — written twice with a sequence number and a checksum on each copy, so a reset during one write always leaves at least one valid record.

Where the Client Runs

Putting download and verification logic in the application is almost always correct: the application already owns the network stack, the file system, the power policy, and the ability to schedule work when the device is idle. Bootloaders that acquire network stacks tend to grow, and every kilobyte of bootloader is difficult or impossible to update safely. The bootloader's job is narrow — read the persisted state, choose an image, verify it, and, where the design uses a trial boot, arm the mechanism that reverts if the new image does not confirm itself. MCUboot, the open-source secure bootloader maintained under the Zephyr project umbrella, models this division: it validates images and manages slots and knows nothing about how the image arrived. Bootloader Development covers the boot-side mechanics this article assumes.

The client also owns policy that decides whether users tolerate the system. A vehicle must not install while moving, an infusion pump must not install during a therapy, and a battery-powered sensor must not download two megabytes on a nearly flat cell. These are preconditions on the transition into Installing, and they are why a device may sit in Pending for days.

Image Layout: A/B Banks Against Single Bank Plus Recovery

The decision that most constrains an update system is where the new image goes while it is being written. Two families of answer dominate, and each buys a different property at a different price.

Dual-Bank, or A/B

An A/B layout carries two complete application slots. The running image executes from one while the update is written to the other; when the write completes and verifies, the bootloader is told to run the other slot at the next boot. The old image stays intact, so any failure before the switch leaves the device running what it was already running. The properties are excellent: the update is atomic by construction, because the switch is a single small write to a state record; rollback is instantaneous and always available; and the device stays in service during the download and the write. Android's A/B, or seamless, update scheme is the most widely deployed instance of the pattern, and it is what allows a phone to apply an update in the background and reboot once.

The cost is storage: roughly twice the application flash the product strictly needs. On a microcontroller with 256 kilobytes of internal flash, doubling the application region is often the difference between a part that fits the bill of materials and one that does not. Mitigations exist. The secondary slot can live in an external serial NOR flash device costing a fraction of on-die flash, with the bootloader copying or swapping into the primary slot at update time. Android's Virtual A/B obtains A/B semantics through a snapshot mechanism instead of permanently duplicating every partition.

Single Bank Plus Recovery

The alternative reserves one application slot and one small, rarely changed recovery image. The update is staged — into external flash, an SD card, or a file system — and then applied by overwriting the application slot. If the overwrite is interrupted, that slot is invalid and the bootloader falls back to the recovery image, which knows just enough to re-download and re-apply. The storage saving is real. The cost is a window during which the device is not running a usable application, and the fact that a defective recovery path loses the device outright. Recovery images receive the least testing precisely because they run almost never, which pairs badly with being the last line of defense; exercise the recovery path regularly, including on production hardware.

A middle position is common on microcontrollers: one application slot plus a secondary staging slot, with the bootloader performing a swap. MCUboot supports several such modes, including swap-using-scratch, which exchanges the slots sector by sector through a scratch region while keeping enough state to resume after a reset, and direct execute-in-place, in which the bootloader runs whichever slot holds the newer valid image and never copies. Direct execution is cheapest in time and flash wear, at the price of an image linked for either slot address or made position independent. Wear matters: NOR flash used for code typically specifies ten thousand to one hundred thousand erase cycles per sector, swap designs write more than copy designs, and a failed and retried update writes twice.

Delta and Differential Updates

Shipping a full image on every release is simple and often wasteful: a typical release changes a small fraction of the binary, and on a metered cellular link the difference between two megabytes and forty kilobytes decides whether a campaign is affordable.

Binary Diffing

A delta update ships a patch computed between the old and new images, and the device reconstructs the new image locally. The classic algorithm for binary executables is bsdiff, published by Colin Percival, which suffix-sorts to find approximate matches and then compresses the differences; it beats generic differencing on compiled code because it tolerates the byte-level perturbations recompilation introduces. Google's Courgette, developed for Chrome updates, goes further: it disassembles the binary, rewrites internal pointers symbolically, differences that representation, and reassembles, collapsing the address changes a small code insertion produces. The Zstandard compressor's --patch-from mode offers a simpler and faster path adequate for many embedded cases.

Why Delta Needs a Deterministic Build

The economics of binary diffing depend entirely on build determinism, and most teams discover this late. If the toolchain embeds a build timestamp, a host name, or an absolute source path, those bytes change on every build. If the linker orders sections or functions differently between builds — because of hash-table iteration order, parallel compilation, or link-time optimization sensitive to file ordering — large regions shift and the diff explodes. A patch that should be forty kilobytes becomes four hundred, and the added complexity stops paying.

Reproducible builds are therefore a prerequisite: pin compiler, linker, and library versions and record them with the artifact; eliminate embedded timestamps or derive them from a source-controlled value; use a path-remapping compiler option; sort inputs deterministically; and verify by building the same commit twice on two machines and comparing digests. Without this discipline, patch sizes vary unpredictably between releases and campaign bandwidth planning becomes impossible.

Source Versions and Fallbacks

Every differencing approach shares a structural constraint: a patch is computed against a specific source. A device running version 4.1.0 cannot apply a patch computed from 4.0.0. A fleet that has drifted across many versions therefore needs a matrix of patches, a smaller set from designated base versions with intermediate hops, or a full-image fallback. The manifest must name the source the patch expects, and the device must verify the digest of that source as well as of the reconstructed image; otherwise a patch applied to the wrong base yields a corrupt image that passes no check until it fails to boot.

Delta updating adds a reconstruction step, scratch storage, a version matrix, and a new class of failure. It pays when bandwidth is scarce or expensive, images are large, and releases are frequent and incremental; it pays poorly for a small microcontroller image on unmetered Wi-Fi, or for releases that change a compiler version or a linker script and perturb the whole binary regardless. A prudent design keeps full images as the always-available path and offers delta only for particular source-to-target pairs.

Atomicity, Power-Fail Safety, and the Commit Handshake

Atomicity is what separates an update system from a way to brick devices at scale. It is achieved not by making the write fast but by arranging that the write never decides which image runs.

Deferring the Point of No Return

In a well-built system exactly one operation changes which image the device executes, and it is small enough to be atomic with respect to power loss: the write of a slot-selector field in an A/B design, or of a trailer record marking a swap complete. Everything before that point is preparation and can be discarded; everything after is committed. A design with no such point — where the bootloader scans both slots and picks the higher version number, say — smears the transition across the whole write, and power-fail behavior becomes a function of which sector was in flight.

Making that write atomic on real flash takes care, because flash pages do not update atomically in general. The standard techniques are to keep the selector within one programmable unit; to arrange the transition as a one-way bit change from erased to programmed, so a torn write leaves an unambiguous state; or to write two copies with sequence numbers and checksums and let the boot code pick the newer valid one. A dual-bank flash controller offering an atomic bank-swap bit should be used where available, since it moves the guarantee into silicon.

The Commit-and-Rollback Handshake

A device that boots is not a device that works, so the switch should be provisional. In the standard trial-boot pattern the bootloader marks the new image as testing, transfers control to it, and arms a mechanism that restores the old image unless the new one affirmatively confirms itself. Confirmation is an explicit act by the new firmware: after completing startup and passing its self-checks, it writes a confirmed marker. Until then the image is on probation, and if the device resets first the bootloader sees the unconfirmed marker on the next boot and reverts.

The mechanism that turns a hang into a reset is the watchdog. A hardware watchdog the new image must service is the only reliable escape from firmware that has stopped making progress without crashing, since a software timer in a hung system does not fire. It must be enabled before control passes to the trial image, and a defective image must not be able to disable it, which is why many microcontrollers offer an independent watchdog with its own oscillator and a write-once configuration. A boot-attempt counter, incremented before each trial boot and cleared on confirmation, catches what a watchdog alone misses: a reset loop in which every individual boot appears to start normally.

Health Checks Worth Trusting

The handshake rests on what the new image checks before confirming. A confirmation in the first line of main reduces the rollback mechanism to decoration; a confirmation requiring an unreasonable condition, such as a server round trip on a device that may be offline for a week, turns it into a reliability hazard, because the device will revert perfectly good firmware.

Good health checks are product specific and test what a firmware defect actually breaks: every expected peripheral answering its identity register, the configuration store parsed, the radio associated with a network, a sensor reading within a plausible range, the task set meeting its deadlines, and heap and stack usage inside budget. The check needs a time bound, because one that never completes is indistinguishable from a hang. Keeping the image provisional across several successful boots catches firmware that boots and then degrades. And in a multi-component device the check must consider the system rather than the part: an updated radio module that confirms itself while the application processor can no longer talk to it has confirmed the wrong thing.

Moving the Bytes: Transport Over Real Links

Transport is where physical reality asserts itself: the laboratory assumptions of stable bandwidth, a session lasting as long as the transfer, and an unmetered link hold for almost no deployed device.

Resumability Is a Requirement

A transfer that restarts from zero after an interruption will never complete on a link that drops every few minutes, and such links are common: a delivery vehicle passing through coverage gaps, a sensor whose radio degrades when a machine starts, a consumer device behind a router that renegotiates. Over HTTP the mechanism is the byte-range request, and the subtlety is that it must be tied to a specific artifact: if the server has published a new build at the same URL, resuming into a partial download of the old one yields a corrupt file the digest check rejects only after the full transfer completes. Immutable, content-addressed artifact URLs solve this; otherwise a strong validator such as an entity tag must be checked on resume.

On constrained links the pattern differs. Devices speaking the Constrained Application Protocol use its block-wise transfer extension to move a large payload as individually acknowledged blocks, which gives natural resumability and bounds the buffer the device must hold. The Lightweight M2M device-management standard from OMA SpecWorks builds a dedicated firmware-update object on that foundation, defining states a management server can drive uniformly across device types. On low-power wide-area networks, where a payload may be a few dozen bytes and a device may transmit a handful of times a day, full-image updating is often infeasible; such fleets rely on very small deltas, on multicast where the radio supports it, or on physical access.

A device with less RAM than the image also cannot verify a signature over the whole image before writing any of it. The standard answer is a manifest carrying per-block digests, itself covered by the signature: the device authenticates the small manifest once, then checks each block as it arrives and writes only verified blocks. That gives streaming verification at constant memory cost and detects corruption early.

Pull Against Push

Whether the device polls the server or the server contacts the device is consequential. Pull is the dominant and more robust pattern: the device opens an outbound connection on its own schedule, which traverses network address translation and firewalls without configuration, requires no inbound reachability, and lets the device apply its own policy about power, cost, and operational state. It degrades gracefully, since a device that cannot reach the server simply tries again. Its weakness is latency: a fleet polling every four hours has a mean detection delay of two hours, uncomfortable when the update fixes an actively exploited vulnerability.

Push shortens latency by holding a connection open — a long-lived MQTT session, a WebSocket, or a message-queue subscription — so the server can notify a device immediately. The costs are a persistent connection at both ends, keepalive traffic a battery-powered device may not afford, and server-side state per device. Most mature systems use a hybrid: a lightweight push channel carries a short notification and the device responds by performing an ordinary pull. The authoritative logic stays in the well-tested pull path, the operator gains a way to accelerate an urgent campaign, and a lost notification is not a failure because the periodic poll still finds the update.

Bandwidth and Metered Links

The device should know whether its connection is metered and should hold a policy — often user-visible in consumer products — about what it downloads over an expensive link; a telematics unit that pulls a hundred-megabyte map update while roaming creates a bill that outlives the feature. The server side needs the complementary control, because a fleet generates a coordinated demand spike no content-delivery arrangement absorbs for free: one hundred thousand devices each fetching a fifty-megabyte image at once is five terabytes of demand. The mitigations are jittered poll intervals, campaign eligibility spread across a window, and artifacts served from a content-delivery network. Devices that reboot together after a power event, or align their poll to a wall-clock hour, will otherwise hit the server together.

Campaign Orchestration

The server side is best understood not as a file server but as a control plane. Its unit of work is the campaign: a defined change, applied to a defined population, at a defined pace, with defined criteria for continuing or stopping. Eclipse hawkBit, an open-source rollout-management server, and the commercial device-management platforms organize themselves around approximately this model.

Targeting and Eligibility

A campaign begins by deciding which devices are eligible, and the answer is rarely all of them. Hardware revision matters, because a respin that moved a sensor to a different bus makes an image incompatible with earlier units. Current version matters, because a delta applies only to a specific source and upgrade paths may require intermediate hops. Region matters, because radio parameters and regulatory certifications differ. Device role matters when a site holds masters and subordinates that must not update in arbitrary order.

These predicates require data the server actually has, so the device must report a rich identity: hardware revision, current version of every component, regulatory profile, and relevant configuration. The manifest must also state its applicability independently, so that a device offered an image meant for another variant refuses it locally rather than trusting the server. Server-side targeting is an operational convenience; device-side applicability checking is a safety property.

Staged and Canary Rollouts

No amount of pre-release testing reproduces a fleet, with its hardware variance, undocumented configuration states, and unsimulated environments. The only reliable way to learn whether a release is good is to give it to a small number of devices and watch.

A canary stage exposes a deliberately small population — often a fraction of a percent — and holds there long enough to observe. How long depends on the failure mode being hunted. A crash on first boot appears within minutes; a memory leak that exhausts the heap appears after days; a defect triggered by a monthly maintenance cycle appears after a month. A canary held for one hour catches only the first class, which is worth remembering when a team reports that its canary passed. Later stages expand by rough order of magnitude with a soak interval between each, and should be representative rather than convenient: a canary drawn from one geography, revision, or customer tests one slice and gives false confidence about the rest.

Halting on a Regression Signal

A staged rollout is useful only if something stops it. Manual review at each gate works during business hours in one time zone and fails at three in the morning, which is when a rollout begun in the afternoon reaches its second stage. Mature systems define halt conditions numerically and enforce them automatically: update success rate below a threshold, rollback rate above one, crash reports rising, product telemetry outside its normal band, and — the strongest indicator that devices have gone dark — the check-in rate of updated devices falling relative to the rest. Compare against the population still running the old version at the same moment rather than a historical baseline, which is the only way to separate a bad release from a bad Tuesday.

The halt must be genuinely effective. Stopping the server from offering the update to new devices is necessary but not sufficient, because devices already sitting in Pending will still install it. A complete halt needs a revocation path that instructs devices holding a pending payload to discard it and, where the product allows, an explicit downgrade campaign. Downgrade interacts awkwardly with rollback protection, and that interaction should be designed before it is needed rather than during an incident.

Rate Limiting and Dependency Ordering

Rate limiting protects two different things: the back end from the demand spike described earlier, and the fleet from itself. The second is easy to overlook. If every device in a substation, a factory cell, or a building reboots to install within the same minute, the simultaneous loss of function is an outage even though every individual update succeeded. Limits aware of topology — one device per site, per redundant pair, or per control loop at a time — are the correct form, and differ sharply from a global limit in devices per hour.

Many products are not one processor. A vehicle holds dozens of electronic control units; an industrial gateway may hold an application processor, a safety microcontroller, a radio module with its own firmware, and programmable logic. When a change spans components, the campaign must express order and coupling. In increasing order of cost: make the change backward compatible so that order does not matter; update in a defined order within one maintenance window, accepting a brief incompatible interval the device tolerates; or define an atomic multi-component transaction in which the device stages every payload, verifies all of them, applies them together, and reverts the set if any part fails. A dependency expression in the manifest lets a device refuse a partial set rather than discover the incompatibility after the reboot.

Trust: Manifests and Compromise-Resilient Frameworks

Security is the heart of an update system, because the update path is by definition a mechanism for making a device run new code, and whoever controls it controls the fleet. The primitives and key-management practices belong to Firmware Update Security; what follows is the system-level question of what is signed, by whom, and what survives a compromise of part of the infrastructure.

What the Manifest Carries

Signing the image alone is insufficient, because an authentic image is not necessarily the right image. An attacker who cannot forge a signature can still replay an old authentic image, deliver one meant for a different hardware variant, or supply a single component of a matched set. The signed object must therefore be a manifest binding the image to its context: the identity and digest of each payload, the device class, the version and the minimum version the device must already hold, dependencies, the expected source image where the payload is a patch, and an expiry indicator. Verification is anchored in hardware — the public key or its digest in read-only memory, fuses, or a root-of-trust block — so that an attacker with write access to flash cannot substitute a key.

SUIT for Constrained Devices

The IETF's Software Updates for Internet of Things working group has standardized this problem for devices too small to run general-purpose update software. RFC 9019, published in April 2021, describes a firmware update architecture for such devices and defines the actors — author, device operator, network operator, and status tracker — and the flows between them. RFC 9124, published in January 2022, specifies the manifest information model: the elements a manifest must express, and the specific threat each counters.

The concrete serialization, a CBOR-based manifest format developed as draft-ietf-suit-manifest, is in the RFC Editor queue rather than published at the time of writing. CBOR encodes compactly and parses with very little code, which matters when the parser must fit in a bootloader. A distinctive decision is that the SUIT manifest is not merely declarative data but a short sequence of commands the device interprets — set a parameter, fetch a component, check a condition, write — so one format expresses fetch-then-install, install-from-local-storage, delta application, and multi-component sequencing.

TUF and the Roles That Survive a Compromise

The Update Framework, a graduated project of the Cloud Native Computing Foundation, addresses what image signing alone does not: a compromise of the update infrastructure itself. It splits authority among roles with separate keys, so that no single compromised key or server suffices to push a malicious image.

Four roles carry the design. The root role distributes and rotates the public keys of the others and anchors trust; its keys stay offline. The targets role signs metadata about the images themselves, including digests and sizes. The snapshot role signs a statement of which metadata files were current together, preventing an attacker from serving a valid but mismatched combination of old and new files. The timestamp role signs a short-lived statement that the client is seeing current metadata, which defeats a freeze attack. Two properties matter operationally: threshold signing requires several keys to agree, so stealing one accomplishes nothing, and key rotation happens inside the framework, since a new root file signed by the old keys tells clients about replacements without touching every device.

Uptane and the Director-Image Repository Split

Uptane adapts TUF to vehicles, where the fleet is heterogeneous, the components are numerous and individually weak, and the consequences of a malicious update are physical. Its defining choice is two repositories with different security properties. The image repository holds images and their signed metadata; it is controlled by human actors, updated infrequently, and its keys can stay offline, because it never responds to an individual vehicle. The director repository is connected to an inventory database and produces signed metadata on demand, telling a specific vehicle's control units which images to install; it must be online and sign continuously, which makes it the more exposed.

The security argument follows from the split. The director knows which vehicle should receive what but cannot introduce an image the image repository has not signed, while the image repository authorizes images but does not decide which vehicle installs them. A unit performing full verification checks both, so compromising the online director at most misdirects a vehicle among already-authorized images — a denial-of-service or mismatched-configuration attack, serious but far short of arbitrary code execution. Uptane also defines partial verification for secondaries with very little memory: such a unit checks only the director's targets metadata and relies on a more capable primary in the vehicle to verify fully on its behalf. The Uptane Standard for Design and Implementation is published openly, with version 2.1.0 current at the time of writing.

Rollback Protection and the Anti-Freeze Problem

Two attacks target the update system by manipulating version and time rather than content, and each needs an explicit countermeasure.

Rollback

A downgrade attack installs an older, authentically signed image containing a vulnerability the manufacturer has since fixed, and signature verification does not stop it. The countermeasure is a monotonic security version counter: each image carries a version that increases when a security-relevant fix lands, the device records the highest version it has ever run, and the bootloader refuses anything lower. The counter must live where the running application cannot rewrite it — fuses, a counter in a secure element, or a region protected by a memory protection unit — because a counter an exploited application can decrement protects nothing. Fuses offer a small, finite number of increments, so the security version usually advances only when a release closes a vulnerability that must not become reachable again.

Rollback protection carries a cost that appears at the worst moment: it makes a deliberate downgrade impossible, which is a problem when a bad release must be pulled. Either recovery means rolling forward to a fixed release, or the design permits a narrow, separately authorized downgrade path. Rolling forward is the safer default, and it implies a commitment to building, signing, and shipping a fix quickly.

Freeze

The mirror-image attack delivers nothing malicious. It prevents the device from learning that an update exists, by replaying the last known-good view of the repository indefinitely; the device believes it is current and the vulnerability stays open. This is cheap for anyone on the network path and leaves no artifact. The countermeasure is metadata expiry: signed metadata carries a short validity window — in TUF and Uptane the timestamp role's job — and a device that sees only expired metadata knows it is not being told the truth, whether from attack or infrastructure failure. Either way the correct response is to raise an alarm rather than proceed quietly.

Expiry requires the device to know the time, which is difficult without a real-time clock or a battery. A device unpowered for a year cannot distinguish current metadata from a year-old replay by its own clock. The answers are partial: take time from a signed source rather than an unauthenticated network time exchange; keep a monotonically advancing highest-time-seen value in persistent storage so regression is detectable; treat the running firmware's build timestamp as a lower bound; and, for vehicles, use the time attestation Uptane defines. The tension has no clean resolution, since a window short enough to blunt a freeze attack also expires on a device that wakes quarterly.

Fleet Realities

A dashboard showing ninety-eight percent success is a satisfying artifact. Understanding the other two percent, and what the ninety-eight actually measures, is where most of the operational work lives.

Partial Connectivity and Devices That Wake Rarely

Fleets are not uniformly online. Agricultural equipment sits in a shed for the winter, a consumer product sits in a drawer for a year, and a battery-powered sensor wakes for four seconds an hour and spends most of that budget sensing. For such populations a rollout that completes in a week is a meaningless notion, and a campaign must stay valid for months.

Long campaign lifetimes have consequences that are easy to miss. Signing keys and metadata must remain valid, or the device that finally wakes will reject an update authorized while it slept. Delta patches must still match the source versions those devices hold, which argues for retaining full-image fallbacks indefinitely. Content-delivery retention must match the campaign horizon rather than a default. And halt criteria must survive time: a rollout stopped for a defect must not silently resume when a long-sleeping device checks in. For the most constrained devices the update must also be planned around the energy budget, since a device with a primary cell sized for a ten-year life may spend a meaningful share of it on one full-image download.

The Long Tail

Every fleet has a tail of units that do not take an update. Some are decommissioned but never deregistered, a category usually larger than operators expect, which quietly depresses every success metric. Some sit where there is no usable connectivity. Some have hardware faults, including flash worn past its endurance. Some belong to users who disabled updates. Some run software so old that the current path no longer reaches it, which happens when an intermediate release changed a manifest format or a key.

The right response is to classify rather than to chase. Separating never seen since manufacture, last seen two years ago, checks in but never downloads, downloads but never installs, and installs but reverts turns an undifferentiated tail into five problems with five owners. The last two indicate a defect in the update system itself. An explicit target — a stated fraction of the reachable fleet within a stated period — is more useful than pursuing a total that includes devices in landfills.

Telemetry, and Why Installed Is Not Working

The update system must report on itself, and the reporting must survive the failures it describes: a device that fails an update has to say so from software that still runs, meaning the bootloader or the reverted old image. Reports need enough structure to aggregate — a specific failure reason rather than a generic error, the version pair, the hardware revision, and the stage at which the failure occurred.

The most useful idea in fleet operations is the distinction between an update that installed and an update that works. A device can complete an update, boot, confirm, and report success while the product it is embedded in has stopped doing its job. The firmware runs; the pump does not dose correctly. The gateway boots; the meter readings it forwards are now scaled wrong. Catching this requires product-level metrics in the halt criteria — successful transactions per hour for a payment terminal, doses delivered for an infusion pump, kilowatt-hours reported for a meter. Identify, before the campaign, the two or three numbers that would move if the release were bad in a way the device cannot itself detect, and compare them between the updated and not-yet-updated populations. A release deployed to the whole fleet at once destroys the control group, and with it the ability to answer the question.

Regulation: Updating Is Increasingly a Duty

For most of the history of embedded products, field updating was optional and largely unregulated. That has changed in several sectors, and the change affects design rather than merely paperwork.

Vehicles and UN Regulation No. 156

UN Regulation No. 156, adopted under the 1958 Agreement administered by the United Nations Economic Commission for Europe and in force since 22 January 2021, sets uniform provisions for approving vehicles with regard to software updates and the management system that governs them. It regulates the manufacturer's process, not only the product: to obtain type approval a manufacturer must hold a certificate of compliance for a Software Update Management System, and that certificate is time limited and subject to periodic reassessment rather than granted once.

The substantive expectations map closely onto the practices described above. A manufacturer must identify the software on each vehicle and its configuration, assess whether an update affects type-approved systems, verify compatibility with the target configuration beforehand, protect the process against manipulation, record the update and its outcome, and inform users appropriately. Where an update changes a type-approved parameter, the approval must be revised before deployment. The requirement that a vehicle be in a safe state to receive an update is the regulatory form of the scheduling preconditions discussed earlier.

The EU Cyber Resilience Act

Regulation (EU) 2024/2847, the Cyber Resilience Act, entered into force on 10 December 2024 and applies to products with digital elements placed on the European Union market. Its main obligations take effect on 11 December 2027, with vulnerability and incident reporting obligations applying earlier, from 11 September 2026.

Its significance is that it converts security updating from a commercial choice into a legal duty with a defined horizon. Manufacturers must handle vulnerabilities across the product lifecycle and must define a support period reflecting the product's expected use time; that period must be at least five years unless the product is genuinely expected to be in service for less, and longer-lived products attract correspondingly longer periods. Security updates issued during the support period must remain available afterward for at least ten years, or for the remainder of the support period if that is longer.

These durations have direct engineering consequences. A device sold in 2028 with a fifteen-year service life implies a signing infrastructure, a build environment, a toolchain, and an update service that must all still function in the 2040s. That argues for reproducible builds and archived toolchains, key management with a documented succession plan, update protocols independent of any one vendor's cloud service, and cryptographic agility.

Medical Device Change Control

Medical devices sit under a change-control regime that predates cybersecurity concerns and must now accommodate them. A change to a marketed device must be assessed for its effect on safety and effectiveness, documented under the quality system, and, if it crosses a defined threshold, submitted to the regulator before distribution. Software changes are explicitly in scope, and a manufacturer must justify why a given change did or did not require a new submission.

The tension with security updating is obvious, since vulnerabilities do not wait for a review cycle. In the United States, section 524B of the Federal Food, Drug, and Cosmetic Act addresses this for cyber devices — those that include software and can connect to the internet. A sponsor must submit a plan to monitor, identify, and address postmarket cybersecurity vulnerabilities and exploits, including coordinated vulnerability disclosure, and must maintain processes that make postmarket updates and patches available: on a reasonably justified regular cycle for known unacceptable vulnerabilities, and as soon as possible out of cycle for critical vulnerabilities that could cause uncontrolled risk. A software bill of materials is required as well, since exposure to a third-party vulnerability cannot be assessed without knowing what the device contains.

For the update system this makes traceability a design requirement rather than an operational nicety: it must answer, on demand and with evidence, which version each device holds, when each change was applied, which change-control record authorized it, and what the outcome was. The rollout controls described earlier also acquire regulatory weight, since a staged rollout with defined halt criteria is a documentable risk-control measure and an uncontrolled push to an entire fleet is difficult to defend.

Conclusion

Over-the-air updating begins as a feature and becomes, across a product's life, one of its load-bearing structures. The device-side work — an explicit state machine, an image layout that makes the switch atomic, a commit-and-rollback handshake enforced by a watchdog, and health checks that test something real — buys the guarantee that a device is never lost. The transport work buys reach across links that fail. The campaign machinery buys steerability: the ability to target, to pace, to observe, and above all to stop. The trust model, built on signed manifests and frameworks such as SUIT, TUF, and Uptane, buys assurance that a stolen key or a compromised server is not enough to take the fleet.

What ties these together is that the interesting failures are not local. A flash write either works or does not, and the bench can tell. A rollout either improves the fleet or degrades it, and only the fleet can tell — measured in stages, against a control population, with a mechanism ready to halt. Two habits follow. Test the failure paths rather than the success path, using rigs that cut power at pseudorandom points during an update, because that is where the residual atomicity defects hide. And make rolling forward fast, because rollback protection and pending installs both limit the ability to undo. Regulation has made the discipline explicit in several sectors, but the engineering case stood on its own well before the legal one arrived. A team that designs for the bad release handles the good ones without noticing; a team that designs only for the good release remembers the bad one for years.

Related Topics