Electronics Guide

Boot ROM and the First-Stage Loader

Execution begins in mask ROM inside the system-on-chip, programmed at the foundry and unmodifiable. The boot ROM samples strapping pins or one-time-programmable fuses to determine the boot device, then reads a small image from a fixed offset of it: a defined sector of eMMC or an SD card, the start of SPI-NOR flash, or the first good block of raw NAND. Most boot ROMs also implement a recovery path over USB or a serial port, which is how a board with blank flash is brought up on the bench and how a bricked unit is revived on the production line.

The critical constraint is that DRAM does not work yet. The memory controller has not been configured and the DDR interface has not been trained, so the first image must execute entirely from on-chip SRAM, typically between sixty-four kilobytes and a few hundred kilobytes. A full bootloader does not fit. U-Boot answers with the secondary program loader, universally abbreviated SPL: a cut-down build that configures the phase-locked loops, initializes the memory controller, runs the vendor's DDR calibration routine where one is required, then copies the full bootloader into DRAM and jumps to it.

On 64-bit Arm platforms the chain also includes Arm Trusted Firmware-A. Its BL31 component installs itself permanently at the highest exception level as the secure monitor, providing the Power State Coordination Interface that Linux later calls to bring secondary cores online, and U-Boot runs after it as the non-secure BL33 payload. A board that hangs before any console output appears is usually failing in the SPL or a Trusted Firmware stage.

U-Boot Proper

U-Boot, formally Das U-Boot, is the de facto bootloader for embedded Linux on Arm, RISC-V, MIPS, and PowerPC. It is distributed under the GNU General Public License version 2 or later and releases roughly quarterly using calendar version numbers, a convention adopted in 2008, so that a release such as v2026.01 names its own date. General bootloader design is covered in Bootloader Development.

Running from DRAM with a working console, U-Boot provides a command interpreter, storage and network drivers, and a scripting mechanism. Its task is to locate a kernel image, a device tree blob, and optionally an initramfs; load them to the correct physical addresses; assemble a kernel command line; and transfer control. The flattened image tree, or FIT, is the container format worth adopting from the outset: one image holds the kernel, device tree blobs, and initramfs, each hashed, with named configurations selecting which combination to boot. Because the whole structure can be signed, FIT is also U-Boot's mechanism for verified boot.

The Environment and the Handover

U-Boot keeps its configuration in an environment of name-value pairs stored in a raw sector of eMMC or SPI-NOR, a UBI volume, or a file. It holds the boot command, the kernel arguments, and, in most update schemes, the variables recording which system slot is active and how many times it has been tried. Because those variables are written by the running system during an update, the environment must be redundant, with two copies and a validity marker, so that a power failure part-way through a save leaves a readable copy behind. Devices bricked by a non-redundant environment are a recurring and entirely avoidable field failure.

The handover follows a documented per-architecture protocol. On 64-bit Arm, U-Boot places the kernel image and the device tree blob in memory, sets register x0 to the blob's physical address, and branches to the kernel entry point with caches and interrupts in a defined state. The kernel command line arrives through the device tree's chosen node. From that instant the bootloader is gone, and everything the kernel knows about the hardware it learns from the blob it was handed.

BusyBox and the Userland

A conventional Unix userland is several hundred separate programs. BusyBox replaces most of them with one multi-call binary that inspects the name by which it was invoked and dispatches to the corresponding applet, reached through a symbolic link per command. Because the applets share code and are compiled for size, a complete shell environment, an init, file and text utilities, and network clients fit in a few hundred kilobytes. BusyBox is licensed under the GNU General Public License version 2 only; Toybox, under permissive 0BSD terms, occupies the same niche.

The C Library Decision

Every program links against a C library, and the choice is one of the few decisions genuinely difficult to revisit, because it fixes the application binary interface of the entire system and is baked into the cross toolchain, whose construction is treated in Cross-Compilation Toolchains.

The GNU C Library, glibc, is the reference implementation and the compatibility baseline. Licensed under the GNU Lesser General Public License version 2.1 or later, it implements the full range of POSIX and GNU extensions and is what essentially all proprietary binaries expect. It is also the largest option by a wide margin. The musl C library, under permissive MIT terms, was written for size, static linking, and strict standards conformance; its cost is compatibility, since glibc extensions may need patching, the name service switch is absent, and proprietary glibc binaries will not run. uClibc-ng persists mainly on processors without a memory management unit. A system running third-party binaries takes glibc; a size-constrained source-built system takes musl.

Init and Device Management

BusyBox init reading an inittab is a few kilobytes and starts a fixed list of programs in order, which suffices for a device with three daemons. Systemd, at the other end, provides dependency-based parallel startup, socket and device activation, service supervision with restart policies, control-group resource limits, watchdog handling, and structured logging, at the cost of a much larger footprint. Supervisors such as runit and s6 sit between. The choice matters because supervision is a reliability feature: a device whose application exits must restart it and escalate to a reboot if that does not help.

Device nodes in /dev are created by devtmpfs and then managed by a userspace helper applying naming rules, permissions, and module autoloading: BusyBox mdev, eudev, or systemd-udevd. An initial RAM file system, a cpio archive embedded in the kernel image or loaded by the bootloader, runs before the real root is mounted and is where integrity verification and encrypted volume setup happen.

Raw Flash: MTD, UBI, and UBIFS

Raw NAND and NOR flash present the bare array to software. Linux exposes them through the memory technology device layer, which handles the erase-program interface and error correction and leaves bad-block management, wear leveling, and power-fail robustness to the layers above. The modern arrangement puts UBI, the unsorted block images layer, directly on the MTD device; UBI presents logical erase blocks whose mapping to physical blocks it manages itself, wear-leveling across the volume set and retiring bad blocks transparently. UBIFS above it supplies write-back caching, compression, and tolerance of unexpected power loss. It was merged into the mainline kernel in Linux 2.6.27, released in October 2008.

Its predecessor JFFS2 remains appropriate on small NOR partitions but scans the entire device at mount, so mount duration grows with capacity and becomes untenable past a few tens of megabytes. UBI has an analogous startup cost, since attaching a volume requires reading erase-block headers, and the fastmap feature exists to shorten it. On a device with a boot-time requirement and a large NAND part, attach time is a line item engineers routinely forget to measure.

Managed Flash: eMMC and UFS

Managed flash puts a controller in the same package as the array. It runs a flash translation layer that performs wear leveling, error correction, and bad-block retirement internally, and presents an ordinary block device to the host. This removes an entire category of software work and is why eMMC dominates designs above a few hundred megabytes, with UFS appearing where bandwidth demands it.

The convenience comes with opacity. The translation layer is vendor firmware, its behavior on power loss is documented only in general terms, and its wear-leveling policy cannot be inspected. Two mitigations are standard: prefer eMMC to removable SD cards for anything the product depends on, since consumer cards vary enormously in controller quality; and run power-cut testing on the production part over thousands of interrupted writes. The standard also supplies hardware-selectable boot partitions and the replay-protected memory block, suited to monotonic counters for anti-rollback protection.

Matching File Systems to Roles

On a block device the choice is driven by whether a partition is written and by how it fails. The ext4 file system is the mature general-purpose choice for writable partitions, and is worth mounting with noatime on flash to eliminate a class of gratuitous writes. F2FS is a log-structured file system designed for storage sitting on a flash translation layer, contributed by Samsung and merged in Linux 3.8 in 2013; by writing sequentially it cooperates with rather than fights the translation layer, and it is widely deployed on Android user data partitions.

SquashFS is a compressed read-only file system long used for system images, storing an entire root file system as one compressed, immutable blob. EROFS is the newer design for the same role. It first appeared in Linux 4.19 and moved out of the staging area in Linux 5.4, and supports LZ4, MicroLZMA, DEFLATE, and Zstandard compression selectable per inode. Its distinguishing choice is fixed-size output compression, producing uniform compressed blocks from variable-sized input, which improves random-read performance relative to schemes that compress fixed-size inputs. The kernel documentation frames its purpose as immutable golden images built once and deployed unchanged.

The Read-Only Root with an Overlay

Standard practice for reliable devices is a read-only root with writable layers stacked on top. The system image is a SquashFS or EROFS volume mounted read-only. Directories that must be writable are provided by overlayfs, using that image as the lower layer and either a tmpfs, for state that need not survive a reboot, or a small ext4 or F2FS partition for state that must. Application and user data live on a separate writable partition.

Four benefits follow. The system image cannot be corrupted by a power failure, because nothing writes to it. It can be verified cryptographically, because it is immutable. It can be replaced atomically during an update, because it is a single blob. And a factory reset becomes an erase of the writable layers. The cost is discipline: every program that expects to write somewhere unexpected must be found during development, which is why the read-only root belongs in the first week of a project and not the last.

Endurance planning belongs here too. The usual cause of premature flash wear in the field is logging, and a daemon writing verbosely can consume the endurance budget of a small partition in months. Keep volatile logs in RAM, rotate what is persisted, and calculate expected write volume per day against the endurance rating of the part before the design is frozen.

Yocto and OpenEmbedded

The Yocto Project supplies the release process, testing infrastructure, and reference distribution around OpenEmbedded, whose core metadata and BitBake task executor do the work. The unit of metadata is the recipe: a file describing where to fetch a component's source, which patches to apply, and how to configure, compile, install, and package it. Recipes are grouped into layers, which stack in a defined priority order, and a layer can add recipes, override variables, or extend an existing recipe through an append file without modifying it.

Two capabilities justify the complexity. Yocto builds packages, in RPM, Debian, or ipk format, and assembles images from them, so a product can ship a package feed and build several image variants from one set of recipes. And because every component enters through a recipe declaring its license and source, the build produces license manifests and can generate SPDX bills of materials as a normal output. Yocto releases twice a year and designates the April release as long-term support: Scarthgap 5.0 of April 2024 carries support to April 2028, and Wrynose 6.0 of April 2026 to April 2030.

The costs are real. The learning curve is steep, and the metadata language has variable-override and task-dependency behaviors that reward study. A first build compiles everything from source, occupying tens of gigabytes of disk and hours of processor time, though the shared-state cache makes later builds incremental and can be shared across a build farm.

Buildroot

Buildroot takes the opposite position. It is a set of makefiles and Kconfig files producing a cross toolchain, kernel, bootloader, and root file system image, configured through the same menuconfig interface the kernel uses. It releases quarterly with calendar version numbers such as 2026.05 and designates one series each year for long-term support.

Its virtues are directness and speed of comprehension. The configuration is one file a person can read, adding a package means a short makefile fragment and a Config.in entry, and a newcomer can produce a booting image on a supported board in an afternoon. The resulting root file system contains only what was selected, with no package manager on the target. For a single product with a fixed image and a small team this is frequently the correct engineering choice.

Its limitations follow from the same design. Because Buildroot does not track per-package dependencies with the granularity a package manager requires, the project's own guidance is that a configuration change may require a full rebuild. There is no package feed, so field updates are whole-image updates. Supporting several products with divergent configurations means maintaining several configurations, where Yocto's layers would express the shared parts once. Compliance is served by the legal-info target, whose output is less structured than Yocto's manifests and SPDX documents.

Choosing Between Them

The honest comparison rests on three axes. On learning curve Buildroot wins decisively; a team can be productive in days rather than weeks. On scaling to a product family Yocto wins decisively, because layers express variation without duplication and package-based images allow per-component updates. On compliance output Yocto is ahead by a clear margin, which matters as customers and regulators increasingly ask for a software bill of materials as a condition of sale. Both build from pinned sources with recorded hashes, though the Yocto Project has invested more in bit-for-bit reproducibility. Other options occasionally fit better: debootstrap for a Debian userland, OpenWrt for network devices, and Android where its application ecosystem is required.

Preemption Models and PREEMPT_RT

The mainline kernel offers a range of preemption models chosen at configuration time, from no involuntary preemption in kernel mode, appropriate for throughput-oriented servers, through voluntary preemption points, to full kernel preemption for low-latency desktop and audio work. Recent kernels can also select among several of these at boot.

PREEMPT_RT is the model beyond those. It converts most kernel spinlocks into sleeping mutexes with priority inheritance, moves nearly all interrupt handlers into schedulable kernel threads, and makes software interrupt processing preemptible. Far less kernel code then runs with preemption disabled, so a high-priority task waiting on an event is delayed by a shorter and far more predictable interval. Maintained out of tree for roughly two decades, it was merged into the mainline kernel in Linux 6.12, released on November 17, 2024; the final enabling piece was a rewrite of the printk subsystem. Real-time capability is therefore now a configuration option in a stock kernel rather than a patch set to forward-port.

Scheduling and Measurement

Linux implements the POSIX real-time policies SCHED_FIFO and SCHED_RR across priorities 1 through 99, which run ahead of all normal tasks, and SCHED_DEADLINE, an earliest-deadline-first policy in which a task declares a runtime, a period, and a deadline and the kernel admits it only if the set remains feasible. One default surprises newcomers: to prevent a runaway real-time task from locking the system out, the kernel throttles the real-time classes to a fraction of each period, conventionally ninety-five percent. A thread that busy-waits is therefore interrupted at a regular interval by what looks like an inexplicable gap.

The standard instrument is cyclictest, from the rt-tests suite. It wakes threads on a fixed period and records how late each wake actually was; the maximum over a long run is the figure quoted as scheduling latency. A latency figure without its test conditions is meaningless. The maximum depends on the processor, the interrupt load, power management settings, and the run length, since the interesting number is a rare outlier. A responsible measurement states the hardware, the kernel configuration, the tuning, the load, and the duration. Two companions matter: hwlatdetect finds latency caused by firmware beneath the operating system, and the osnoise and timerlat tracers attribute latency to its source.

Stated carefully, a well-tuned PREEMPT_RT system on suitable hardware achieves worst-case scheduling latencies in the range of tens of microseconds, where a general-purpose configuration on the same hardware shows outliers one or two orders of magnitude larger. Tuning is not optional to reach that range. It normally includes isolating processor cores from the general scheduler, directing interrupts away from those cores, disabling frequency scaling and deep idle states, locking the application's memory with mlockall and pre-faulting its stacks, and eliminating dynamic allocation, page faults, and unbounded blocking from the critical path. The throughput cost is real and should be budgeted.

What Linux Still Cannot Do

Two limits remain after all tuning. The first is analytical: there is no worst-case execution time analysis of the Linux kernel, and there will not be one. A measured maximum over a long run is strong evidence, not a bound, and the difference matters when a deadline miss is a safety event. The second is evidentiary: the certification packages that safety standards expect are far harder to assemble for a codebase of Linux's size and rate of change.

Where those limits bite, the architecture changes rather than the tuning. The common answer is asymmetric multiprocessing on one chip: an application cluster running Linux alongside a real-time core, typically a Cortex-R or Cortex-M class processor, running an RTOS or bare-metal firmware on its own memory. Linux loads and starts the companion firmware through the remoteproc framework and exchanges messages over shared memory and hardware mailboxes using RPMsg, with the OpenAMP project supplying a common implementation on both sides. The deterministic work runs where determinism is achievable, and Linux handles connectivity, storage, and the user interface.

The stronger version is a separate microcontroller in its own package, connected over SPI, UART, or CAN. It costs a part and a second firmware image and buys complete isolation: the controller keeps running while Linux reboots, and no shared cache or interconnect couples the two. For certified safety functions this separation is frequently the only architecture a certification body accepts without argument. A third approach, the co-kernel design exemplified by Xenomai, runs a small real-time core beneath Linux and treats Linux as its lowest-priority task, at the cost of a separate patch set and a distinct programming interface.

A/B Partitions

The dominant scheme keeps two complete copies of the system, conventionally slots A and B. The device runs from one slot while an update is written to the other, so the running system is never modified. When the write completes and its integrity is verified, the bootloader is told to try the inactive slot at the next boot and a boot-attempt counter is set. If the new system reaches a healthy state the application confirms it and the slot is marked good; if it does not, the counter runs out and the bootloader falls back to the previous slot, still intact.

The cost is two copies of the system partition. The benefit is that no single failure, including a power loss at any instant or a new image that boots but does not work, leaves an unbootable device. Where a technician visit costs more than the device, this is not a luxury. Cheaper variants exist: a single slot with a minimal recovery image saves storage but leaves a real window during which a power failure strands the device in recovery, and delta updates cut bandwidth by shipping a binary difference. Android's virtual A/B uses copy-on-write snapshots to obtain the safety of two slots without the full storage cost.

Mechanism and Frameworks

The bootloader holds the state machine. In a U-Boot system the slot selection, the boot-attempt counter, and the update-pending flag are environment variables that the running system writes and the bootloader reads and decrements, which is what makes a redundant environment mandatory. Mature open-source frameworks implement the full flow, including SWUpdate, RAUC, and Mender; OSTree offers a different model in which the file system is versioned like a repository. Two points close the topic. Updating the bootloader is the riskiest operation a device performs, which is partly why hardware-switchable eMMC boot partitions exist. And the update path must be exercised throughout development, from the oldest shipped version forward.

Source Obligations

The kernel is licensed under version 2 of the GNU General Public License, BusyBox under the same license, U-Boot under version 2 or later, glibc under the Lesser General Public License version 2.1 or later, and musl under MIT terms. Distributing a device containing copyleft software is distribution of that software, and the corresponding source must be made available. Version 2 offers three routes, and the second is the one embedded products almost always take: accompanying the product with a written offer, valid for at least three years, to supply the complete corresponding source to any third party for no more than the cost of distribution.

That three-year term is a term of the license itself, and its consequence is underestimated. The obligation attaches to each shipped version, so the source archive for every release must be preserved and retrievable for three years after the last unit carrying it shipped, and it must be the exact source, including patches and build scripts sufficient to reproduce the binary. Reconstructing that archive years later from an undocumented build environment is the failure mode, and archiving at release time prevents it. The Lesser General Public License adds a requirement for glibc: the user must be able to relink against a modified library, which dynamic linking satisfies.

Compliance is best treated as a build output rather than a document. Yocto produces per-image license manifests and can generate SPDX bills of materials; Buildroot's legal-info target produces a license report and collects package source. Scanning tools such as FOSSology and ScanCode verify what the metadata claims, and the OpenChain specification, published as ISO/IEC 5230, describes a process standard for an auditable compliance program. Enforcement is not hypothetical: the BusyBox litigation of the late 2000s established that these obligations are pursued.

Long-Term Support Kernels

The kernel community designates certain releases as long-term support, maintained with backported fixes long after mainline has moved on. As of this writing the maintained longterm branches are 6.18, released November 30, 2025, and 6.12, released November 17, 2024, both projected to December 2028; 6.6, released October 29, 2023, and 6.1, released December 11, 2022, both projected to December 2027; and 5.15 and 5.10, released in October 2021 and December 2020, both projected to December 2026. The projections are explicitly not guarantees and have been extended before.

Read that against a typical industrial product lifetime of ten to fifteen years and the problem is immediate. A device shipping today on a vendor board support package built from a 5.10 kernel runs a base whose upstream maintenance ends within months, after which the manufacturer inherits the whole burden. Vulnerability reporting sharpens the point: since becoming a CVE numbering authority in early 2024, the kernel community has assigned identifiers at a volume that makes per-vulnerability triage impractical for most organizations. Tracking the stable branch of a maintained LTS kernel is therefore both more defensible and cheaper than cherry-picking fixes into a frozen fork.

Three practices follow. Choose silicon whose mainline support lets the product track an upstream LTS kernel rather than a vendor fork. Schedule and budget a kernel migration during the product's life, since a planned migration costs a fraction of an emergency one. And treat the security-update commitment as a specification with a stated duration: the European Union's Cyber Resilience Act, which entered into force in December 2024 with obligations phasing in over the following years, makes support for security updates across a product's expected lifetime a legal requirement in that market.