Electronics Guide

Central Processing Units (CPUs)

CPUs serve as the general-purpose workhorses in heterogeneous systems, handling control flow, operating system functions, and irregular computational tasks. Modern embedded CPUs, typically based on Arm Cortex-A architectures or RISC-V designs, provide sophisticated out-of-order execution, branch prediction, and cache hierarchies that excel at sequential code with complex control dependencies. In heterogeneous systems, CPUs orchestrate the overall computation, manage task scheduling across accelerators, and handle workloads that are difficult to parallelize or accelerate.

Arm's big.LITTLE architecture pioneered heterogeneous CPU clusters, combining high-performance cores with energy-efficient cores on the same chip. This approach allows the system to use powerful cores for demanding tasks while shifting to efficient cores for lighter workloads, optimizing both performance and battery life. Arm's DynamIQ technology generalized the scheme by placing cores of different types in a single cluster behind a shared coherent interconnect, which enabled the three-tier arrangements now common in mobile processors: one or two very wide cores for peak single-thread demand, several mid-tier cores for sustained throughput, and a group of small cores for background work. Because every core in such a cluster implements the same instruction set, the operating system scheduler can migrate a thread between tiers without recompilation.

Graphics Processing Units (GPUs)

GPUs provide massive parallel processing capability through arrays of simpler processing elements designed for throughput-oriented workloads. Originally developed for graphics rendering, GPUs excel at any task involving parallel operations on large data sets, including image processing, scientific computing, and increasingly, machine learning inference. Mobile GPUs from vendors such as Arm (Mali and the higher-tier Immortalis line), Qualcomm (Adreno), Imagination (PowerVR), and Apple have evolved to support general-purpose computing through APIs such as OpenCL, Vulkan compute shaders, and Metal.

In embedded heterogeneous systems, GPUs handle compute-intensive tasks that can be expressed as parallel operations on arrays of data. Their single-instruction, multiple-thread (SIMT) architecture achieves high throughput by executing the same operation across many data elements simultaneously. However, GPUs are less efficient for tasks with divergent control flow or irregular memory access patterns, making them complementary rather than universal replacements for CPUs.

Digital Signal Processors (DSPs)

DSPs are specialized processors optimized for the repetitive mathematical operations common in signal processing applications. Features like single-cycle multiply-accumulate operations, circular buffers, and specialized addressing modes make DSPs highly efficient for filtering, transforms, and other signal processing algorithms. In heterogeneous systems, DSPs typically handle audio processing, sensor data conditioning, and communication signal processing where their architectural optimizations provide significant efficiency advantages over general-purpose processors.

Modern DSPs have evolved beyond pure signal processing to support broader computational workloads. Vendors like Texas Instruments, Qualcomm, and Cadence offer DSP architectures with vector processing capabilities, making them suitable for computer vision and machine learning tasks in addition to traditional signal processing. The Qualcomm Hexagon DSP, for example, combines scalar units with wide vector extensions (HVX) and, in recent generations, tensor units, so that a single block covers audio, sensor fusion, and neural network work. Cadence's Tensilica families follow a similar path, offering configurable cores whose instruction sets can be extended for a specific algorithm.

Field-Programmable Gate Arrays (FPGAs)

FPGAs provide reconfigurable hardware that can be customized to implement any digital logic function. In heterogeneous systems, FPGAs serve as flexible accelerators that can be programmed to implement custom data paths optimized for specific algorithms. This customization enables FPGAs to achieve efficiency approaching that of dedicated hardware while maintaining the flexibility to adapt to changing requirements or support multiple algorithms.

Embedded FPGAs excel at tasks requiring custom interfaces, precise timing control, or algorithms that do not map efficiently to fixed processor architectures. Applications include protocol processing for custom or legacy interfaces, real-time control systems with deterministic latency requirements, and pre-processing stages that prepare data for other accelerators. Devices that place hardened processor subsystems beside programmable fabric make this acceleration far more accessible: AMD's Zynq family, acquired with Xilinx in 2022, pairs Arm Cortex-A and Cortex-R cores with FPGA logic, its Versal devices add hardened AI and DSP engine arrays alongside the fabric, and the Agilex families from Altera (an independent, Silver Lake-controlled company since September 2025, when it separated from Intel) span comparable ground from low-power to high-end parts. Microchip's PolarFire SoC takes the same approach with RISC-V application cores, and lower-cost devices from Lattice serve power-constrained edge designs.

Neural Processing Units (NPUs)

NPUs are purpose-built accelerators for neural network computations, optimized for the matrix multiplications and tensor operations that dominate machine learning workloads. By implementing specialized architectures like systolic arrays and supporting reduced-precision arithmetic, NPUs achieve orders of magnitude better efficiency for AI inference than general-purpose processors. Modern system-on-chips for smartphones, automotive systems, and edge computing increasingly include dedicated NPUs alongside CPUs and GPUs.

The integration of NPUs in heterogeneous embedded systems enables sophisticated AI capabilities at practical power levels. Applications range from always-on voice recognition and face detection to real-time object detection for autonomous systems. NPUs typically support specific neural network operations and data types, with software frameworks responsible for mapping neural network models to efficient NPU implementations. Most embedded NPUs are built around 8-bit integer arithmetic, since quantizing a trained network from 32-bit floating point to INT8 usually costs little accuracy while cutting both arithmetic energy and memory traffic substantially; 4-bit and block floating-point formats extend the same idea further for suitable models.

Vendor headline figures deserve scrutiny. The commonly quoted TOPS rating (tera-operations per second) describes peak arithmetic throughput at an assumed precision and says nothing about achieved utilization, which depends on how well a given model's layer shapes match the accelerator's data path and on whether external memory can sustain the required bandwidth. Measured inference throughput and energy per inference on the actual target model are the meaningful figures. An operation the accelerator does not implement forces a fallback to the CPU or GPU, and the resulting round trip can erase the accelerator's advantage entirely, which makes operator coverage as important as raw throughput when selecting a platform.

System-on-Chip Integration

Modern heterogeneous embedded systems typically integrate multiple processor types on a single system-on-chip (SoC). This integration enables tight coupling between processing elements, with shared memory systems, high-bandwidth interconnects, and unified power management. Leading mobile SoCs from Apple, Qualcomm, Samsung, and MediaTek integrate CPU cores, GPUs, NPUs, DSPs, and specialized accelerators for tasks like image signal processing and video encoding, all sharing a common memory subsystem.

SoC integration offers significant advantages in power efficiency, latency, and cost. On-chip communication consumes far less energy than off-chip interfaces, enabling frequent data exchange between processing elements. Shared memory eliminates the overhead of copying data between accelerators, though careful management is required to avoid contention and maintain coherency. The physical proximity of integrated components also enables sophisticated power management, with fine-grained control over operating frequencies, voltages, and power states for each processing element.

Memory Architecture

Memory architecture is critical to heterogeneous system performance, as data movement often dominates both execution time and energy consumption. Heterogeneous systems employ various memory organizations, from fully shared memory where all processors access a common address space to distributed approaches where each accelerator has private memory with explicit data transfers between elements.

Shared memory simplifies programming by allowing processors to communicate through common data structures, but maintaining cache coherency across heterogeneous processors adds complexity and overhead. Many heterogeneous systems adopt hybrid approaches with coherent memory for CPU-GPU communication and dedicated, high-bandwidth memory for specialized accelerators. In embedded and mobile products, LPDDR5 and LPDDR5X supply most of the external bandwidth, chosen for their low interface energy per bit as much as for peak throughput; High Bandwidth Memory (HBM) appears only in the largest edge and automotive compute modules, where its silicon interposer and packaging cost can be justified. Tightly coupled on-chip scratchpad memories remain common for accelerators with predictable access patterns, because a local buffer read costs a small fraction of the energy of an external DRAM access. At the system level, Compute Express Link (CXL) rides on PCI Express electricals to attach coherent memory and accelerators at the board level; it is a data-center technology first, and chiplet-scale integration is instead the province of die-to-die standards such as UCIe.

Interconnect Design

Interconnects provide the communication fabric linking heterogeneous processing elements. Network-on-chip (NoC) architectures have replaced simple bus structures in modern heterogeneous SoCs, providing scalable bandwidth and supporting multiple simultaneous communications. NoCs use packet-based communication through routers and links, enabling efficient connectivity among many processing elements without the scaling limitations of shared buses.

Interconnect design must balance bandwidth, latency, and energy efficiency while supporting the diverse traffic patterns of heterogeneous workloads. Quality-of-service mechanisms ensure that latency-sensitive traffic receives priority, while bandwidth allocation prevents any single element from monopolizing shared resources. Cache coherency protocols extend across the interconnect, enabling processors with caches to share data efficiently. Arm's AMBA specifications supply the dominant protocol vocabulary here: AXI for high-throughput non-coherent traffic, AHB and APB for simpler peripheral access, ACE and CHI for coherent traffic between caching agents, and ACE-Lite for accelerators that must observe processor caches without maintaining their own. Configurable network-on-chip generators, including Arm's CoreLink and NI-700 products and Arteris FlexNoC, let designers synthesize a topology matched to a specific floorplan and traffic profile rather than assembling one by hand.

Programming Challenges

Programming heterogeneous systems presents significant challenges beyond those of traditional embedded development. Developers must partition applications across multiple processor types, each with different programming models, instruction sets, and optimization strategies. Data must be marshaled between processing elements, with careful attention to memory allocation, data layout, and synchronization. The complexity of heterogeneous programming has historically limited adoption, driving extensive research and development in programming models and tools.

Performance optimization in heterogeneous systems requires understanding the capabilities and limitations of each processing element. Tasks must be matched to appropriate accelerators based on their computational characteristics, data sizes, and communication requirements. The overhead of offloading computation to an accelerator may exceed the benefit for small tasks, requiring careful granularity analysis: a kernel launch, a cache maintenance operation, and a synchronization point each carry fixed cost, so a work item must be large enough to amortize them. Amdahl's law sets the ceiling on the whole exercise, because the portion of an application that remains on the host caps the achievable speedup no matter how fast the accelerator becomes. Different accelerators may also apply different numerical precision, requiring attention to algorithm accuracy across heterogeneous implementations, and results that differ between backends complicate testing when a system must produce identical output regardless of where a task ran.

Standard Programming Interfaces

Standard programming interfaces provide portable abstractions for heterogeneous programming, enabling code to target multiple platforms without complete rewrites. OpenCL (Open Computing Language) offers a cross-platform framework for writing programs that execute across heterogeneous systems, supporting CPUs, GPUs, FPGAs, and other accelerators through a common programming model. OpenCL programs consist of kernels written in a C-like language that are compiled for target devices at runtime.

SYCL provides a higher-level C++ abstraction over OpenCL concepts, enabling single-source programming where host and device code coexist in standard C++. This approach simplifies development by eliminating the need for separate kernel files and string-based kernel compilation. Both OpenCL and SYCL are maintained by the Khronos Group. The oneAPI initiative that Intel began builds on SYCL to provide a unified programming model across CPUs, GPUs, and FPGAs, with domain-specific libraries for common workloads; its specification is now stewarded by the UXL Foundation under the Linux Foundation, which broadens vendor participation. Vendor-proprietary toolchains such as NVIDIA CUDA remain widespread where a single supplier's hardware is assumed, and they generally expose device features earlier than the portable standards do. Portability and peak efficiency therefore pull in opposite directions, and many embedded projects settle on a portable framework for the bulk of the code with vendor-specific paths for the few kernels that dominate runtime.

Domain-Specific Frameworks

Domain-specific frameworks hide heterogeneous complexity behind high-level APIs tailored to particular application domains. Machine learning runtimes dispatch neural network operations to available accelerators, selecting CPUs, GPUs, or NPUs based on operation type and model characteristics. LiteRT, the runtime Google renamed from TensorFlow Lite in September 2024, routes work through pluggable delegates that target a specific backend. ExecuTorch fills the equivalent role for models authored in PyTorch, and ONNX Runtime uses execution providers to spread a single graph across several backends. Computer vision libraries such as OpenCV likewise leverage GPU and DSP acceleration through optional backends.

These frameworks encapsulate the expertise needed to efficiently utilize heterogeneous hardware, providing optimized implementations for common operations that would be difficult for application developers to match. Framework developers work closely with hardware vendors to tune performance for specific platforms, and vendor-specific backends enable full utilization of proprietary accelerator features. For many embedded applications, domain-specific frameworks provide the most practical path to heterogeneous acceleration.

Runtime Systems and Scheduling

Runtime systems manage the execution of heterogeneous applications, handling task scheduling, memory allocation, and resource management across processing elements. Effective runtime scheduling is essential for heterogeneous performance, as poor scheduling can leave accelerators idle while tasks wait in queues or create memory contention that degrades throughput.

Advanced runtime systems employ dynamic scheduling that adapts to system state and workload characteristics. Tasks may be migrated between processors based on current load, power constraints, or thermal conditions. Machine learning approaches are increasingly applied to scheduling decisions, learning from execution history to predict optimal task placements.

Platform history illustrates how difficult a single portable abstraction over diverse accelerators can be. The Android Neural Networks API (NNAPI), introduced in Android 8.1, offered one interface through which a runtime could enumerate accelerators and partition a model across them. Google deprecated NNAPI in Android 15, citing the pace of on-device machine learning development and the uneven quality of vendor implementations, and directed developers toward LiteRT delegates and updatable runtimes delivered through Google Play services. Existing NNAPI code continues to function for backward compatibility. The lesson generalizes: a lowest-common-denominator interface across accelerators is easy to target but tends to leave vendor-specific capability unused, so the industry has drifted toward thin, replaceable backends behind a stable framework API.

Dynamic Power Management

Power management in heterogeneous systems must coordinate across multiple processing elements with different power characteristics. Dynamic voltage and frequency scaling (DVFS) adjusts operating points for each processor type independently, enabling fine-grained power-performance trade-offs. Power gating disables unused accelerators entirely, eliminating leakage power when they are not needed. Effective power management requires understanding workload patterns and predicting future demands to balance responsiveness against energy efficiency.

Heterogeneous systems offer unique power optimization opportunities through workload migration. Tasks can be shifted from high-performance processors to more efficient alternatives when performance requirements allow. A video decoding task might run on a GPU during playback but shift to a dedicated video decoder for better efficiency. Machine learning inference might use an NPU for neural network operations but fall back to a CPU for unsupported operations. These migrations require careful management to avoid performance degradation while maximizing battery life.

Thermal Considerations

Thermal management presents particular challenges in heterogeneous embedded systems, where multiple high-performance processing elements share limited thermal dissipation capacity. The compact packaging of mobile devices, automotive systems, and industrial equipment constrains heat removal, potentially forcing performance throttling to maintain safe operating temperatures. Thermal-aware scheduling must consider the spatial distribution of heat sources, as neighboring processors contribute to each other's thermal environment.

Sophisticated thermal management strategies balance workload distribution to spread heat generation across the chip. Rather than driving a single accelerator at maximum throughput, distributing work across multiple elements can achieve higher sustained performance by avoiding thermal concentration. Predictive thermal management uses models of heat generation and dissipation to anticipate thermal issues before they trigger emergency throttling, enabling smoother performance that avoids sudden slowdowns.

Workload Analysis

Successful heterogeneous system design begins with thorough workload analysis to understand the computational requirements and characteristics of target applications. Profiling reveals which portions of applications consume the most time and energy, identifying candidates for acceleration. Analysis of computational patterns, data sizes, and memory access characteristics guides the selection of appropriate accelerator types for each workload component.

Workload diversity across the intended application space influences architectural decisions. Systems targeting a narrow application domain may employ specialized accelerators optimized for specific algorithms, while general-purpose platforms require more flexible acceleration that can support varied workloads. Understanding the expected mix of applications and their relative importance helps prioritize acceleration investments and guides trade-offs between specialization and generality.

Architecture Exploration

Architecture exploration evaluates alternative heterogeneous configurations to identify designs that best meet system requirements. Simulation and modeling tools enable rapid assessment of different processor combinations, memory configurations, and interconnect designs. Trade-off analysis considers performance, power consumption, silicon area, and cost across the design space, identifying Pareto-optimal configurations that offer the best combinations of these metrics.

Early architecture exploration uses high-level models that enable evaluation of many alternatives without detailed implementation. As the design converges, increasingly detailed models provide more accurate performance and power estimates. Virtual prototyping using software models of the complete system enables software development before silicon availability, reducing time-to-market and enabling software-hardware co-optimization. The Arm Fast Models and Synopsys Virtualizer platforms provide virtual prototyping capabilities commonly used in heterogeneous SoC development.

Verification and Validation

Verifying heterogeneous systems requires testing not just individual components but their interactions and the behavior of complete workloads executing across multiple processors. Verification must cover correct data transfer between elements, proper synchronization, and consistent behavior under various scheduling scenarios. The combinatorial complexity of heterogeneous verification often requires extensive simulation, emulation, and formal verification to achieve confidence in design correctness.

Validation confirms that the integrated system meets performance, power, and functional requirements with real applications. Silicon validation on early devices identifies issues that escaped pre-silicon verification, including performance anomalies, thermal problems, and software compatibility issues. The complexity of heterogeneous systems makes comprehensive validation challenging, requiring systematic test strategies that cover the range of workloads, operating conditions, and processor combinations the system will encounter in deployment.

Mobile and Consumer Electronics

Mobile devices exemplify the benefits of heterogeneous computing, with smartphones and tablets integrating diverse accelerators to deliver rich functionality within severe power constraints. A typical flagship smartphone SoC includes CPU clusters with multiple core types, a GPU for graphics and compute, an NPU for machine learning, a DSP for audio and sensor processing, and dedicated accelerators for image signal processing, video encoding and decoding, and security functions.

The heterogeneous architecture of mobile SoCs enables capabilities that would be impossible with homogeneous designs. Real-time computational photography combines image signal processor output with GPU-based processing and NPU-accelerated scene analysis. Always-on voice assistants use low-power DSP processing to detect wake words before engaging higher-power processors for speech recognition. These applications demonstrate how heterogeneous computing enables both performance and efficiency in battery-powered devices.

Automotive Systems

Automotive applications increasingly rely on heterogeneous computing for advanced driver assistance systems (ADAS) and autonomous driving. These systems must process data from multiple cameras, radar, and lidar sensors in real-time, fusing information to build environmental models and make driving decisions. The computational demands of perception, localization, path planning, and vehicle control exceed what any single processor type can efficiently deliver.

Automotive heterogeneous platforms from vendors like NVIDIA, Qualcomm, and Mobileye combine high-performance CPUs for control and decision-making, GPUs for parallel sensor processing, and dedicated accelerators for neural network inference. Safety-critical automotive applications add requirements for functional safety certification, redundancy, and deterministic behavior that strongly influence heterogeneous architecture choices. ISO 26262 assigns an Automotive Safety Integrity Level to each function, and the highest level, ASIL D, is difficult to reach with a throughput-oriented accelerator whose scheduling and memory behavior are hard to bound. A common resolution is to split the architecture by criticality: lockstep Arm Cortex-R or equivalent safety cores handle the certified control path, while the GPU and neural accelerators perform perception under the supervision of that safety island. Freedom from interference between the two domains must then be demonstrated, which places heavy demands on memory protection units, interconnect access control, and quality-of-service isolation. The NVIDIA DRIVE and Qualcomm Snapdragon Ride platforms are representative heterogeneous solutions built along these lines.

Industrial and Edge Computing

Industrial systems adopt heterogeneous computing for applications ranging from factory automation to infrastructure monitoring. Edge computing platforms bring heterogeneous processing to distributed installations, enabling local processing of sensor data, machine learning inference, and real-time control without continuous cloud connectivity. Industrial applications often require long product lifecycles, wide temperature ranges, and deterministic real-time behavior that influence heterogeneous platform selection.

Industrial heterogeneous systems may combine traditional industrial processors with modern accelerators. FPGAs provide the flexibility to implement custom protocols and interfaces common in industrial equipment, while NPUs enable machine learning applications like visual inspection and predictive maintenance. The convergence of operational technology (OT) and information technology (IT) in Industry 4.0 initiatives drives increasing adoption of heterogeneous platforms that bridge traditional industrial control with modern computing capabilities.

Chiplet-Based Heterogeneous Systems

Chiplet architectures decompose monolithic SoCs into multiple smaller dies interconnected in advanced packages. This approach enables heterogeneous integration of dies manufactured using different process technologies, with compute chiplets on leading-edge nodes and I/O or memory chiplets on more mature, cost-effective processes. Chiplets also enable flexible product configurations, with different combinations of processing elements assembled to meet varied market requirements without full redesign.

Advanced packaging technologies like 2.5D interposers and 3D stacking enable high-bandwidth, low-latency connections between chiplets, approaching the integration density of monolithic designs. The Universal Chiplet Interconnect Express (UCIe) standard aims to enable chiplet interoperability across vendors, potentially allowing heterogeneous systems to be assembled from best-in-class components from multiple sources. It defines both a standard package variant for organic substrates and an advanced variant for silicon interposers and comparable dense packaging; the UCIe 3.0 specification, released in August 2025, raised the per-lane rate to 48 and 64 GT/s from the 32 GT/s of the preceding revision and added runtime recalibration and management features. Embedded systems are beginning to adopt chiplet approaches as packaging costs decrease, though the economics still favor monolithic integration for the moderate die sizes typical of embedded parts, and known-good-die testing remains a practical obstacle to a genuinely open multi-vendor marketplace.

Specialized AI Acceleration

AI acceleration continues to drive heterogeneous innovation, with increasingly specialized accelerators targeting specific neural network types or deployment scenarios. Transformer accelerators optimize for the attention mechanisms dominating large language models. Graph neural network accelerators support irregular computation patterns poorly served by conventional NPUs. Tiny ML accelerators enable neural network inference on microcontrollers with microwatt power budgets.

The diversity of AI workloads suggests that future heterogeneous systems may include multiple AI accelerator types, each optimized for different model architectures or precision requirements. Adaptive accelerators that can reconfigure for different neural network types offer an alternative approach, trading some efficiency for flexibility. The rapid evolution of AI algorithms continues to challenge hardware designers, requiring heterogeneous platforms that can accommodate both current models and anticipated future developments.

Security-Integrated Heterogeneous Architectures

Security considerations increasingly influence heterogeneous architecture design, with dedicated security processors and isolated execution environments becoming standard components. Trusted execution environments provide hardware-enforced isolation for sensitive computations, protecting cryptographic keys and secure boot processes from compromise. Arm TrustZone partitions a processor between secure and non-secure worlds and propagates that distinction across the AMBA interconnect, so that peripherals and memory regions can be restricted to secure accesses; RISC-V systems achieve comparable separation through physical memory protection, with vendor extensions such as SiFive's WorldGuard extending the idea across the interconnect. Dedicated security subsystems, often a small isolated core with its own ROM and key storage, anchor the chain of trust and integrate cryptographic accelerators with tamper-resistant storage.

Heterogeneous security architectures must protect not just data at rest but also data as it moves between processing elements. Encrypted memory interfaces prevent physical attacks from extracting sensitive information. Access control mechanisms ensure that accelerators can only access authorized memory regions. The integration of security throughout heterogeneous systems reflects the growing importance of protecting embedded devices from increasingly sophisticated attacks.