Electronics Guide

The Basic Protocol

A federated training run proceeds in rounds. In each round the server selects a subset of eligible participants, ships them the current model parameters, and waits. Each selected participant runs some number of local optimization steps on its own examples, then returns the difference between its locally trained parameters and the ones it received. The server aggregates those differences, typically as a weighted average, applies the result to the global model, and starts the next round. Rounds continue for hundreds or thousands of iterations, and a production run against a large mobile population may take days.

Federated Averaging

The defining design choice in Federated Averaging is that participants perform multiple local gradient steps before reporting, rather than returning a single gradient. Communication is the scarce resource, so the algorithm spends local computation to reduce the number of rounds. The trade-off is that local models drift apart when participants hold dissimilar data, and averaging drifted models degrades convergence. Round count, local epoch count, and local batch size are therefore tuned against each other, and the correct settings depend on how heterogeneous the population is. The original paper reported a ten- to hundredfold reduction in the number of communication rounds relative to synchronized stochastic gradient descent, with the factor depending on the model and on how the data are distributed.

What the Server Actually Sees

In the naive protocol the server receives, from each participant, a full-length vector of parameter deltas attributable to one identifiable client. This is a far richer signal than most engineers expect. It is not a summary statistic; it is a differentiable function of the participant's private examples, of the same dimensionality as the model. Everything described in the sections on secure aggregation, differential privacy, and trusted execution exists to narrow what that vector reveals.

Cross-Device and Cross-Silo Settings

Two deployment regimes differ so sharply that they are best treated as separate engineering problems. The survey by Peter Kairouz, Brendan McMahan, and a large group of co-authors, published as Advances and Open Problems in Federated Learning in Foundations and Trends in Machine Learning in 2021, formalized the distinction.

The cross-device setting involves an enormous number of unreliable, resource-constrained clients: phones, wearables, vehicles, or sensor nodes. Population sizes reach hundreds of millions, but any individual device participates rarely, may vanish mid-round, is stateless between rounds, and cannot be indexed or addressed on demand. Devices are assumed to be honest but the operator cannot audit them. Bandwidth is metered and expensive, and power comes from a battery.

The cross-silo setting involves a small number of organizations, typically between two and a hundred: hospitals, banks, manufacturers, or research institutions. Each holds a substantial dataset, has reliable connectivity and server-class hardware, participates in every round, is addressable and stateful, and is bound by a legal agreement. The constraint here is not battery or radio but regulation and commercial confidentiality. Silos may also be mutually distrustful competitors, which raises the required security model considerably, and the data may be partitioned by feature rather than by sample, so that several institutions hold different attributes of the same individuals. Cross-device work concentrates on communication efficiency and straggler tolerance; cross-silo work concentrates on strong cryptographic guarantees and auditability.

Why Updates Leak: Gradient Inversion

The claim that federated learning is private because data never moves is false as stated, and the attack that disproves it is the reason the rest of the privacy machinery exists.

Deep Leakage from Gradients

In 2019 Ligeng Zhu, Zhijian Liu, and Song Han published Deep Leakage from Gradients at NeurIPS, showing that an adversary holding a client's gradient and the model that produced it can recover the training input almost exactly. The method is an optimization: initialize a dummy input and dummy label, compute the gradient the model would produce from them, and adjust the dummy pair by gradient descent to minimize the distance between the dummy gradient and the observed one. For small batches and modest images the reconstruction converges to a pixel-level match of the original.

Jonas Geiping and colleagues extended the attack in Inverting Gradients at NeurIPS 2020, replacing the Euclidean gradient distance with a cosine-similarity objective and adding a total-variation image prior. Their version reconstructs images faithfully at high resolution rather than at the small sizes the earlier method required. They also prove that any input to a fully connected layer can be recovered analytically, independent of the rest of the architecture, and show numerically that averaging the gradient over several iterations or over several images does not by itself protect a participant. Subsequent work extended inversion to text, where recovering token sequences from a language-model update is in some respects easier because the embedding-layer gradient is sparse and directly indexes which tokens appeared.

What Makes Inversion Harder

Attack difficulty rises with batch size, because the reported gradient is an average and the attacker must disentangle contributions; with the number of local steps, because a multi-step update is not a single gradient and the intermediate states are unknown; with model width, in some regimes; and with any lossy transformation applied before reporting, including quantization, sparsification, and noise. None of these is a security guarantee. They raise cost rather than establishing a bound, and an adversary who controls the model architecture or the initialization can deliberately weaken them, for instance by inserting layers that isolate individual examples in the averaged gradient. The distinction matters: a defense that merely makes the published attacks fail is not the same as a defense with a proof.

Membership Inference and Memorization

Even where exact reconstruction fails, weaker inferences often succeed. Membership inference determines whether a particular record was in the training set, which is itself a disclosure when the dataset is a list of patients with a given diagnosis. Large models also memorize rare training sequences and can be induced to emit them at inference time. Because these leaks travel through the model itself rather than around any protection, no amount of transport encryption or hardware isolation prevents them. Only a statistical guarantee on the output does.

On-Device Training on Constrained Silicon

Federated learning requires participants to run a backward pass, and that requirement collides with how edge inference silicon is actually built.

The Backward Pass Is Not a Second Forward Pass

Inference streams activations through a network and discards each layer's output once the next layer has consumed it, so peak memory is bounded by the largest pair of adjacent tensors. Backpropagation cannot do this. Computing the gradient of a layer's weights requires that layer's input activation, so activations from the entire forward pass must be retained until the backward pass reaches them. Peak memory therefore scales with network depth multiplied by activation size, and for a convolutional network of any size this is the dominant cost, exceeding the weights themselves. Optimizer state adds more: plain stochastic gradient descent with momentum stores one extra copy of the parameters, and Adam stores two. A model that fits comfortably in an inference budget can need an order of magnitude more memory to train.

Neural Processing Units Are Built for Inference

Most edge neural processing units are designed around quantized inference. The datapath is an integer multiply-accumulate array, commonly eight-bit with a wider accumulator, and the toolchain expects a frozen graph compiled ahead of time. Training needs several things such a unit does not provide. It needs a transposed-convolution or equivalent data movement pattern for the gradient with respect to activations, which is awkward on a dataflow scheduled for forward convolution. It needs sufficient dynamic range in the gradient computation, and gradients span a far wider and less predictable range than activations, so a naive eight-bit backward pass diverges. It needs a way to accumulate weight updates in higher precision than the weights are stored. And it needs runtime control flow, since the layers being updated may vary. The practical consequence is that on many devices the backward pass falls back to the CPU or the general-purpose vector unit while the NPU sits idle, which changes the energy and latency arithmetic entirely. Newer parts covered under edge AI processors increasingly expose higher-precision modes and gradient support precisely because on-device adaptation has become a product requirement, but the installed base skews heavily toward inference-only silicon.

Training Under a Few Hundred Kilobytes

At the microcontroller end the constraints are severe enough to require algorithm and system co-design. Ji Lin and colleagues at MIT demonstrated in On-Device Training Under 256KB Memory, published at NeurIPS 2022, that convolutional network training is possible within 256 KB of SRAM and 1 MB of flash with no auxiliary memory. Two ideas carry the result. Quantization-aware scaling rescales the gradients of quantized layers, whose magnitudes otherwise differ wildly between weight and bias tensors, so that eight-bit training remains stable. Sparse update skips the gradient computation for less important layers and for selected channels within layers, which removes both the arithmetic and the stored activations those gradients would have required. The implementation, a tiny training engine, prunes the backward graph and moves automatic differentiation from runtime to compile time. Measured training memory falls from 303 MB under PyTorch to 149 KB, a reduction of roughly two thousand times, at matched transfer-learning accuracy on the visual wake words task. The figure is for MobileNetV2-w0.35 at batch size one and 128 by 128 resolution, and the system was demonstrated on an STM32F746 with 320 KB of SRAM and 1 MB of flash. Work of this kind is the precondition for federated learning on the class of hardware discussed under ultra-low-power computing.

Eligibility, Thermals, and Duty Cycle

On handsets the binding constraint is often policy rather than capability. Production systems train only when the device is charging, idle, and connected to an unmetered network, because a training round is a sustained high-power workload that would otherwise be visible to the user as battery drain and a warm case. Sustained matrix arithmetic pushes a mobile system-on-chip into thermal throttling within seconds to minutes, so the achievable throughput is set by the thermal design power and the chassis, not by the peak numbers on the datasheet. The techniques catalogued under low-power design techniques and the state-of-charge reporting described under battery management systems are what the eligibility check consults before a device accepts a round.

The Communication Bottleneck and Its Energy Cost

Federated learning inverts the usual traffic pattern of a consumer device. Instead of downloading content, the device uploads a payload the size of a model, repeatedly, over a link engineered on the assumption that uplink demand is light.

Payload Size

The update a client returns has the same dimensionality as the model it trained, so a model with ten million float32 parameters implies a forty-megabyte upload per participating round before compression. Even the compact recurrent and transformer models used for keyboard prediction run to several megabytes. Multiplied across the population and across thousands of rounds, this is the dominant system cost, and it falls on the least capable half of the link. Residential and cellular access are asymmetric by design, with uplink capacity a fraction of downlink, and shared cells degrade further when many devices transmit at once.

Radio Energy Dominates

The reason communication is expensive is physical rather than architectural. Pottie and Kaiser made the point memorably in Communications of the ACM in 2000: transmitting one kilobit over one hundred meters costs roughly three joules, whereas a processor achieving 100 MIPS per watt could execute three million instructions for the same energy. Both sides of that comparison have improved enormously in the intervening quarter century, and the absolute figures are long obsolete, but the ratio has proved durable because computation benefits from process scaling in a way that radiated power does not. Local computation is cheap relative to transmission, which is the fundamental justification for the whole federated architecture and equally for compressing updates aggressively before sending them.

Cellular links add a second cost that byte counts do not capture. Radio resource control state machines keep the modem in a high-power state for a tail period after the last packet, so a small transmission can consume energy for many seconds afterward. A federated client that dribbles its update out in fragments therefore pays repeatedly for the same radio promotion. Practical implementations batch the entire update into one transfer, prefer Wi-Fi, and align transmission with other scheduled network activity so that the radio wake-up is amortized. The same reasoning drives the aggregation strategies described under gateway and edge computing and edge computing systems.

Compressing the Update

Jakub Konečný and colleagues set out the standard toolkit in 2016. Structured updates constrain the client to learn an update of a restricted form, such as a low-rank factorization or a random-mask sparse pattern, so that only the free parameters need transmission. Sketched updates train freely and then compress the result through subsampling, probabilistic quantization, and random rotation, the last of which spreads the signal across coordinates so that coarse quantization does less damage. Reported reductions reach two orders of magnitude with limited accuracy loss. Error feedback, in which the client retains the residual it failed to transmit and folds it into the next round, recovers much of the accuracy lost to aggressive quantization.

Compression interacts badly with the privacy machinery, and this is one of the recurring design tensions in the field. Sparsification saves bandwidth only if the aggregator can exploit the sparsity, but secure aggregation masks every coordinate, so the transmitted ciphertext is dense regardless. Quantization to few bits is desirable for bandwidth and mandatory for the modular arithmetic that secure aggregation uses, yet the differential privacy noise must be added in the quantized domain without breaking either the privacy analysis or the accuracy. A design that optimizes any one of the three in isolation usually degrades the other two.

Secure Aggregation

Secure aggregation lets the server learn the sum of many client updates while learning nothing about any individual update. It is a specialized instance of secure multi-party computation, narrowed to a single function, summation, and to a single-server topology with a very large number of unreliable participants. That specialization is what makes it fast enough to deploy where general multi-party protocols would not be.

Pairwise Masking

The construction that made this practical is due to Keith Bonawitz and colleagues, presented at ACM CCS in 2017. Every pair of participating clients agrees on a shared secret through Diffie-Hellman key exchange and derives from it a pseudorandom mask vector. Each client adds the masks it shares with lower-indexed peers and subtracts those it shares with higher-indexed peers, so that in the sum over all clients every mask cancels exactly against its partner. Individually, each masked update is indistinguishable from random; summed, the masks vanish and the true total appears.

Client dropout breaks this, because a departed client's masks no longer cancel. The protocol handles it with threshold secret sharing: before the round, each client distributes shares of its own secrets to the others, so that the server can reconstruct the masks of clients that failed to report, provided enough clients survive. A second, independent self-mask is layered on top to close the race condition in which a client is declared dropped after its update has already arrived, which would otherwise let the server recover that client's update in the clear. The security model tolerates a curious server and a bounded fraction of colluding clients.

Cost and Scaling

The original protocol requires each client to establish a key with every other client in the round, giving communication linear in the number of participants and computation quadratic in it, with server cost quadratic as well. That confines cohorts to a few hundred or low thousands of clients. James Bell and colleagues removed the bottleneck at CCS 2020 by replacing the complete pairwise graph with a random regular graph in which each client masks against only a logarithmic number of neighbors, chosen so that the graph remains connected and resistant to adversarial dropout with high probability. The result is polylogarithmic communication and computation per client, which is what allows cohorts to grow large enough for the differential privacy analysis to be favorable.

What It Costs the Device

On the client, secure aggregation adds public-key operations for the key agreement, symmetric expansion to generate the mask vectors, and modular arithmetic over the full update length. The arithmetic is the part that hurts on constrained hardware: masking operates in a finite field, so the quantized update must be lifted into a modulus large enough to hold the sum of all clients' contributions without wraparound, which inflates each coordinate relative to the raw quantized value and can undo part of the compression gain. Choosing the modulus is a genuine design parameter, traded against bandwidth on one side and against the probability of arithmetic overflow corrupting the round on the other. The cryptographic primitives themselves are well served by the accelerators discussed under embedded security and cryptography.

The Limit of Secure Aggregation Alone

Secure aggregation hides individual contributions but publishes their sum, and a sum still reflects its terms. With a small cohort, or with a server that manipulates cohort membership across rounds so that a target client's contribution can be differenced out, the aggregate leaks. Secure aggregation is therefore necessary but not sufficient, and its real function in a modern design is to enable a better differential privacy trade-off rather than to provide the guarantee itself.

Differential Privacy on the Updates

Differential privacy supplies the bound that architecture alone cannot. Applied to federated learning it operates at the level of the participant rather than the individual example: the guarantee is that the released model is statistically almost the same whether or not any one client took part, which is the unit of privacy users actually care about.

Clipping and Noise

The mechanism has two parts. Each client clips its update to a fixed norm, which bounds the sensitivity of the aggregate to any single participant, and calibrated noise proportional to that clip norm is added to the sum. Choosing the clip norm is the practical difficulty. Too small and the update is dominated by clipping bias; too large and the required noise swamps the signal. Adaptive schemes track a quantile of the observed update norms and set the threshold from it, though the quantile estimate must itself be computed privately.

Cross-device federated learning cannot use the DP-SGD analysis that data-center training relies on, because that analysis depends on privacy amplification by uniform subsampling, and no server can uniformly sample from a population of devices that appear and disappear according to their own charging and connectivity state. The DP-FTRL family, which uses correlated noise from a tree-aggregation structure rather than fresh independent noise each round, removes the sampling assumption and yields guarantees that hold under the participation patterns that actually occur.

Distributed Noise

Where the noise is added determines whom the user must trust. Central differential privacy has the server add noise to the aggregate, which gives good accuracy but requires trusting the server with individual updates. Local differential privacy has each client add enough noise to protect itself alone, which trusts nobody but destroys utility at any reasonable privacy level. Secure aggregation opens a middle path: each client adds a small share of the total noise, and because the server sees only the sum, the noise adds up to the central-model amount while no individual update is ever exposed. The complication is that secure aggregation works in modular arithmetic over integers, so continuous Gaussian noise cannot be used directly. Peter Kairouz, Ziyu Liu, and Thomas Steinke resolved this at ICML 2021 with the distributed discrete Gaussian mechanism, which discretizes each client's update, adds discrete Gaussian noise on the device, and provides a privacy analysis that accounts for both the quantization and the modular wraparound. They report accuracy essentially matching central differential privacy with fewer than sixteen bits of precision per coordinate, which is the result that makes the combination practical.

Production Guarantees

Google reported deployed results at the ACL 2023 industry track in Federated Learning of Gboard Language Models with Differential Privacy. Using DP-FTRL with a participation criterion designed for real device availability, and quantile-based adaptive clipping, the team trained and launched more than twenty Gboard language models with formal zero-concentrated differential privacy guarantees at ρ between 0.2 and 2, with two of the models additionally trained under secure aggregation. The authors state that every next-word-prediction neural network language model in Gboard now carries a differential privacy guarantee, and that future launches of such models will require one. These are among the few production neural networks trained on user data that ship with a stated formal privacy parameter rather than a policy assertion.

Trusted Execution for the Aggregator

A third approach protects the server side in hardware. Running the aggregator inside a trusted execution environment means individual client updates are decrypted only inside an attested enclave or confidential virtual machine, where neither the machine operator nor a compromised host operating system can read them.

What Attestation Adds

The mechanism is the standard key-release pattern described under remote attestation systems. Before a client uploads, it verifies a hardware-signed measurement of the code and configuration running in the aggregator, checks that measurement against a published reference, and only then encrypts its update to a key bound to that environment. The client is therefore not trusting a promise about how its data will be handled; it is trusting a signature over the specific binary that will handle it. Google described such a system in Confidential Federated Computations in 2024, combining confidential virtual machines with publicly released aggregator binaries and reproducible builds, so that an outside party can compile the published source, derive the expected measurement, and confirm that the attested server is running exactly that code and nothing else. The design also enforces a ledger over uploaded data so that a given contribution can be used only for the computations the client authorized.

Trade-Offs Against Cryptography

Compared with secure aggregation, a trusted execution environment is far cheaper. Aggregation runs at close to native speed, cohort sizes are unconstrained, dropout requires no recovery protocol, and the server can inspect individual updates for robustness checks that masked updates make impossible. The cost is a different trust assumption: correctness now depends on a processor vendor's implementation, its firmware, and its attestation key hierarchy. Transient-execution attacks, fault injection through voltage manipulation, and ciphertext side channels have all been demonstrated against production enclaves, and each is met with microcode updates and revised security version numbers that attestation lets a client refuse to accept. Secure aggregation, by contrast, rests on cryptographic hardness assumptions independent of any manufacturer, at substantially higher cost. Layered designs use both, so that defeating the system requires breaking the hardware isolation and the cryptography, and add differential privacy so that even a total compromise of the aggregator bounds what the published model reveals. Broader treatment of the hardware side appears under confidential computing.

System Heterogeneity and Stragglers

A federated round completes when enough participants report, and in a population spanning flagship handsets and five-year-old budget devices the completion time is set by the slow tail rather than the median.

Over-Selection and Deadlines

The production system design published by Bonawitz and colleagues at MLSys in 2019 handles this with a reporting deadline and deliberate over-provisioning. The server selects roughly 130 percent of the target number of devices, aggregates whatever arrives before the deadline, and abandons the round if fewer than a configured minimum report in time. That margin is sized against measured loss: the same paper puts the share of devices that drop out from computation errors, network failures, or a change in eligibility at 6 to 10 percent on average, and the remainder of the margin covers the stragglers the operator chooses to discard. The factor is tuned from the empirical distribution of device reporting times. Discarding stragglers is not free: devices that are systematically slow tend to be systematically different, so the updates that are dropped are not a random sample, and the model quietly comes to fit the faster half of the fleet.

Statistical Heterogeneity

Client datasets are not independent and identically distributed. A keyboard model sees different vocabularies by region and language; a hospital sees a patient population shaped by its catchment area and specialty. Under Federated Averaging, local models trained on skewed data drift toward their local optima, and averaging drifted models produces an update that serves nobody well. FedProx, from Tian Li and colleagues at MLSys 2020, adds a proximal term penalizing local divergence from the global model, which both dampens the drift and permits partial work from clients that cannot finish the full local computation, converting a straggler from a discarded round into a reduced contribution. Personalization approaches go further, keeping some layers local to each device so that a shared representation is combined with a locally fitted head.

Asynchrony

Buffered asynchronous aggregation lets the server apply updates as they arrive into a staging buffer rather than waiting for a synchronized cohort, which raises device utilization and removes the straggler deadline entirely. The difficulty is that both secure aggregation and the standard privacy amplification arguments assume a well-defined synchronous cohort. Asynchrony therefore trades throughput against the privacy machinery, and reconciling the two remains an active research problem rather than a settled engineering practice.

Participation Bias

Eligibility rules impose a selection effect that is easy to overlook and hard to correct. Requiring a charging device on an unmetered network with sufficient free storage skews participation toward newer hardware, toward households with home broadband, and toward the overnight hours in each time zone. The resulting model is trained on a population that is not the user population. This is a fairness and reliability problem rather than a privacy one, and it belongs in the same analysis as the failure modes discussed under artificial intelligence system reliability.

Robustness and the Untrusted Client

Distributing training also distributes the attack surface. A participant that submits a crafted update rather than an honest one can degrade the global model or implant a backdoor that misclassifies a chosen trigger while leaving overall accuracy intact. Defenses generally rely on inspecting the distribution of submitted updates, discarding outliers by coordinate-wise median or trimmed mean, or bounding each contribution by clipping its norm.

These defenses conflict directly with the privacy mechanisms. A server operating under secure aggregation sees only the sum and cannot inspect individual updates for anomalies, and norm clipping performed by the client cannot be verified by a server that cannot see the pre-clipping value. Resolving this requires proving properties of a hidden value, which is exactly what zero-knowledge proof systems provide: a client accompanies its masked update with a succinct proof that the underlying vector is well formed and within the permitted norm, and the aggregator rejects contributions whose proofs fail without ever seeing the contribution itself. Verification cost at the server and proving cost at the client are the practical obstacles, and proving cost is precisely the expensive half of that asymmetry. An enclave-based aggregator sidesteps the problem by inspecting updates inside the protected boundary, which is one of the stronger arguments for the trusted-execution approach.

Deployments, Frameworks, and Standards

Mobile keyboards remain the largest cross-device deployment. Google first described Gboard training on decentralized user data in 2017, and the deployment has since covered query suggestion, next-word prediction, emoji suggestion, and out-of-vocabulary word discovery, together with the differentially private language models reported in 2023 and described above. Apple has applied federated methods to keyboard and speech personalization, combining them with local differential privacy for the analytics side.

The most cited cross-silo result is medical. Sarthak Pati, Ujjwal Baid, and a large international group reported in Nature Communications in 2022 that a federated study spanning 71 sites across six continents trained a glioblastoma boundary detector on 6,314 patients and 25,256 magnetic resonance scans, the largest such dataset assembled for the disease, without any site sharing patient images. Against a model trained only on public data, the federated model improved delineation of the surgically targetable tumor by 33 percent and of the complete tumor extent by 23 percent. Rare-disease work of this kind is the clearest demonstration that the architecture buys statistical power that regulation would otherwise forbid, and it connects to the regulatory questions treated under artificial intelligence in medical devices. Comparable consortia operate in pharmaceutical compound screening, in interbank fraud and anti-money-laundering analysis, and in industrial predictive maintenance across equipment fleets owned by different operators.

Open-source frameworks have consolidated around a few options: TensorFlow Federated for research on the Google stack, Flower as a framework-agnostic coordinator that runs against PyTorch, TensorFlow, or bare NumPy, NVIDIA FLARE for cross-silo deployments with an emphasis on healthcare, and OpenFL, hosted by the Linux Foundation and grown out of the Intel collaboration with the University of Pennsylvania group that built the federated tumor-segmentation platform behind the glioblastoma study above. Google has consolidated its own components under the parfait organization, which hosts TensorFlow Federated, the federated-compute runtime, and the trusted-execution binaries used for confidential aggregation. On the standards side, IEEE 3652.1-2020, the IEEE Guide for Architectural Framework and Application of Federated Machine Learning, provides the reference architecture, a taxonomy of horizontal, vertical, and transfer federated settings, evaluation criteria, and a survey of applicable regulatory requirements. It is a guide rather than a conformance standard, but it supplies the shared vocabulary that cross-organization agreements need. The obligations that make such agreements necessary are covered under data privacy and information protection and artificial intelligence and machine learning regulation.

Engineering the Trade-Offs

A federated deployment is defined by four quantities that cannot all be optimized at once: model accuracy, the privacy parameter, the energy and bandwidth cost per device, and the wall-clock time to convergence. Tightening the privacy budget requires more noise, which requires larger cohorts or more rounds to average it away, which costs energy and time. Compressing updates saves bandwidth but adds quantization error that stacks with the privacy noise. Adding secure aggregation improves the privacy-accuracy trade-off but inflates the payload and pins the design to a synchronized cohort. Relaxing the eligibility rules widens participation and reduces bias but drains batteries.

Some guidance is nonetheless stable. Measure the uplink cost first, since it usually dominates and it is the quantity users notice. Set the unit of privacy explicitly and early, because a guarantee stated per example rather than per client is far weaker than it sounds and the choice propagates through every subsequent design decision. Assume that any update visible to the server in the clear is recoverable, and design as though gradient inversion will succeed. Treat straggler policy as a modeling decision rather than an operational one, since who gets dropped determines what the model learns. And prefer layered defenses, because secure aggregation, differential privacy, and trusted execution fail in different ways and under different assumptions, which is precisely why combining them is worth the cost.

Conclusion

Federated learning moves computation to data instead of data to computation, and in doing so converts a data-governance problem into a distributed systems problem on some of the least cooperative hardware in existence. The architecture on its own provides no privacy guarantee: model updates are invertible, and gradient inversion attacks recover training inputs when nothing stands in the way. What makes a deployment defensible is the stack built on top. Secure aggregation ensures the server sees only sums. Differential privacy, with noise distributed across clients so that no central party must be trusted, bounds what the published model reveals about any participant. Trusted execution protects the aggregator against its own operator and makes the running code externally verifiable. Underneath all of it sits the hardware reality: a backward pass that costs an order of magnitude more memory than inference, edge accelerators designed for a forward pass only, and a radio whose energy per bit still dwarfs the energy per arithmetic operation. Progress in the field has come as much from training engines that fit in a few hundred kilobytes and aggregation protocols with logarithmic overhead as from any advance in the learning algorithm itself.

Related Topics