Electronics Guide

The Problem: Drift, Sparse References, and a Circular Dependency

Position estimation admits two broad strategies, and both fail on their own in characteristic ways.

Dead Reckoning and Unbounded Drift

Dead reckoning integrates motion. Wheel encoders count revolutions, an inertial measurement unit integrates acceleration and angular rate, a camera or lidar estimates the incremental transform between consecutive observations. Every one of these is a relative measurement, and the position estimate is their accumulated sum.

Errors accumulate along with the signal. A gyroscope bias of one degree per hour, respectable for a tactical-grade device, integrates into an attitude error that grows linearly with time, and an attitude error tilts the accelerometer frame so that a component of gravity leaks into the horizontal channel. Because position is the double integral of acceleration, an inertial-only solution accumulates position error roughly with the cube of elapsed time. Wheel odometry has no cubic term but suffers from slip, tire deformation, and uncertainty in the effective wheel radius, producing a scale error that grows with distance traveled. Visual odometry drifts too, and its rotation errors are especially damaging, because a small heading error becomes a lateral position error proportional to everything driven afterward.

Dead reckoning is therefore locally excellent and globally hopeless. Over a second it is the most accurate thing available; over an hour it is unbounded, and better sensors move the constant without changing the shape of the curve.

Absolute References and the Circular Dependency

The complementary strategy measures against something fixed. A satellite fix, a surveyed beacon, an ultra-wideband anchor, or a fiducial marker provides a position whose error does not grow with time. Absolute references cure drift completely, but they are unavailable exactly where robots most need them. Instrumenting a warehouse costs money and must be redone when the racking moves, satellite signals are attenuated indoors and reflected in urban canyons where multipath produces confident and wrong fixes, and a vehicle in a tunnel, a drone in a collapsed building, and a rover on another planet share the same predicament: no infrastructure, and no prospect of any.

SLAM resolves the impasse by observing that an unmapped environment is still full of geometry, and geometry that stays put can serve as a reference even when nobody surveyed it. A wall corner observed from two positions constrains the relative motion between them; observed a hundred times around a loop, it constrains the whole loop. This creates the circular dependency that gives the field its reputation for difficulty, because placing a landmark in a global frame requires knowing the pose it was observed from, and knowing that pose requires landmarks already placed. The resolution is not to break the circle but to accept it, estimating trajectory and map jointly as one system of coupled unknowns.

Why the Joint Problem Is Genuinely Hard

The unknowns are correlated, and the correlations matter. If the robot pose is uncertain, every landmark placed from that pose inherits the uncertainty and is therefore correlated with every other. An estimator that treats landmarks as independent discards the very structure that makes loop closure work. The recognition, in the stochastic-map work of Randall Smith, Matthew Self, and Peter Cheeseman in the mid-1980s, that these correlations must be maintained is what turned SLAM from a heuristic into an estimation problem with a defensible answer.

The problem is high-dimensional and grows without bound. The state holds the robot pose, six degrees of freedom in three dimensions, plus three numbers per point landmark, and it grows with time if the trajectory is estimated rather than marginalized away. A modest indoor map holds thousands of landmarks and an outdoor lidar map millions of points, so any algorithm costing worse than roughly linear in map size hits a wall on real data.

The models are nonlinear and the noise is not Gaussian. A camera projects points through a perspective transform, and rotations live on a manifold rather than a vector space. Linearizing about a wrong estimate injects error that the estimator mistakes for information, which is the mechanism behind the overconfidence that plagued early filters.

Data association is a discrete problem hidden inside a continuous one. Before a measurement can be used, the system must decide which landmark produced it. Get that wrong and the estimator fuses a constraint between two places that are not the same place. Almost all catastrophic SLAM failures trace to data association rather than to the numerical optimization.

The Probabilistic Formulation

The standard formalism treats SLAM as inference over a posterior distribution given the control inputs and the observations. Two variants are distinguished. Online SLAM estimates the posterior over the current pose and the map, marginalizing past poses as they leave the window; this keeps the state bounded but commits permanently to the linearizations made when each pose was discarded. Full SLAM, or smoothing, estimates the entire trajectory and the map together, so a later observation can revise an early pose and every linearization point can be improved by iterating. Full SLAM is more expensive and strictly more accurate, and the field moved decisively toward it once the computational objections were answered.

Both rest on a motion model and an observation model. If the noise on each is Gaussian and independent, maximizing the posterior is equivalent to minimizing a sum of squared residuals weighted by the inverse covariance of each measurement. That equivalence between probabilistic inference and weighted nonlinear least squares connects the filtering literature to the optimization literature, and every framework below attacks the same objective by a different route.

The Estimation Frameworks and What Each Fixed

The history of SLAM is legible as a sequence of frameworks, each removing a specific limitation of the last.

Extended Kalman Filter SLAM

The first workable formulation stacked the robot pose and all landmark positions into a single Gaussian state with a full covariance matrix and propagated it with an extended Kalman filter. Each observation corrected both the pose and, through the off-diagonal covariance blocks, every correlated landmark, which is how a single sighting of an old landmark corrects the entire map at once.

Two weaknesses proved crippling. The first is cost: the covariance has an entry per pair of state variables, so its size is quadratic in landmark count and the update touches all of it. A few hundred landmarks are comfortable; a few thousand are not. The second is consistency. The filter linearizes about an estimate that is systematically wrong, which injects spurious information, particularly about global heading. The reported covariance shrinks below the true error, the filter becomes overconfident, and it begins rejecting correct measurements as outliers. Later analysis showed this inconsistency to be structural, arising because linearization at different times uses different estimates and thereby erodes the unobservable directions of the problem.

Particle Filters, Rao-Blackwellization, and FastSLAM

Particle filters represent a distribution by weighted samples, which handles nonlinearity and multimodality gracefully. Applied naively to SLAM they fail at once, because the number of particles needed to cover a space grows exponentially with its dimension and the state includes every landmark.

The FastSLAM family, introduced by Michael Montemerlo and colleagues in the early 2000s, escaped this with the most elegant idea in the filtering branch of the subject. Conditioned on a known trajectory, the landmarks become independent: if the poses are given, each landmark is estimated from its own observations alone. The joint posterior therefore factors into a distribution over trajectories multiplied by a product of small independent landmark distributions. This is a Rao-Blackwellized particle filter, in which each particle carries one hypothesized trajectory plus its own set of tiny Kalman filters rather than one enormous joint covariance, and cost becomes roughly linear in landmark count. The same factorization underlies grid-based methods such as GMapping, where each particle carries an entire occupancy grid, and such filters remain the standard solution for two-dimensional indoor lidar mapping.

Their limitation is particle depletion. Resampling discards low-weight particles along with the trajectory hypotheses they carried, so after a long traverse the survivors often share a recent common ancestor. The filter has forgotten the diversity it needed to correct an old error when a loop finally closes. Filtering, in short, cannot easily revise the past.

Graph-Based SLAM and the Sparsity Insight

The formulation that now dominates abandons filtering. It represents the problem as a graph whose nodes are the quantities to be estimated — robot poses at selected times, and optionally landmark positions — and whose edges are constraints derived from measurements: an odometry or scan-matching edge between consecutive poses, an observation edge between a pose and a landmark, a loop-closure edge between two poses far apart in time but near each other in space. Each edge carries a measured relative transform and an information matrix expressing how strongly to trust it. Solving the graph means finding the node configuration that minimizes the total weighted squared disagreement across all edges, a nonlinear least-squares problem attacked with Gauss-Newton or Levenberg-Marquardt.

The insight that made large-scale SLAM tractable is that the resulting linear system is extremely sparse, and sparse in a structured way. Each pose is constrained only by its immediate neighbors and the landmarks it actually saw; each landmark only by the handful of poses that observed it. The information matrix therefore has a nonzero block only where a real measurement links two variables, a vanishing fraction of all pairs. Sparse Cholesky or QR factorization, with a variable ordering chosen to limit fill-in, solves such systems in time far closer to linear than to the cubic cost of a dense factorization. The same structure supports the Schur complement trick used in bundle adjustment, where the landmark variables are eliminated in a block operation, leaving a much smaller system in the poses alone.

It is worth being precise about the claim. The equations did not change, and the objective is the one an ideal filter would optimize. What changed was the realization that the matrix had been dense only because marginalization made it dense, and that keeping past poses in the problem keeps it sparse and therefore cheap. That recognition converted SLAM from a method limited to a few hundred landmarks into one that routinely handles trajectories of many kilometers.

Incremental Solvers and Practical Libraries

Re-solving the whole graph after every measurement is wasteful, since a new observation usually perturbs only a local part of the solution. Incremental smoothing maintains a factorization and updates only the affected parts, relinearizing selectively where the estimate has moved enough to matter. The iSAM2 algorithm of Michael Kaess and colleagues, published in 2012, organizes this around a data structure called the Bayes tree and is the basis of the GTSAM library. Three open-source back ends carry most of the field's practical weight: GTSAM from the Georgia Institute of Technology, the g2o graph optimizer widely used in visual SLAM, and Google's Ceres Solver, adopted for bundle adjustment and pose-graph work. All three provide differentiation, robust loss functions, and optimization over rotation manifolds directly, rather than through parameterizations that would drift off the manifold.

Front End and Back End

Nearly every modern system splits into two parts with a clean interface between them. The front end is sensor-specific: it extracts features or preprocesses scans, tracks them between frames, selects keyframes, and performs data association, deciding which observation corresponds to which landmark and detecting when the robot has returned to a known place. Its output is a set of constraints with uncertainties. The back end is sensor-agnostic: it takes constraints and produces the configuration of poses and landmarks that best satisfies them, knowing nothing about pixels or laser returns.

The division matters for reliability. The back end is well understood and, given correct constraints, essentially solved. The front end is where the difficulty lives, because data association is a discrete decision under ambiguity, and a wrong decision hands the back end a constraint that is not merely noisy but false. Robustness work therefore concentrates on the front end, and on making the back end tolerant of its mistakes.

Robust Estimation in the Back End

Least squares assumes Gaussian noise, giving a large residual quadratic influence and letting one bad constraint dominate. Robust kernels replace the quadratic penalty with one that grows more slowly past a threshold: the Huber loss becomes linear, while the Cauchy and Geman-McClure losses saturate entirely, so a sufficiently wrong constraint contributes almost no gradient. Kernels alone do not survive a confidently wrong loop closure, which may not produce a large residual until the optimizer has already been dragged toward it. Switchable constraints, introduced by Niko Sünderhauf and Peter Protzel, attach a continuous switch variable to each loop closure and let the optimizer turn a constraint off, penalized by a prior that discourages doing so without cause. Max-mixture formulations model each constraint as a mixture of a nominal and a broad null hypothesis and select the more likely component, and dynamic covariance scaling scales each constraint's information matrix by its current residual. All three let the back end reject a bad edge instead of averaging it into the answer.

Lidar SLAM

Lidar measures range directly, removing the scale ambiguity that troubles cameras. The associated signal chain, from raw returns through point clouds, is treated under LIDAR signal processing.

Scan Matching and Iterative Closest Point

The core operation is scan matching: finding the rigid transform that best aligns a new scan with a previous scan or with the accumulated map. The iterative closest point algorithm, described independently by Paul Besl and Neil McKay and by Yang Chen and Gérard Medioni around 1991 and 1992, alternates two steps: associate each point with its nearest neighbor in the reference, then compute the transform minimizing the squared distances between the pairs. Repeat until the transform stops changing.

The variants matter. Point-to-point ICP converges slowly on smooth surfaces, because sliding along a wall costs nothing. Point-to-plane ICP minimizes the distance from each source point to the tangent plane at its target, converges far faster on structured indoor scenes, and is the usual default. Generalized ICP treats both clouds as locally planar, while the normal distributions transform models the reference as a grid of Gaussians and maximizes the likelihood of the new scan, avoiding nearest-neighbor search entirely.

ICP is a local method. It needs an initial guess good enough to land in the correct basin of attraction, which is why it is invariably seeded with odometry or an inertial prediction, and why it fails on large rotations. Nearest-neighbor search dominates its cost, so implementations rely on k-d trees, voxel hashing, or projective association exploiting the sensor's scan pattern.

Feature-Based and Lidar-Inertial Odometry

Matching every point is expensive. The LOAM approach of Ji Zhang and Sanjiv Singh, presented at Robotics: Science and Systems in 2014, classifies points by local smoothness into edge and planar features and matches only those, associating edge points to lines and planar points to planes. It also separates a fast odometry thread running at scan rate from a slower mapping thread refining against the accumulated map, a two-rate structure many later systems adopted.

A spinning lidar takes tens of milliseconds per revolution, during which a moving platform displaces the sensor and skews the scan. Correcting this motion distortion requires knowing the motion during the sweep, which an inertial measurement unit supplies directly. Tightly coupled lidar-inertial systems therefore both de-skew the scan and use the inertial data as the registration prior. LIO-SAM formulates the problem as a factor graph with pre-integrated inertial factors alongside lidar odometry and loop-closure factors. FAST-LIO2, published by Wei Xu and colleagues at the University of Hong Kong, registers raw points directly against the map with no feature extraction and maintains the map in an incremental k-d tree, the ikd-Tree, supporting insertion, deletion, and dynamic re-balancing; its authors report odometry and mapping at rates up to one hundred hertz in large outdoor environments at lower computational load than comparable systems.

Characteristic Failure Modes

Lidar SLAM fails where geometry fails to constrain the solution. A long straight tunnel, a featureless corridor, or an open parking lot leaves translation along one axis undetermined, since every candidate position produces the same scan. This geometric degeneracy is detectable, because the Hessian of the registration problem becomes ill-conditioned along the unconstrained direction, and the remedy is to freeze the degenerate directions or defer to another sensor rather than let the optimizer fill them with noise.

Beyond geometry, dust, rain, snow, and fog produce spurious returns. Reflective surfaces and glass return from the wrong place or not at all, which is why glass-walled lobbies are a recognized hazard for indoor lidar robots. Vegetation moving in wind produces returns that are geometrically real but not static, violating the rigid-world assumption underlying every scan matcher.

Visual SLAM

Cameras are cheap, light, low-power, and dense in information. They also measure bearing only, not range, which changes the character of the problem. Visual SLAM splits along a well-defined line between two families.

Feature-Based Methods

Feature-based methods detect repeatable keypoints, describe their local appearance, match descriptors across frames, and then discard the images. Estimation runs on the sparse correspondences, minimizing reprojection error: the pixel distance between where a landmark is predicted to appear and where it was observed. SIFT and SURF offered strong invariance at high cost; ORB, combining the FAST corner detector with a rotation-aware variant of the BRIEF binary descriptor, offers adequate invariance far more cheaply, and its binary form allows matching by Hamming distance, which a processor computes with a population-count instruction. That efficiency made ORB the default for real-time systems on embedded hardware.

The reference implementation of the family is the ORB-SLAM line from the University of Zaragoza. ORB-SLAM3, described by Carlos Campos, Richard Elvira, Juan J. Gómez Rodríguez, José M. M. Montiel, and Juan D. Tardós and published in IEEE Transactions on Robotics in 2021, supports monocular, stereo, and RGB-D cameras with and without an inertial unit, handles pinhole and fisheye models, and maintains multiple maps: when tracking is lost it starts a new map and merges it with an earlier one on revisiting known territory. The authors report their stereo-inertial configuration achieving an average accuracy of 3.6 centimeters on the EuRoC drone sequences and describe the system as two to five times more accurate than previous approaches. As with any published result, those figures characterize the datasets tested rather than guaranteeing field performance.

Outliers are inevitable, and the standard defense is RANSAC, the random sample consensus scheme of Martin Fischler and Robert Bolles from 1981: repeatedly fit a model to a minimal random subset, count the correspondences that agree, and keep the model with the largest consensus. The minimal solvers for camera geometry are well known, including the five-point algorithm for the essential matrix and the perspective-three-point solution for pose from known landmarks.

Direct Methods

Direct methods skip features and work on pixel intensities, estimating motion by minimizing photometric error so that a patch of one image, warped into another, matches as closely as possible. LSD-SLAM produced semi-dense maps by tracking pixels with sufficient gradient; DSO, the direct sparse odometry of Jakob Engel and colleagues, selects a well-distributed sparse set of points and jointly optimizes their inverse depths with camera poses and photometric calibration.

The trade is legible. Direct methods use regions carrying gradient but no corner, so they work in scenes too textureless for feature detectors and produce denser reconstructions. They also assume brightness constancy, which automatic exposure, rolling shutter, and changing illumination all violate, and their convergence basin is narrower. Feature-based methods tolerate large baselines and illumination change but throw away most of the image.

Monocular Scale Ambiguity

A single camera cannot recover metric scale. A large object far away and a small object nearby project identically, so the reconstruction is determined only up to an unknown scale factor, and that factor drifts over long trajectories because nothing anchors it. A stereo pair fixes scale through a known baseline, with depth uncertainty growing as the square of range, so a baseline of a few tens of centimeters gives usable depth to a few tens of meters and little beyond. An RGB-D camera measures depth directly, by structured light or by time of flight, typically over a few meters indoors and degrading badly in sunlight; the underlying sensing is covered under time-of-flight systems. An inertial measurement unit fixes scale by providing acceleration in metric units. A known object size or a known height above a flat ground plane serves as a partial remedy in automotive systems.

Visual-Inertial Odometry

Pairing a camera with an inertial measurement unit is the most productive sensor combination in the field, because their weaknesses do not overlap.

The inertial unit is fast, sampling at two hundred hertz to one kilohertz, and never loses track. It measures specific force and angular rate in metric units, supplying the scale a monocular camera lacks, and because it senses gravity it makes roll and pitch observable absolutely, anchoring two rotational degrees of freedom to the world. It works in darkness, through motion blur, and during rapid rotations that break feature tracking. What it cannot do is hold still: its biases drift with temperature and time, and integrating them produces unbounded error.

The camera is slow, typically twenty to sixty hertz, and fails in darkness or blur. But it observes static structure, and a static landmark observed twice gives a constraint that does not degrade with time. The camera arrests inertial drift and, in the process, estimates the inertial biases themselves.

Two architectures dominate. Filter-based systems descend from the multi-state constraint Kalman filter of Anastasios Mourikis and Stergios Roumeliotis, which keeps a sliding window of past camera poses in the state and uses each feature track to constrain them without adding the feature to the state vector, giving cost linear in feature count. Optimization-based systems such as OKVIS and VINS-Mono keep a sliding window of keyframes and solve a small bundle adjustment over it, using inertial pre-integration to compress the hundreds of samples between two keyframes into a single relative-motion factor with a propagated covariance. Pre-integration, formulated on the rotation manifold, made tightly coupled visual-inertial optimization practical, because it avoids re-integrating raw samples whenever the linearization point changes.

These systems have a distinctive requirement: initialization. Scale, gravity direction, initial velocity, and inertial biases are observable only under sufficient excitation. A platform translating at constant velocity gives no acceleration signal to separate bias from gravity, and a purely rotating camera gives no parallax to triangulate landmarks, which is why augmented-reality applications prompt the user to move the phone at startup. Four degrees of freedom also remain permanently unobservable: global position and rotation about the gravity vector. Well-designed estimators ensure their linearization does not accidentally make those directions appear observable, a correction known as observability-constrained or first-estimates-Jacobian filtering. The complementary discipline of inertial navigation is treated under inertial navigation systems.

Radar, Event Cameras, and Degraded Conditions

Radar penetrates conditions that stop optical sensors and measures radial velocity directly through the Doppler shift, information neither a camera nor a conventional lidar provides. Scanning millimeter-wave radar produces a rotating range-azimuth image supporting scan matching in the same spirit as lidar, and automotive units in the 76 to 81 gigahertz band produce sparse point clouds with Doppler per point. The difficulties are equally distinctive: angular resolution is coarse, multipath and specular reflection produce ghost targets at plausible ranges, and radar cross-section varies with aspect angle, so returns are far less repeatable frame to frame than optical features. Radar odometry remains attractive because its failure conditions differ from everything else in the suite, and Doppler measurements allow ego-velocity to be estimated from a single scan with no data association.

Event cameras replace the global shutter with per-pixel circuits reporting brightness changes asynchronously. The survey by Guillermo Gallego and colleagues, published in IEEE Transactions on Pattern Analysis and Machine Intelligence in 2020, describes the stream as encoding the time, location, and sign of each brightness change, with microsecond temporal resolution, very low power consumption, and dynamic range around 140 decibels against roughly 60 decibels for a conventional camera. For SLAM this means no motion blur during aggressive maneuvers and usable output where a standard sensor would be saturated at one end of the frame and black at the other.

The cost is that every algorithm must be rebuilt. There are no frames, so nothing can be run through a corner detector without first accumulating events into an image-like representation, and the accumulation window becomes a tuning parameter trading latency against signal. Events fire only where brightness changes, so a stationary camera viewing a static scene reports nothing. Event-based SLAM remains a research area rather than a deployed default. Whatever the modality mix, combining several sensors is itself a design discipline, treated under multi-sensory fusion.

Loop Closure and Place Recognition

Everything so far still drifts, because odometry constraints relate only nearby poses and their errors accumulate along the chain. Loop closure breaks the chain. When the robot recognizes a place it has visited, it adds a constraint directly between two poses separated by a long stretch of trajectory, and the optimizer distributes the accumulated error backward across the whole loop. A trajectory that had drifted several meters over a few hundred snaps into consistency in a single optimization. Without loop closure there is odometry; with it, there is SLAM.

Appearance-Based Place Recognition

Recognition normally proceeds by appearance rather than geometry, because a geometric search over the whole map is too expensive. The classical approach is the bag of visual words: quantize local descriptors into a vocabulary learned offline by clustering, represent each image as a histogram of word occurrences weighted by inverse document frequency, and query an inverted index. The DBoW2 library of Dorian Gálvez-López and Juan Tardós made this practical with binary descriptors and a hierarchical vocabulary tree, and it underlies the loop-closure module of ORB-SLAM and many derivatives. FAB-MAP, from the University of Oxford, added a probabilistic model of word co-occurrence that reasons explicitly about how distinctive an appearance is, since a picture of a blank corridor is evidence of very little.

Learned global descriptors now compete strongly. NetVLAD, introduced by Relja Arandjelović and colleagues in 2016, trains a network with a differentiable analogue of the classical vector of locally aggregated descriptors, producing one compact vector per image compared by dot product. Learned local features and matchers, notably SuperPoint and SuperGlue, improved the verification step that follows. Lidar has its own descriptors, of which Scan Context, encoding a scan as a polar height map, is the most widely used.

Geometric Verification and the Asymmetry of Errors

Appearance matching alone is not trustworthy, so every serious system verifies a candidate geometrically: compute a relative transform from the feature correspondences under RANSAC and require a minimum number of inliers consistent with a single rigid motion. Systems commonly add temporal consistency, requiring several consecutive queries to agree on the same region, and a covisibility check requiring that the matched keyframes share observed landmarks.

The most important practical fact about loop closure is that its two error types differ enormously in cost. A missed detection is a lost opportunity: the drift that would have been corrected persists, and the next visit may catch it. A false positive is a catastrophe: the optimizer receives a constraint asserting that two different places are the same, expressed with the same confidence as a true one, and folds the map onto itself. A corridor may be duplicated or bent through a wall, and the damage is not local, because the erroneous constraint propagates through the whole graph.

Every design decision in the pipeline follows from this asymmetry. Thresholds are conservative, verification is layered, acceptance requires agreement across independent checks, and the back end is armed with the robust methods described earlier so that a false positive slipping through can still be switched off rather than obeyed. The underlying hazard is perceptual aliasing: many real environments contain places that genuinely look identical, including repeated office bays, warehouse aisles, and forest paths. A system relying on appearance alone will eventually be fooled.

Map Representations

The word "map" covers several distinct data structures, chosen by what the map is for: a map used only to relocalize needs different content from one used to plan a collision-free path.

Landmark and feature maps store a sparse set of three-dimensional points with descriptors. They are compact, support relocalization directly, and are what a feature-based visual system naturally produces, but they are useless for obstacle avoidance, because the absence of a landmark says nothing about whether space is free. Point clouds store raw or downsampled measurements: simple and lossless within the sampling resolution, but unbounded in growth and carrying no notion of free space.

Occupancy grids discretize space into cells holding the probability of occupancy, updated in a log-odds form so that evidence accumulates by addition, with each sensor ray marking the cells it passes through as free and its endpoint as occupied. This explicit representation of free space makes occupancy grids the standard input to path planning. In two dimensions they are cheap; in three a dense grid becomes prohibitive, since memory grows with the cube of the inverse resolution. OctoMap, from the University of Freiburg, addresses this with an octree storing only observed regions and pruning uniform subtrees, and it distinguishes occupied, free, and genuinely unknown space, a distinction that matters for exploration.

Signed distance fields store at each voxel the distance to the nearest surface, with sign indicating inside or outside. Truncated variants, storing the value only within a band around the surface, became the standard representation for dense depth-camera fusion after the KinectFusion work of 2011. Their appeal is threefold: fusing a new depth image is a weighted average per voxel, so noise averages out across views; the surface is extracted at sub-voxel precision as the zero crossing; and the distance value is exactly what a trajectory optimizer needs to keep a robot clear of obstacles.

Topological and hybrid maps abandon a single global metric frame, storing places as nodes and connections as edges with local metric maps attached. This scales gracefully and matches how navigation actually works: a delivery robot needs to know precisely where the door frame is and only approximately where the building is. Learned implicit maps are the newest entrant, with neural radiance fields and three-dimensional Gaussian splatting representing geometry and appearance as differentiably rendered parameters, coupled since 2022 to pose estimation for photorealistic dense SLAM. Their standing is unsettled: compute and memory demands are high, behavior outside observed viewpoints is not guaranteed, and the loop-closure and consistency machinery that classical representations inherited from decades of work is only beginning to be rebuilt for them.

Computation, Memory, and Power on Embedded Hardware

A system running at thirty frames per second on a workstation may be infeasible on the processor inside a vacuum cleaner or a drone, and these constraints shape designs more often than algorithmic elegance does.

Latency, not throughput, sets the safety limit. A pose estimate arriving two hundred milliseconds late describes where the robot was, not where it is, and at two meters per second that is forty centimeters of error from scheduling alone. Systems therefore split into a fast tracking thread that must meet a deadline every frame and a slower mapping or optimization thread that may take longer, with tracking never blocking on the optimizer. ORB-SLAM's separation of tracking, local mapping, and loop closing into three threads is the canonical arrangement.

Memory bounds the map. A dense three-dimensional map at five-centimeter resolution consumes gigabytes quickly, so practical systems bound growth aggressively: culling redundant keyframes, voxel-downsampling incoming clouds, pruning landmarks observed too few times, and sliding-window marginalization that converts old variables into a prior factor rather than keeping them.

Arithmetic is heterogeneous. Feature detection and descriptor extraction are regular and parallel, mapping well onto vector instructions, a graphics processor, or a vision accelerator. Sparse factorization is irregular and memory-bound and runs best on the general-purpose cores. Robot computers are therefore heterogeneous by design, pairing application cores with a vision processing unit or neural accelerator; the hardware side is covered under edge AI processors. Purpose-built visual-inertial modules appear periodically, such as Intel's RealSense T265 tracking camera, which paired fisheye cameras and an inertial unit with an onboard vision processor; Intel discontinued the line, a useful reminder that dedicated SLAM silicon has repeatedly struggled against general-purpose compute.

Power turns into heat and shortens missions. On a battery platform every watt spent on perception is a watt unavailable for motion, and inside a sealed enclosure it is also a thermal problem. Consumer robot vacuums perform useful SLAM within a few watts by choosing cheap sensing and modest map resolution, a legitimate engineering answer rather than a compromised one. Toolchains and simulators that make these trade-offs testable before hardware exists are surveyed under robotics development platforms.

Dynamic Environments and Long-Term Autonomy

Classical SLAM assumes a static, rigid world observed by a moving sensor. Real environments contain people, vehicles, doors, pallets, and seasons, and every one of them violates the assumption.

Moving Objects and Map Maintenance

A moving object generates measurements no static map can explain. Fused, they corrupt both the map and the pose estimate; a robot following a truck at constant relative velocity can convince itself that it is stationary. Three responses are common. Statistical rejection treats inconsistent measurements as outliers, which works when moving objects occupy a small fraction of the view and fails in a crowd. Semantic masking uses a learned segmentation network to remove classes known to move, effective but discarding a parked car that would have been an excellent landmark. Explicit tracking estimates the state of each dynamic entity alongside the robot, more capable and considerably more expensive.

A subtler problem is semi-static structure: the chair moved overnight, the pallet stack present on Tuesday and gone on Wednesday. Such objects are static within a session and unreliable across sessions, and the usual treatment is a persistence score per map element, downweighting elements repeatedly observed to be absent. A map is in general a perishable asset, since lighting changes between morning and afternoon, seasons change vegetation, and buildings are renovated. Appearance-based relocalization degrades sharply under such change, which is why lidar and radar maps age better than visual ones, and why experience-based approaches store several appearances of the same place rather than a single canonical one.

Relocalization and the Kidnapped Robot

The kidnapped-robot problem asks what happens when a robot with an existing map is moved without being told, or wakes up somewhere in a map it has not yet localized within. It differs from ordinary tracking because there is no prior on the pose: the search is global rather than local. The same machinery used for loop closure serves here, since recognizing where you are in a known map and recognizing a return to a known place are the same computation with a different trigger.

Detecting the condition matters as much as recovering from it. A system that has silently lost track and continues to report poses is more dangerous than one reporting failure, so implementations monitor inlier counts, residual magnitudes, and agreement between independent motion sources, and declare loss of tracking rather than reporting a confident and wrong pose. Multi-map approaches such as ORB-SLAM3's turn the failure into a manageable event by starting a fresh map on loss and merging it later.

Multi-Session and Multi-Robot Mapping

Merging maps built at different times, or by different robots, requires finding correspondences between them and computing the transform aligning their frames, followed by a joint optimization over the union of the graphs. The recognition problem is the place recognition described earlier, but the consequences of an error are worse, because a false match between two robots' maps corrupts both.

Multi-robot SLAM adds a communication constraint that reshapes the algorithms. Sending raw sensor data over a shared radio link does not scale, so teams exchange compressed descriptors and candidate constraints, and distributed optimization lets each robot solve its own portion of the graph while exchanging only boundary variables. Double counting is a genuine hazard: if robot A tells robot B something B originally told A, and neither tracks provenance, the shared estimate becomes overconfident. Consistent distributed estimation therefore requires either careful bookkeeping of information sources or conservative fusion rules. These issues connect to the coordination problems treated under swarm robotics.

Calibration and Time Synchronization

More SLAM deployments fail on calibration than on algorithms, and the failures are frustrating because they present as vague inaccuracy rather than as an obvious fault.

Intrinsic calibration describes each sensor in isolation: focal lengths, principal point, and lens distortion for a camera; scale factors, biases, axis misalignments, and temperature coefficients for an inertial unit. A calibration error of a few pixels at the image edge produces a systematic reprojection residual the estimator cannot distinguish from motion, so it absorbs it into the trajectory.

Extrinsic calibration describes the rigid transform between sensors, and it is where most of the pain lives, because the errors grow with range. A camera-to-lidar rotation error of one degree displaces a point at fifty meters by nearly a meter, and an inertial-to-camera translation error of one centimeter produces a spurious acceleration whenever the platform rotates, since the lever arm converts angular acceleration into linear acceleration at the sensor. Toolboxes such as Kalibr, from the Autonomous Systems Lab at ETH Zurich, estimate camera intrinsics, camera-to-inertial extrinsics, and the temporal offset between the streams from a single recorded sequence of a calibration target.

Temporal calibration is the requirement most often underestimated. Fusing a camera frame with an inertial sample is meaningful only if both timestamps refer to the same clock and the same physical instant. A constant offset of ten milliseconds, easily acquired through driver buffering, produces an error proportional to velocity: at two meters per second that is two centimeters of systematic displacement on every measurement, and the estimator will invent a bias to explain it. The remedies are architectural. Hardware triggering drives exposure from the same signal that timestamps the inertial samples, timestamps are recorded at the mid-exposure instant rather than on arrival at the host, and networked sensors are disciplined to a common clock with the Precision Time Protocol of IEEE 1588 or its Ethernet profile in IEEE 802.1AS. Where none of that is possible, the offset is treated as an unknown and estimated online, which many modern visual-inertial systems now do as a matter of course.

Rolling shutter deserves separate mention. Such a camera exposes rows sequentially over several milliseconds, so one image is not one viewpoint, and ignoring this on a fast platform introduces skew the estimator interprets as scene geometry. The options are a global-shutter sensor, the usual recommendation for any serious system, or explicit modeling of the row-dependent exposure time in the projection function.

Evaluation and Where the Field Stands

Two error metrics are standard. Absolute trajectory error aligns the estimated and reference trajectories with a single rigid transform and reports the residual position error, capturing global consistency and dominated by loop-closure quality. Relative pose error measures drift accumulated over fixed intervals of time or distance, capturing local odometry quality and insensitive to a global misalignment. A system may be excellent on one and poor on the other.

Public benchmarks made the field comparable. The KITTI suite from Karlsruhe Institute of Technology and the Toyota Technological Institute at Chicago provided synchronized stereo, lidar, and satellite-inertial ground truth from an instrumented car and became the reference for outdoor driving. The EuRoC micro aerial vehicle datasets from ETH Zurich became the standard for visual-inertial evaluation, and the TUM RGB-D benchmark from the Technical University of Munich served the indoor depth-camera community. Later collections, including the Newer College dataset and the Hilti SLAM challenges, deliberately targeted the conditions that break systems: construction sites, stairwells, darkness, and geometric degeneracy.

An honest reading of the current state runs roughly as follows. The estimation back end is mature: sparse nonlinear least squares over a factor graph, with robust kernels and incremental relinearization, is well understood with several solid open-source implementations, and it is rarely the reason a system fails. Within a bounded, static, well-textured, well-lit environment, with calibrated and synchronized sensors, SLAM is essentially a solved engineering problem, which is why robot vacuums, warehouse vehicles, and smartphone augmented-reality frameworks such as Apple's ARKit and Google's ARCore ship it to consumers by the million.

The unsolved parts are the ones that led the 2016 survey by Cesar Cadena, Luca Carlone, and colleagues to call for a "robust-perception age" rather than declare victory: operation over months rather than minutes, environments that change, failure detection with performance guarantees, semantic understanding of what the map contains, and reasoning about where to go next in order to map well. Those problems remain open a decade later, because they are not primarily estimation problems; they concern data association, representation, and the ability of a system to know when it is wrong.

The learned-component turn is the most consequential recent shift. Learned feature detectors and descriptors, place-recognition embeddings, depth priors that give a monocular system usable scale, dynamic-object masks, and end-to-end differentiable architectures have all shown gains, and hybrid systems that keep a geometric back end while learning the front end are now common. The gains are real, and so is the evaluation difficulty. A learned component is characterized by the data it was trained on, and the standard benchmarks are small enough that a system tuned against them may not generalize to a different building, camera, or climate. Reported accuracy on a public sequence is therefore a weaker guarantee than it appears, and cross-dataset generalization, honest uncertainty estimates, and behavior under distribution shift deserve scrutiny before a learned front end is trusted near people.

Conclusion

SLAM exists because the two obvious ways to know where you are both fail. Integrating motion is accurate over seconds and unbounded over hours; measuring against fixed references is exact but requires infrastructure absent wherever autonomy is most valuable. The field's contribution is to make the environment its own reference, and its central technical insight, arrived at over three decades, is that the joint estimation problem this creates is sparse, and that keeping the full trajectory in the problem is what keeps it sparse.

The frameworks form a coherent progression. The extended Kalman filter established that correlations between landmarks must be maintained, and revealed the quadratic cost and linearization inconsistency that follow from maintaining them densely. Rao-Blackwellized particle filters escaped the cost by conditioning on trajectories, at the price of being unable to revise the past. Graph-based formulations moved to smoothing over the full trajectory, where sparse solvers make large problems affordable, and around that back end grew a division of labor between a sensor-specific front end and a sensor-agnostic optimizer, with robust kernels and switchable constraints as insurance against front-end mistakes.

The sensor choices are complementary rather than competing, and the practical answer in almost every serious system is a fused suite whose value depends less on any one sensor than on calibration and synchronization good enough that the measurements refer to the same instant and the same frame. Loop closure converts drift into bounded error, and it is where the risk concentrates, because a false positive damages the map far more than a missed detection costs. The remaining engineering — bounded memory, layered threading, dynamic-object handling, map maintenance, relocalization, and multi-session merging — is what separates a demonstration that runs for two minutes from a product that runs for two years. That distinction, rather than any new estimator, is where most current work in the field is honestly located.

Related Topics