Electronics Guide

Sources of Verification Requirements

The primary source of verification requirements is the design specification, which describes the intended functionality of the hardware. Each functional requirement in the specification implies a corresponding verification requirement to demonstrate that the implementation meets the specification. However, specifications often contain ambiguities, implicit assumptions, and undefined corner cases that must be clarified during requirements analysis.

Architectural documents provide verification requirements related to performance, power consumption, timing constraints, and interface protocols. These requirements often span multiple design blocks and require system-level verification approaches. Compliance with industry standards such as USB, PCIe, or DDR memory interfaces imposes specific verification requirements defined by the standard bodies, often including mandatory compliance test suites.

Customer requirements may add verification obligations beyond what the specification explicitly states. Automotive customers, for example, may require functional safety verification per ISO 26262, while aerospace applications demand verification according to DO-254. These domain-specific requirements significantly impact verification methodology and resource requirements.

Historical data from previous projects provides valuable verification requirements based on past experience. Bug patterns from similar designs indicate areas requiring focused attention. Field failures from previous generations highlight verification gaps that must be addressed. Lessons learned documents capture verification techniques that proved effective or ineffective.

Requirements Decomposition and Traceability

High-level requirements must be decomposed into verifiable items that can be mapped to specific tests or checks. A requirement stating "the processor shall execute all ARM instructions correctly" must be broken down into requirements for each instruction class, addressing mode, and exception condition. This decomposition continues until reaching requirements specific enough to guide test development.

Traceability links each verification activity back to the requirements it addresses. This bidirectional traceability ensures that every requirement has corresponding verification coverage and that every verification effort serves a clear purpose. Traceability matrices document these relationships and enable impact analysis when requirements change.

Requirements management tools facilitate this process by maintaining the requirements database, tracking decomposition hierarchies, and managing traceability links. These tools integrate with verification environments to automatically update coverage status as tests execute, providing real-time visibility into verification progress against requirements.

Requirements Prioritization

Not all requirements carry equal weight, and verification resources must be allocated according to priority. Critical requirements affect safety, security, or core functionality and demand the most rigorous verification. These requirements typically require multiple independent verification approaches, such as simulation combined with formal verification.

Risk-based prioritization considers both the probability of defects and the impact of those defects. Complex features with intricate state machines are more likely to contain bugs than simple combinational logic. Features used in safety-critical applications have higher impact if defective. The product of probability and impact guides resource allocation.

Schedule constraints influence prioritization by identifying features on the critical path. Features required early for software development or system integration must be verified first, even if they are not the highest risk. Conversely, features with later integration dates can be verified in later project phases.

Code Coverage

Code coverage measures how thoroughly tests exercise the RTL implementation. Statement coverage tracks which lines of code have been executed. Branch coverage ensures that all conditional branches have been taken in both directions. Expression coverage verifies that all combinations of conditions in complex expressions have been evaluated. Toggle coverage confirms that all signals have transitioned between logic states.

While code coverage is relatively easy to measure automatically, high code coverage does not guarantee verification quality. Achieving 100% code coverage means every line of code has executed, but not that every line executes correctly under all relevant conditions. Code coverage should be viewed as a necessary but not sufficient condition for verification completeness.

Code coverage exclusions require careful management. Dead code that cannot be reached due to constant propagation or disabled features should be explicitly excluded with documented justification. Legitimate exclusions include unused configuration options, synthesis-only code paths, and defensive code intended to catch impossible conditions.

Functional Coverage

Functional coverage measures whether tests have exercised the design functionality as intended, independent of implementation details. Unlike code coverage which is automatically derived from the code structure, functional coverage is explicitly defined based on design intent and verification requirements.

Coverpoints define individual items to be covered, such as specific values of signals, states of state machines, or ranges of parameters. A coverpoint for an ALU might track which operations have been tested: ADD, SUB, AND, OR, XOR, and so forth. Bins within coverpoints group related values, such as edge cases (minimum, maximum, zero) versus typical values.

Cross coverage captures combinations of coverpoints that must occur together. For the ALU example, cross coverage might track which operations have been tested with which source operand ranges, ensuring that ADD has been tested with small numbers, large numbers, and overflow-inducing combinations. Cross coverage exposes interaction effects that individual coverpoints miss.

Transition coverage tracks sequences of states or values, important for verifying protocol compliance and state machine behavior. A bus protocol might require coverage of all legal transaction sequences: read following read, write following read, read following write, and write following write, each with various timing relationships.

Assertion Coverage

Assertion coverage measures how effectively tests exercise embedded assertions. Assertions encode design intent and correct behavior as executable checks, and assertion coverage ensures that tests create conditions where assertions can detect violations.

Assertion success coverage tracks assertions that have been triggered and passed. An assertion stating "grant shall follow request within 10 cycles" should be covered by tests that exercise the request-grant protocol, confirming that the design correctly generates grants in response to requests.

Assertion failure coverage, paradoxically, verifies that assertions can actually detect failures. By temporarily injecting errors or using formal techniques, verification engineers confirm that assertions would catch the bugs they are intended to detect. An assertion that never fires might indicate either a bug-free design or an ineffective assertion.

Cover properties, a variant of assertions, specify conditions that should be reachable and serve as verification targets. Unlike assertions that check invariants, cover properties confirm that interesting scenarios have been explored. A cover property might specify "request occurred while buffer is full" to ensure testing of back-pressure handling.

Setting Coverage Goals

Coverage goals define the numeric thresholds that indicate verification completion. These goals should be specific, measurable, and aligned with project risk tolerance. A goal of "100% functional coverage" means nothing without defining what functional coverage items must be covered.

Goal setting requires balancing completeness against feasibility. Demanding 100% coverage of all cross-products may be mathematically impossible to achieve in available test time. Goals should identify mandatory coverage items that must reach 100% and optional items where lower thresholds are acceptable with justification.

Coverage holes, items not yet covered, require explicit closure. Each uncovered item must be either covered by additional tests, proven unreachable through analysis, or explicitly waived with documented rationale and risk acceptance. Coverage reports without disposition of holes provide incomplete verification evidence.

Test Architecture

The test architecture defines the structural organization of tests. Unit tests verify individual modules in isolation, providing fast feedback during development and detailed debug visibility. Integration tests verify interactions between modules, catching interface mismatches and integration bugs. System tests verify end-to-end functionality in a realistic context.

The testing pyramid principle suggests that most tests should be low-level unit tests, with fewer integration tests and still fewer system tests. Unit tests are fast to run, easy to debug, and provide precise fault localization. System tests are slower and harder to debug but verify real-world behavior more accurately. A balanced test portfolio leverages the strengths of each level.

Directed tests explicitly specify input sequences to exercise particular scenarios. They provide predictable, reproducible verification of specific requirements but require significant development effort and may miss unexpected corner cases. Random tests generate stimuli algorithmically, potentially discovering bugs that directed tests miss, but require more sophisticated checking and may duplicate coverage.

Constrained random testing combines the strengths of both approaches by generating random stimuli within specified constraints that ensure legal, interesting scenarios. Constraints might specify that all transactions are valid, that certain configurations are exercised, or that stress conditions are created. Coverage-driven random testing uses coverage feedback to guide random generation toward uncovered scenarios.

Test Sequence Organization

Tests must be organized for efficient execution and meaningful reporting. Test suites group related tests for batch execution, such as all tests for a particular feature or all regression tests that must pass before check-in. Test sequences define ordering when tests have dependencies or when progressive testing is desired.

Smoke tests provide quick sanity checking that basic functionality works before investing time in comprehensive testing. These tests should run in minutes and catch obvious failures that would doom longer tests. Nightly regression suites run more comprehensive tests that complete overnight, catching bugs introduced during the day's development.

Full regression tests exhaustively exercise all functionality and may run for days or weeks. These tests typically run before major milestones or tape-out and must be carefully scheduled to complete within project constraints. Incremental coverage analysis identifies tests that add unique coverage, enabling pruning of redundant tests from long-running regressions.

Test Case Specification

Each test case should be documented with sufficient detail for implementation and review. The test case specification includes the test objective, prerequisites, stimulus description, expected results, and coverage contribution. Clear specification enables test review before implementation and maintenance after initial development.

Test objectives link test cases back to verification requirements, establishing traceability. A test objective might state "verify that interrupt requests are prioritized correctly when multiple interrupts occur simultaneously," directly addressing a specific functional requirement.

Expected results must be unambiguous and checkable. Self-checking tests embed expected results in the test itself, automatically flagging failures without manual inspection. Reference models provide expected results by implementing the specification independently from the design, enabling comparison of design output against reference output.

Environment Architecture

Modern verification environments typically follow the Universal Verification Methodology (UVM) or similar structured approaches. The environment architecture defines major components: drivers that convert transactions to signal-level activity, monitors that observe interfaces and create transactions, scoreboards that compare actual results against expected results, and coverage collectors that track verification progress.

Reusable verification components, known as Verification IP or VIP, provide pre-verified environment infrastructure for standard interfaces. Rather than developing a PCIe or USB verification environment from scratch, teams leverage VIP from commercial vendors or internal libraries. Environment planning must identify required VIP, evaluate options, and plan integration.

Environment configuration mechanisms enable the same environment to support multiple test scenarios. Configuration objects parameterize environment behavior, such as enabling different protocol modes, adjusting timing, or selecting between directed and random stimulus. A flexible configuration architecture reduces the number of distinct environments that must be maintained.

Debug infrastructure must be planned alongside functional infrastructure. Waveform dumping, transaction logging, assertion messaging, and coverage visualization all require implementation and integration. Debug capabilities are often neglected during initial planning, leading to expensive retrofitting when bugs prove difficult to diagnose.

Stimulus Planning

Stimulus planning defines what types of input sequences tests will apply and how those sequences will be generated. For random testing, this includes defining constraints, weighting distributions, and coverage-driven feedback mechanisms.

Protocol compliance requirements dictate many stimulus characteristics. A memory controller test environment must generate only legal DDR transactions with correct timing relationships. An Ethernet environment must generate frames with valid format, including preambles, addresses, and checksums. Stimulus generators must be validated against protocol specifications before testing begins.

Corner case identification guides stimulus development toward high-value scenarios. Boundary values, maximum sizes, minimum intervals, and unusual but legal combinations often reveal bugs that typical scenarios miss. The stimulus plan should explicitly list corner cases to be covered and how they will be generated.

Error injection capabilities enable testing of error handling paths. The stimulus plan should specify what errors can be injected, how injection is controlled, and what design responses are expected. Error injection might include data corruption, timeout simulation, protocol violations, and resource exhaustion.

Checking Strategy

The checking strategy defines how correct design behavior will be verified. Passive checking monitors design outputs and flags deviations from expected behavior. Active checking compares design outputs against an independent reference model that computes expected results.

Reference models may be implemented at various abstraction levels. Transaction-level models are fastest but may miss timing-related bugs. Cycle-accurate models catch timing issues but are slower to develop and execute. C or SystemC models can often be shared with software teams, enabling co-verification of hardware and software.

Assertion-based checking embeds correctness properties in the design or environment. Assertions continuously monitor design behavior and immediately flag violations. Effective assertion development requires design understanding and specification analysis to identify properties that should always hold.

Protocol checkers verify compliance with interface specifications. These checkers may be part of VIP or developed specifically for internal interfaces. Protocol checking catches interface violations that might otherwise propagate into confusing downstream failures.

Environment Development Schedule

Environment development must complete in time to support test execution, but environment work competes with other activities for engineering resources. The environment schedule should identify dependencies and critical path items.

Many environment components are needed before design code is stable. Drivers, monitors, and basic environment infrastructure can be developed based on interface specifications before RTL is available. This early development enables testing to begin immediately when RTL arrives, accelerating the verification schedule.

Incremental environment delivery enables early testing while development continues. An initial environment might support basic functionality with limited checking, while subsequent versions add more sophisticated stimulus generation, comprehensive checking, and coverage collection. This incremental approach provides early bug detection while the environment matures.

Staffing Requirements

Verification staffing must account for the diverse skills required across the verification lifecycle. Environment development requires expertise in verification methodology, programming, and design understanding. Test development requires feature knowledge and testing creativity. Debug and analysis require deep design understanding and systematic problem-solving skills.

Staff ramp-up time affects when resources become productive. Engineers new to a project require time to learn the design, environment, and methodology before contributing effectively. Training requirements for new methodologies or tools also consume schedule time. Resource plans should account for these productivity ramps.

Peak staffing typically occurs during intensive verification phases before major milestones. Projects should plan for staff augmentation during these phases through contractors, temporary reassignment, or offshore resources. Managing distributed teams introduces coordination overhead that should be included in resource estimates.

Compute Infrastructure

Simulation throughput often limits verification progress. Compute resource planning must estimate simulation requirements and ensure sufficient capacity. Factors affecting compute requirements include design size, simulation speed, test count, and required turnaround time.

Regression execution typically runs on compute farms shared across projects. Resource planning must ensure adequate allocation of farm resources and plan for peak demand periods when multiple projects approach milestones simultaneously. Cloud computing provides elastic capacity for handling peak loads.

Formal verification and emulation have distinct compute requirements. Formal tools require specialized licenses and may benefit from high-memory servers. Emulators represent major capital investments with capacity constraints that must be scheduled across projects.

Tool and Methodology Requirements

Verification requires numerous EDA tools including simulators, formal verification tools, coverage analysis tools, debug tools, and regression management systems. Resource planning must identify tool requirements and ensure license availability.

Tool evaluation and deployment consume calendar time before verification can begin. New tool versions may require environment updates and revalidation. Methodology infrastructure such as VIP and standard libraries requires procurement or development. These preparation activities should be scheduled early in the project.

Training investments improve tool and methodology effectiveness. Teams adopting new methodologies like UVM or new techniques like formal verification require training before becoming productive. Training costs include course fees, engineering time, and productivity impact during learning curves.

Budget Allocation

Verification budgets must cover staffing costs, tool licenses, compute infrastructure, VIP procurement, and training. Budget planning at project start enables appropriate resource commitment and reduces mid-project surprises.

Trade-offs between investment and schedule are often possible. Additional staff can accelerate verification but increase cost. Emulation can reduce verification time but requires capital investment. Resource planning should identify these trade-offs and present options to project leadership.

Standard Verification Milestones

Common verification milestones aligned with design milestones include RTL freeze, gate-level verification complete, and tape-out. Each milestone has specific entry and exit criteria that define what must be completed and verified before the milestone is achieved.

Environment ready milestones confirm that verification infrastructure is in place before intensive testing begins. An environment might be declared ready when it successfully runs basic directed tests, generates legal random traffic, and collects coverage without errors. Environment readiness gates the transition from environment development to test execution.

Feature complete milestones mark when verification of specific features has achieved required coverage. These milestones enable integration decisions, such as declaring a block ready for chip-level integration or a feature ready for software development. Feature complete criteria typically include functional coverage goals, code coverage goals, and no open high-priority bugs.

Regression stable milestones confirm that the test suite executes reliably and that design quality has reached a predictable level. A stable regression shows consistent pass rates, no tests timing out or hanging, and new bugs appearing at a manageable rate. Regression stability typically precedes final verification push.

Milestone Criteria

Each milestone requires explicit entry and exit criteria. Entry criteria define prerequisites that must be satisfied before milestone activities begin. Exit criteria define achievements that must be demonstrated before the milestone is declared complete.

Quantitative criteria provide objective measurement. Examples include: functional coverage exceeds 95%, code coverage exceeds 90%, all critical assertions pass, no priority-one bugs open, and regression pass rate exceeds 99%. These numeric thresholds reduce ambiguity in milestone decisions.

Qualitative criteria address aspects not easily quantified. Examples include: all verification requirements traced to tests, all coverage exclusions reviewed and approved, verification plan reviewed and updated, and stakeholder sign-off obtained. Qualitative criteria require judgment but capture important verification quality aspects.

Milestone reviews bring together verification leads, design leads, and project management to assess criteria achievement and make milestone decisions. Reviews should follow a standard agenda addressing each criterion, with clear documentation of status and any waivers granted.

Schedule Integration

Verification milestones must align with overall project schedule and dependencies. Design milestones that produce testable RTL enable verification milestones that consume that RTL. Verification milestones that declare features complete enable integration milestones that depend on verified features.

Schedule buffers should protect critical path milestones from typical verification schedule risks. Bug discovery rates are inherently unpredictable, and late-found bugs can delay milestones significantly. Explicit buffer allocation acknowledges this uncertainty and protects downstream activities.

Milestone tracking provides early warning of schedule problems. If interim metrics show coverage growth rate insufficient to meet milestone goals, corrective action can be taken. Regular milestone progress reviews enable adaptive management response to emerging risks.

Technical Risks

Design complexity risks arise from features that are difficult to verify thoroughly. Complex state machines, intricate timing relationships, and subtle protocol requirements all increase the probability of verification gaps. Mitigation includes allocating additional resources to high-complexity areas and employing multiple verification techniques.

New technology risks affect projects using unfamiliar design techniques, tools, or methodologies. Teams verifying their first ARM processor, first use of formal verification, or first implementation of a new protocol face learning curves and unexpected problems. Mitigation includes early training, prototype activities, and expert consultation.

Verification environment risks include the possibility that the environment itself contains bugs that mask design bugs or create false failures. Environment validation through known-good designs, targeted fault injection, and environment self-checks reduces this risk.

Coverage adequacy risks reflect uncertainty about whether defined coverage goals truly represent thorough verification. Coverage metrics can be gamed or may not capture important scenarios. Mitigation includes coverage model review by independent engineers and correlation of coverage with bug detection effectiveness.

Resource Risks

Staff availability risks include key personnel leaving, reassignment to higher-priority projects, or extended absence. Single points of failure where only one person understands a critical area create vulnerability. Mitigation includes cross-training, documentation, and succession planning.

Compute capacity risks affect projects with large simulation requirements. Shared compute farms may become overloaded during peak periods. Mitigation includes early capacity planning, cloud burst capability, and simulation optimization to reduce compute requirements.

Tool and license risks include tool bugs that block verification progress, license server failures, and inadequate license counts for team size. Mitigation includes vendor support relationships, backup tools, and license usage monitoring.

Schedule Risks

Design instability risks affect verification when design changes occur faster than tests can be updated or when frequent design bugs create verification backlog. Mitigation includes design quality gates, change control processes, and environment architecture that tolerates design changes.

Integration risks arise when block-level verification succeeds but system-level problems emerge during integration. Interface mismatches, assumption conflicts, and emergent behaviors may require significant rework. Mitigation includes early integration testing and interface verification between blocks.

Late bug discovery risks include the possibility of finding critical bugs close to tape-out with insufficient time for proper fixes. Mitigation includes front-loading verification effort, continuous regression testing, and schedule reserves for late bug fixing.

Risk Tracking and Response

Risk registers document identified risks, their probability and impact assessments, planned mitigations, and current status. Regular risk review meetings assess risk status, identify new risks, and adjust mitigations as needed.

Risk triggers are observable events that indicate a risk is materializing. For example, declining coverage growth rate might trigger schedule risk response. Defined triggers enable prompt response before risks fully impact the project.

Contingency plans define responses when risks materialize despite mitigation. These plans might include schedule adjustments, scope reductions, or resource reallocation. Having contingencies prepared enables rapid response when needed.

Coverage Closure

Coverage closure demonstrates that all defined coverage goals have been achieved. Functional coverage should meet specified thresholds with all coverage holes either covered or explicitly waived. Code coverage should meet thresholds with excluded code documented and justified.

Coverage closure documentation provides auditable evidence of coverage achievement. This includes coverage reports, hole analysis, exclusion justifications, and waiver approvals. The documentation should enable an independent reviewer to understand what was covered, what was not, and why.

Coverage correlation validates that coverage metrics accurately represent verification quality. Correlation with bug detection rates provides confidence that high coverage corresponds to thorough verification. Low correlation might indicate coverage model deficiencies requiring correction.

Bug Closure

Bug closure criteria define acceptable bug status for sign-off. Typically, all priority-one bugs must be fixed and verified, and all priority-two bugs must be fixed or have approved waivers. Lower priority bugs may remain open with documented plans for post-tape-out resolution.

Bug trend analysis provides confidence that bug discovery has converged. A design approaching sign-off should show declining bug discovery rate, indicating that major issues have been found and fixed. Rising or flat bug rates suggest verification is not complete.

Escape analysis estimates the probability of bugs remaining undiscovered. Based on bug discovery history and coverage data, statistical methods can estimate residual bug counts. Acceptable escape rates depend on product requirements and cost of field failures.

Regression Status

Regression pass rates must meet defined thresholds. A sign-off criterion might require 100% pass rate on the full regression suite, with any failures investigated and shown to be test issues rather than design bugs. Flaky tests that pass sometimes and fail sometimes should be resolved or excluded.

Regression stability over time provides confidence that the design is mature. The same regression suite should produce consistent results across multiple runs. Intermittent failures may indicate timing sensitivities, race conditions, or other lurking bugs.

Performance regression ensures that timing and throughput meet requirements. Beyond functional correctness, performance verification confirms that the design meets speed, latency, and bandwidth specifications under realistic operating conditions.

Review and Approval Process

Sign-off reviews formally assess criteria achievement and authorize design release. Reviews should include verification leads, design leads, project management, and quality assurance representatives. The review agenda systematically addresses each sign-off criterion.

Documentation requirements for sign-off typically include the verification plan showing completion, coverage reports with closure analysis, bug tracking reports, regression summary, and risk assessments. This documentation becomes part of the project record and may be required for regulatory compliance.

Waivers for unmet criteria require explicit approval with documented rationale. Waivers might be granted for low-risk items when schedule pressure is high, but the waiver process ensures that deviations from criteria are visible decisions rather than oversights. Waiver authority should be defined and limited to appropriate management levels.

Post-silicon validation planning acknowledges that silicon testing may reveal issues not caught during pre-silicon verification. The sign-off process should confirm that post-silicon validation plans are in place and that resources are committed to address any issues discovered after fabrication.

Document Structure

A typical verification plan includes sections for design overview, verification scope, verification approach, coverage strategy, resource requirements, schedule, risks, and sign-off criteria. Each section should be detailed enough to guide implementation while remaining readable for stakeholders who need an overview.

Requirements traceability matrices link design requirements to verification activities. These matrices may be maintained as separate documents or spreadsheets that accompany the main plan. The traceability documentation enables impact analysis when requirements change and provides evidence that all requirements are addressed.

Test plans may be integrated into the verification plan or maintained as separate documents for each major feature or block. Detailed test plans describe specific test cases, stimuli, checking mechanisms, and coverage contributions. Separating detailed test plans from the overall verification plan enables teams to work on test details without affecting the main document.

Plan Reviews

Verification plan reviews validate that the plan adequately addresses verification requirements and is feasible with available resources. Reviewers should include verification engineers, design engineers, project managers, and quality representatives. External review by verification experts can identify gaps based on experience from other projects.

Review findings should be tracked to closure. Comments requiring plan updates should be dispositioned with either plan changes or documented rationale for not changing. The review record demonstrates that the plan has been vetted and approved.

Plan Maintenance

The verification plan is a living document that evolves throughout the project. Design changes may require verification plan updates. Resource changes may require schedule adjustments. Discovered risks may require mitigation additions. The plan should be updated to reflect current reality.

Version control of the verification plan enables tracking of changes and understanding of plan evolution. Major plan changes should be reviewed and approved. Change history documents the rationale for plan modifications.

Progress tracking against the plan provides ongoing visibility into verification status. Regular reporting compares actual progress against planned progress, highlighting variances that require management attention. Dashboards and metrics enable stakeholders to understand verification status at a glance.