Test Framework

Purpose and Scope

The test framework provides infrastructure for running standardized Ethereum JSON test fixtures to validate core protocol behavior.
Its goal is to verify that transaction processing, block validation, state transitions, and consensus rules operate correctly across a range of hard-fork configurations.

The test fixtures use data from the tests directory included in the StableNet codebase and are executed directly on top of the StableNet implementation.

For EVM-specific testing tools, see EVM Testing Tools.

For general build and CI/CD flows, see Build and CI/CD.

Test Type Overview

The test framework supports four major test categories to validate different aspects of the protocol.

Test Type	Purpose	JSON Structure	Execution
State Tests	Validate transaction processing without block context	stJSON containing pre-state, transaction, and post-state	Execute a single transaction on top of genesis
Block Tests	Validate full blocks including header/body and state transitions	btJSON containing genesis and a block sequence	Insert blocks sequentially
Transaction Tests	Validate RLP encoding and sender derivation	Transaction RLP with expected sender per fork	Decode and verify sender
Difficulty Tests	Validate difficulty calculation rules	Parent header and expected difficulty	Compare against consensus engine output

Test Infrastructure Architecture

testMatcher Pattern Matching

testMatcher controls test selection, configuration overrides, and expected-failure handling via regular-expression rules.

Pattern Type	Method	Purpose
skiploadpat	skipLoad(pattern)	Skip JSON loading itself to reduce memory usage
slowpat	slow(pattern)	Exclude in -short mode or on 32-bit environments
failpat	fails(pattern, reason)	Mark known failing tests
configpat	config(pattern, cfg)	Override chain config for specific tests
runonlylistpat	runonly(pattern)	Run only tests matching the specified pattern

Running State Tests

State tests execute a single transaction against a pre-state and validate the resulting state root and logs, allowing transaction processing logic to be verified in isolation.

StateTest Implementation Overview

State tests generate and run subtests for each fork combination and input index.

Key methods:

Method	Purpose
Subtests()	Enumerate all fork/index combinations
Run(...)	Execute and verify results
RunNoVerify(...)	Execute without verification (for benchmarking)
checkError(...)	Validate whether an error was expected

Execution Flow Summary

1. Parse fork string and load chain config
2. Create genesis block and set up pre-state
3. Convert the transaction into a core.Message
4. Create EVM context and execute the transaction
5. Commit state and compute state root
6. Compare against expected state root, logs, and exceptions

Running Across Storage Schema Variants

Each State test is executed across the following four combinations:

• hash / trie: hash-based scheme without snapshots
• hash / snap: hash-based scheme with snapshots
• path / trie: path-based scheme without snapshots
• path / snap: path-based scheme with snapshots

This validates consistency across both StateDB and snapshot layers.

Running Block Tests

Block tests insert a block sequence into an actual chain and validate the final state and header chain to confirm end-to-end blockchain behavior.

BlockTest Implementation Overview

Block tests include the following tasks:

• Create a genesis block from test data
• Insert test blocks sequentially
• Handle valid vs invalid blocks
• Validate final state and header chain

If a block JSON lacks BlockHeader, transactions, and UncleHeaders, that block is treated as invalid, and insertion failure is the expected behavior.

Execution Flow Summary

1. Commit genesis state to the DB
2. Initialize the blockchain including the consensus engine and caches
3. Insert test blocks sequentially
4. Validate account state against the final StateDB
5. Validate header consistency by walking backward from the chain head

Transaction Tests

Transaction tests validate that RLP decoding and sender derivation behave correctly under different fork rules.

The framework verifies:

1. Whether RLP decoding succeeds
2. Sender recovery via ECDSA signatures
3. Validity of intrinsic gas calculation
4. Transaction hash consistency

Fork Configuration

The test framework predefines chain configurations for all Ethereum hard forks.

Examples include:

Fork	Key Characteristics
Frontier	Initial release
Byzantium	Difficulty bomb delay
Istanbul	Gas cost adjustments
London	EIP-1559 BaseFee
Merge	Transition to PoS
Cancun	Blob transactions
Anzeon	WBFT consensus

Fork strings may include EIP overrides such as "Berlin+1884+1283", and are configured dynamically at runtime.

Test Directory Structure

Test data is organized using a standardized directory structure.

tests/
├── testdata/
│   ├── BlockchainTests/
│   ├── GeneralStateTests/
│   ├── TransactionTests/
│   ├── RLPTests/
│   ├── BasicTests/
│   └── LegacyTests/
├── spec-tests/
│   └── fixtures/
│       ├── blockchain_tests/
│       └── state_tests/
└── evm-benchmarks/
    └── benchmarks/

EVM Command Integration

The cmd/evm tool provides a CLI interface for running Block tests.

blocktest Command Overview

Key flags include:

• --run: select tests via regular expression
• --machine: output JSON-based EVM traces
• --dump: dump final state after execution

Benchmarking Support

State tests support benchmark execution for performance measurement.

Benchmarks measure:

• Gas throughput (mgas/s)
• Total execution time
• Effective throughput accounting for gas refunds

This enables quantitative analysis of how EVM and StateDB changes impact performance.

Error Handling and Verification

Expected Exception Handling

Tests determine success by comparing expected exceptions with actual outcomes.

Expected	Actual	Result
No exception	No error	Pass
Exception expected	Error occurs	Pass
No exception	Error occurs	Fail
Exception expected	No error	Fail

The current implementation does not strictly compare error strings; it focuses primarily on whether an error occurred.

Diagnostic Output on Failure

When a test fails, the following information is provided:

• Full EVM execution trace for the failing test
• Step-by-step operation logs in JSON format
• Gas-limit-based constraints to prevent excessive output

This enables precise analysis of protocol behavior issues.