Testing TypeScript

Learn how to run tests and write new test cases for the TypeScript compiler

Running Tests

Quick Test Run

Run all tests in parallel (recommended):

hereby runtests-parallel

Or use the npm script:

npm test

This builds the test infrastructure and runs all test suites using multiple worker threads.

By default, hereby uses one worker per CPU core. Adjust with --workers=<number>.

Serial Test Execution

Run tests sequentially for debugging:

hereby runtests

Test Suites

TypeScript has several test suites organized by type:

Compiler Tests

Tests in tests/cases/compiler/ verify compiler behavior:

hereby runtests --tests=compiler

These test compilation, emit, and error detection.

Conformance Tests

Tests in tests/cases/conformance/ verify spec compliance:

hereby runtests --tests=conformance

Organized by spec section (types, expressions, statements, etc.).

FourSlash Tests

Tests in tests/cases/fourslash/ test language service features:

hereby runtests --tests=fourslash

Includes completions, refactorings, navigation, and more.

Project Tests

Tests in tests/cases/projects/ test multi-file scenarios:

hereby runtests --tests=project

Tests project references and build mode.

Running Specific Tests

By Name Pattern

Run tests matching a regex:

hereby runtests --tests=<pattern>

Examples:

# Run tests with "async" in the name
hereby runtests --tests=async

# Run a specific test file
hereby runtests --tests=2dArrays

# Run tests in a specific suite
hereby runtests --tests=conformance/types

By Runner

Specify which test runner to use:

hereby runtests --runner=<runnerName>

Available runners:

• conformance - Conformance suite
• compiler - Compiler suite
• fourslash - Language service suite
• project - Project suite

From Failed Tests File

Rerun previously failed tests:

hereby runtests --failed

Failed tests are tracked in .failed-tests.

Test Options

Common Options

# Light mode (faster, fewer verifications)
hereby runtests-parallel --light

# Keep failed tests in .failed-tests even if they pass
hereby runtests --keepFailed

# Don't clean test output directories first
hereby runtests --dirty

# Custom timeout (default: 40000ms)
hereby runtests --timeout=60000

# Disable colors
hereby runtests --no-color

Advanced Options

# Custom stack trace limit
hereby runtests --stackTraceLimit=50
hereby runtests --stackTraceLimit=full

# Generate code coverage with c8
hereby runtests-parallel --coverage

# Run on a specific shard (for CI)
hereby runtests-parallel --shards=4 --shardId=1

# Use built compiler instead of LKG
hereby runtests --built

# Specify number of parallel workers
hereby runtests-parallel --workers=8

Reporters

# Default parallel reporter (minimal output)
hereby runtests-parallel --reporter=min

# Five-line format (serial runs)
hereby runtests --reporter=mocha-fivemat-progress-reporter

# Mocha's spec reporter
hereby runtests --reporter=spec

# Mocha's dot reporter
hereby runtests --reporter=dot

Watch Mode

Automatically rerun tests when files change:

hereby runtests-watch --tests=<pattern>

You must specify --tests or --failed to use watch mode.

Example:

hereby runtests-watch --tests=asyncAwait

This watches:

• Test infrastructure (built/local/run.js)
• Test cases (tests/cases/**)
• Test libraries (tests/lib/**)

Debugging Tests

Inspector Mode

Run tests with Node.js inspector:

hereby runtests --tests=<pattern> -i

Example:

hereby runtests --tests=2dArrays -i

Then open chrome://inspect in Chrome or attach from VS Code.

VS Code Debugging

1. Create Launch Configuration

   Copy .vscode/launch.template.json to .vscode/launch.json:

   cp .vscode/launch.template.json .vscode/launch.json
   
2. Open Test File

   Open the .ts test file you want to debug in VS Code.

3. Launch Debugger

   Press F5 or select Run > Start Debugging.

   The configuration runs Mocha with the currently open test file name.

4. Set Breakpoints

   Set breakpoints in either:

   • Test files (tests/cases/**)
   • Compiler source (src/**)

The debugger uses source maps, so breakpoints in TypeScript source work directly.

Test Baselines

Most tests generate baseline files to detect output changes:

Baseline Files

Compiler tests generate:

• .js - Emitted JavaScript
• .d.ts - Emitted declarations (in .js file)
• .errors.txt - Compiler errors
• .types - Type of each expression
• .symbols - Symbol for each identifier
• .js.map - Source maps (if requested)

Comparing Baselines

After running tests, compare changes:

hereby diff

Or manually with git:

git diff --no-index tests/baselines/reference tests/baselines/local

Set the DIFF environment variable to use your preferred diff tool:

export DIFF="code --diff"
hereby diff

Accepting Baselines

If the baseline changes are correct:

hereby baseline-accept

This copies tests/baselines/local/* to tests/baselines/reference/.

Carefully review baseline changes before accepting! Unexpected changes may indicate bugs.

Writing New Tests

Adding a Compiler Test

1. Create Test File

   Add a .ts file in tests/cases/compiler/:

   tests/cases/compiler/myNewFeature.ts

   // @strict: true
   // @target: es2020
   
   // Test code that demonstrates the bug fix or new feature
   function example() {
       const x: string = "hello";
       return x.toUpperCase();
   }
   
2. Add Metadata

   Use comment tags to configure test behavior:

   // @strict: true              // Enable strict mode
   // @target: es2020            // Set compilation target
   // @module: commonjs          // Set module system
   // @noEmit: true              // Don't emit output
   // @noImplicitAny: true       // Specific compiler option
   
3. Run the Test

   hereby runtests --tests=myNewFeature
   
4. Accept Baselines

   hereby baseline-accept
   

Multi-File Tests

Use @filename tags to simulate multiple files:

// @filename: types.ts
export interface User {
    name: string;
    age: number;
}

// @filename: main.ts
import { User } from "./types";

const user: User = {
    name: "Alice",
    age: 30
};

Conformance Tests

For spec compliance tests, add files to appropriate subdirectories:

tests/cases/conformance/
├── types/               # Type system tests
├── expressions/         # Expression tests
├── statements/          # Statement tests
├── es6/                 # ES6 feature tests
├── es2015/              # ES2015 tests
└── ...

Conformance test names must be unique across all test directories.

Test Infrastructure

The test runner is built from src/testRunner/:

# Build test infrastructure
hereby tests

# Or use npm script
npm run build:tests

Output: built/local/run.js

Test Harness

The harness code in src/harness/ provides utilities:

• Compiler host implementations
• Virtual file systems
• Baseline management
• Test configuration parsing

ESLint Rule Tests

TypeScript includes custom ESLint rules with their own tests:

hereby run-eslint-rules-tests

Or:

npm run test:eslint-rules

These tests are separate from the main compiler test suite.

Continuous Integration

Sharded Testing

CI systems can split tests across multiple machines:

# Run shard 1 of 4
hereby runtests-parallel --shards=4 --shardId=1

# Run shard 2 of 4
hereby runtests-parallel --shards=4 --shardId=2

Coverage Reports

Generate test coverage with c8:

hereby runtests-parallel --coverage

Coverage reports are written to the coverage/ directory.

Troubleshooting

Tests fail with 'Cannot find module'

Build the test infrastructure:

hereby tests

Baseline comparison shows no differences but test fails

Regenerate baselines:

hereby runtests --tests=<test> --dirty
hereby baseline-accept

Test times out

Increase timeout:

hereby runtests --tests=<test> --timeout=120000

Watch mode doesn't detect changes

Watch mode is experimental. Use manual reruns:

hereby runtests --tests=<pattern>

Debugger doesn't hit breakpoints

Ensure you’ve built tests with source maps:

hereby tests

Source maps are enabled by default in Herebyfile.mjs.

Test Guidelines

Test Requirements

Every PR should include:

• At least one test that fails without your changes
• Reasonable permutations of the fix/feature
• Updated baselines if output changes
• Test names that clearly describe what they test

Next Steps

Debugging

Learn debugging techniques

Building

Build system reference