chipintegrity.org GitHub
TAKE PART

Test your own chip

The same tests run on an NVIDIA GPU, an Apple GPU or a CPU with one command. This page shows how to install and run them, and what is planned next.

ON THIS PAGE YOU WILL LEARN
  1. How to install the tester and run every probe on your own chip
  2. What is planned next on the roadmap
TAKE PART

Run it on your chip

One command, ten result files, one pull request. Works on CUDA, Apple MPS and CPU today.

# install once, in its own virtual environment (Python 3.10 or newer)
python3 -m venv chip-env && source chip-env/bin/activate
pip install ai-chip-integrity

# a one-minute smoke test: every probe and its self-test at small sizes
chip-integrity run --quick

# the full run: every probe and its self-test, ten OCP result files
chip-integrity run --name NAME

# or, on an NVIDIA GPU host with Docker, with nothing else installed
docker run --rm --gpus all -v "$PWD:/results" ghcr.io/tech4biz-yasha/ai-chip-integrity:0.4.2 run --name NAME --out-dir /results

On many Macs the built-in python3 is version 3.9, which pip skips. Install a newer Python from python.org or with Homebrew first, then create the environment with its versioned command, for example python3.12 -m venv chip-env.

The same ten files come from running each probe by hand from a clone of the repository, for example python screen.py --out NAME_clean.jsonl and then python screen.py --inject 5 --out NAME_inject.jsonl for its self-test. The README lists all ten commands.

  1. Open a pull request with the ten files, the device name, driver version and PyTorch version.
  2. Rows are added only from attached files. The table is regenerated from them, so a row cannot say more than its file does.
  3. Found a fault? Open an issue with the file. A real finding on real hardware is the most useful thing anyone can contribute.

The output follows the OCP Test and Validation spec v2.0 through the official ocptv library, and the test suite validates it against the published schema.

TAKE PART

Roadmap

Twelve probes, each aimed at one part of the chip. Six are built: 1, 2, 3, 6, 9 and 11.

1. Matrix multiplyThree shapes, four precisions plus FP8, a proven or exact reference and a bit-exact repeat check. Done. FP4 once a Blackwell card is measured.
2. Memory pattern sweepSix patterns over most of device memory, exact read-back, address and bit mask on every miss. Done. On-chip SRAM next.
3. Transformer kernelsSoftmax, layer norm, GELU, rotary embeddings and the fused attention kernel against a float64 reference with derived bounds. Done.
4. Full model forward passA small LLM, every layer checked against a CPU float64 reference. Each layer is fed the device's own input, so its bound stays as tight as a single kernel's.
5. Reductions, all-reduce and copy enginesSums across threads, blocks and then GPUs over NVLink and PCIe, with checksums that must agree at both ends, plus copies of known patterns between host and device.
6. Checksum-protected matrix multiplyHuang and Abraham's row and column checksums, which detect and locate a wrong element inside the multiply itself, with proven thresholds and the blind spot measured. Done.
7. Hours under loadThe same probes for hours at full power, with temperature and clocks logged next to every result.
8. Clock and voltage margin sweepLower the margin step by step where the driver allows it and record where each unit starts to fail. Likely needs bare-metal access.
9. Data pattern libraryWide exponents, full mantissas, cancellation, alternating signs, sparsity, subnormals with flush detection, near-overflow and int8 extremes, for the matrix multiply. Done. Patterns for the transformer kernels next.
10. Fault injection inside the computationFlip bits in registers during the multiply to measure how often a flip becomes a wrong answer. The basis for a space radiation column.
11. ECC and error countersEvery run of every probe reads the chip's counters before and after and records whether the hardware noticed what the probe noticed. Done. First real test on the H100 on 10 October 2026: every ECC and row-remap counter quiet. On 11 October the RTX 4090 and RTX A5000 ran with ECC off, which added the ecc-off verdict.
12. RecoveryAfter a detected fault, reset and rerun on the same card to see whether the fault clears. Likely needs bare-metal access.
Fleet modeRun every probe across many devices automatically and collect the files. Fault rates are per thousand units, so volume is how they get found.
Vendor comparisonRun NVIDIA DCGM and AMD AGFHC on the same device and record what each catches.