ONNX
ONNX represents model computations as graphs with typed inputs, outputs and versioned operators. It is an interchange format, not a promise that every runtime or execution provider supports every graph or produces acceptable task results.
Validate the artifact in layers: graph structure, runtime admission, numerical agreement, preprocessing/postprocessing, task quality and operating behavior. A file that passes a checker has crossed only the first boundary.
TL;DR
- Record graph IR, operator-domain opsets, shapes and dtypes.
- Match runtime and execution-provider support to the actual graph.
- Compare outputs using justified tolerances and representative inputs.
- Validate preprocessing, postprocessing and task quality as part of parity.
- Treat model artifacts and custom operators as a trust boundary.
Quick Example
This complete local Python fixture constructs a tiny graph rather than exporting a trained model. It uses ONNX 1.23.2 and ONNX Runtime 1.31.1 on CPU, deliberately specifying older IR/opset versions supported by this simple operation. No downloaded model is loaded.
The broad exception capture only checks the expected runtime exception class in this fixture; unexpected errors are re-raised. This is not a converter, accelerator, trained-model accuracy or performance test.
Core Concepts
Graph and operator versions
A graph declares operator-domain opset versions. An operator uses the applicable definition for that domain/version, while IR version describes the model representation. Changing a version number is not a general conversion strategy. ONNX concepts
Runtime and provider
An execution provider implements operations for a target such as CPU or an accelerator. Runtime loading and actual execution need testing with the chosen versions and provider configuration. Provider availability alone does not prove the entire graph executes there. Runtime Python workflow
Parity
Numerical parity compares outputs under a defined input and tolerance. Task parity asks whether the deployed workflow makes acceptable decisions. Input normalization, tokenization, image layout, thresholds and output interpretation can change task behavior even when the exported graph is correct.
Validate an export through distinct gates
Keep the source model, exporter, graph, runtime and representative input set identifiable. Use simple known cases to diagnose conversion mistakes, then cover the distribution and edge cases that matter to the task.
Choose tolerances from the model’s use, not simply a convenient number that makes the test pass. Report maximum and relevant distributional differences where useful, and inspect cases that change decisions. Quantization or reduced precision may require a separate quality evaluation.
Include input contract failures. Reject wrong shape/dtype and bound dynamic dimensions according to the application’s resource policy. A permissive dynamic axis can admit a request too large for the target even when it is structurally valid.
Preserve artifact trust and deployment evidence
Record the artifact hash, external weight files if used, source/export versions, operator domains and custom dependencies. A custom operator or supporting loader can expand the execution surface. Use trusted artifacts and a controlled loading environment rather than treating interchange files as inherently harmless.
Test the packaged deployment with the actual provider, drivers, device and input pipeline. Separate cold initialization from steady-state requests and count failed/unsupported work. A CPU arithmetic fixture says nothing about accelerator speed or memory demand.
Retain a known-good artifact and a tested recovery route. Verify the application’s output schema and postprocessing remain compatible when changing versions. Replacing the graph file alone may not restore a previous tokenizer, feature transformation or decision threshold.
Best Practices
Version the entire inference contract
Keep preprocessing and output interpretation with the model, not only the graph. Confirm the input producer and result consumer agree.
Keep failed cases in the corpus
Preserve examples that exposed conversion, shape or decision differences. They are more useful regression evidence than repeatedly checking one ordinary input.
Measure the actual target
Use the provider and workload intended for deployment. Structure validation, CPU correctness and target performance are separate results.
Common Mistakes
Bad: Changing an opset number to silence an unsupported-operator error.
Correct: Use a supported conversion/export path and rerun compatibility and parity checks.
Bad: Calling checker success an accuracy result.
Correct: Compare numerical outputs and task behavior using representative inputs.
Bad: Ignoring preprocessing differences.
Correct: Version and test the complete input-to-decision pipeline.
FAQ
Was a trained model exported here?
No. A small constructed graph passed structural, CPU output and wrong-shape checks.
Does ONNX guarantee accelerator support?
No. Verify the graph against the actual runtime, provider, device and driver combination.
What should block deployment?
A required unsupported operation, unacceptable decision change, or failed operating constraint should stop acceptance until resolved.
Related Topics
- MLOps & Model Deployment — Connect artifact validation to deployment.
- MLflow — Retain run and model evidence.
- Deploying PyTorch Models — Review exporter and serving choices.