Convert existing AIMET encodings to ONNX QDQ

encodings_to_onnx_qdq() converts an ONNX model and its AIMET encodings file into an ONNX QDQ model, which carries its quantization parameters in the graph itself as QuantizeLinear and DequantizeLinear node pairs rather than alongside it in a separate file. It runs directly in ONNX Runtime and is the form downstream toolchains consume.

Use it for models already exported with their encodings – the two files that QuantizationSimModel.export() writes. Because it reads the encodings directly, the encodings alone decide which tensors are quantized and how: no quantization simulation configuration is consulted, so supergroups, quantizer tying, and runtime constraints cannot add or remove a quantizer behind your back.

If you still hold a calibrated simulation in memory, use QuantizationSimModel.to_onnx_qdq() instead.

This API is only available in aimet-onnx.

Note

Export encodings in version 2.0.0. It is the only version that records the per-channel axis explicitly, so conversion never has to infer it from the model. See the Encoding Format Specification page for the format itself.

Workflow

Prerequisites

An ONNX model and its encodings file, as written by QuantizationSimModel.export() in the Post-Training Quantization workflow, for example quantized_mobilenet_v2.onnx and quantized_mobilenet_v2.encodings.

Procedure

Step 1: Load the exported artifacts

import numpy as np
import onnx
import onnxruntime as ort
from aimet_onnx import encodings_to_onnx_qdq

# An AIMET export produces these two files side by side.
model = onnx.load_model('./quantized_mobilenet_v2.onnx')
encodings = './quantized_mobilenet_v2.encodings'

Step 2: Convert to QDQ

The model is not modified in place; the converted graph is returned.

qdq_model = encodings_to_onnx_qdq(model, encodings)
onnx.save_model(qdq_model, './quantized_mobilenet_v2_qdq.onnx')

Step 3: Run the QDQ model

session = ort.InferenceSession(qdq_model.SerializeToString())
dummy_input = np.random.randn(1, 3, 224, 224).astype(np.float32)
outputs = session.run(None, {'input': dummy_input})

Limitations

  • An encoding naming a tensor that is not in the model is an error, since there is nothing to attach it to.

  • float16 and bfloat16 are plain casts rather than quantized data types, and onnx QDQ has no representation for them. Scaled float formats such as FP8 do carry a scale and are converted.

  • The model itself must be float32. A float16-typed or bfloat16-typed graph is rejected.

API

aimet_onnx.encodings_to_onnx_qdq(model, encodings, *, prequantize_constants=False, force_activation_as='unsigned')[source]

Convert an ONNX model and its AIMET encodings into an ONNX QDQ model.

This is a direct translation: the encodings determine exactly which tensors are quantized and how. No QuantizationSimModel is created, so no quantsim configuration (supergroups, op tying, encoding constraints) can add or remove quantizers relative to what the encodings specify.

[b]float16 is not a quantized data type and onnx QDQ has no representation for it, so plain-cast float encodings are omitted with a warning, whether they name a parameter or an activation. Scaled float formats such as FP8 do carry a scale, and are emitted as Q/DQ.

An encoding naming a tensor that is not in the model is an error.

Parameters:

  • model (ModelProto | ONNXModel | str) – ONNX model as a ModelProto or ONNXModel, or a path to one. The model is not modified in place.

  • encodings (dict | str) – AIMET encodings as a dict, or a path to an encodings file. Versions 0.6.1, 1.0.0, 2.0.0, and 2.1.0 are accepted.

  • prequantize_constants (bool) – If True, store quantized parameters as integer initializers rather than emitting QuantizeLinear for them. (default: False)

  • force_activation_as (Optional[Literal['unsigned', 'signed']]) – Cast all activation encodings to signed or unsigned, a workaround for QNN converter limitations. Pass None to leave them as-is. (default: “unsigned”)

Return type:

ModelProto

Returns:

The equivalent ONNX QDQ model.

Examples

>>> qdq_model = encodings_to_onnx_qdq("model.onnx", "model.encodings")
>>> onnx.save_model(qdq_model, "model_qdq.onnx")