aimet_onnx.quantsim.set_lpbq_for_params

Top level APIs

aimet_onnx.quantsim.set_lpbq_for_params(sim, bitwidth, block_size, *, op_types=None, nodes_to_exclude=None, nodes_to_include=None, strict=None)[source]

Set weight quantizers of specified nodes to use low-power blockwise quantization.

Overloads:

  • sim (QuantizationSimModel), bitwidth (int), block_size (int), nodes_to_include (Set[str])

  • sim (QuantizationSimModel), bitwidth (int), block_size (int), op_types (Union[str, Set[str]]), nodes_to_exclude (None), strict (bool)

This function is overloaded with the following signatures:

aimet_onnx.quantsim.set_lpbq_for_params(sim, bitwidth, block_size, *, nodes_to_include=None)[source]
Parameters:

  • sim (QuantizationSimModel) – Quantsim to set weight quantizers for

  • bitwidth (int) – Compressed bitwidth for lpbq quantization

  • block_size (int) – Block size for affine quantization. The block size will be applied to the weight’s input features dimension, while per-channel will be used for the weight’s output features dimension

  • nodes_to_include (Optional[Set[str]]) – Set of onnx node names to include for blockwise weight quantization.

aimet_onnx.quantsim.set_lpbq_for_params(sim, bitwidth, block_size, *, op_types=None, nodes_to_exclude=None, strict=False)[source]
Parameters:

  • sim (QuantizationSimModel) – Quantsim to set weight quantizers for

  • bitwidth (int) – Compressed bitwidth for lpbq quantization

  • block_size (int) – Block size for affine quantization. The block size will be applied to the weight’s input features dimension, while per-channel will be used for the weight’s output features dimension

Parameters:

  • op_types (Union[str, Set[str], None]) – Operator types for which to enable grouped blockwise weight quantizaiton

  • nodes_to_exclude (Optional[Set[str]]) – Set of onnx node names to exclude from blockwise weight quantization.

  • strict (bool) – If False, only enable blockwise quant for layers with dimensions evenly divisible by block_size. If True, throw an error for layers with incompatible shapes.

Examples

>>> sim = QuantizationSimModel(...)
>>> set_lpbq_for_params(sim, bitwidth=4, block_size=64, op_types={"Gemm", "MatMul", "Conv"})
>>> # or
>>> set_lpbq_for_params(sim, bitwidth=4, block_size=64, nodes_to_include={"/lm_head/MatMul", ...})