Prerequisites

Qualcomm Cloud AI 100 accelerator with QAIC Platform/Apps SDK >= 1.22.0 on Linux (Ubuntu 22.04+), Python 3.12.

Platform Requirements

Requirement	AOT Mode	PYT Mode
OS	Ubuntu 22.04+	Ubuntu 22.04+
Python	3.12	3.12
QAIC Platform SDK	>= 1.22.0	>= 1.22.0
QAIC Apps SDK	>= 1.22.0	>= 1.22.0 (with --install-torch-qaic)
torch	2.7.0+cpu	2.10.0+cpu
vLLM	0.15.0	0.15.0
QEfficient	main	—

Hardware

Platform	Description
Qualcomm Cloud AI 100 Ultra	4 QIDs per card, 16 NSP cores per QID
Qualcomm Cloud AI 100 Standard	Data center inference accelerator
Qualcomm Cloud AI 080	Entry-level Cloud AI accelerator

Step 1: Install Cloud AI SDK

Follow the official SDK installation guide:

1. Install Cloud AI SDK and Drivers
2. Verify your installation

PYT mode only

Run the Apps SDK installer with --install-torch-qaic to build torch_qaic wheels into /opt/qti-aic/integrations/torch_qaic/py312/.

Step 2: Create Python 3.12 Environment

condauv (fastest)venv

conda create -n vllm-qaic python=3.12
conda activate vllm-qaic

uv venv .venv --python 3.12
source .venv/bin/activate

python3.12 -m venv .venv
source .venv/bin/activate

Step 3: Verify Device Access

# Check QAIC devices are visible
qaic-util | grep -i "qid\|status"

# Set device visibility
export QAIC_VISIBLE_DEVICES=0   # comma-separated QIDs

QEfficient Reference

The QEfficient library handles model compilation for AOT mode. See the QEfficient Installation Guide for additional context on the compilation stack.