Deploying LLMs on Qualcomm NPU using LiteRT LM

Overview

LiteRT LM is Google’s on-device inference library for large language models. On Qualcomm Snapdragon devices, it leverages the Hexagon NPU to execute models locally without any cloud connectivity.

This article walks through deploying the LiteRT LM sample Android application on a Snapdragon device, covering both the command-line and Android Studio deployment paths.

The .litertlm Model Format

The .litertlm format is a model binary pre-compiled for a specific Qualcomm chipset by Qualcomm AI Hub. Unlike general-purpose formats such as ONNX or GGUF, a .litertlm file is ready for direct NPU execution — no runtime compilation is required.

Important

Each .litertlm file targets a single chipset. Download the file that matches your device’s SoC identifier (e.g. sm8750 for Snapdragon 8 Elite). Loading an incompatible file will result in a runtime error.

Supported Models

Model

Chipset Target

Capabilities

Gemma 4 E2B-it

SM8750 (Snapdragon 8 Elite), SM8850 (Snapdragon 8 Elite Gen5)

Text chat, voice input

FastVLM-0.5B

SM8750 (Snapdragon 8 Elite)

Text chat, image understanding

Prerequisites

Device

A device with Snapdragon 8 Elite (SM8750) or Snapdragon 8 Elite Gen5 (SM8850)

USB Debugging

Enabled under Developer Options on the target device

Android Studio

Panda 4 (2025.3.4) or newer — download

ADB

Included with Android Studio’s platform tools

Git

To clone the sample repository

Step 1 — Clone the Sample Repository

git clone https://github.com/google-ai-edge/litert-samples.git

Navigate to the NPU chatbot project:

cd litert-samples/compiled_model_api/qualcomm/llm_chatbot_npu

This directory is a self-contained Android Studio project requiring no source modifications.

Step 2 — Download the Model

Download the .litertlm file for your device from the corresponding Hugging Face repository. Ensure the filename matches your device’s SoC identifier — for example:

gemma-4-E2B-it_qualcomm_sm8750.litertlm   # Snapdragon 8 Elite
FastVLM-0.5B.qualcomm.sm8750.litertlm      # Snapdragon 8 Elite

Note

Model files range from 1 GB to 4 GB. Ensure sufficient storage is available on both the host machine and the target device before proceeding.

Step 3 — Build and Deploy the Application

  1. Open Android Studio and select File → Open.

  2. Navigate to litert-samples/compiled_model_api/qualcomm/llm_chatbot_npu and click OK.

  3. Wait for the Gradle sync to complete. Resolve any SDK licence prompts if shown.

  4. Connect the target device via USB. Confirm it appears in the device selector toolbar.

  5. Select Run → Run ‘app’ (or press Shift+F10). Android Studio will build the APK and deploy it directly to the connected device.

Step 4 — Push the Model to the Device

Copy the downloaded .litertlm file into the application’s data directory using ADB:

adb push gemma-4-E2B-it_qualcomm_sm8750.litertlm \
  /sdcard/Android/data/com.example.qnn_litertlm_gemma/files/model.litertlm

The application scans this directory at startup and loads any .litertlm file it finds. No additional configuration is required.

Step 5 — Launch and Verify

Open the application on the device. On first launch, the model is loaded onto the Hexagon NPU, after which the chat interface becomes available. Submit a text prompt to confirm the model is responding correctly.

Application Capabilities

Mode

Description

Supported Models

Text Chat

Conversational text input and streamed response output

Gemma 4, FastVLM

Image + Text

Image selected from device gallery with an accompanying text query

FastVLM-0.5B

Voice Input

Microphone-based prompt entry

Gemma 4 (2B variant only)

References