How to Run a Local LLM on Android (No Internet Needed)

Can you really run an AI chatbot on your phone offline?

Yes. A modern Android phone with 8 GB of RAM can run a small large language model (LLM, the kind of AI behind ChatGPT) entirely on its own chip. Once the model file is downloaded, you can switch on airplane mode and keep chatting. Your prompts never touch a server, there is no subscription, and it works on a plane, in a basement, or anywhere without signal.

The trade-off is size. Phone models have 1 to 4 billion parameters instead of the hundreds of billions behind cloud chatbots, so they are good at summarising, rewriting, brainstorming, quick explanations and simple coding help, and weaker at deep reasoning and recent facts. This guide shows you which app and model to pick for your phone, how to set each one up, and how to prove nothing leaves the device.

What you need before you start

You do not need root, a custom ROM, or a gaming phone. You need enough memory, some free storage, and Wi-Fi for the one-time model download.

A simple rule: the model needs its file size in free RAM, plus another 0.5 to 1.5 GB for the app and the conversation. Google's own AI Edge Gallery enforces similar floors: 6 GB for 1B to 1.5B models, 8 GB for Gemma 4 E2B, and 12 GB for Gemma 4 E4B (from the app's model allowlist, version 1.0.19, September 2026).

Storage matters too. Plan for the model file plus about 1 GB of headroom, and more if you want to keep a few models around. If your phone is nearly full, clear some room first with our guide to freeing up space on Android.

Pick a model

Every model below is free to download from Hugging Face (the main public library of AI models). File sizes are for the Q4_K_M version, which is what you want on a phone.

Sizes from the Hugging Face API (unsloth GGUF builds and Google's LiteRT builds), checked September 25, 2026.

If you are unsure, start with Qwen3.5 2B on a 6 GB phone or Gemma 4 E2B on an 8 GB phone. Both punch well above their size. Qwen3.5 is still the newest small Qwen: the later 3.6 and 3.8 releases only come in 27B and larger, which is far too big for a phone. For more on what Gemma 4 can do once it is running locally, see our Gemma 4 offline projects guide.

Pick an app

These are the apps worth using in September 2026. All of them run the model on your phone; none need an account.

Versions and store listings checked on GitHub and Google Play, September 25, 2026.

01 / EASIEST

PocketPal AI

Best overall

PocketPal is the one to install first. It runs any GGUF model through llama.cpp, lets you search Hugging Face from inside the app, and handles gated models (ones that need you to accept a licence) if you paste in a Hugging Face token. It can use the Adreno GPU and, on recent Snapdragon chips, the Hexagon NPU, falling back to the CPU when neither is available.

It is very actively maintained. Version 1.18.0 landed on September 25, 2026. The only data that leaves your phone is opt-in: benchmark uploads and feedback, per the project's README. There is also a built-in benchmark and a public device leaderboard if you want to see how your phone compares.

Replaces

ChatGPT app for private chats

Runs on

Depends on the model (see RAM table)

Difficulty

Easy

Licence

MIT (open source)

Skip if

You only want Google's Gemma models

Official siteSource code

02 / GEMMA

Google AI Edge Gallery

Best for Gemma 4

Google's own showcase app, still labelled an experimental beta, is the fastest way to run Gemma 4. It uses Google's LiteRT runtime, which is tuned for phone GPUs, and goes beyond plain chat: Ask Image describes photos, Audio Scribe transcribes voice clips, and Agent Skills lets the model call simple tools. All of it runs offline once the model is downloaded.

You pick models from a curated list, not any file you like. The app hides models your phone does not have enough RAM for. One privacy note: the app includes Firebase Analytics, so usage stats can be sent when you are online, even though your prompts are processed on the device. Airplane mode or a firewall rule stops that.

Replaces

Gemini for offline tasks

Runs on

6 GB RAM, Android 12+

Difficulty

Easy

Licence

Apache 2.0 (open source)

Skip if

You want to load your own GGUF files

Official siteSource code

03 / SIMPLE

Maid

No telemetry

Maid (Mobile Artificial Intelligence Distribution) is a clean, no-frills llama.cpp front end with a curated Hugging Face model list. Its README promises no telemetry and no ads. The Play Store build (updated September 2, 2026) is newer than the last GitHub release, so install it from the Play Store.

Runs on

Depends on the model (see RAM table)

Difficulty

Easy

Licence

MIT (open source)

Skip if

You need GPU speed-ups

Official siteSource code

04 / FAST

MNN Chat

Alibaba's runtime

MNN Chat runs models converted to Alibaba's MNN format, with a built-in catalogue that includes Qwen3.5 and Gemma 4. It supports OpenCL and Vulkan GPU acceleration and has its own benchmark screen. The developers say they have only tested it on a OnePlus 13 and a Xiaomi 14 Ultra, and warn that low-spec phones may not work at all, so treat it as a flagship option.

Runs on

Recent flagship recommended

Difficulty

Easy

Licence

Apache 2.0 (open source)

Skip if

Your phone is older or low-end

Official siteSource code

05 / TINKER

ChatterUI

Roleplay and characters

ChatterUI is for people who want character cards, custom system prompts and fine control over sampling settings. It is not on the Play Store: download the APK from its GitHub releases page (v0.10.0, September 22, 2026). Import a GGUF file you already downloaded. It also has a Remote Mode for cloud APIs, so make sure you are in local mode if privacy is the point.

Runs on

Depends on the model (see RAM table)

Difficulty

Medium

Licence

AGPL-3.0 (open source)

Skip if

You want a Play Store install

Source code

Layla is a paid alternative ($19.99 on the Play Store, with a free APK on its website). It is closed source and asks for 6 GB of RAM at minimum, 8 GB or more recommended. MLC Chat still shows up in older guides, but its last Android APK is from September 2024, so skip it.

Method 1: Set up PocketPal AI (5 minutes)

This is the quickest route and works on any phone that meets the RAM table above.

1. Install PocketPal AI from the Google Play Store and open it.
2. Tap the menu and open Models. You will see a short list of suggested models.
3. Tap the + button, then Add from Hugging Face. Search for Qwen3.5-2B GGUF (or Llama-3.2-1B GGUF on a 4 GB to 6 GB phone).
4. Pick the file ending in Q4_K_M.gguf and tap download. Stay on Wi-Fi: it is roughly 0.8 to 1.3 GB.
5. When it finishes, tap Load next to the model. The first load takes a few seconds.
6. Open PocketPal's Settings and turn on GPU acceleration if your phone offers it. If the app crashes or slows down, switch it back off.
7. Go back to the chat screen and send a message.

Now test that it is truly offline: swipe down, turn on airplane mode (and make sure Wi-Fi is off too), force-close PocketPal, reopen it, and chat again. If it answers, you are running AI entirely on your phone.

Method 2: Run Gemma 4 with Google AI Edge Gallery

Pick this route if you have 8 GB of RAM or more and want Gemma 4's image and audio features.

1. Install Google AI Edge Gallery from the Play Store (Android 12 or newer).
2. Open AI Chat. Choose Gemma 4 E2B (8 GB phones) or Gemma 4 E4B (12 GB and up).
3. If asked, sign in to Hugging Face and accept the Gemma licence. This is a one-time step for gated models.
4. Download the model (about 2.6 GB for E2B, 3.7 GB for E4B).
5. Tap the model settings and select GPU as the accelerator. On a Galaxy S26 Ultra, Gemma 4 E2B runs at about 52 tokens per second on the GPU versus 47 on the CPU, and the GPU version uses far less RAM (about 676 MB versus 1.7 GB), per Google's LiteRT model card.
6. Turn on airplane mode and try Ask Image with a photo from your gallery.

Method 3: Termux with llama.cpp or Ollama (power users)

Termux is a Linux terminal for Android. Running llama.cpp there gives you the latest model support the day it ships, full control over every setting, and a local web chat you can open in your browser.

Install llama.cpp from the Termux package repository (version 0.5.0, built with Vulkan and OpenCL, as of September 25, 2026):

pkg update && pkg upgrade
pkg install llama-cpp
termux-setup-storage

Download a GGUF model in your phone's browser (it lands in Downloads), then start a chat:

llama-cli -m ~/storage/downloads/Qwen3.5-2B-Q4_K_M.gguf -cnv

Prefer a proper chat window? Run the built-in server and open http://127.0.0.1:8080 in Chrome on the same phone:

llama-server -m ~/storage/downloads/Qwen3.5-2B-Q4_K_M.gguf --port 8080

If you already use Ollama on a PC, it is in Termux too (version 0.31.1, a few releases behind upstream):

pkg install ollama
ollama serve &
ollama run llama3.2:1b

The ollama run step needs internet the first time to pull the model. After that it runs offline. If you run Ollama on a home server as well, our home lab apps guide covers that side.

How to confirm nothing leaves your phone

Airplane mode is the simple test, but a few extra checks make it airtight.

• Download the model, then turn on airplane mode with Wi-Fi and Bluetooth off.
• Force-stop the app from Settings, reopen it, and confirm it still answers.
• Check the app's Data safety section on its Play Store page.
• For permanent blocking, deny the app network access with a firewall app such as NetGuard (no root needed).
• If you use ChatterUI, confirm Remote Mode is off.

How fast is it?

Speed is measured in tokens per second (a token is roughly three quarters of a word). Anything above 10 feels like fast reading. Below 5 feels slow but usable.

Two things slow you down in practice. Long prompts: on the midrange phone above, a long pasted document pushed the wait for the first word to about 11 seconds. And heat: the same test saw Gemma 4 E2B drop from 7.0 to 5.8 tokens per second once the phone warmed up. Running a model is heavy work, so expect the phone to get warm and the battery to drain faster than normal. Plug in for long sessions, and see our battery-saving Android settings for everyday use.

Fixing common problems

FAQs