Most AI assistants have a quiet dependency you only notice when it breaks: the cloud. Every question your users ask travels to someone else's server, gets processed there, and comes back — along with a per-request bill and a growing pile of data that has left the device. On-device AI agents flip that model. The intelligence runs on the phone itself, so the data stays put and the marginal cost of a conversation is zero.
This post explains what an on-device AI agent actually is, when it makes sense, and how LyboAI Edge — our runtime for private, on-device agents — puts one inside your mobile app.
What "on-device" really means
An on-device agent is a small language model plus the machinery around it — retrieval, tools, memory, guardrails — running entirely on the user's phone, tablet or laptop. Modern devices are more capable here than most teams assume. Quantised models in the 1–8B parameter range now handle summarisation, extraction, routing and focused chat well, and every major platform ships its own native model: Gemini Nano on Android, Apple's Foundation Models on iOS, and Phi Silica on Windows.
The practical differences from a cloud assistant come down to three things.
First, privacy by architecture rather than by policy. When inference happens locally, raw user content never needs to leave the device unless the user explicitly consents to share it. That is a much easier story to tell customers — and regulators — than "we send it to a server, but we promise to be careful".
Second, cost. Cloud inference is metered; on-device inference uses compute the user already owns. For apps with frequent, lightweight AI interactions — note-taking, personal organisation, field work, coaching — zero inference billing changes what is economically viable.
Third, resilience. An agent that lives on the device keeps working on a plane, at a mine site, or anywhere connectivity is patchy. With deterministic fallback tiers, behaviour stays predictable offline instead of silently degrading.
Where the model runs is the whole difference: cloud-only assistants send raw content off-device on every request; on-device agents keep it local.
Where on-device agents fit (and where they do not)
On-device is not the answer to everything, and it is worth being clear-eyed about the boundary. Small local models will not out-reason a frontier cloud model on open-ended, complex tasks. If your product depends on long-horizon reasoning over huge contexts, you will still want the cloud in the loop.
Where local agents shine is bounded, repeatable work: answering questions over a known set of documents, extracting structured data from text, routing a request to the right workflow, summarising content the user is looking at, or walking someone through a defined task. Most of what mobile apps actually need from AI falls in this territory — and it is exactly the territory LyboAI Edge is built for.
How LyboAI Edge approaches it
LyboAI Edge is a developer framework for React Native and Expo apps that packages the hard parts of on-device agents into a runtime of roughly 2.4 MB of native library per platform. A few design choices matter most.
Bounded agent patterns instead of a blank canvas. Edge ships eight agent patterns — chat, retrieval, ReAct, router, extraction, summarisation, planner–executor and deep task. Rather than handing a small model an open-ended loop and hoping, you pick the pattern that matches the job. Bounded patterns are how small models stay reliable.
The eight bounded patterns Edge ships. You choose the one that fits the task rather than improvising an open-ended loop.
Local hybrid RAG. Edge indexes your app's content at install time and retrieves over it locally, combining keyword and semantic search. Your agent can answer questions about your product manual, policy pack or catalogue without a single network call.
A workflow engine with checkpoints and rewind. Multi-step agent work is declared as a durable workflow. If a step fails or the app is killed mid-task, work resumes from the last checkpoint rather than starting over — and human-in-the-loop approval gates let users confirm actions before they happen.
Governed device tools. Thirteen device capabilities — camera, mail, speech-to-text, text-to-speech, OCR and more — are exposed to agents through governed handlers, so tool access is explicit and auditable rather than improvised.
Signed capability packs. When you want to update prompts, retrieval content or agent behaviour after release, signed packs let you ship changes without republishing the app, with integrity checks on what the runtime will load.
On top of the runtime, Lybo OS adds a layered approach that uses the platform's native models where they fit — Gemini Nano, Apple Foundation Models, Phi Silica — and augments them with skills and governance when the job needs more. Telemetry throughout is redacted, aggregate and policy-gated, so you can observe your agents without collecting what your users typed.
The Edge stack end to end: your app calls the runtime, which orchestrates bounded agents on top of the device's own native model.
To see how those pieces work together, here is a single request making its way through an on-device agent — without leaving the phone at any step.
A full agent loop stays inside the device boundary; raw content only crosses it if the user explicitly chooses to share.
Getting started
If you are weighing up an AI feature for your app, start by asking which parts of it are bounded and repeatable — those are your on-device candidates. Prototype the pattern (retrieval and extraction are usually the quickest wins), test it against your real content, and keep the cloud for the tasks that truly need it.
Edge Studio, our developer members area, is where you manage Edge Runtime licence keys, follow the quickstarts for Expo, Node and Python, and work with the visual app Builder. And if you would rather talk through fit first, the VALUE-AI framework on our site is how we help teams assess AI feasibility before committing to a build.
On-device AI is not a niche any more — it is the sensible default for a growing share of mobile AI work. Explore LyboAI Edge and the rest of the toolkit at lyboai.app, and check the quickstarts to have a local agent running in your app this week.