Real browser inference · know the boundaries

Run an LLM in your browser with WebLLM

LLMinONE has a genuine local inference mode powered by WebLLM. It downloads a supported model and uses your device’s GPU through WebGPU to generate text in the browser. It is separate from both the Instant simulation and links to external AI providers.

This guide reflects the configured application flow, not a device benchmark. A model appearing in the picker does not guarantee that your browser, GPU or available memory can load it.

Three modes that should not be confused

Instant: generates scripted local responses styled as model voices. It is a simulation, not inference from the named provider. Concept previews do not give access to unreleased models.
In-browser · real: loads one of the five configured models below through WebLLM and generates tokens locally. It does not run ChatGPT, Claude or Gemini on your device.
Official provider links: open external services with their own accounts, costs and privacy terms. Start with the ChatGPT, Claude or Hugging Face tool pages.

How to start a real local chat

Open the gateway’s chat section in a WebGPU-capable browser. The app suggests Chrome or Edge when WebGPU is unavailable; support still depends on browser version, operating system and GPU.
Select In-browser · real in the chat mode controls.
Open the model picker beside Ask and choose a local model. The default for a fresh browser profile is Llama 3.2 · 3B; a saved selection may be restored.
Choose New session if you do not want earlier conversation content included. Changing modes or models alone does not clear the conversation.
Enter a short, non-sensitive prompt and select Ask. The first request triggers engine loading and the model download; selecting a model alone does not load it. Watch the loading percentage and the “running on your device” status.

Configured models and download estimates

These are the application’s current picker labels and approximate size hints, not measured downloads or minimum memory requirements. Runtime memory use can differ from file size.

Llama 3.2 · 3B: approximately 2.3 GB.
Qwen 2 · 7B: approximately 5.1 GB.
Phi-3.5 mini: approximately 3.7 GB.
autoLLLM R1 · 7B: approximately 5.1 GB; the configured model is an open R1-Distill-Qwen model.
Qwen Math · 1.5B: approximately 1.6 GB.

The library is imported on demand from a CDN. Its model registry and hosted assets can change, so a configured model can fail to load. There is no paid inference API used by this local chat path, but downloads, device resources and connectivity are still required.

Privacy and offline limits

The local inference path passes your conversation to the browser’s WebLLM engine rather than to a hosted chat completion endpoint. This is narrower than a guarantee that the whole website makes no network requests: loading the library and model assets requires external requests.

Chat sessions and selections are saved in this browser’s local storage when available. They are not an encrypted vault or a synced account. On a shared device, avoid entering sensitive information and use browser site-data controls when you need to remove stored data. New session starts another conversation; it does not delete previous sessions.

Do not rely on fully offline operation. Asset caching and browser storage vary, and this application does not guarantee an offline-ready reload or download-free future sessions.

If loading fails

WebGPU warning: try an up-to-date WebGPU-capable browser on supported hardware. The app checks whether WebGPU is exposed; that is not a full compatibility or memory test.
Loading stalls or errors: check connectivity and available device resources, close other GPU-heavy tabs, then try a smaller configured model. The generic loading error does not identify a single cause.
Still unavailable: use an official cloud provider link, or switch to Instant only if a scripted demonstration is what you want. Instant is not a substitute for a real model answer.

The built-in local path does not attach a web-search tool. Asking for citations does not make it browse or verify facts. For sourced web research, see Perplexity versus ChatGPT; for cloud writing options, see ChatGPT alternatives.

Official technical references

WebLLM overview describes browser inference and WebGPU. The official WebLLM repository documents supported models and integration. Those capabilities do not imply every listed upstream model is configured in LLMinONE.

Open chat and choose In-browser · real Browse official provider links