Chrome's Built-in LLM (Prompt API) Is Not a Free API
It runs on the visitor's GPU and its result can't be trusted, so use it to assist, not to enforce
These two lines run in Chrome.
const session = await LanguageModel.create()
const answer = await session.prompt("Is this comment abusive?")
No API key, no server. At first glance it looks like a free GPT API. It isn't. It's client-side computation on the visitor's GPU, with a result you can't trust. After reading this you'll have one rule for deciding where the API belongs and where it doesn't.
It started with a Korean YouTube video (Coding Apple, about the "2GB file Chrome installs quietly"). I wanted to see for myself, so I added a section to a JS test app.
Download and use are separate steps
availability() downloads nothing, so calling it on page load is safe. Real preparation starts at create(), and if something needs fetching, you have to call it inside a user click (user activation). A session keeps its conversation history, so for things like judging comments, where each call must be independent, fork it with clone(). Pass a JSON Schema as responseConstraint and the output format is enforced.
What I measured (2026-09-19, macOS, Chrome 152, no flags)
The model was already there, months earlier. Under OptGuideOnDeviceModel/ in the profile folder, weights.bin was 4.27GB, dated 2026-04-27. Five months before I pressed any button. That's the "quietly installed file" from the video.
downloadable didn't mean "no file". It said downloadable on page load, yet after create() the date on weights.bin hadn't changed. Nothing was downloaded. Instead create() took 18 seconds and program_cache.bin appeared at that moment. It looks like GPU preparation (shader compilation), not a download.
Regular Chrome gave me Gemini Nano. The model name in the manifest was v3Nano. I couldn't confirm the Gemma 4 (about 2GB) mentioned in the video on my setup, so this post makes no claims about Gemma 4 quality.
Translation pivots through English. I asked for Korean to Japanese and got two packs: en_ko (75MB) and en_ja (76MB). Korean goes to English, then to Japanese. Nice that N languages need only 2N models. The catch is that nuance English doesn't have, like Korean and Japanese politeness levels, can get lost in the middle. Not great for Korean-Japanese.
The abuse check contradicted itself.
{"toxic": true, "category": "insult",
"reason": "Uses a positive expression about appearance toward a specific person, implying an insult."}
(Translated from the Korean output.) A "positive expression" flagged as an "insult". responseConstraint kept the JSON shape intact, but the content can't be trusted. This is the rule below in action.
Compared with a server API
| Server LLM API | Chrome built-in LLM | |
|---|---|---|
| Cost | You pay per token | Zero for you. The visitor pays in power, GPU and disk |
| Input data | Leaves the device | Stays on the device |
| Trust in the result | Comes from your server | Anyone can change it in DevTools |
| Who can use it | Any browser | Desktop Chrome with hardware requirements. Not on Android or iOS yet |
| Model | Your choice | Whatever the browser ships (Gemini Nano for me) |
The hardware bar is high. Per the official docs you need 22GB free on the profile's disk and more than 4GB of VRAM. Don't assume every visitor has it.
One rule: does anyone lose if the client fakes the result?
| Use | Verdict | Why |
|---|---|---|
| "This sounds harsh, send anyway?" before posting | Use | Faking it only hurts the poster. Keep server checks separately |
| Blocking abuse instead of a server check | Don't | One line of JS gets around it |
| Suggesting alt text for blog images | Use | The author fixes mistakes |
| Translation or summaries for the reader | Use | The person computing is the person reading |
| Points, rankings, billing | Don't | Faking it pays |
| Building a whole feature on it | Don't | Non-Chrome users get nothing. You need a fallback |
In one line: use it to assist, never to enforce.
If you need enforcement, do it on the server. For fast, cheap server-side decisions there are options like Jev.
If you're deciding whether a feature should run in the browser or on a server, negabaro is happy to talk it through. Drawing these lines is work we've done on real services.
Key Concepts
Call only availability() on page load. It downloads nothing
Call create() inside a user click. If downloadable, preparation starts here
For independent calls like judging, fork the session with clone()
Enforce the format with responseConstraint, but don't trust the content
Always keep a fallback for non-Chrome users
Pros
- ✓ Zero inference cost for you: no server, no key, no billing
- ✓ Input never leaves the device
Cons
- ✗ The result can't be trusted. The client can change it
- ✗ Desktop Chrome on high-spec machines only
- ✗ Translation pivots through English and can lose politeness nuance