own-proxy decision, terraform provisioner, system diagram, gateway SSE notes, server-budget comment and roadmap 0.4.0 now use Caddy with ACME-managed TLS instead of nginx plus certbot
49 lines
2.0 KiB
Markdown
49 lines
2.0 KiB
Markdown
---
|
||
type: Project
|
||
title: In-browser language model
|
||
description: A tiny language model trained by Noel that runs entirely in the visitor's browser and shows its internals live.
|
||
tags: [architecture, ml, typescript, learning]
|
||
status: draft
|
||
generated: { by: claude-code/claude-opus-5-5, at: 2026-10-11T12:15:30Z }
|
||
---
|
||
|
||
# Goal
|
||
|
||
A from-scratch learning project on the same domain: a small transformer whose
|
||
attention patterns, next-token probabilities and temperature are visible and
|
||
adjustable in the browser.
|
||
|
||
# Decision (2026-10-15)
|
||
|
||
Browser-only, confirmed by Noel: the model runs in the visitor's browser.
|
||
A **server-run inference variant is cut** — the visitor's device carries the
|
||
compute and the server only serves the weights file. This is part of the
|
||
planned post-6-months stretch of the [learning contract](/decisions/learning-contract.md).
|
||
|
||
# Architecture
|
||
|
||
No backend. A `/lm` page in the Nuxt build plus a weights file, both served
|
||
statically by the [gateway](/architecture/gateway.md). It adds nothing to the cluster.
|
||
|
||
# Plan
|
||
|
||
| Step | What | Guide |
|
||
|---|---|---|
|
||
| Base model | About 5M parameters, layers written by hand, trained with PyTorch on TinyStories with word-piece tokens | Karpathy "Zero to Hero", llama2.c |
|
||
| Browser inference | Forward pass in plain TypeScript | llama2.c |
|
||
| Visuals | Attention heatmaps, next-token probabilities, temperature slider | |
|
||
| Chat | Fine-tune at 15-30M parameters on simple dialogues in a fixed user/bot format | |
|
||
| Own engine | Replace PyTorch gradients with Noel's own autograd | micrograd |
|
||
|
||
# Expectations
|
||
|
||
- TinyStories gives coherent short stories. Grimm gives stylish nonsense.
|
||
- The chat handles greetings and small talk, not knowledge questions or long conversations.
|
||
|
||
# Constraints
|
||
|
||
- **Download size:** 5M parameters in fp32 is about 20 MB, 30M is about 120 MB.
|
||
Ship int8 weights (about 4× smaller) and load them lazily.
|
||
- **Training happens on Noel's own computer.** The [server](/architecture/server-budget.md) only serves the weights file.
|
||
- **PyTorch is new tooling** and needs Noel's approval under the stack rule.
|