own-proxy decision, terraform provisioner, system diagram, gateway SSE notes, server-budget comment and roadmap 0.4.0 now use Caddy with ACME-managed TLS instead of nginx plus certbot
2.0 KiB
2.0 KiB
type, title, description, tags, status, generated
| type | title | description | tags | status | generated | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Project | In-browser language model | A tiny language model trained by Noel that runs entirely in the visitor's browser and shows its internals live. |
|
draft |
|
Goal
A from-scratch learning project on the same domain: a small transformer whose attention patterns, next-token probabilities and temperature are visible and adjustable in the browser.
Decision (2026-10-15)
Browser-only, confirmed by Noel: the model runs in the visitor's browser. A server-run inference variant is cut — the visitor's device carries the compute and the server only serves the weights file. This is part of the planned post-6-months stretch of the learning contract.
Architecture
No backend. A /lm page in the Nuxt build plus a weights file, both served
statically by the gateway. It adds nothing to the cluster.
Plan
| Step | What | Guide |
|---|---|---|
| Base model | About 5M parameters, layers written by hand, trained with PyTorch on TinyStories with word-piece tokens | Karpathy "Zero to Hero", llama2.c |
| Browser inference | Forward pass in plain TypeScript | llama2.c |
| Visuals | Attention heatmaps, next-token probabilities, temperature slider | |
| Chat | Fine-tune at 15-30M parameters on simple dialogues in a fixed user/bot format | |
| Own engine | Replace PyTorch gradients with Noel's own autograd | micrograd |
Expectations
- TinyStories gives coherent short stories. Grimm gives stylish nonsense.
- The chat handles greetings and small talk, not knowledge questions or long conversations.
Constraints
- Download size: 5M parameters in fp32 is about 20 MB, 30M is about 120 MB. Ship int8 weights (about 4× smaller) and load them lazily.
- Training happens on Noel's own computer. The server only serves the weights file.
- PyTorch is new tooling and needs Noel's approval under the stack rule.