--- type: Project title: In-browser language model description: A tiny language model trained by Noel that runs entirely in the visitor's browser and shows its internals live. tags: [architecture, ml, typescript, learning] status: draft generated: { by: claude-code/claude-opus-5-5, at: 2026-10-11T12:15:30Z } --- # Goal A from-scratch learning project on the same domain: a small transformer whose attention patterns, next-token probabilities and temperature are visible and adjustable in the browser. # Architecture No backend. A `/lm` page in the Nuxt build plus a weights file, both served statically by the [gateway](/architecture/gateway.md). It adds nothing to the cluster. # Plan | Step | What | Guide | |---|---|---| | Base model | About 5M parameters, layers written by hand, trained with PyTorch on TinyStories with word-piece tokens | Karpathy "Zero to Hero", llama2.c | | Browser inference | Forward pass in plain TypeScript | llama2.c | | Visuals | Attention heatmaps, next-token probabilities, temperature slider | | | Chat | Fine-tune at 15-30M parameters on simple dialogues in a fixed user/bot format | | | Own engine | Replace PyTorch gradients with Noel's own autograd | micrograd | # Expectations - TinyStories gives coherent short stories. Grimm gives stylish nonsense. - The chat handles greetings and small talk, not knowledge questions or long conversations. # Constraints - **Download size:** 5M parameters in fp32 is about 20 MB, 30M is about 120 MB. Ship int8 weights (about 4× smaller) and load them lazily. - **PyTorch is new tooling** and needs Noel's approval under the stack rule.