A tiny language model trained by Noel that runs entirely in the visitor's browser and shows its internals live.
architecture
ml
typescript
learning
draft
by
at
claude-code/claude-opus-5-5
2026-10-11T12:15:30Z
Goal
A from-scratch learning project on the same domain: a small transformer whose
attention patterns, next-token probabilities and temperature are visible and
adjustable in the browser.
Architecture
No backend. A /lm page in the Nuxt build plus a weights file, both served
statically by the gateway. It adds nothing to the cluster.
Plan
Step
What
Guide
Base model
About 5M parameters, layers written by hand, trained with PyTorch on TinyStories with word-piece tokens
Karpathy "Zero to Hero", llama2.c
Browser inference
Forward pass in plain TypeScript
llama2.c
Visuals
Attention heatmaps, next-token probabilities, temperature slider
Chat
Fine-tune at 15-30M parameters on simple dialogues in a fixed user/bot format
Own engine
Replace PyTorch gradients with Noel's own autograd
micrograd
Expectations
TinyStories gives coherent short stories. Grimm gives stylish nonsense.
The chat handles greetings and small talk, not knowledge questions or long conversations.
Constraints
Download size: 5M parameters in fp32 is about 20 MB, 30M is about 120 MB.
Ship int8 weights (about 4× smaller) and load them lazily.
Training happens on Noel's own computer. The server only serves the weights file.
PyTorch is new tooling and needs Noel's approval under the stack rule.