42 lines
1.7 KiB
Markdown
42 lines
1.7 KiB
Markdown
---
|
||
type: Project
|
||
title: In-browser language model
|
||
description: A tiny language model trained by Noel that runs entirely in the visitor's browser and shows its internals live.
|
||
tags: [architecture, ml, typescript, learning]
|
||
status: draft
|
||
generated: { by: claude-code/claude-opus-5-5, at: 2026-10-11T12:15:30Z }
|
||
---
|
||
|
||
# Goal
|
||
|
||
A from-scratch learning project on the same domain: a small transformer whose
|
||
attention patterns, next-token probabilities and temperature are visible and
|
||
adjustable in the browser.
|
||
|
||
# Architecture
|
||
|
||
No backend. A `/lm` page in the Nuxt build plus a weights file, both served
|
||
statically by the [gateway](/architecture/gateway.md). It adds nothing to the cluster.
|
||
|
||
# Plan
|
||
|
||
| Step | What | Guide |
|
||
|---|---|---|
|
||
| Base model | About 5M parameters, layers written by hand, trained with PyTorch on TinyStories with word-piece tokens | Karpathy "Zero to Hero", llama2.c |
|
||
| Browser inference | Forward pass in plain TypeScript | llama2.c |
|
||
| Visuals | Attention heatmaps, next-token probabilities, temperature slider | |
|
||
| Chat | Fine-tune at 15-30M parameters on simple dialogues in a fixed user/bot format | |
|
||
| Own engine | Replace PyTorch gradients with Noel's own autograd | micrograd |
|
||
|
||
# Expectations
|
||
|
||
- TinyStories gives coherent short stories. Grimm gives stylish nonsense.
|
||
- The chat handles greetings and small talk, not knowledge questions or long conversations.
|
||
|
||
# Constraints
|
||
|
||
- **Download size:** 5M parameters in fp32 is about 20 MB, 30M is about 120 MB.
|
||
Ship int8 weights (about 4× smaller) and load them lazily.
|
||
- **Training happens on Noel's own computer.** The [server](/architecture/server-budget.md) only serves the weights file.
|
||
- **PyTorch is new tooling** and needs Noel's approval under the stack rule.
|