The Chatbot Nobody Trained
WPLM is a small transformer chatbot whose weights were programmed, not trained. No optimizer ever touched it: every weight in its five blocks was written into an empty network from a plain-text edit list, one line at a time. When it needs words, it asks a ghost table, an N-gram memory built once from a corpus and never trained. What to ask, what to trust, what to say and when to stop are all decided by the programmed weights.
Architecture: the CPU–GPU dual LM from The Mystery of the Unlearned Parameters. A ghost table offers candidates with counts; a small transformer weighs them and chooses. That post asks how little a network needs to learn. WPLM learns nothing: even the transformer is programmed.
Chat with WPLM
Pick a message or type one. Every reply was recorded from the real bot. Press “How?” under a reply for its activation trace: every neuron and attention head at every step, and the exact weights behind any cell you click.
Rename the bot by editing weights
Version 1.0 called itself Gram and said its weights were “hand-written”. We renamed it WPLM and reworded those lines by editing the weight file only: no code change, no retraining, the same neurons and tensor shapes. Switch versions and watch the actual weights move.
The whole model, raw
Every nonzero number in WPLM, at its real row and column. Pick a matrix; zoom with the buttons, a double-click, or Ctrl/⌘ + scroll (pinch on a trackpad); drag to pan; hover for the exact value. Everything not drawn is exactly zero.
How the weights were programmed
The network side of the dual LM is a decoder transformer: a fixed token-code embedding, five blocks (attention heads in the first three, a ReLU MLP in every block), a fixed output read-out, and no LayerNorm. Every block parameter starts at zero. A generator script writes the edit list, and loading the list fills in the weights. Nothing is learned.
One neuron, six lines
This neuron decides “ask the ghost table for replies now”. It fires only at the start of a reply, when the message is not a joke request or a question about the bot, and before any lookup has run.
Weights or ghost table?
Same message, same ghost table. One reply went through WPLM’s programmed weights. The other is what the ghost table gives with no weights at all: one random matching line. Which one came from the weights?
Not always charming
Real recorded replies. Most happen when the ghost table has a bad match and the weights trust it; sometimes the weights misread the message.
What this shows, and its limits
Shows
Limits
The ghost table
Most of WPLM’s words come from its ghost table. The architecture is the CPU–GPU dual LM from The Mystery of the Unlearned Parameters: a statistical LM built once from a corpus offers candidates with counts, and a small neural LM weighs them and makes the choice. The ghost table is not the point of this page, so here it is in one table.
WPLM vs naive infini-gram
Naive infini-gram.
Both sides use the same ghost table and one seeded sample per message; nothing in a match is picked by hand. Bad matches can still get through WPLM, as the bloopers show.