Interactive · concept 1
Type anything. Watch your words become tokens.
Before an AI can do anything with your sentence, it chops it into chunks called tokens. Not letters, not quite words. Type below and watch it happen, live, with the real tokenizer.
0 tokens
Real GPT tokenizer (the one GPT-4 uses) · a · marks a space inside a token.
Common everyday words get their own token. Longer or rarer ones, like “tokenization” or your own name, break into pieces. Either way, chunks are all a model ever sees.
Get the calm version of AI news.
One email a day on what is actually happening in AI, in plain English. No hype, no doom.
Free. One calm email a day. No hype, no doom.
The book behind this newsletter
Just Predicting Words
How ChatGPT, Claude, and Modern AI Actually Work
The trick is small. The world it built is not.