Your own LLM: Train your own LLM in 5 minutes on Google Colab

You know that feeling when you use ChatGPT and it feels like magic you can't explain? Every LLM tutorial either hand-waves the fundamentals or drowns you in theory without code you can run. You've heard about transformers and tokenizers, but you've never watched a loss curve drop on a model you actually built — and that gap makes all of AI feel like a black box.

Think of it like teaching a child to finish your sentences — you show them thousands of examples, they spot patterns, eventually they predict what comes next. GuppyLM does this with text: you feed it 60K fish conversations (synthetically generated from templates), it learns which words tend to follow others. The architecture is a vanilla transformer — 6 layers, 384 hidden dimensions, 4096-token vocabulary — small enough to train in 5 minutes. You run one Colab notebook, watch loss drop from 10.0 to 0.384, and chat with your creation.

• 5-minute training on free GPU — run the full pipeline (data, tokenizer, model, inference) on Colab's free T4 tier with zero setup
• Complete from-scratch pipeline — synthetic data generation, BPE tokenizer training, vanilla transformer, training loop, and chat interface all in one repo
• Educational architecture choices — creator documents what they removed (system prompts, chain-of-thought, multi-turn, fancy activations) and why each failed at 9M params
• Fork-and-build-your-own — swap the fish personality for any character by editing one Python file with topic generators
• Pre-trained model on HuggingFace — skip training entirely and chat with Guppy in 30 seconds via the use_guppylm notebook
• MIT licensed everything — code, model weights, and 60K dataset all permissively licensed for any use

Who It Is For

If you've used GPT-4 but couldn't explain what a transformer actually does, this is your entry point. Perfect for developers who learn by doing — you'll touch every layer of the stack. Not for you if you need production-ready multi-turn chat or factual Q&A; this is intentionally tiny and domain-locked.

Worth Exploring?

Yes — specifically if you want to demystify LLMs. The creator's honest documentation of failed experiments (system prompts, chain-of-thought, multi-turn) is worth more than the code itself. This is educational infrastructure, not production tooling. Expect to spend an afternoon, walk away understanding tokenizers, embeddings, attention, and loss curves viscerally.