Tools › AI Models & ML › LLMs From Scratch

LLMs From Scratch

Build a ChatGPT-like LLM in PyTorch step by step

🧪 AI Models & ML ★ 108.6k +44 this week Intermediate

What it is

LLMs From Scratch is a hands-on book-style repository that teaches how to implement a ChatGPT-like large language model in PyTorch from the ground up, covering tokenization, attention, training and fine-tuning. It is for developers who want to understand and build LLMs rather than just call APIs.

How it works

Jupyter notebooks walk through each stage: preparing text data, implementing attention and the transformer block, pretraining, then fine-tuning for classification and instruction following. You run the notebooks sequentially, optionally scaling to GPU, and end with a small working LLM.

Use cases

  • Learn transformer internals by coding attention and training a small model.
  • Fine-tune a small LLM for a specific classification or instruction task.
  • Teach a paid workshop on how LLMs actually work under the hood.
  • Prototype a lightweight model when API costs or privacy rule out hosted LLMs.

💰 Business opportunities

  • Sell a cohort course teaching LLM internals with GPU-backed notebooks.
  • Offer fine-tuning services for small businesses that need a domain-specific model.
  • Build an internal training program for engineering teams adopting LLMs.

Featured in 1 video

16 Cool GitHub Repos You WILL Use (no pressure)

The Coding Sloth · 2025-06-23

Similar tools

MiniMind

Train a tiny language model from scratch on regular hardware

🧪 AI Models & ML ★ 66.1k +533 wk Python

Microsoft AI for Beginners

Free 12-week AI curriculum with runnable notebooks

🎓 Education & Learning ★ 71k +51 wk Jupyter Notebook

Made With ML

Learn to build, deploy and iterate production ML apps

🧪 AI Models & ML ★ 50.1k +11 wk Jupyter Notebook

TradingAgents

A pretend trading firm staffed entirely by AI agents

💼 Finance & Business ★ 123k +100 wk Python

Ruflo

Multi-agent coding teams with roles, models and memory

🤖 AI Agents & Assistants ★ 82k +76 wk TypeScript

Daily Stock Analysis

LLM-powered multi-market stock analysis with dashboards and alerts

💼 Finance & Business ★ 73.1k +229 wk Python