Text LLM training from scratch with PyTorch
A clean, readable codebase that implements the full LLM training pipeline (pretraining, SFT, DPO, GRPO/RL) using only PyTorch primitives, avoiding high-level abstractions. It helps developers understand the underlying math and mechanics of LLM training.
Visit project ↗Entities
Related
Interactive app teaching LLM pipeline from pattern matching to transformer training
An interactive web app that walks through every stage of the LLM pipeline, from basic pattern matching to training a transformer from scratch. It includes working code that users can run locally, making it an educational tool for developers and learners.
Software engineer publishes final part of LLM-from-scratch series covering inference and decoding
A software engineer published the fourth and final part of a blog series explaining LLMs from the ground up, focusing on token-by-token generation, KV cache, and decoding strategies (temperature, top-k, top-p). The series aims to help other software engineers understand the internals of LLMs.
Developer trains 113M-parameter earthquake LLM from scratch
A developer trained a 113M-parameter language model specifically for earthquake-related text, building it from scratch. The project, shared on GitHub, demonstrates a focused application of LLMs in geoscience.
Developer completes Python-for-AI course covering agents and LLM evals
A developer completed a comprehensive Python-for-AI course that covered core Python, data structures, tooling, and agent-specific material including LLM evals, the Analyze-Measure-Improve cycle, and building a basic AI coding agent from scratch. The course also covered first-principles agent architecture with intelligence layer, memory, tools, validation, and control.
AgentMaker: a new Python framework for building LLM agents and multi-agent systems
AgentMaker is a general-purpose Python framework for building LLM agents and multi-agent systems, featuring tools, memory, RAG, context engineering, guardrails, human-in-the-loop, and observability. It is released under MIT license on GitHub and PyPI.