Foundation Model Engineering
Menu Table of Contents 1. The Evolution of Intelligence 1.1 Symbolism vs. Connectionism 1.2 The Power of Representation 1.3 Deep Learning Paradigms 1.4 The Bitter Lesson 2. The Sequence Modeling Era 2.1 Markov Chains to RNNs 2.2 Vanishing/Exploding Gradients 2.3 The Dawn of Attention 2.4 CNNs for NLP 3. The Transformer Deep Dive 3.1 Self-Attention Mathematics 3.2 Multi-head Attention (MHA) 3.3 Position Encoding Strategy 3.4 Layer Normalization & Residuals 3.5 Complexity Analysis 4. LLM Architectures & Paradigms 4.1 Encoder-only (BERT-style) 4.2 Decoder-only (GPT-style) 4.3 Encoder-Decoder (T5/BART) 4.4 Hybrid & Prefix LM 4.5 Various LLM Architectures & Latest Trends 5. Scaling Mixture of Experts (MoE) 5.1 Sparse vs Dense Models 5.2 Routing Algorithms 5.3 Expert Parallelism 5.4 Collapsing & Load Balancing 5.5 Case Study 6. Foundation Model Pre-training 6.1 Data Engineering at Scale 6.2 Tokenization Science 6.3 Large-scale Training Stability 6.4 Infrastructure 6.5 Synthetic Data for Pre-training 7. Training Optimization & Systems 7.1 Data Parallelism (DP/DDP) 7.2 ZeRO (Zero Redundancy Optimizer) 7.3 Model & Pipeline Parallelism 7.4 Flash Attention 1, 2, 3 8. Scaling Laws & Compute Optimality 8.1 The Power Law 8.2 Chinchilla Optimality 8.3 Over-training vs Optimal-training 8.4 Transfer Learning & Generalization 8.5 Continued Pre-training & Domain Adaptation 9. Post-training: SFT & Instruction Tuning 9.1 SFT Fundamentals 9.2 Dataset Quality vs Quantity 9.3 Parameter-Efficient Fine-Tuning (PEFT) 9.4 Prompt Engineering as SFT 9.5 Synthetic Instructions & Self-Instruct 10. Alignment: RLHF & Direct Preference 10.1 Human Feedback Loop 10.2 PPO (Proximal Policy Optimization) 10.3 DPO (Direct Preference Optimization) 10.4 KTO & IPO 10.5 Alignment Tax 11. Multimodal Learning 11.1 Vision-Language Bridges 11.2 Audio & Speech Integration 11.3 Unified Multimodal (Any-to-Any) 11.4 Image Diffusion Models 11.5 Video Generation Foundations 11.6 Commercial Video Models & Future Directions 12. LLM Inference Optimization 12.1 KV Cache Management 12.2 PagedAttention (vLLM) 12.3 Continuous Batching 12.4 Speculative Decoding 12.5 Long Context Serving 12.6 Serving Policies, SLOs, and Fallbacks 13. Model Compression & Quantization 13.1 PTQ vs QAT 13.2 Quantization Methods 13.3 Weight Sparsification 13.4 Knowledge Distillation 13.5 Advanced Quantization 14. RAG (Retrieval Augmented Generation) 14.1 From Lexical to Semantic Search 14.2 Vector Indexing & DB Solutions 14.3 Advanced Retrieval & Reranking 14.4 RAG Orchestration 14.5 GraphRAG & Ontology 14.6 RAG Failure Modes and Operational Design 15. Reasoning & Search-time Scaling 15.1 Chain of Thought (CoT) 15.2 Tree/Graph of Thoughts 15.3 Search-time Compute 15.4 Verifiers & Reward Models 16. Agentic AI & Tools 16.1 Function Calling & Tool Use 16.2 Autonomous Agents 16.3 Self-Improving Agents 16.4 Multi-agent Collaboration 16.5 Long-term Memory for Agents 16.6 Agent Reliability, Recovery, and Guardrails 17. AI Evaluation & Benchmarking 17.1 Academic Benchmarks 17.2 LLM-as-a-Judge 17.3 Elo Rating & Leaderboards 17.4 Contamination Issues 17.5 Production Evaluation and Release Gates 17.6 Commercial Model Benchmarks 18. AI Safety & Alignment Research 18.1 Red Teaming 18.2 Jailbreaking & Defense 18.3 Hallucination Detection 18.4 Scalable Oversight 19. Interpretability & Science of LLMs 19.1 Mechanistic Interpretability 19.2 Logit Lens & Attention Visualization 19.3 Probing Classifiers 19.4 Sparse Autoencoders (SAE) 20. Next Generation: SSM & Beyond 20.1 State Space Models (SSM) 20.2 Mamba & S6 20.3 Linear Attention 20.4 Neural Networks as Programs 20.5 Multi-Token Prediction 20.6 Diffusion-based LLMs 20.7 Path to AGI & World Models Foundation Model Engineering EN English (EN) Korean (KO) Last update: 2026-08-16 Foundation Model Engineering is a technical textbook for readers who want to understand how modern foundation models actually work, why the stack evolved the way it did, and what engineering trade-offs appear when those ideas meet real systems. This project is written primarily for AI engineers and research-oriented readers who want to move past surface-level API usage and build a deeper mental model of architectures, training pipelines, inference systems, retrieval stacks, evaluation loops, and agentic workflows. The goal is not to provide scattered tips or isolated definitions. The goal is to explain the historical flow, mathematical ideas, and systems constraints that connect topics like attention, MoE, RLHF, multimodality, long-context serving, RAG, and agents into one engineering narrative. Why read this If you have ever wondered why the field moved from RNNs to Transformers, why some models are dense while others are sparse, why inference systems care so much about KV cache and batching, or why evaluation and alignment are product problems rather than just research topics, this book is meant to help you connect those dots. Instead of treating each topic as an isolated trend, the book tries to show how modeling ideas, systems constraints, and product requirements shape one another. The payoff is not just more terminology. It is better engineering judgment. Who this is for AI EngineersReaders building or evaluating LLM systems, inference stacks, RAG systems, or agentic products. Research-Oriented ReadersReaders who want a broad but technically grounded understanding of the foundation model landscape, including current architecture and systems trends. What to expect You will find rigorous conceptual explanations, concept-focused PyTorch examples, short quizzes for consolidation, and interactive visualizers for topics that are easier to understand by manipulating them directly. The material is designed to help you reason about quality, memory, throughput, latency, scaling, and alignment trade-offs, not just memorize terminology. This is not a lightweight beginner introduction. If you are looking for a first overview of AI or a prompt-engineering-only guide, this book will probably feel denser than necessary. It is intentionally written for readers who want depth. A Living Document AI changes extremely quickly, so some details in a project like this may need revision as new papers, systems, and products appear. If you spot an outdated section, an awkward explanation, a typo, or a better reference, contributions are always welcome. Pull requests that improve accuracy, pedagogy, examples, localization, or overall clarity are appreciated. The goal is for this to remain a useful long-term resource, not a frozen snapshot. Table of Contents 1. The Evolution of Intelligence 1.1 Symbolism vs. Connectionism1.2 The Power of Representation1.3 Deep Learning Paradigms1.4 The Bitter Lesson 2. The Sequence Modeling Era 2.1 Markov Chains to RNNs2.2 Vanishing/Exploding Gradients2.3 The Dawn of Attention2.4 CNNs for NLP 3. The Transformer Deep Dive 3.1 Self-Attention Mathematics3.2 Multi-head Attention (MHA)3.3 Position Encoding Strategy3.4 Layer Normalization & Residuals3.5 Complexity Analysis 4. LLM Architectures & Paradigms 4.1 Encoder-only (BERT-style)4.2 Decoder-only (GPT-style)4.3 Encoder-Decoder (T5/BART)4.4 Hybrid & Prefix LM4.5 Various LLM Architectures & Latest Trends 5. Scaling Mixture of Experts (MoE) 5.1 Sparse vs Dense Models5.2 Routing Algorithms5.3 Expert Parallelism5.4 Collapsing & Load Balancing5.5 Case Study 6. Foundation Model Pre-training 6.1 Data Engineering at Scale6.2 Tokenization Science6.3 Large-scale Training Stability6.4 Infrastructure6.5 Synthetic Data for Pre-training 7. Training Optimization & Systems 7.1 Data Parallelism (DP/DDP)7.2 ZeRO (Zero Redundancy Optimizer)7.3 Model & Pipeline Parallelism7.4 Flash Attention 1, 2, 3 8. Scaling Laws & Compute Optimality 8.1 The Power Law8.2 Chinchilla Optimality8.3 Over-training vs Optimal-training8.4 Transfer Learning & Generalization8.5 Continued Pre-training & Domain Adaptation 9. Post-training: SFT & Instruction Tuning 9.1 SFT Fundamentals9.2 Dataset Quality vs Quantity9.3 Parameter-Efficient Fine-Tuning (PEFT)9.4 Prompt Engineering as SFT9.5 Synthetic Instructions & Self-Instruct 10. Alignment: RLHF & Direct Preference 10.1 Human Feedback Loop10.2 PPO (Proximal Policy Optimization)10.3 DPO (Direct Preference Optimization)10.4 KTO & IPO10.5 Alignment Tax 11. Multimodal Learning 11.1 Vision-Language Bridges11.2 Audio & Speech Integration11.3 Unified Multimodal (Any-to-Any)11.4 Image Diffusion Models11.5 Video Generation Foundations11.6 Commercial Video Models & Future Directions 12. LLM Inference Optimization 12.1 KV Cache Management12.2 PagedAttention (vLLM)12.3 Continuous Batching12.4 Speculative Decoding12.5 Long Context Serving12.6 Serving Policies, SLOs, and Fallbacks 13. Model Compression & Quantization 13.1 PTQ vs QAT13.2 Quantization Methods13.3 Weight Sparsification13.4 Knowledge Distillation13.5 Advanced Quantization 14. RAG (Retrieval Augmented Generation) 14.1 From Lexical to Semantic Search14.2 Vector Indexing & DB Solutions14.3 Advanced Retrieval & Reranking14.4 RAG Orchestration14.5 GraphRAG & Ontology14.6 RAG Failure Modes and Operational Design 15. Reasoning & Search-time Scaling 15.1 Chain of Thought (CoT)15.2 Tree/Graph of Thoughts15.3 Search-time Compute15.4 Verifiers & Reward Models 16. Agentic AI & Tools 16.1 Function Calling & Tool Use16.2 Autonomous Agents16.3 Self-Improving Agents16.4 Multi-agent Collaboration16.5 Long-term Memory for Agents16.6 Agent Reliability, Recovery, and Guardrails 17. AI Evaluation & Benchmarking 17.1 Academic Benchmarks17.2 LLM-as-a-Judge17.3 Elo Rating & Leaderboards17.4 Contamination Issues17.5 Production Evaluation and Release Gates17.6 Commercial Model Benchmarks 18. AI Safety & Alignment Research 18.1 Red Teaming18.2 Jailbreaking & Defense18.3 Hallucination Detection18.4 Scalable Oversight 19. Interpretability & Science of LLMs 19.1 Mechanistic Interpretability19.2 Logit Lens & Attention Visualization19.3 Probing Classifiers19.4 Sparse Autoencoders (SAE) 20. Next Generation: SSM & Beyond 20.1 State Space Models (SSM)20.2 Mamba & S620.3 Linear Attention20.4 Neural Networks as Programs20.5 Multi-Token Prediction20.6 Diffusion-based LLMs20.7 Path to AGI & World Models Next → 1.1 Symbolism vs. Connectionism Created by Seongeun So Developed with AI assistance

Foundation Model Engineering serves as a technical resource designed to provide a deep understanding of how modern foundation models function, tracing the historical evolution of the field, the underlying mathematical principles, and the engineering trade-offs encountered when applying these concepts to real-world systems. The book aims to bridge the gap between theoretical modeling ideas and practical system implementation across the entire lifecycle of foundation models, covering everything from architecture and training to inference, alignment, and deployment strategies.

The narrative begins with the evolution of intelligence, contrasting symbolism and connectionism, and establishing the power of representation, culminating in deep learning paradigms and concepts like the Bitter Lesson. This historical context transitions into the sequence modeling era, exploring the development from Markov Chains to Recurrent Neural Networks (RNNs), the challenges posed by vanishing and exploding gradients, and the pivotal introduction of the attention mechanism. The core of the book then dives into the Transformer architecture, dissecting the mathematical foundations of self-attention, multi-head attention, position encoding, normalization techniques, and complexity analysis.

The subsequent sections detail various LLM architectures, examining encoder-only, decoder-only, and encoder-decoder paradigms, alongside emerging structures like hybrid models and prefix learning methods. Scaling techniques are explored through the Mixture of Experts (MoE) paradigm, focusing on sparse versus dense models, routing algorithms, and expert parallelism. Foundation model pre-training introduces critical areas such as large-scale data engineering, tokenization science, training stability, infrastructure management, and the use of synthetic data. Training optimization and systems focus on efficient parallelization strategies, including data parallelism, ZeRO, model parallelism, and techniques like Flash Attention.

The process moves toward refinement and deployment by covering post-training stages, including supervised fine-tuning (SFT), prompt engineering, parameter-efficient fine-tuning (PEFT), and the crucial alignment phase using Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO). Multimodal learning is addressed through vision-language bridges, audio integration, and diffusion models, covering the foundations for image and video generation.

The engineering demands of deploying these models are detailed through LLM inference optimization, focusing on managing the key components like the KV cache, implementing efficient serving solutions such as PagedAttention, continuous batching, and speculative decoding for long context serving. Model compression and quantization methods are analyzed, comparing various techniques, including weight sparsity and knowledge distillation, to manage model size and computational cost. Retrieval Augmented Generation (RAG) is treated as a critical application layer, detailing the shift from lexical to semantic search, vector indexing, advanced retrieval, orchestration, and failure modes.

The scope expands into reasoning and agentic systems, covering methodologies for search-time scaling, such as Chain of Thought, Tree of Thoughts, and the use of verifiers, alongside the design of autonomous agents involving function calling, self-improving capabilities, multi-agent collaboration, and reliable memory systems. Essential operational concerns are addressed through rigorous evaluation and benchmarking, including LLM-as-a-Judge, contamination detection, and establishing production release gates.

AI safety and interpretability form another major pillar, investigating mechanisms like red teaming, hallucination detection, and scalable oversight. Interpretability methods explore mechanistic interpretability, visualizing attention, probing classifiers, and sparse autoencoders. Finally, the book looks toward the next generation of models, focusing on State Space Models (SSM), Mamba and S6, linear attention mechanisms, and the vision of neural networks as programs, leading toward the trajectory of Artificial General Intelligence and world models.