Lectures

Week   Topic(s)                                                     Additional Resources
Week 01
  Introduction [pdf]

  
Week 02
 Tokenization, Word Embeddings and Representational Learning
- Tokenization [pdf]
- Word Emdeddings [pdf]
- Contextual Embeddings[pdf]

- BERT Embedding Example 1 (ipynb)
- BERT Embedding Example 2 (ipynb)
- https://github.com/SumanthRH/tokenization/
Week 03 - LLMs Basics [pdf] - Simple Neural Networks and Neural Language Models [pdf]
- Large Language Models explained briefly by 3Blue1Brown [video]
- SLP3 Book Chapter 7 [pdf]
Week 04 Attention in Transformers
- Attention [pdf]
- Numerical Example [pdf]		 - Attention Is All You Need [pdf]
- The Illustrated Transformer [html]
Week 05 - Transformer Arhitecture [pdf] - Transformers, the tech behind LLMs [video]
- Attention in transformers, step-by-step [video]
- How might LLMs store facts [video]
- LLM Visualization [html]

- Efficient Attention Mechanisms for Large Language Models: A Survey [pdf]
Week 06 Transformer-Based Architectures and Models [pdf]

Mixture of Experts [pdf]		 -BERT: Pre-training of Deep Bidirectional Transformers for Language
-Understanding Contextual Word Representations: A Contextual Introduction
- The Illustrated BERT, ELMo, and co.
- Jurafsky and Martin Chapter 10 (Masked Language Models)
- The Llama 3 Herd of Models
Week 07 LLM Training
- Pre-Training [pdf]
- Post-Training
   - Intruction Tuning [pdf]
   - Preference Alignment [pdf] 		- Post-training: Instruction Tuning, Alignment, and Test-Time Compute
- Datasets for large language models: a comprehensive survey [pdf]
Week 08 Midterm Week  
Week 09 - LLM Inferencing [pdf]
- Prompting, ICL, RAG, FT [pdf]
- Prompt Enginneering [pdf]
 		 - https://github.com/dlops-io/llm-rag?tab=readme-ov-file#agents
- https://developers.openai.com/api/docs/guides/prompt-engineering
- https://developers.openai.com/cookbook/articles/related_resources
- https://developers.openai.com/api/docs/guides/prompt-guidance
Week 10 - Information Retrieval and
  Retrieval Augmented Generation - RAG [pdf]
 - https://web.stanford.edu/~jurafsky/slp3/11.pdf
- GitHub - Tongji-KGLLM/RAG-Survey · GitHub

- PyTorch Tutorial from CS224n [ipynb]
Week 11 - Parameter Efficient Fine Tuning [pdf]
- LMM Model Compression [pdf]		 - Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning [pdf]
- https://huggingface.co/blog/samuellimabraz/peft-methods
- https://sumanthrh.com/post/distributed-and-efficient-finetuning/
- https://github.com/stas00/ml-engineering
- https://huggingface.co/docs/transformers/v4.20.1/en/training
- https://huggingface.co/docs/transformers/v4.20.1/en/tasks/token_classification

- Hugging Face Transformers Tutorial from CS224n [pdf, ipynb]
Week 12 - Agentic AI I [pdf]
- Agentic AI II [pdf]
- Agentic AI Frameworks [pdf]		 - https://language-agent-tutorial.github.io/
- https://mitsloan.mit.edu/ideas-made-to-matter/agentic-ai-explained
- https://arxiv.org/pdf/2505.10468
- https://www.geeksforgeeks.org/artificial-intelligence/agentic-ai-tutorial/
Week 13 Model Context Protocol (MCP) [pdf] - https://www.anthropic.com/news/model-context-protocol
- https://github.com/modelcontextprotocol
- https://en.wikipedia.org/wiki/Model_Context_Protocol
- https://www.geeksforgeeks.org/artificial-intelligence/model-context-protocol-mcp/
- https://www.youtube.com/watch?v=c7yl0GS2mJQ
- https://github.com/kmkarakaya/mcp_tutorial
Week 14 Multi-Modal LLMs [pdf] - https://magazine.sebastianraschka.com/p/understanding-multimodal-llms
- https://www.ibm.com/think/topics/multimodal-llm
- https://arxiv.org/pdf/2408.01319v1
     
     
 
Similar Courses and  Resources :
   - CS224N: Natural Language Processing with Deep Learning [html]
   - CME 295 - Transformers & Large Language Model [html]
   - CS336: Language Modeling from Scratch [html]
  - COS 484: Natural Language Processing [html]
   - COS 597G (Fall 2022): Understanding Large Language Models [html]
   - CS 194/294-267 Understanding Large Language Models: Foundations and Safety [html]
   - LLM Courses [html]
   - HuggingFace LLM Course [html]
   - M. Labonne-llm-course [html]
   - CSC 2541 Winter 2025: Large Models [html]
   - CSC6203: Large Language Model [html]
   - CMU 11868 Large Language Model Systems [html]
   - CS 886: Recent Advances on Foundation Models - Waterloo [html]
   - AC215 : MLOps & LLMOps: Production AI Systems - AC215 [html]
   - CS294/194-196: Agentic AI [html]

CS 194/294-267 Understanding Large Language Models: Foundations and Safety