| Week | Date | Topic | Readings | Note | |
|---|---|---|---|---|---|
| W1 | 1/13 | L1 | Course Overview [slides] | ||
| 1/15 | L2 | Text Classification [slides] |
Logistic Regression Neural Networks |
||
| W2 | 1/20 | Martin Luther King, Jr. Day (No Class) | |||
| 1/22 | L3 | Word Representations [slides] |
Word2Vec GloVe fastText |
Assignment 0 Released [link] | |
| W3 | 1/27 | L4 | Word Representations, Tokenization, Language Modeling [slides] |
Byte-Pair Encoding Smoothing Neural Language Models |
Assignment 1 Released [link] |
| 1/29 | L5 | Convolutional Neural Network, Recurrent Neural Network [slides] |
TextCNN LSTM |
Assignment 0 Due | |
| W4 | 2/3 | L6 | Sequential Labeling, Sequence-to-Sequence, Attention [slides] |
Sequence-to-Sequence Attention-Based RNN |
Project Sign-Up [link] |
| 2/5 | L7 | Transformers [slides] |
Attention Is All You Need The Annotated Transformer The Illustrated Transformer |
||
| W5 | 2/10 | L8 | Transformers [slides] |
Longformer Relative Positional Encoding RoFormer |
|
| 2/12 | L9 | Contextualized Representations, Pre-Training [slides] |
ELMo BERT RoBERTa BART |
||
| W6 | 2/17 | L10 | Pre-Training, Model Distillation [slides] |
T5 GPT-2 Model Distillation |
Assignment 1 Due Quiz 1 |
| 2/19 | L11 | Parameter-Efficient Fine-Tuning, Large Language Models [slides] |
Prompt Tuning Prefix Tuning Adapter MoE LoRA |
||
| W7 | 2/24 | L12 | Large Language Models, Instruction Tuning [slides] |
In-Context Learning Chain-of-Thought |
Assignment 2 Released [link] |
| 2/26 | L13 | Human Preference Alignment [slides] |
RLHF/PPO DPO KTO SimPO GRPO |
||
| W8 | 3/3 | L14 | Alignment, Text Similarity, Retrieval-Augmented Generation [slides] |
SimPO GRPO Sentence-BERT SimCSE RAG |
Project Proposal Due |
| 3/5 | Project Highlight | Presentation Slides [link] | |||
| W9 | 3/10 | Spring Break (No Class) | |||
| 3/12 | Spring Break (No Class) | ||||
| W10 | 3/17 | L15 | Multilingual NLP [slides] |
NLLB XLM-R XTREME Corss-Lingual ICL Multilingual LLMs Thinking |
Assignment 2 Due Quiz 2 |
| 3/19 | L16 | Vision-Language Models [slides] |
VisualBERT CLIP BLIP-2 LLaVA |
||
| W11 | 3/24 | L17 | Adversarial Attack and Defense [slides] |
Word Replacement Attack Paraphrase Attack Jailbreaking LLMs Data Poisoning Attack Certified Robustness |
Assignment 3 Released [link] |
| 3/26 | L18 | AI-Generated Text Detection [slides] |
Grover DetectGPT Fast-DetectGPT Watermarking |
||
| W12 | 3/31 | Invited Talk (Minhao Cheng) | Zoom [link] | ||
| 4/2 | L19 | Bias Detection and Mitigation [slides] |
Bias in Word Embeddings WinoBias Geo-Bias |
Midterm Report Due | |
| W13 | 4/7 | L20 | Hallucinations [slides] |
FActScore Hallucination Snowball SelfCheckGPT Context-Aware Decoding |
|
| 4/9 | L21 | Non-Autoregressive Generation [slides] |
Medusa NAT SynST Diffusion-LM Insertion Transformer |
||
| W14 | 4/14 | L22 | Information and Knowledge Extraction [slides] |
OneIE LM as Knowledge Base Knolwedge Locating |
Assignment 3 Due Quiz 3 |
| 4/16 | Invited Talk (Pan Lu) | Zoom [link] | |||
| W15 | 4/21 | Project Presentation | Presentation Slides [link] | ||
| 4/23 | Project Presentation | Presentation Slides [link] | |||
| W16 | 4/28 | Project Presentation | Presentation Slides [link] | ||
| 4/30 | Reading Day (No Class) | Project Report Due |