| Week | Date | Topic | Readings | Note | |
|---|---|---|---|---|---|
| W1 | 1/13 | L1 | Course Overview | ||
| 1/15 | L2 | Text Classification |
Logistic Regression Neural Networks |
||
| W2 | 1/20 | Martin Luther King, Jr. Day (No Class) | |||
| 1/22 | L3 | Word Representations |
Word2Vec GloVe fastText |
Assignment 0 Released | |
| W3 | 1/27 | L4 | Word Representations, Tokenization, Language Modeling |
Byte-Pair Encoding Smoothing Neural Language Models |
Assignment 1 Released |
| 1/29 | L5 | Convolutional Neural Network, Recurrent Neural Network |
TextCNN LSTM |
Assignment 0 Due | |
| W4 | 2/3 | L6 | Sequential Labeling, Sequence-to-Sequence, Attention |
Sequence-to-Sequence Attention-Based RNN |
Project Sign-Up |
| 2/5 | L7 | Transformers |
Attention Is All You Need The Annotated Transformer The Illustrated Transformer |
||
| W5 | 2/10 | L8 | Transformers |
Longformer Relative Positional Encoding RoFormer |
|
| 2/12 | L9 | Contextualized Representations, Pre-Training |
ELMo BERT RoBERTa BART |
||
| W6 | 2/17 | L10 | Pre-Training, Model Distillation |
T5 GPT-2 Model Distillation |
Assignment 1 Due Quiz 1 |
| 2/19 | L11 | Parameter-Efficient Fine-Tuning, Large Language Models |
Prompt Tuning Prefix Tuning Adapter MoE LoRA |
||
| W7 | 2/24 | L12 | Large Language Models, Instruction Tuning |
In-Context Learning Chain-of-Thought |
Assignment 2 Released |
| 2/26 | L13 | Human Preference Alignment |
RLHF/PPO DPO KTO SimPO GRPO |
||
| W8 | 3/3 | L14 | Alignment, Text Similarity, Retrieval-Augmented Generation |
SimPO GRPO Sentence-BERT SimCSE RAG |
Project Proposal Due |
| 3/5 | Project Highlight | Presentation Slides | |||
| W9 | 3/10 | Spring Break (No Class) | |||
| 3/12 | Spring Break (No Class) | ||||
| W10 | 3/17 | L15 | Multilingual NLP |
NLLB XLM-R XTREME Corss-Lingual ICL Multilingual LLMs Thinking |
Assignment 2 Due Quiz 2 |
| 3/19 | L16 | Vision-Language Models |
VisualBERT CLIP BLIP-2 LLaVA |
||
| W11 | 3/24 | L17 | Adversarial Attack and Defense |
Word Replacement Attack Paraphrase Attack Jailbreaking LLMs Data Poisoning Attack Certified Robustness |
Assignment 3 Released |
| 3/26 | L18 | AI-Generated Text Detection |
Grover DetectGPT Fast-DetectGPT Watermarking |
||
| W12 | 3/31 | Invited Talk (Minhao Cheng) | Zoom | ||
| 4/2 | L19 | Bias Detection and Mitigation |
Bias in Word Embeddings WinoBias Geo-Bias |
Midterm Report Due | |
| W13 | 4/7 | L20 | Hallucinations |
FActScore Hallucination Snowball SelfCheckGPT Context-Aware Decoding |
|
| 4/9 | L21 | Non-Autoregressive Generation |
Medusa NAT SynST Diffusion-LM Insertion Transformer |
||
| W14 | 4/14 | L22 | Information and Knowledge Extraction |
OneIE LM as Knowledge Base Knolwedge Locating |
Assignment 3 Due Quiz 3 |
| 4/16 | Invited Talk (Pan Lu) | Zoom | |||
| W15 | 4/21 | Project Presentation | Presentation Slides | ||
| 4/23 | Project Presentation | Presentation Slides | |||
| W16 | 4/28 | Project Presentation | Presentation Slides | ||
| 4/30 | Reading Day (No Class) | Project Report Due |