Research foundations of modern LLMs

What modern LLMs inherited from earlier NLP research, and why those ideas reorganized into today’s stack instead of disappearing.

Author

Yee Seng Chan

Published

2026 · March 11

Encoders, retrieval, fine-tuning, and information extraction were each treated as the main event at some point. Each is now a component inside a larger system. The series traces where those ideas went and why the rearrangement matters.

Read in order:

Pretraining objectives: why decoder-only won

How unification, not raw capability, settled the architecture race.

2026 · March 18

The encoder didn’t die. It became the embedding model

BERT-style encoders moved into retrieval, reranking, and classification.

2026 · March 23

Retrieval is older than RAG: from DPR to end-to-end

What RAG inherited from DPR, and what it quietly skipped.

2026 · March 27

The fine-tuning stack: one loss, different data

What actually changes behavior across SFT, instruction tuning, and preference tuning.

2026 · March 31

Information extraction didn’t disappear. It moved inside the workflow.

Classic IE was absorbed into prompts, schemas, and pipelines.

2026 · April 5