
Recursive Language Models and Advanced LLM Architectures
MP4 | Video: h264, 1920×1080 | Audio: AAC, 44.1 KHz, 2 Ch
What you’ll learn
Explain the three meanings of recursive in deep learning: tree networks, looped depth and Recursive Language Models
Build a small decoder-only transformer from scratch and use it as a measured baseline for every other architecture
Implement recursive networks, Tree-LSTM and a weight-tied looped transformer, and compare them at equal parameters
Build a Recursive Language Model loop by hand with a Python REPL and a local model, and read every step of its trace
Add sliding-window and linear attention to the baseline and measure memory, speed and quality against sequence length
Implement a selective state space layer, train a tiny Mamba-style model and compare pretrained Mamba with Pythia on CPU
Build mixture-of-experts layers, watch expert load with and without balancing, and inspect a real MoE router
Measure the KV cache, manage it with the transformers cache classes and keep generation stable with attention sinks
Build and evaluate a production-style RAG pipeline with chunking, FAISS, hybrid search, re-ranking and faithfulness checks
Run fair architecture comparisons and write an architecture selection report backed by a complete results table
Requirements
Working Python and basic PyTorch: you can read a training loop and run a script from the command line
A computer with a multi-core CPU. No GPU is needed: the course bench is an 8-CPU machine and every lab is measured on it
Description
New architecture papers arrive every week, and most claims are made on hardware and at scales you will never reproduce. This course teaches you to understand the main ideas beyond the plain transformer, implement small versions of them yourself, and measure them fairly on an ordinary CPU, so you can decide what is worth using.
The middle of the course works through the main efficiency ideas. Sparse, sliding-window and linear attention are implemented and timed; FlashAttention, MQA, GQA and latent attention are explained with honest notes on what needs a GPU. You implement a selective state space layer, train a tiny Mamba-style model, and compare pretrained Mamba and Pythia on the same data. RWKV, RetNet, Gated DeltaNet and today’s hybrid stacks are explained, and an associative recall task shows why hybrids keep an attention layer.
Mixture of experts gets its own section: you build a router, watch expert load with and without the balancing loss, cause router collapse on purpose, and inspect the routing choices of a real small MoE model token by token. Memory follows, from external memory networks to the KV cache, which you measure, size from the model configuration and manage with the transformers cache classes and attention sinks.
Retrieval is treated as external memory you can update. You compare chunking strategies, choose a small embedding model and check its licence, build FAISS indexes, combine BM25 and dense search, add a cross-encoder re-ranker, and score retrieval with recall, MRR and nDCG on a labelled set. A complete local RAG pipeline answers with citations and a faithfulness check, and is compared with an RLM and a long prompt on the same questions.
The last sections are about decisions. You learn the rules of fair comparison, the pitfalls of perplexity, how to measure throughput and time to first token properly, and what a toy experiment can and cannot say about scaling. The capstone compares three architectures on one task with one harness and ends with a written architecture selection report. Eleven downloadable documents come with the course, including a paper-claim checklist, the fair comparison protocol, a CPU bench setup guide, KV cache sizing and MoE checklists, a RAG evaluation template and the selection report template. All numbers in the course are measured on the course bench and are presented as such.
Who this course is for
ML and LLM engineers who want to understand what comes after the plain transformer and when each design is worth it
Software engineers building LLM applications who need to choose between long context, RAG and recursive inference
Students and researchers who want hands-on, reproducible implementations instead of paper summaries


RapidGator
https://www.keeplinks.org/p27/6af30bd58c6f8
DDownload
https://www.keeplinks.org/p27/6af30c6f0b734
NitroFlare
https://www.keeplinks.org/p27/6af30e7f557f8

Scrivi un commento