Building a Transformer (part 3)

About this event

**Agenda:** Join us as we work through Assignment 1 of Stanford's CS336: Language Modeling from Scratch. We'll continue going over the implementation and theory behind each block of the transformer architecture! We'll cover: einsum notation, Linear and Embedding modules, RMSNorm, the SwiGLU feed-forward network, RoPE, softmax and scaled dot-product attention, causal multi-head self-attention, and assembling the pre-norm transformer block into the full LM. We'll finish by training the model on some sample corpus’ and generate some sample output. https://github.com/coventry/css336-assignment-1/tree/main/cs336_basics https://github.com/stanford-cs336/assignment1-basics/blob/main/cs336_assignment1_basics.pdf **Venue:** Boston Public Library (Kirstein Business Library & Innovation Center (KBLIC): Alcove 4) 700 Boylston St, Boston, MA 02116 **Time:** 6:00 - 8:00 PM EST