Deep Learning Architectures & Systems

Let's continue with our survey of developments since GPT-3. Useful starting points: [A Visual Tour of LLM of Modern LLM Architectures](https://www.youtube.com/watch?v=CepbWmGie0E) (youtube 38min) A slop summary of Reiner Pope's interview on how LLMs are actually trained and served. (Has some useful diagrams) * The Deepseek family of models are the most well documented of the lot for the last two years. Informal, all levels welcome. Suggestions for future papers and topics welcome.