Paper Club #5: Position Embeddings & Long Context

Welcome to DEVx's next AI Paper Club! How does encoding token position improve long context and structural understanding of inputs, if at all? This paper explores position embeddings -- RoPE, NoPE, and QK-Norm. I will discuss motivations, intuitions, architectural designs, and some applications of position embeddings. Paper: https://proceedings.neurips.cc/paper_files/paper/2025/file/5c9ab393551b7a39b4c02d88fe5e7e69-Paper-Conference.pdf All are welcome to come and learn. Questions and curiosity encouraged! 📍 LOCATION CHANGE AI Paper Club will be at a new location this time: 11988 El Camino RealSan Diego, CA 92130 We’ll be in the seminar space right off the lobby. Please double-check the address before heading over, especially if you’ve attended a previous Paper Club at our old location. See you there!