Long Short-Term Memory
This page provides study materials on Long Short-Term Memory (LSTM) networks. To prepare the lecture, please watch the flipped classroom-style lecture video. For more auditive learners, I also included an AI-generated podcast based on the lecture. You can download all the lecture materials and study at your own speed.
To test and solidify your knowledge, I also provide a small interactive quiz, and a Jupyter notebook with a coding exercise that you can do locally. Finally, if you have questions you want to discuss during our in-person session, please submit them via the online question form.
Watch the Lecture
Download the Lecture Slides
đź“„ Please click the image to download/view the slides (PDF).
Listen to the Podcast
Using the lecture notes, I made an AI-generated podcast with NotebookLM.
Explore the Topic by Coding
I believe that any topic is best learned hands-on. Therefore, I prepared a Jupyter notebook where you can explore LSTMs by coding. My notebook provides the skeleton code which generates some training data and does the operational logic for you. You have three TASKs to fulfill based on your understanding of the material to make the network train. We will not use any abstractions like the torch.torch.nn.LSTM, but we will use PyTorch tensors and autograd, so we do not have to derive gradients manually. The notebook is view-only, please make an editable copy for yourself to start working.
Quiz
You can test your understanding of the lecture with this interactive quiz. The correct answers are displayed on the last page. If you got less than 7 out of the 10 questions correctly, I recommend re-watching the lecture video and/or looking into the additional study material. You can also submit any question that remains unclear in the form below.
Submit Questions for the In-Person Session
If you have questions that you want to discuss during the in-person session, please submit them through the form.
Find Additional Study Materials
Find below a listing on additional study materials that might help you to get more familiar with the topic. I included both short and long references, depending on how deep of a dive you want to take into the topic. I also linked some recent research around the topic for those who are interested in the development of the field.
Brief Dive
Deep Dive
- Full lecture on Simple and LSTM RNNs (Stanford CS224N NLP)
- Paper introducing LSTMs
Going Above & Beyond: Check More Recent Research
- NeurIPS 2024 (LSTM extension): xLSTM: Extended Long Short-Term Memory
- NeurIPS 2017 (Transformers as a modern-day alternative to LSTMs): Attention is all you need