The last couple of months I went down the rabbit hole of understanding the workings of GPT-3, the language model that got a lot of hype in the summer of 2020. During my research to understand the technical implementations of GPT-3, I got most fascinated by this blog post by Gwern about the