OpenAI released a whitepaper on their work to train deep reinforcement learning agents, the OpenAI Five to beat world champions at DOTA2. This feels like a big achievement, but it begs the question: what did we learn? lessons To my eye, there’s a few key results from the paper: Scaling compute for training can solve a lot of problems. The authors emphasize the scaling point explicitly: The key ingredient in solving this complex environment was to scale existing reinforcement learning systems to unprecedented levels, utilizing thousands of GPUs over multiple months.

I decided I want to get back into prototyping more game ideas and polled twitter for advice. In case others find it helpful, I figured I’d write up the results. YMMV. Compared to the top web hits my personal criteria are a bit strange: Programming first, not visual scripting. I’m more comfortable writing code and I find it awkward to jump between my code and a visual editor. Comfortable programming language.