Just day 1 of NIPS, and we're off to the races with a couple of big announcements. I'm thinking about shotgunning just a few quick updates every day or so through the conference, but we'll see how it goes.
OpenAI announced and released Universe today. Remember OpenAI Gym, the collection of video games that OpenAI curated and prepped for training reinforcement learning algorithms? OpenAI's Universe is basically a huge expansion of that. At its core, its a way of creating a virtual desktop environment, allowing you to convert any runnable program into a gym, and thus ready for reinforcement learning. Along with that they're also releasing 1,000+ environments for different programs and games. This is huge - it's going to massively expand the number of standardized training scenarios that RL algorithms can play with. Andrej Karpathy also speculated that the large variety of games which may have similar interfaces at times will also open the doors to better transfer learning in RL algorithms.
While a lot of the environments are still in "coming soon", the list is very impressive, including games like Civilizations 5, StarCraft II, GTA V, Portal and even Shovel Knight. (Granted, I'm guessing that many of these will just be isolated portions of the game, though we'll have to see). Also interesting is the World of Bits environments. The argument goes that we already do so much online (buy plane tickets, check directions, order food), and that's done through a digital interface, so we may as well train an AI on those same interfaces/tasks as well.
Open Sourcing DeepMind Lab
by Google DeepMind
Unfortunate for DeepMind that OpenAI's Universe came out today, because DeepMind is also open-sourcing its reinforcement learning environment. Unlike Universe which is a massive collection of random environments, DeepMind Lab is a single purpose-built 3D platform that allows algorithms to be trained on a variety of tasks. It also allows users to custom-build tasks to train on, and those can be contributed back to the open-source project. The GitHub repo will be active in a week. It's been a good 24-hours for reinforcement learning.
Tutorial Slides on Generative Adversarial Networks
by Ian Goodfellow
I can't go into every talk I attend this week, but I'll highlight the ones that I thought were particularly good. Ian Goodfellow's tutorial on Generative Adversarial Networks was stunningly clear and powerful and conveyed his excitement for the topic. It's no surprise that Yann LeCun said in his own keynote speech later that day that "I think this is the best idea in Machine Learning in the last 10 years. Actually maybe 20 years." In the tutorial, Ian goes through the motivation behind GANs, the current and future applications of them, and their theoretical foundation and limitations. I'm also hoping his talk got recorded, cause some minor drama happened between him and Jürgen Schmidhuber that was somewhat amusing. (And I'm still upset that I couldn't get my book autographed.)