Machine Learning Mailing List - Issue 2

Jason Phang, Wed 09 November 2016, Machine learning mailing list

deep-learning, machine-learning, mlml

A whole bunch of papers for ICLR 17 just got uploaded, and I haven't had nearly enough time to comb through them. I'll wait for the dust to settle and for Reddit to tell me what to read.

Here're some cool things I saw over the last week:

Rewrite: Neural Style Transfer For Chinese Fonts
     by Yuchen Tian
Super neat project involving neural styles. The idea is that you use convolutional networks to learn the visual "style" of a font - this has been done before, but this time it's applied to a pretty meaningful problem. Basically, there are a ton of Chinese characters (in the order of 10,000s). For a Chinese "font" to be complete, it has to cover the full set of characters. This is a monumental task, even if you're being smart by decomposing the characters into the constituent parts. The idea here is to train a network to translate the style from one font to another using just a subset of the complete character set, and then let it loose on the rest of the characters you want to translate into the new style. The author put in some really nice visualizations too - here's an animation of the characters "settling" into the new style. Just another example of how much low-lying fruit there is left in the deep learning space of applications.

DeepMind and Blizzard to release StarCraft II as an AI research environment]
     by Google DeepMind
This is a really ambitious move by DeepMind and Blizzard. Video games present a good platform for testing more general artificial intelligence algorithms as they provide contexts for deep and complex problem-solving within a controlled environment. Broad interest in this area, as well as deep reinforcement learning, likely started with Google's paper on playing Atari games in Nature. Subsequently, OpenAI collated a set of simple simulations and games to act as the starting point and benchmark for reinforcement learning algorithms. This project is something similar, except it's leapfrogging all the way from tackling Pong to tackling StarCraft 2. (There's almost 40 years of history between those two games.) Granted, it looks like they'll be adapting StarCraft 2 to a simpler API to learn from, but it's still a very impressive project. The environment is expected to be open to public sometime next year.

Supercharging Style Transfer
     by Google Research
Multistyle Pastiche Generator - GitHub - arXiv
     by Magenta (Google Brain)
Two related blogposts from Google about visual style transfer in images. If you haven't already seen the original Neural Algorithm of Style works, you need to check it out. Anyway, these two posts continue to build on that work. As it turns out, the original algorithm is actually pretty slow, as what it does is basically bounce around between the high-level (content) and low-level (visual-style) features of the two content- and style-images until if finds a good middle ground between the two. There is an alternative, less general approach that's less iterative but pins you down to only one style per algorithm/network. This new work, using a technique called conditional instance normalization, projects each style into a point in an embedding space. This means that not only is a single network able to learn multiple styles, it is also able to arbitrarily mix multiple styles into one image by moving around the embedding space. Definitely check out the demo.

Deconvolution and Checkerboard Artifacts
A detailed and highly visual look into one of the quirks of current image generation / modification deep learning algorithms. It gets pretty into the weeds with the esoteric behavior of convolutional layers, but it also has fun interactive demos. Basically convoluational layers are fairly discretely layered / strided, and this introduces checkerboard artifacts that could be avoided with some intelligent smoothing. The fix itself still seems like a small improvement, but it could lead to at least some nicer looking pictures from image-generating algorithms.

Microsoft Concept Graph - For Short Text Understanding
     by Microsoft Research
Microsoft has been doing some work on getting machines to learn "concepts", or the different kinds of connections between ideas. They're starting by releasing a large dataset on the "IsA" relations between instances and classes (i.e. "instance is a class"). This is the first of several releases, with later releases incorporating more contextual information. They're also releasing an API for quickly querying this dataset. I think the concepts themselves are pretty rudimentary, but the size of the dataset mean that it could be useful as an ingredient in a larger natural language processing project.

Neural Architecture Search with Reinforcement Learning
     by Barret Zoph, Quoc V. Le (Google Brain)
Let me introduce some concepts here. So we have Neural Networks, which form the basis for Deep Learning. They've been incredibly successful in the last couple years, dominating machine learning fields as varied as image recognition, text translation, and as mentioned above, playing games. Note however that the architecture (structure) of the neural networks applied to each task tends to differ wildly - e.g. image recognition tends to heavily use Convolutional Neural Networks (CNNs), while text or speech focused networks use Recurrent Neural Networks(RNNs). And researchers continue to innovate on new techniques and components to introduce into networks, such as BatchNorm or ResNets.

Now, taking a step back, the networks for learning to play video games are fairly specific, and the technique normally used is called Deep Reinforcement Learning. Unlike in a simpler task like image recognition where you provide an input, the algorithm supplies a prediction and you can immediately tell how right or wrong it is, in games the reward or evaluation is actually fairly far from every point at which the algorithm has to made a decision about what next move to take. Because there's a much larger distance between decision and evaluation, it is a much harder problem to solve and thus requires a more sophisticated training technique, involving policy functions and multi-armed bandits and what have you. It's hard work but it's provided good results, and now our algorithms can learn to play (some) video games.

Where I'm getting at is this - this paper applies reinforcement learning to learn new network architectures. In other words, this is the algorithm that learns to make better versions of learning algorithms. Yes, we're going there - have fun folks.