Tag: ai

Mastering the Game of Go with Deep Neural Networks and Tree Search

Today we published our [cached]paper on beating the human state of the art in Go, the only major board game where humans (or at least top professionals) could still beat computers. No more. Our program AlphaGo achieved a 99% winning rate against the strongest existing Go programs, and defeated the human European champion by 5 games to 0.

(That's me playing at 0:10)

The first major breakthrough in computer Go - after remaining at weak amateur level for decades - came with the advent of [cached]Monte Carlo Tree Search (MCTS) around 2007, massively improving playing strength. Still, Go programs …

Computing Machinery and Intelligence

I was just reading [cached]Quantum Computing since Democritus (an excellent book!), when it referred me to Turing's [cached]Computing Machinery and Intelligence.

This argument is very, well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. "Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain-that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its …

Adventures in Reinforcement Learning

You may or may not have noticed that I've been working for DeepMind for a while, causing me to get exposed to lots and lots of cutting edge machine learning research. Most of that I can't share here, but there's plenty that's already public.

Reinforcement Learning

Firstly, yes, that's what all the fuzz is about. There's a great book by Sutton and Barto, [cached]Reinforcement Learning: An Introduction, with an in-progress version of the second edition available for free from their website!

The book is very good at introducing and explaining RL itself, but does not cover how to combine …

Brains, Sex, and Machine Learning

A great explanation of why Dropout is really good for training large neural networks, and why it's actually the same thing your brain is doing:

Recent advances in machine learning cast new light on two puzzling biological phenomena. Neurons can use the precise time of a spike to communicate a real value very accurately, but it appears that cortical neurons do not do this. Instead they send single, randomly timed spikes. This seems like a clumsy way to perform signal processing, but a recent advance in machine learning shows that sending stochastic spikes actually works better than sending precise real …

Neural Turing Machines

DeepMind just published a new paper [cached]Neural Turing Machines with some very interesting implications. For the ones pertaining to AI just read the paper; here I want to focus on what it means for the human brain.

The main achievement of the NTM paper is coupling a neural network to external traditional computer memory, implementing both content and address based access. This implies something similar could potential work with the human brain.

There's already precedent for [cached]implanting electrodes and using them for control. The big open question here is the training - for NTM this was achieved by gradient …

More on Deep Learning

This is more or less just a random collection of links I've come across while researching Deep Learning, I hope they are as useful to you as to me.

[cached]Neural Networks, Manifolds, and Topology is a great post on how to visualize deep neural networks and get an intuition for them. Reading it was the first time I truly appreciated how the successive layers of NNs just transform the topology of the input data, until finally it becomes linearly separable.

[cached]So You Wanna Try Deep Learning? has a good collection of papers, articles and tips to get started …

Learning Machine Learning

Fortunately, it is getting easier and easier to get into machine learning. Just today, there was a post on HackerNews about a [cached]JavaScript neural network library. Can't get much easier than this:

var net = new brain.NeuralNetwork();

net.train([{input: [0, 0], output: [0]},
           {input: [0, 1], output: [1]},
           {input: [1, 0], output: [1]},
           {input: [1, 1], output: [0]}]);

var output = net.run([1, 0]);  // [0.987]

There's more and more [cached]free online courses, and even a [cached]good introduction to Deep Learning.

I've also come across a very neat …

Playing Games with AI

[cached]Playing Atari with Deep Reinforcement Learning is certainly the most catchy paper I've read in years. But before I start talking about the paper, you should definitely watch this video from it's presentation:

The basic idea is very simple - to train and evaluate the AI, let it play a variety of computer games. This has several advantages:

Training and Evaluation can happen faster than real time
Using several very different games can prevent over-fitting
Games are usually created to be challenging for humans, thus containing an implicit requirement for intelligence.

DeepMind achieves this by letting it's Deep Reinforcement Learning …

Towards General Artificial Intelligence

I you haven't slept under a rock for the past months you've surely heard of Google's acquisition of DeepMind, an AI company that is still quite young. For general facts about them I'll just refer you to Wikipedia or the usual news articles, but much more interestingly they've actually published quite a few papers about their methodology: