Tag: programming

Getting into Machine Learning

If you are interested in getting started with Machine Learning, the TensorFlow Playgroundcache is a good way of building some intuition.

For a first introduction to the field, I recommend Neural Networks and Deep Learningcache, followed by the Deep Learningcache book.

To dive into a specific topic quickly, the TensorFlow tutorialscache are also great, Dive into Deep Learning has a lot of detail as well.

For Reinforcement Learning specifically, the standard text is Reinforcement Learning: An Introductioncache. Dave's UCL Course on RLcache is great too (playlist of all lectures).


AlphaGo Zero

Usually in software, version numbers tend to go up, not down. With AlphaGo Zero, we did the opposite - by taking out handcrafted human knowledge, we ended up with both a simpler and more beautiful algorithm and a stronger Go program.

We provide a full description in our paper, Mastering the game of Go without human knowledge, which you can also read online.

At the core is a self-improvement loop based on self-play and Monte Carlo Tree Search (MCTS): We start with a randomly initialized network, then use this network in the MCTS to play the first games. The network is …


AlphaGo in China

You might have heard about our recent games with AlphaGocache in China, at the Future of Go summit. No only did we play the legendary Ke Jie, but there were also two new and exciting formats: Team Go and Pair Go.

This match was also very exciting on the technical side because we had improved AlphaGo to the point where we ran it on a single machine in the Google Cloudcache - that's one tenth of the computation powercache compared to the distributed version we used in the last match!

Personally, I also really enjoyed the Pair Go …


Error handling with Status(Or)

I just read an article linked on Hacker News advocating the use of Either<L, R> to signal errors in functions. While I think this is a good start, in my experience Either is too general, the lack of standardizing of the error value makes error handling abstractions hard. How did that file function signal again that a file was not found?

What works a lot better in my experience is a more specialized type Status, with a standardized enum of error codes and a free form error message.

Here's an excerpt from the Status type we use at Google …


Usability Testing on 10 Cents a Day

I just read Steve Krug's Don't Make Me Think and while I really liked the whole book, what especially stuck out was the chapter on usability testing as often as possible.

It doesn't need to be an expensive or time intensive process - just grab 3 people for 45 minutes each, pay them 50$, show them your website or app and ask them to use it while thinking aloud. Maybe give them a few simples tasks to do.

Repeat this process regularly - say once a month - and you should be able to catch a significant fraction of the usability problems on …


AlphaGo - Lee Se-dol

After 5 long and exciting games AlphaGo finally managed to win 4:1 against the legendary Lee Se-dol, the first time in history a computer program managed to defeat a 9 dan professional player in an even match. And not just any 9 dan player, probably the best player of the decade. It was even awarded an honorary rank of 9 dan professional itself!

Obviously we are all extremely proud of this achievement, you can find out more about the details in our Nature paper. Most importantly, we still used roughly the same amount of hardware! This was a true …


Mastering the Game of Go with Deep Neural Networks and Tree Search

Today we published our paper on beating the human state of the art in Gocache, the only major board game where humans (or at least top professionals) could still beat computers. No more. Our program AlphaGo achieved a 99% winning rate against the strongest existing Go programs, and defeated the human European champion by 5 games to 0.

(That's me playing at 0:10)

The first major breakthrough in computer Go - after remaining at weak amateur level for decades - came with the advent of Monte Carlo Tree Searchcache (MCTS) around 2007, massively improving playing strength. Still, Go programs …


Computing Machinery and Intelligence

I was just reading Quantum Computing since Democrituscache (an excellent book!), when it referred me to Turing's Computing Machinery and Intelligencecache.

This argument is very, well expressed in Professor Jefferson's Lister Oration for 1949, from which I quote. "Not until a machine can write a sonnet or compose a concerto because of thoughts and emotions felt, and not by the chance fall of symbols, could we agree that machine equals brain-that is, not only write it but know that it had written it. No mechanism could feel (and not merely artificially signal, an easy contrivance) pleasure at its …


Adventures in Reinforcement Learning

You may or may not have noticed that I've been working for DeepMind for a while, causing me to get exposed to lots and lots of cutting edge machine learning research. Most of that I can't share here, but there's plenty that's already public.

Reinforcement Learning

Firstly, yes, that's what all the fuzz is about. There's a great book by Sutton and Barto, Reinforcement Learning: An Introductioncache, with an in-progress version of the second edition available for free from their website!

The book is very good at introducing and explaining RL itself, but does not cover how to combine …


Lua Rocks

Inspired by reading Lua - an extensible extension languagecache, specifically the section on using fallbacks to implement an expression parser, I wanted to see if I could do so in a safe manner. With safe meaning parsed expressions must not be able to execute arbitrary code.

Half an hour later, I have to say it is surprisingly easy! Less than 50 lines (including comments) is all it takes:

function parse(expression)
  local op, create

  -- Calculate each subexpression at most once, otherwise just return the cached
  -- result. Note that this is not clever enough to take advantage of
  -- commutativity, i.e …

© Julian Schrittwieser. Built using Pelican. Theme by Giulio Fidente on github. .