Questions about AlphaGo Development

I was asked some questions about the development of AlphaGo and figured my answers might be of wider interest, so here we go:

How low-level does the code need to be in order to get the maximum performance both in training and, possibly, live game?

To maximize playing strength it is necessary to balance maximum utilization of the accelerators (GPUs or TPUs) and focusing on exploring and evaluating the most promising line of play. Unfortunately there are several conflicting tendencies:

• The more board positions we can evaluate the better our estimates of how good each ...

Being Productive

Recently I was asked which habits and practices were the secret to my productivity. I figured my answers might be of wider interest. Let me know what works for you!

Why should you listen to my advice? Aside from my papers, in each year of the past decade I've authored ~1k pull requests with ~200k LoC of deltas, with a similar number reviewed:

You can tell when I'm on holiday :)

Learning

My first recommendation is to learn as much as possible, about the entire stack you work with - in AI, ...

Custom Static Site Generator

Once again I've migrated this blog to a new backend - originally I had been using Octopress, then later I migrated to Pelican after I got fed up with the complicated setup and dependencies. Well, recently I had issues getting Pelican to work after some updates, so I decided to bite the bullet and write my own small static site generator: 開板.

Really it's just an excuse to play around with Rust and make it easier to customize my site, but if it means no more painful upgrades that's a nice side effect :D It also gave me a ...

Saving Aranet4 data to Raspberry Pi

Recently I was curious about the amount of CO2 present while sleeping and working; I ended up buying an Aranet4 monitor. It's great to see the reading live, but I would also like to compare trends over time. The Aranet4 itself can only store a maximum of 7 days of data, so for longer term analysis I need to periodically export the data.

This is where a Raspberry Pi comes in handy! First, install the necessary packages:

sudo apt install bluetooth pi-bluetooth bluez blueman

Use bluetoothctl to pair the Aranet4:

sudo bluetoothctl
> ...

Competitive Programming with AlphaCode

I'm excited to share more on our latest project, AlphaCode - a system to write programs, able to reach roughly median human performance in coding competitions. And that's median competitive programmer performance, not median programmer or median human! In addition to our paper and official blog post, you can also find my personal take below.

Problem Setup

Coding competitions are difficult even for experienced programmers. Before writing the first character of a program, the first step is to understand the natural language problem description: often spanning several paragraphs in length, problem descriptions do not directly describe the ...

Values, Pointers and References in C++

If you've primarily used high level languages like Python, you may not be used to explicitly thinking about the ownership or representation of your values in memory.[^cache] In system languages like C++ or Rust, we have direct control over these aspects, and are able to use the type system to explicitly represent when a function takes ownership of a value, vs when it only takes a (temporary) reference.[^typing]

First, different types of ownership, in order of preference:

• T t. A normal owned value of type T, uniqlue owned. If declared as a variable it is stored on ...

Complexity Budget

No matter how smart, there's a limit to how much you can keep in mind at once before you start forgetting something. In the context of software engineering, a common term for this is complexity budget: the maximum complexity of a system you can understand at once.

Just as fitting an entire task into the cache of your CPU is orders of magnitude faster than paging in from RAM (or worse, disk!), fitting the entire problem you are working on into your brain can make you many times more productive.

Every line of code you write, every feature ...

MuZero talk - ICAPS 2020

I gave a detailed talk about MuZero at ICAPS 2020, at the workshop "Bridging the Gap Between AI Planning and Reinforcement Learning".

In addition to giving an overview of the algorithm in general, I also went into more detail about reanalyse - the technique that allows MuZero to use the model based search to repeatedly learn more from the same episode data.

I hope you find the talk useful! I've also uploaded my slides for easy reference....

Static & Dynamic Typing

I've written and been paid to write code in a wide variety of languages - functional and imperative, statically and dynamically typed, verbose and concise. Over time, I came to appreciate the benefits of certain language features.

The one I want to talk about today is static typing. My programs in statically typed languages seem to be systematically more likely to work correctly on the first try, more robust against bugs introduced by refactoring or adding new features, and easier to reason about. In fact, the benefits are so large that I now use pytype even for Python scripts ...

A simple way to run Rust WebAssembly in a browser

The Rust WebAssemply book has a detailed introduction to WebAssembly in Rust; unfortunately it's example setup is somewhat complicated and requires the use of npm just to run show a simple Hello World! message in the browser.

Luckily, there's a simpler way to get started if you don't care about npm modules.

First, follow the setup instructions to install the rust toolchain, wasm-pack and cargo-generate. You don't need npm.

Clone the example project template:

cargo generate --git https://github.com/rustwasm/wasm-pack-template

which will prompt you for a project name, in the following we'll assume you ...