Tag: ai

Learning and Planning in Complex Action Spaces

We've been working on making MuZero as easy to use as possible, allowing it to be applied to any RL problem of interest. First and foremost, this required us to make some algorithmic extensions.

The original MuZero algorithm achieved good results in discrete action environments with a small action space - classical board games and Atari, with up to a few thousand different actions. However, many interesting problems have action spaces that cannot be easily enumerated, or are not discrete at all - just think of controlling a robot arm with many degrees of freedom, smoothly moving each joint. So far, MuZero …


Rates of Growth

I've been enjoying [cached]A Map that Reflects the Territory, a collection of the best LessWrong essays from 2018. While in general the essays are thought provoking and interesting, one set of essays gave me pause: [cached]Hyperbolic Growth and related chapters on fast vs slow takeoff.

The discussion repeats tropes that are common in the rationalist and futurist community, describing how economic and technological [cached]growth have been accelerating and suggesting that they will soon increase so quickly that we will be unable to follow:

world gdp, growing exponentially ([cached]data source, csv)

However, we have to be careful not to mix up …


Stack Overflow Podcast

Last week, I had a chance to chat with [cached]Ben Popper and [cached]Cassidy Williams about MuZero and AI on the [cached]Stack Overflow podcast.

I had a lot of fun, thanks to Ben and Cassidy for being such great hosts!


MuZero Intuition

To celebrate the publication of our MuZero paper in [cached]Nature ([cached]full-text), I've written a high level description of the MuZero algorithm. My focus here is to give you an intuitive understanding and general overview of the algorithm; for the full details please read the paper. Please also see our [cached]official DeepMind blog post, it has great animated versions of the figures!

MuZero is a very exciting step forward - it requires no special knowledge of game rules or environment dynamics, instead learning a model of the environment for itself and using this model to plan. Even though it …


MuZero talk - ICAPS 2020

I gave a detailed talk about MuZero at ICAPS 2020, at the workshop "Bridging the Gap Between AI Planning and Reinforcement Learning".

In addition to giving an overview of the algorithm in general, I also went into more detail about reanalyse - the technique that allows MuZero to use the model based search to repeatedly learn more from the same episode data.

I hope you find the talk useful! I've also uploaded my slides for easy reference.


Getting into Machine Learning - 2020

My previous Getting into Machine Learning post is one of my most popular; since then much has changed.

There's a new kid on the block: [cached]JAX. A thin, but powerful layer over Autograd and XLA, it makes it easy to concisely express algorithms with the same syntax as numpy while getting the full performance of TPUs and GPUs.

Especially in combination with higher level libraries such as [cached]Haiku, JAX makes it fun and easy to try new ideas. I've migrated all my own research to JAX and can only recommend it!

The resources I recommended in my previous …


The Case Against the Singularity

From Musk's "Potentially more dangerous than nukes." tweet, increased funding for the Machine Intelligence Research Institute (MIRI) to the founding of cross-industry groups like the Partnership on AI, AI is being taken more seriously.

One worry that is sometimes cited, as in the book [cached]Superintelligence by Nick Bostrom, is that once we reach human-level AI, it might rapidly improve itself past anything humans can envision, becoming impossible to control. This is called "Singularity", because anything after such a point is unforseeable.

The argument for a Singularity rests on the fact that a hypothetical AI could devote all its resources …


MuZero - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

I'm excited to finally share some more details on what we've been working on since AlphaZero.

Recently, we made our latest paper - Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, aka MuZero - available on arXiv:

Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess and Go, where a perfect simulator is available. However, in real-world problems the dynamics governing the environment are often complex and unknown. In this work we present the MuZero …


Getting into Machine Learning

Update: I've published a newer version of this post.

If you are interested in getting started with Machine Learning, the [cached]TensorFlow Playground is a good way of building some intuition.

For a first introduction to the field, I recommend [cached]Neural Networks and Deep Learning, followed by the [cached]Deep Learning book.

To dive into a specific topic quickly, the [cached]TensorFlow tutorials are also great, Dive into Deep Learning has a lot of detail as well.

For Reinforcement Learning specifically, the standard text is [cached]Reinforcement Learning: An Introduction. Dave's [cached]UCL Course on RL is great too …


AlphaGo Documentary

As some of you may have noticed, the [cached]AlphaGo documentary is now available on [cached]Play Movies and [cached]Netflix!

It's a great documentary and really captures the history of AlphaGo very well - every time I watch it it takes me right back to the excitement of those months! If you are interested in AI, Go, or just like documentaries in general I really recommend you give it a try.

© Julian Schrittwieser. Built using Pelican. Theme by Giulio Fidente on github. .