Brains, Sex, and Machine Learning

A great explanation of why Dropout is really good for training large neural networks, and why it's actually the same thing your brain is doing:

Recent advances in machine learning cast new light on two puzzling biological phenomena. Neurons can use the precise time of a spike to communicate a real value very accurately, but it appears that cortical neurons do not do this. Instead they send single, randomly timed spikes. This seems like a clumsy way to perform signal processing, but a recent advance in machine learning shows that sending stochastic spikes actually works better than sending precise real …

Reading Drive: The Surprising Truth About What Motivates Us

After mentioning that I was reading Influence to my good friend [cached]Adrienne, she recommended [cached]Drive as my next book. The theme is quickly explained: While there are three different ways to motivate us - biological urges like hunger or sex, external reward & punishment, and intrinsic reward from performing a task - only intrinsic reward can consistently foster creative behavior.

Pink starts out by showing how traditional external motivation - cash bonuses et al. - overly constrain our focus, interfere with creativity, extinguish internal rewards and even lead to unethical behavior (think doctoring sales numbers to meet a bonus target). Only in special …

Relevant Reading

How much of what you read today would you still want to read if it was a week from now? A year? A decade, a century?

Too much of the information hammering us is ephemeral infotainment, not enough brings lasting value.

The test I apply is simple: would I still want to read this if I had to wait a year, even if it was the only thing I could read that day?

Taking Notes

I always thought taking notes was a waste of time. Surely you could just look back at the slides if you forgot something?

But I realized I might have had it backwards all this time. What if you take notes not to have something to refer to, but because it forces you to listen with greater attention and strengthens your memory?

To really take notes you need to understand the subject well enough to pick out the key phrases and concepts in real time, all while the lecturer is speaking on and on. It's similar to how actually doing the …

Reading Influence: The Psychology of Persuasion

The theme of [cached]Influence is clear: unable to keep up with the onslaught of information and decisions we fall back to mental shortcuts and learned behavior.

Robert Cialdini orders them into six large groups:

• Reciprocation. Giving a gift or doing a favor makes us much more likely to comply with a subsequent request, even if the favor was unwanted. The same concept applies to making an initial high demand and then "conceding" to a lower one. Example: charities giving you a free gift.
• Commitment and Consistency. We will change our opinions and desires to be in line with our …

Neural Turing Machines

DeepMind just published a new paper [cached]Neural Turing Machines with some very interesting implications. For the ones pertaining to AI just read the paper; here I want to focus on what it means for the human brain.

The main achievement of the NTM paper is coupling a neural network to external traditional computer memory, implementing both content and address based access. This implies something similar could potential work with the human brain.

There's already precedent for [cached]implanting electrodes and using them for control. The big open question here is the training - for NTM this was achieved by gradient …

A Walk around Umbrella Revolution

The Umbrella Revolution (from the umbrellas used to protect against tear gas) has been going on for nearly a month, yet publicity is scarce; fading out almost completely in the last two weeks.

Which is a shame - here's a vibrant city struggling for a most basic right, fair and independent elections, and yet we in the west, so proud of our democracies, stand by and watch?

By chance I've been in Hong Kong for the past few days, and meeting these protesters - both young and old - really struck a chord in me. They risk their freedom and lives for something …

I Travel

I travel to explore the world, I travel for the thrill of the unknown, to broaden my mind, to taste the unexpected, to learn and keep growing. I travel for chance encounters and randomly meeting again halfway across the globe. I travel because it's the easiest way to multpily my (remembered) life span. I travel to break the daily routine, to upset hardened molds, to escape boredom, to make new friends and meet old ones again.

I travel not to feel alone, to meet new people and fall in love for a brief romance, to share moments and explore together …

Censorship and the Great Firewall

I just arrived in China for my vacation and noticed one thing immediately: All kinds of useful sites are blocked. Google Search and Maps, Gmail and Facebook. This is a major pain in the ass for doing anything from looking up directions to uploading pictures for those back at home, not even to speak of any human rights aspects.

My initial approach was using OpenVPN on my phone, but that doesn't seem to work - Facebook is still blocked, etc. What does help is ssh -D <PORT> <HOST>, but even that seems to get slowed down after some use. It's enough …

Backups Simplified

I spent quite a bit of time today looking at different options for making sure my laptop was all backed up. Starting with [cached]rdiff-backup - which seems great if you are doing local backups and have LVM setup -, remembering I had been using [cached]rsnapshot previously and then wondering how to configure it for targets that are only available sometimes: [cached]push backups.

That was when it hit me: I already have all my data in [cached]Dropbox. Including most of my dotfiles, I just symlink them out of my Dropbox folder into the appropiate location, making it easy to …