Recently I was asked which habits and practices were the secret to my productivity. I figured my answers might be of wider interest. Let me know what works for you!
Why should you listen to my advice? Aside from my papers, in each year of the past decade I've authored ~1k pull requests with ~200k LoC of deltas, with a similar number reviewed:
You can tell when I'm on holiday :)
Learning
My first recommendation is to learn as much as possible, about the entire stack you work with - in AI, that's from transistor design and CPU architecture to data centers, internet infrastructure, from assembly to frontend TS. Be able to do any job in the company!
Having the ability to implement and test an idea without needing to convince anyone else that the idea is worthwhile is a productivity multiplier. A solid understanding of all parts of the system also makes it much easier to design algorithms that map well onto the hardware, and to debug tricky problems.
So how to learn?
There's a lot of valuable information out there, the main challenge is actually filtering the flood. One great filter are books - they provide information in condensed form, and reviews make it easy to find good books. Due to the slow process of publishing, books are best suited to topics that change slowly, if ever. Think fundamental principles and theories of a field, basic algorithms, etc. For more up to date information, I'm subscribed to the RSS feeds of several dozen high signal-to-noise blogs (<< 1 post/day each), as well as a daily newsletter with the top 30 posts from Hacker News.
Don't just stick to your own field though, read widely - mathematics, physics, biology, etc. You'll be surprised how often this provides unexpected inspiration!
Focus
The best knowledge is useless if you can't consistently convert it into action. Good sleep is key. I get 8 to 9 hours every night, making sure I'm refreshed and full of energy by the time I wake up. Optimize your space for this! Blackout curtains or blinds for complete darkness, good windows to ensure it's quiet, a slightly cooler temperature, avoid too much mental stimulation before bed. This is a great time to read books!
I also make liberal use of tea and coffee - they taste great and get me really focused. However, do not use caffeine to compensate for insufficient sleep! Instead, sleep as much as needed, then boost your attention further. I haven't experimented with any other nootropics; caffeeine seems hard to beat.
Avoid distractions - social media, TikTok, Instagram, YouTube, Twitter all provide rapidly diminishing value.
Unblocking Output
Once you have all the right knowledge and are fully focused on your problem you want to iterate as fast as possible while minimizing distractions.
Learn to touchtype to the point where it's never the bottleneck and you never need to hunt for keys no matter the character. Learn Dvorak or other improved keyboard layouts, and use ergonomic or split keyboards to reduce strain on your hands, arms and shoulders.
Optimize the iteration cycle of your project - minimize build times; automatically format, type-check and recompile on save; auto-run unit tests; auto-reload the running binary.
Unfortunately even after your best effort there may be residual iteration latency that you cannot reduce further. Running ML experiments or training jobs can be slow, debugging a complex issues may require running a slow test, etc. In this case it helps massively to always work on multiple ideas at once, switching when logical units of work are done and you are blocked on some results. Like hyper-threading for your brain!
Don't overdo it though, swapping state between ideas is expensive. To minimize the state you need to keep in your head lean heavily on your infrastructure. Use types to catch mistakes at compile time and make invalid states impossible to encode; liberally sprinkle assertions and comments through your code; strategically log the internal state of your program to make it easy to recover its execution trace; keep notes about your ideas and experiment results.
When working with others, you should a) liberally use code reviews, b) review incoming requests as quickly as possible (<<1h). Reviews are great to share knowledge, but nothing is worse than blocking progress with a slow review.
Humility
Most research ideas don't work, but the best ideas often make something work that was previously thought impossible. Approach a new idea with the excitement and optimism necessary to push it hard, but be ready to abandon it if it does not work out. Knowing how long and hard to push on an idea before giving up is an art - you can only learn the right kind of stubborn from experience.
Always ask, how could I prove that this idea is bad? If you have good results, what else could explain them? Did you really push your baseline as hard as you could? If you've run out of alternative explanations and your results still look promising, then maybe, just maybe, you have a good idea on your hands.
Tags: productivity, programming, work