This is more or less just a random collection of links I've come across while researching Deep Learning, I hope they are as useful to you as to me.
Neural Networks, Manifolds, and Topologycache is a great post on how to visualize deep neural networks and get an intuition for them. Reading it was the first time I truly appreciated how the successive layers of NNs just transform the topology of the input data, until finally it becomes linearly separable.
A Gentle Introduction to Backpropagationcache provides an unusual (but maybe easier to understand) look at back propagation, the essential algorithm for updating weights when training neural networks.
ImageNet Classiﬁcation with Deep Convolutional Neural Networkscache is a very interesting landmark paper on the use of Deep Learning in Computer Vision.
Visualizing and Understanding Convolutional Networkscache helps immensely to understand how a convulutional network represents its features and how it generalizes in successive layers.