In my experience, the gap between a conceptual understanding of how a machine learning model “learns” and a concrete, “I can do this with a pencil and paper” understanding is large. This gap is further exacerbated by the nature of popular machine learning libraries which allow you to use powerful models without knowing how they really work. This isn’t such a bad thing. But knowledge is power. In this post, I aim to close the gap above for a vanilla neural network that learns by gradient descent: we will use gradient descent to learn a weight and a bias for a single neuron. From there, when learning an entire network of millions of neurons, we just do the same thing a bunch more times. The rest is details. The following assumes a cursory knowledge of linear combinations, activation functions, cost functions, and how they all fit together in forward propagation. It is math-heavy with some Python interspersed.

Reactive Stream vs Akka Stream

Originally appeared in the Guardian’s Media and Tech Network

S Agreste, S Catanese, P De Meo, E Ferrara, G Fiumara. Network structure and resilience of Mafia syndicates. Information Sciences, 2016

The White House. Photo: By Zach Rudisin (Own work) [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0)%5D, via Wikimedia CommonsIt looks like Artificial Intelligence has really gone mainstream with the White House taking notice and starting to act.

Wikipedia is one of the top 10 most widely used websites globally and a rich source of information. There are pages for every conceivable topic; the English version alone has over 5 million. Using Wikipedia it is both possible to rapidly get information about a topic and then to also delve deeper into the details.

Gotta Catch‘em all!

A while back, I discussed Recurrent Neural Networks (RNNs), a type of artificial neural network in which some of the connections between neurons point “backwards”. When a sequence of inputs is fed into such a network, the backward arrows feed information about earlier input values back into the system at later steps. One thing that I didn’t describe in that post was how to train such a network. So in this post, I want to present one way of thinking about training an RNN, called unrolling.