Today an Op-ed I wrote has been published in the Guardian Media & Tech Network.

Over the past 11 blogs in this series, I have discussed how to build machine learning models for Kaggle’s Denoising Dirty Documents competition.

This is straightforward. Certain types of input create certain types of hidden layers. Certain types of hidden layers create certain types of output layers. It’s kindof a closed system. Memory changes this. Memory means that the hidden layer is a combination of your input data at the current timestep and the hidden layer of the previous timestep.

E Ferrara, Z Yang. Measuring Emotional Contagion in Social Media. PLoS ONE, 2015

James Bond: Do you expect me to talk?Auric Goldfinger: No, Mr. Bond, I expect you to die!

The idea of writing a collection of short stories has been on my mind for a while. This post is my first ever half-serious attempt at a story, and what better way to kick things off than with a story on AI and what that might look like if you extrapolate our current technology and make the (sensible) assumption that we might achieve much more progress with scaling up supervised learning than any other more exotic approach.

This is the third part of a three part post. The first part covered the early history of Monte Carlo and the rejection sampling method, the second part covered sequential Monte Carlo.

Note : I originally prepared this post as an article for Predictive Analytics Times, geared towards a broad audience with a business/practitioner leaning. I ended up writing this article on knitr instead. In a subsequent post, I will discuss the shiny app I created to interactively explore and visualize association rules.

When I give talks about probabilistic programming and Bayesian statistics, I usually gloss over the details of how inference is actually performed, treating it as a black box essentially. The beauty of probabilistic programming is that you actually don’t have to understand how the inference works in order to build models, but it certainly helps.