In a basic neural language model, we optimise a fixed set of parameters based on a training corpus, and predictions on an unseen test set are a direct function of these parameters. What if instead of a static model we constantly measured the types of errors the model is making and adjust the parameters accordingly? It would potentially be more closer to how humans operate, constantly making small adjustments in their decisions based on feedback.

In this post I discuss several real life examples of strategies that cannot be achieved through mere practice of an inferior strategy. Backpropagation is an algorithm akin to such “mere practice” in that backpropagation develops skill at a specific strategy (i.e., it learns a specific local minimum). Like practice, backpropagation alone cannot result in a switch to a superior strategy. I look at how strategy switches are achieved in real examples and ask what algorithm might allow machines to effectively switch strategies.

The 0.2.0 release of the spark-ts package includes includes a fleshed-out Java API, among other things.

In the series “The Mathematics Behind” I will explain mathematical concepts behind commonly used technologies. In this post, I will explain the mathematics behind polynomial curve fitting MATLAB.

Bayes’ Rule is one of the fundamental Theorems of statistics, but up until recently, I have to admit, I was never very impressed with it. Bayes’ gives you a way of determining the probability that a given event will occur, or that a given condition is true, given your knowledge of another related event or condition. All the examples that I’ve read or heard about seemed somewhat contrived and unrelated to the sorts of data analysis I was interested in. But it turns out there’s also an interpretation of Bayes’ Theorem that’s not only much more geometric than the standard formulation, but also fits quite naturally into the types of things that I’ve been discussing on this blog. So in today’s post, I want to explain how I came to truly appreciate Bayes’ Theorem.

Shapeless is a Scala library for generic programming. The name “Shapeless” comes from a famous Bruce Lee quote: