Automatic Differentiation in Mixture Models

Samuel Berchuck is a Postdoctoral Associate in Duke University’s Department of Statistical Science and Forge-Duke’s Center for Actionable Health Data Science.

Euclid’s Elements is a series of 13 books produced in 300 BC that forms a collection of mathematician Euclid’s proofs and definitions. In 1847, Oliver Byrne recreated the first six books “in which coloured diagrams and symbols are used instead of letters for the greater ease of learners.” Nicholas Rougeux recreated Byrne’s work with an online interactive version:

TV commentator Carlson in 2018 recently raised a stir by saying that immigration makes the United States “poorer, and dirtier, and more divided,” which reminded me of this rant from literary critic Alfred Kazin in 1957:

Sponsored Post.

Synthetic Gradient (SG) Artifical Neural Network are a particular class of learning system modeled after biological neural functions with an interesting penchant for Hebbian learning, that is ‘neurons that wire together, fire together’. However, unlike their natural counterparts, artificial neural networks have a close and stringent coupling between the modules of neurons in the network. This coupling or locking imposes upon the network a strict and inflexible structure that prevent layers in the network from updating their weights until a full feed-forward and backward pass has occurred. Such a constraint though may have sufficed for a while, is now no longer feasible in the era of very-large-scale machine learning, coupled with the increased desire for parallelization of the learning process across multiple computing infrastructures. To solve this problem, synthetic gradients (SG) with decoupled neural interfaces (DNI) are introduced as a viable alternative to the backpropagation algorithm. This paper performs a speed benchmark to compare the speed and accuracy capabilities of SG-DNI as over to a standard neural interface using multilayer perceptron MLP. SG-DNI shows good promise, in that it not only captures the learning problem, it is also over 3-fold faster due to it asynchronous learning capabilities. …

Jump Process A jump process is a type of stochastic process that has discrete movements, called jumps, with random arrival times, rather than continuous movement, typically modelled as a simple or compound Poisson process. In finance, various stochastic models are used to model the price movements of financial instruments; for example the Black-Scholes model for pricing options assumes that the underlying instrument follows a traditional diffusion process, with continuous, random movements at all scales, no matter how small. John Carrington Cox and Stephen Ross:145-166 proposed that prices actually follow a ‘jump process’. Robert C. Merton extended this approach to a hybrid model known as jump diffusion, which states that the prices have large jumps interspersed with small continuous movements. …

vtreat‘s purpose is to produce pure numeric R data.frames that are ready for supervised predictive modeling (predicting a value from other values). By ready we mean: a purely numeric data frame with no missing values and a reasonable number of columns (missing-values re-encoded with indicators, and high-degree categorical re-encode by effects codes or impact codes).

“The 4 Types of Data Analytics• Descriptive: Answers the question, ‘What Happened?’.• Diagnostic: Commonly used in engineering and sciences to diagnose ‘what went wrong?’.• Predictive: Used to predict for future trends and events based on statistical or mathematical modeling of current and historical data.• Prescriptive: Used to tell you what to do to achieve a desired result. Based on the findings of predictive analytics.” Lillian Pierson ( May 11, 2015 )

In this previous post, I showed how one can scrape top-level NBA game data from BasketballReference.com. In the post after that, I demonstrated how to scrape play-by-play data for one game. After writing those posts, I thought to myself: why not do both? And that is what I did: scrape all the box scores for the 2017-18 NBA season and save them to an RDS object.