Power Laws in Deep Learning 2: Universality
In a previous post, we saw that the Fully Connected (FC) layers of the most common pre-trained Deep Learning display power law behavior. Specifically, for each FC weight matrix W , we compute the eigenvalues lambda of the correlation matrix X: X = W^T W; Xv = lambdav. For every FC matrix, the eigenvalue frequencies or Empirical Spectral Density (ESD), can be fit to a power law where the exponents all lie in [2,4]. Remarkably, the FC matrices all lie within the Universality Class of Fat Tailed Random Matrices!
How to Avoid For Loop in R
Batch normalization: theory and how to use it with Tensorflow
Not so long ago, deep neural networks were really difficult to train, and making complex models converge in a reasonable amount of time would have been impossible. Nowadays, we have a lot of tricks to help them converge, to achieve faster training and to solve any kind of troubles that arise when we want to train a Deep Learning model. This article is going to explore one of those tricks: batch normalization.
An introduction to quantile regression
Ordinary least square regression is one of the most widely used statistical methods. However, it is a parametric model and relies on assumptions that are often not met. Quantile regression makes no assumptions about the distribution of the residuals. It also lets you explore different aspects of the relationship between the dependent variable and the independent variables.
Generative Deep Learning : Let’s seek how AI Extending, not Replacing Creative Process
In this article I will provide a high level overview of how AI is used currently to extend not replace, the creative process through generative deep learning. In this post I will discuss what is generative deep learning, what is a Discriminative model and how it differ from Generative model. I’ll even provide some concrete examples of the application of generative deep learning which will further help anybody and everybody to increase their understanding towards the fantastic possibilities that these Generative models is offering to all of us.
Combing LDA and Word Embeddings for topic modeling
Latent Dirichlet Allocation (LDA) is a classical way to do a topic modelling. Topic modeling is a unsupervised learning and the goal is group different document to same ‘topic’. Typical example is clustering a news to corresponding category including ‘Finance’, ‘Travel’, ‘Sport’ etc. Before word embeddings we may use Bag-of-Words in most of the time. However, the world changed after Mikolov et al. introduce word2vec (one of the example of Word Embeddings) in 2013. Moody announced lda2vec which combing LDA and word embeddings together to tackle topic modelling problem. After reading this article, you will understand: ‘ Latent Dirichlet Allocation (LDA); ‘ Word Embeddings’; lda2vec
Simple Method of Creating Animated Graphs
After the publication of one of my latest articles, many people asked me for tips on how to create animated charts in Python. Indeed, there are often situations when a static chart is no longer sufficient and in order to illustrate the problem we are working on we need something more powerful. There are of course many libraries that allow us to make animated and sometimes even interactive graphs like Bokeh, Pygal or my personal favorite Plotly. This time however, we will go old school?-?I will show you how to create really impressive charts using only ‘simple’ Matplotlib and a few command line tricks. Inside the article I will place only the most important parts of the code. But on my GitHub, you can find full notebooks that were used to create shown visualizations.
Clearing the Confusion: AI vs Machine Learning vs Deep Learning Differences
Raise your hand if you’ve been caught in the confusion of differentiating artificial intelligence (AI) vs machine learning (ML) vs deep learning (DL)… Bring down your hand, buddy, we can’t see it! Although the three terminologies are usually used interchangeably, they do not quite refer to the same things.
Like this:
Like Loading…
Related