You might have come across Judea Pearl’s new book, and a related interview which was widely shared in my social bubble. In the interview, Pearl dismisses most of what we do in ML as curve fitting. While I believe that’s an overstatement (conveniently ignores RL for example), it’s a nice reminder that most productive debates are often triggered by controversial or outright arrogant comments. Calling machine learning alchemy was a great recent example. After reading the article, I decided to look into his famous do-calculus and the topic causal inference once again.

Advancing your skills is an important part of being a data scientist. When starting out, you mostly focus on learning a programming language, proper use of third party tools, displaying visualizations, and the theoretical understanding of statistical algorithms. The next step is to test your skills on more difficult data sets.

This technical article was written for The Data Incubator by Paul Paczuski, a Fellow of our 2016 Spring cohort in New York City who landed a job with our hiring partner, Genentech as a Clinical Data Scientist.

At the PyCon 2018 conference, I presented a tutorial called “Using pandas for Better (and Worse) Data Science”. Through a series of exercises, I demonstrated best practices with pandas to help students become more fluent at using pandas to answer data science questions and avoid data science errors.

The proliferation of e-commerce websites such as Amazon and content-based subscription services like Netflix and Spotify has made it essential that the right product or item is delivered to the right customer. This is one area where big data shines. In a nutshell, we  want to answer the following question: What product should we recommend to the customer?

Ed Hawkins, a climate scientist, tweeted the following animated visualization in 2017 and captivated the world:

This post is authored by Gopi Kumar, Principal Program Manager at Microsoft.

Being a PHP developer with six years experience, I knew it was time to move to something bigger, such Big Data, Machine Learning, AI, Mobile Apps Development, Data Analysis. But, as with many newbies in the tech industry, choosing the right career path is never easy.

On the range of dates 14.-16. May 2018, the European R users meeting (eRum) was held in Budapest. I was there as an active participant since I had the presentation about time series data mining. The eRum 2018 was a very successful event and I want to thank organizers of this event for a great organization of it.