By Reinforce ConferenceSponsored Post.2018 was another hot year for Data Science and AI. Here we picked out 5 highlights, which in our opinion shaped the field in the past year. Comment below and let us know how you would pick!
Use code: KDNuggets for 20% discount on tickets to Reinforce Conf - which gives product managers, data scientists and engineers insights into success stories, lessons learned, best practices and new approaches, Budapest, March 20-22, 2019
1. OpenAI Five achieves human-level DOTA team gameplay
OpenAI has started to achieve professional-level 1v1 playback in 2017, but 2018 was the year when team gameplay became human level. In DOTA, two teams of 5 players (or bots) battle for domination of resources on a map and try to kill each other off. OpenAI uses Reinforcement Learning with an LSTM network and self-play for learning, playing 180 years against itself every day on 256 GPUs and 128,000 CPU cores.
img source
At a high level, one interesting aspect of OpenAI’s success is that no deep theoretical breakthrough was necessary (so far) to achieve this; the basic ideas of the AI are similar to Google DeepMind’s AlphaGo, which defeated Lee Sedol in 2016. But Go, although a very deep, strategic game, still seemed very “discrete” compared to a Real Time Strategy game like DOTA. It will be interesting to see how far the OpenAI architecture will scale: can it defeat the top professional human team in 2019? We wouldn’t bet against the AI.
More resources:
2. Human-level performance in Quake 3 Arena
Staying with games, 2018 was also the year when DeepMind achieved human-level performance in Quake 3, using only pixels and game points as input. Although this result received less coverage than the earlier AlphaGo and the OpenAI successes, for people who grew up playing Doom and Quake, this is perhaps an even cooler result.
/cdn.vox-cdn.com/uploads/chorus_image/image/60264409/deepmind_capture_the_flag_quake_iii_crop.0.png)
img source
Like the other game playing bots, this one is also based on Reinforcement Learning, but here thousands of bots are “alive” at any given time, playing in teams against themselves in randomly generated environments. Each agent in the population learns its own internal reward signal to complement the sparse delayed reward from winning and selects actions using a hierarchical representation that enables the agent to reason at multiple timescales. The videos show that the bots display human-like behaviors such as navigating, following, and defending:
3. Self-driving cars did not arrive in 2018
Self-driving cars are the “killer-app” of the current AI hype cycle, and many people look at it as an indicator tent pole. Everybody understand this application, most people’s life would be affected, and it’s hard to argue with the cool-factor. The stakes are high, many big players are working on this disruptive advancement: who will supply the brain of these self-driving cars? who will sell the most self-driving cars? which self-driving car service will dominate? and equally importantly, what will drivers who no longer have to drive do? whose content will they consume?
Progress is steady, with Tesla pushing out new versions over the air, and other major car manufacturing also experimenting with self-driving features, but it seems Google’s Waymo is the most far ahead in terms of commercial deployment: in 2018 Waymo launched its first commercial self-driving car service called “Waymo One”, where users in the Phoenix metropolitan area can use an app to request the service.
But in 2018, we still were not able to get in a self-driving car, put our kids in the back, and get from San Francisco to Los Angeles without touching the wheel.
Instead, we had to cope with sad news. There were fatal crashes related to self-driving technology of Uber and Tesla, which triggered widespread debate about how the technology should be marketed, tested and deployed.
More resources:
4. PyTorch reached 1.0
Data Scientists and ML Researchers have a wide variety of tools and frameworks to chose from, with Tensorflow, Keras and PyTorch being the most widely used Deep Learning ones. PyTorch stands out to us because it is a “hacker’s framework”. Tensorflow came out of Google, where most ML jobs operate on vast amounts of data, which lead to design decisions to trade ease of use and debuggability for raw performance.
For smaller scale, more lean problems and teams, we believe PyTorch is a better fit. PyTorch is easier to work with than Tensorflow, the execution model is nicely integrated with Python, for example, step-by-step print()s work—at the expense of some performance. But unless you’re working at the Google scale, the speed of iterating on your model and code should come before the speed of training. To get an introduction to PyTorch, check out the excellent tutorial lecture by Stefan Otte:
5. Reinforce AI Conference announced!
Reinforce Conf gives product managers, data scientists and engineers insights into success stories, lessons learned, best practices and new approaches, in the beautiful European city of Budapest, March 20-22, 2019.
Reinforce brings together a diverse group of leading experts and practitioners to share their knowledge and experience on the fast-moving field of Machine Learning and Artificial Intelligence. Some of the speakers:
-
Andrea Pasqua, Data Science Manager, Uber
-
Christian Szegedy, Staff Research Scientist, Google
-
Cibele Montez Halasz, Machine Learning Engineer, Twitter
-
Kush R. Varshney, Principal Research Staff Member, and Manager, IBM Watson
-
Patrick van der Smagt, Director Artificial Intelligence, Volkswagen
With the code: KDNuggets you can save 20% on regular tickets. (340 EUR –>272 EUR) Don’t miss it! Register now!