Last month, experts from the AI world came together for the Artificial Intelligence Conference at San Francisco to discuss insights, opportunities, challenges, and trends related to the rapidly expanding field of AI. The conference included hands-on trainings, tutorials, startup showcase (which was won by Clobotics), keynotes, sessions, expo, and social events.
Across the keynotes and sessions (on 9/6 and 9/7), the following points appeared in multiple talks and do provide a sense of current prevailing trends:
Domain Specific Architectures – the next big thing
Despite recent advances, the CPU/GPU architectures are falling behind to keep up with the immense computation appetite of training tasks for AI projects. DSAs provide a great opportunity for innovation as hardware and software are designed from the scratch focused on a very specific goal.
Emerging China – evolving from copying ideas to true innovation
For decades China lagged far behind the west in the field of technology innovation. However, in the field of AI China has made formidable progress in the last few years including several unicorn startups and initiatives from major technology leaders (such as Alibaba and Baidu). In the next 5 years, it is seeming increasingly possible that China can outpace US in the field of AI.
Addressing Risks in AI – Security, Privacy, and Ethics
As more and more AI solutions make their way to mainstream adoption, the inherent risks become increasingly important and harder to ignore. The concerns here include not just data breaches and privacy violations but also AI systems being hacked to execute malicious intent leading to fatal incidents.
Here are the key takeaways from Day 1 (Thursday, September 6, 2018):
Kai-Fu Lee, Chairman and CEO, Sinovation Ventures gave an interesting keynote on China: AI superpower. He started his talk with a demo of ML-based voice-simulation system (developed by iFLYTECH, startup based in China) talking in the voice of President Trump. He classified AI advances in four waves. First wave was enabled by Internet leading to the generation and collection of vast amounts of data. Second wave was about revenue generation or cost saving from leveraging properly structured data warehouses. Third wave was about digitizing the physical world through autonomous stores such as Amazon Go and perception technologies such as face recognition. Third wave is about AI becoming the sensory inputs such as eyes and ears. Fourth wave is about being not just sensory input system but also having the ability take action – having arms and legs – so that, robots can act and manipulate physical world. Applications include autonomous driving and robots running industrial assembly lines.
Comparing US and China, Dr. Lee noted that US leads in AI Science and Research and US is dominant in a lot of technology fields such as PC (Wintel), Software (Oracle), Portal (Yahoo), Search (Google), Smartphone (Apple), and Social (Facebook). However, a major event occurred in China about 10 years ago as the following innovation loop became operational.
Chinese entrepreneurs take lead in terms of hard work, typically working on 9-9-6 schedule (working from 9am to 9pm, 6 days a week). In 10 years, China has transitioned from copying innovations from US to innovations coming truly from Chinese startups. He claimed that WeChat is better product than Facebook Messenger and Weibo is a better product than Twitter.
Compared to US entrepreneurs, Chinese entrepreneurs face a much more competitive market which has made them a lot more tenacious, innovative, and fast. Another major distinction between the two markets is that China’s AI capital leads the world – in 2017 48% of world-wide AI investments came from China vs 38% from US. Sinovation Ventures has created 5 unicorns (total $23B in valuation) in last 2 years.
China is compensating for its lack in research talent through a strong talent and regulatory environment for implementation of AI technologies. He asserted that in the age of AI, China is the Saudi Arabia in Data. Finally, he mentioned that today US is ahead of China, but China is very likely to overcome US in next 5 years.
Dr. Lee has written a book titled “AI Super-Powers – China, Silicon Valley, and the New World
Order” explaining these points in greater detail.
Greg Brockman, Co-founder and CTO, OpenAI delivered a highly inspiring keynote on OpenAI and the path towards safe AGI. He strongly asserted that while highly uncertain, near-term AGI (Artificial General Intelligence) should be taken as a serious possibility. He shared evidence from the following 4 areas to support his assertion:
Lessons from history of science:
In 1901, Simon Newcomb proved that heavier-than-air flight is impossible. Even when Wright brothers managed a successful flight in 1908, Simon said that it will not be commercially viable.
After World War 2, both US and Russia were thinking about extending V-2 (14-ton rocket) technology to 5-ton payload (200-ton rocket) i.e. scaling it by 10x. US rejected the idea calling it impossible. Russia experimented and built the giant rocket which helped it become the first to enter Space.
Lessons from history of AI:
Historical view of AI had been that AI is full of fads which come and go every few years such as Perceptron in 1960s, expert systems in 1970s, back-propagation in 1980s, SVMs in 1990s, and so on. The idea of Perceptron generated great hype about its potential leading to unsustainably high short-term expectations. This led to hostility from the other group of scientists who were focused on building bigger machines and were afraid of all funding being taken away by neural network research. In 1980s, there was a revival of neural networks led not by people from computer science (focused on scale), but rather people from cognitive science who were rather happy to have the machines solve basic problems in an autonomous way. Overall, as we look back it is clear that the fads were more political than technical.
Fundamental limits on deep learning:
Time and again, experts have publicly announced that there are fundamental limits on deep learning such as inability to do long-term planning or train using sparse sampling, and these limits have been shattered within a few years. The deep learning algorithm has been making strong advances across disciplines – speech recognition, machine translation, object recognition, etc.; much stronger progress compared to the domain-specific solutions. Previously, it was believed that deep learning is only about static datasets, but after DQN (2013), it was clear that RL (Reinforcement Learning) + deep networks might be able to observe and act in the world. Similarly, it was believed that RL cannot solve hard tasks, and it was proved wrong with the success of AlphaGo in 2016. Also, it was previously believed that AI progress is driven by labeled datasets, but a lot of unsupervised NLP papers in 2018 have proved that unlabeled data can be even more important than labeled.
Practical limits on compute:
Compute technology is doubling every 3.5 months. To understand the scale comparison here is an analogy – if your phone battery lasted 1 day in 2012 and had similar growth rate, it would last 800 years in 2018, and would last 100 million years in 2023.
He concluded his keynote asking for proactively thinking about safety and policy for AGI to address scenarios such as: machines pursuing goals mis-specified by their operator, malicious humans subverting deployed systems, or out-of-control economy that grows without resulting in improvement to human lives.
Ben Lorica, Chief Data Scientist, O’Reilly Media and Roger Chen, CEO, Computable Labs delivered an insightful talk on Unlocking Innovation in AI, focused on Compute and Data aspects. More and more companies are beginning to explore deep learning, and their biggest bottleneck is the lack of talent. Sharing data about usage on Safari, O’Reilly Online Learning platform, for 2017 and 2018, they highlighted the strong increase in learning for Deep Learning, Neural Networks, and Artificial Intelligence – particularly, Keras, PyTorch, and Reinforcement Learning.
A survey of about 8,000 IT leaders across 84 countries (CIO Survey by Harvey Nash and KPMG) revealed a growing interest in AI and automation. Location proved out to be an important metric, as the interest in RPA (Robotic Process Automation) is particularly more in China compared to rest of world.
In 2018, we are seeing an average of 90+ ML papers per day on arxiv. Compared to a few years ago, there is a strong trend of democratization in ML/AI/DL research, particularly in open-source ML libraries.
The hardware sector is no more limited to traditional players such as Intel and AMD. We are seeing hardware participation from leading tech companies.
We are witnessing a steady increase in startups focused on hardware for edge devices across China, US, and rest of the world.
Despite great advances in hardware, the major bottleneck for AI (particularly for deep learning) continues to be labelled data. The AI advances and ‘Enormous Data’ could make Tech Giants harder to topple. By 2020, China will hold 20% of the global data, which is expected to reach 44 trillion gigabytes.
Apart from hardware and software advancements, some startups are now focusing on data sharing – enabling it through cryptography, blockchain, and secure communication.
Related: