This post is authored by Tara Shankar Jana, Senior Technical Product Marketing Manager at Microsoft.
Among our exciting announcements at //Build, one of the things I was thrilled to launch is the AI Lab – a collection of AI projects designed to help developers explore, experience, learn about and code with the latest Microsoft AI Platform technologies.
What is AI Lab?
AI Lab helps our large fast-growing community of developers get started on AI. It currently houses five projects that showcase the latest in custom vision, attnGAN (more below), Visual Studio tools for AI, Cognitive Search, machine reading comprehension and more. Each lab gives you access to the experimentation playground, source code on GitHub, a crisp developer-friendly video, and insights into the underlying business problem and solution. One of the projects we highlighted at //Build was the search and rescue challenge which gave the opportunity to developers worldwide to use AI School resources to build and deploy their first AI model for a problem involving aerial drones.
AI Lab is developed in partnership with Microsoft’s AI School and the Microsoft Research (MSR) AI organization.
AI Lab Experiments
We released the following experiments from Microsoft at //Build:
1. DrawingBotAt the core of Microsoft’s drawing bot is a technology known as a Generative Adversarial Network, or GAN. The network consists of two machine learning models, one that generates images from text descriptions and another, known as a discriminator, that uses text descriptions to judge the authenticity of generated images. The generator attempts to get fake pictures past the discriminator; the discriminator never wants to be fooled. Working together, the discriminator pushes the generator toward perfection. GANs work well when generating images from simple text descriptions such as a blue bird or an evergreen tree, but the quality stagnates with more complex text descriptions such as a bird with a green crown, yellow wings and a red belly. As humans draw, we repeatedly refer to the text and pay close attention to the words that describe the region of the image we are drawing. To capture this human trait, the researchers created what they call an attentional GAN, or AttnGAN, that mathematically represents the human concept of attention. It does this by breaking up the input text into individual words and matching those words to specific regions of the image. This drawing bot is developed by researchers from MSR AI lab is programmed to pay close attention to individual words when generating images from caption-like text descriptions. This deliberate focus produced a nearly three-fold boost in image quality compared to the previous state-of-the-art technique for text-to-image generation, according to results on an industry standard test reported in a research paper, which is published at the top conference CVPR in computer vision. To learn more, click here.
2. JFK Files Cognitive search is the backbone of the JFK Files experiment. We announced Cognitive Search at //Build, an AI-first approach to content understanding. Cognitive Search is powered by Azure Search with built-in Cognitive Services capabilities – it can pull data from almost any source and apply a set of composable cognitive skills to extract knowledge out of it. This knowledge is organized and stored in an index, enabling new experiences for exploring the data using Search. When we first applied Cognitive Search to the JFK files, it was incredible to see what emerged. Not only could we many interesting questions, we could also see the answers and relationships in the context of the original documents. When we first showed the JFK Files demo to customers, they were immediately able to see how they would apply it to their own domains, using it to answer their own questions. With Cognitive Search, it’s never been easier to bring the full power of the Cloud and AI to your data. You can get started in just a few minutes from the portal today – click here to learn more.
.
3. Style Transfer To create this application, we used Visual Studio Tools for AI to train the deep learning models and include them in our app. Visual Studio Tools for AI improved our productivity by easily enabling stepping through our Keras+Tensorflow model training code on our local dev machine, then submitting to Azure VMs with powerful Nvidia GPUs. By starting with a pre-trained model like VGG-19, we were able to speed up the model training and still maintain semantic similarities to the original input image by preserving objects like people, buildings, cars and more. Additionally, Visual Studio Tools for AI also generated C# code from our trained TensorFlow models to include them in our application without having to write the code. Using the new Microsoft.ML.Scoring library it’s easy to include TensorFlow or ONNX models in applications that run on your devices or in the cloud. Style transfer applications are just one type of application that use trained machine learning models, but the process to infuse these new ML experiences is always the same. You can train models yourself using frameworks like Tensorflow or CNTK, or you can use pre-trained AI capabilities like Azure Cognitive Services. You too can get started with our AI platform and become an AI developer today – to learn more, click here.
4. Machine Reading Comprehension (MRC)Machine Reading Comprehension (MRC) is all about answering a query about a given context paragraph. MRC requires modeling complex interactions between the context and the query. Using a novel neural network architecture called the Reasoning Network (ReasoNet), Microsoft Researchers were able to mimic the inference process of human readers. With a question in mind, ReasoNets read a document repeatedly, each time focusing on different parts of the document until a satisfying answer is found or formed. Microsoft Researchers today have been able to surpass human-level parity on the SQUAD dataset using an unique MRC algorithm called R-NET: Machine Reading Comprehension With Self-Matching Networks. R-NET applies a self-matching attention mechanism to refine the representation by matching the passage against itself, which effectively encodes information from the whole passage. When we applied these MRC algorithms to the book Future Computed by Harry Shum and Brad Smith. It was incredible to see how many interesting questions we were able to answer using this technique. We can apply this to enterprise data and help customers answer their domain-specific questions. To learn more, click here.
5. Drones + AirSim (//Build challenge) For this search and rescue scenario, we created a 3D-generated environment in AirSim to simulate the soccer field on Microsoft campus and placed stuffed animals on the field. We then created a Python script to fly a drone around the simulated environment and take pictures of the animals. We then pushed the images into the Custom Vision service and trained a model to identify each type of animal in the field. From there we exported the trained model into TensorFlow format and pushed it into Docker containers. These containers were then deployed to Azure IoT Edge and then pushed to a drone running a custom board and a Nvidia GPU. The drone is then able to fly around and send an alert to Azure IoT Hub every time it successfully identifies an animal. This is a great showcase of how real-time custom AI can run on edge devices, such as a drone. To learn more, click here.
Coming Soon – AI Lab Community Submission
The next phase of the AI Lab, which will be released soon, will enable community submission. The idea is to engage developers and get your feedback about the Microsoft AI platform. While we work to create the portal submission experience, we can’t wait to hear about your ideas and what you have been doing with Microsoft AI – please share your thoughts via the commenting feature below.
Summary
To quickly recap, this post was about how you can:
-
Start building your AI skills by learning, coding and experiencing the latest from the Microsoft AI Platform via the AI Lab (www.ailab.microsoft.com).
-
Submit experiments using the AI Lab platform and help the community through the power of your ideas.
-
Use resources from the AI School and Microsoft Research AI to take your AI skills to the next level.
We hope this post helps you get started with AI and motivates you to become an AI developer!
Tara
