Building Intelligent Cloud Applications Develop Scalable Models Using Serverless Architectures with Azure (Vicente Herrera García, John Biggs)（Z-Library）

Page 1

Building Intelligent Cloud Applications Develop Scalable Models Using Serverless Architectures with Azure Vicente Herrera Garcia & John Biggs

Page 2

Vicente Herrera García and John Biggs Building Intelligent Cloud Applications Develop Scalable Models Using Serverless Architectures with Azure Boston Farnham Sebastopol TokyoBeijing

Page 3

978-1-492-05221-0 [LSI] Building Intelligent Cloud Applications by Vicente Herrera García and John Biggs Copyright © 2019 O’Reilly Media, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Editors: Rachel Roumeliotis and Nicole Tache Production Editor: Kristen Brown Copyeditor: Octal Publishing Services, Inc. Proofreader: Rachel Head Indexer: Judith McConville Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Rebecca Demarest April 2019: First Edition Revision History for the First Edition 2019-04-25: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492052326 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Building Intelligent Cloud Applications, the cover image of an azure jay, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This work is part of a collaboration between O’Reilly and Microsoft. See our statement of editorial inde‐ pendence.

Page 4

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Part I. Cloud-Based Development 1. Machine Learning and Deep Learning Models in the Cloud. . . . . . . . . . . . . . . . . . . . . . . . 3 An Introduction to Machine Learning 3 An Introduction to Deep Learning 7 Neural Networks 7 Difficulties Defining Structure and Training Machine Learning Models 8 An Introduction to Serverless Machine Learning 10 Using Containers with Machine Learning Models 12 The Benefits of Serverless Computing for Machine Learning 13 2. Functions-as-a-Service and Event-Driven Programming. . . . . . . . . . . . . . . . . . . . . . . . . 15 Software-as-a-Service, Cloud Computing, and Serverless 15 Microservices Architecture 18 The Rise of Functional Programming 19 Using Functions Instead of Objects 22 Asynchronous Programming 24 Serverless 25 Implementing Functions 28 Event-Driven Architecture 29 Implications of Real-Time Processing 30 Summary and Look Ahead 32 3. Serverless Application Programming Interfaces in Microsoft Azure. . . . . . . . . . . . . . . 33 APIs in Serverless Platforms 33 iii

Page 5

An Introduction to Azure 34 Azure General Services 35 Part II. Adding Intelligence 4. Getting Started with Microsoft Azure Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Azure Functions 43 Creating a Function App by Using Azure Portal 44 Local Development Environment 48 Creating a Functions Project Using Core Tools 52 Using Visual Studio Code 60 Debugging Python by Using Visual Studio Code 63 Azure Blob Storage 66 Creating a Storage Account 66 Transferring Blobs 68 Summary and Look Ahead 70 5. Using Machine Learning and Deep Learning Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Azure Cognitive Services 71 Setting Up and Using a Service in Cognitive Services 72 Using a Cognitive Service from a Serverless Azure Functions Project 74 General Machine Learning Tools, Libraries, and Frameworks 75 Microsoft Cognitive Toolkit 75 ML.NET 77 Jupyter Notebook 78 TensorFlow 78 Keras 79 Scikit-learn 79 Cloud Machine Learning Solutions with Azure 81 Microsoft Machine Learning Studio 81 Azure Machine Learning Service 86 Part III. Deployment and Continuous Delivery 6. Deployment and Scaling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 Deployment Options 93 Azure DevOps 95 Docker 97 Azure Container Registry 98 Kubernetes 99 iv | Table of Contents

Page 6

Machine Learning Tools for Kubernetes 102 Single-Container Machine Learning 105 Distributed Machine Learning with TFJob 108 7. Security. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Azure Functions Authorization Levels 111 API Management 113 Azure Security 117 Operations 117 Applications 118 Storage 119 Network Layer Controls 119 Networking 120 8. Monitoring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Azure Monitor 123 Metrics and Logs 125 Data Sources 126 Application Insights 128 Insights for Containers (and VMs) 128 Log Analytics 129 Data Explorer Query Language 129 Alerts 130 Smart Groups 130 Autoscale and Metrics Alerts 131 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 Table of Contents | v

Page 7

(This page has no text content)

Page 8

Preface The past few years have brought massive changes to the way we think about applica‐ tions. As products scaled to a global level, developers worked hard to keep up, creat‐ ing new paradigms that left behind the complex client–server systems that were then in place. The resulting systems are called serverless applications, and they will be the primary focus of this book. Most applications require computing resources, databases, and hardware. The assumption behind a program in Django, for example, is that it will be deployed in an environment that is similar or superior to the environment in which it was coded. It’s also assumed that the program will be on all the time, using resources and running up a bill. A serverless program is, in essence, a system that requires no infrastructure manage‐ ment, allowing the programmer to focus primarily on the business logic or function‐ ality of the program. It is a bold change in the world of program design, and it’s becoming more and more popular. To understand the implications of this model, you need to have an understanding of the basic ideas that underlie functional programming. We’ll review them in Chap‐ ter 2. Intelligent Serverless Applications There are many different meanings, often overlapping, attributed to the terms “artifi‐ cial intelligence” and “machine learning.” In this book, we concern ourselves with the ways in which we can use algorithms to find patterns hidden in large datasets—pat‐ terns that would be difficult or impossible for humans to detect without the help of computers. There are various ways in which we can set algorithms to adjust them‐ selves to improve their performance upon each iteration. This is called machine learning. vii

Page 9

Serverless applications and functional programming lend themselves perfectly to the exploration of these systems, and in this book you will learn how to build machine learning applications using a serverless architecture. How This Book Is Organized First, we should note that this book focuses on Azure, Microsoft’s cloud platform. The concepts we discuss here are universal, but our examples describe how to imple‐ ment them in Azure. The book comprises eight chapters divided into three parts. Part I, Cloud-Based Development, lays out the idea of “serverless applications” and explains how event-driven apps are built up from self-contained function calls hosted in the cloud. This part explains the APIs that are the interface between your locally running program and the cloud-based machine learning and data-manipulation services. Part II, Adding Intelligence, discusses the role of data and how you make it available to your programs. The very essence of “intelligence” in cloud-based applications comes from the use of various machine learning techniques and models to interrogate large datasets. You don’t need to build your own models; you just need to understand which ones best address the problem you’re trying to solve. The functions have already been written—you just need to understand when and how to employ them. Part III, Deployment and Continuous Delivery, explains how to release your serverless applications into the world in a reliable, efficient, cost-effective, secure, testable man‐ ner that allows you to monitor their performance and continually improve them. Who This Book Is For This book is for experienced programmers who would like to familiarize themselves with functional programming through the lens of machine learning. As we said, the book uses Microsoft’s Azure platform to build these systems, and it uses Python as the primary programming language. Goals for the Book When you are done with this book, we hope that you will be proficient in serverless programming in Azure as it relates to machine learning. viii | Preface

Page 10

Using Python in Our Code Examples Examples in this book are primarily in Python 3.6. Python is a widely used open source language, with runtimes, compilers, and tools available for all operating sys‐ tems. It is well suited to use with the Azure platform. Python implements an object- oriented and functional paradigm and is one of the main languages used for machine learning projects, with many open source libraries available. All serverless platforms/ providers support Python. You should be aware, however, that there are many other functional languages, for example JavaScript and Clojure, that are also well suited to this kind of programming. If you know nothing about machine learning in Python, don’t worry. We’ll guide you through the core concepts, and the example code is meant to be short and supportive. If you find them difficult to fully understand, we encourage you to look for a short introductory Python course to get up to speed. This book is here to help you get your job done. In general, if example code is offered in this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing a CD-ROM of examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Building Intelligent Cloud Applica‐ tions by Vicente Herrera García and John Biggs (O’Reilly). Copyright 2019 O’Reilly Media, Inc., 978-1-492-05221-0.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Preface | ix

Page 11

Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. This element signifies a general note. This element indicates a warning or caution. O’Reilly Online Learning For almost 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help compa‐ nies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in- depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com. x | Preface

Page 12

How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) To comment or ask technical questions about this book, send email to bookques‐ tions@oreilly.com. For more information about our books, courses, conferences, and news, see our web‐ site at http://www.oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://www.youtube.com/oreillymedia Preface | xi

Page 13

(This page has no text content)

Page 14

PART I Cloud-Based Development

Page 15

(This page has no text content)

Page 16

CHAPTER 1 Machine Learning and Deep Learning Models in the Cloud It doesn’t seem that long ago that artificial intelligence (AI) was a dream. The idea that a machine could simulate and even beat humans at games of skill, image recogni‐ tion, and predictions was preposterous 20 years ago. Now, the average user brushes up against some form of machine learning every day and everywhere—from our cars to stores to doctors’ offices, and throughout our homes. We are living at the dawn of thinking machines. But how do they think? What do they use to build models of the world? And how can we, as developers, use these tools to make our systems smarter, more responsive, and more lifelike? Our goal in this book is to discuss the basics of machine learning and to show you, in a step-by-step introduction, how to implement and code machine learning into your projects using serverless systems and pretrained models. We can think of machine learning as a tool for interacting with an ever-changing world using models that change and grow as they experience more of that world. In other words, it’s how we teach computers new things without explicitly programming them to do anything. An Introduction to Machine Learning The AI discipline, of which machine learning is a part, was born in the 1950s, during the Cold War, as a promise to develop systems that could solve complex problems. At that time, computers were not powerful enough for the task. Over the years, AI began to encompasses many different subdisciplines, from algorithms used to find a maze exit to systems that could recognize human emotions, drive cars, and predict future outcomes. 3

Page 17

Machine learning, also often referred to as ML, is the study and creation of algo‐ rithms that become better at a specific task when they receive more data about that task. Most machine learning models begin with “training data” such as a large group of emails or a huge folder of images, which the machine processes and begins to “understand” statistically. Because machine learning assumes no active human pro‐ gramming, the machine algorithm itself changes and grows as it addresses the train‐ ing data. After a while, the algorithm is ready for “real” data. It continues to evolve as it processes new information, ultimately leading to an answer or solution without outside intervention. Machine learning is important for the simple reason that not all problems consist of a closed set of variables and routines. For example, a machine learning model could be tasked to analyze spots on the skin for cancer. Because every spot is different, the machine learning model will categorize each spot based on its own statistically chosen criteria and, when queried, will return its best guess as to whether a spot is cancerous—which, in many cases, is as good or better than a human guess. Tradi‐ tional algorithms solve very specific, bounded problems. Machine learning lets us train a model to solve something that we initially might not know how to do. We must also remember the difference between algorithms and training. You can have an algorithm for facial recognition that uses a set of training data passed into the system by the developer. But if your goal is something else—to look for vehicle license plates in pictures of cars, for example—you might use the same simple pattern recognition algorithm with the new dataset. Depending on your use case, you could either look for a model that has the algorithm and present new data on which to train it, or look for a model that’s already trained for the problem that you are trying to solve. Models aren’t always perfect, however. Researcher Victoria Krakovna created a popu‐ lar list of machine learning “mistakes” in which the machine learned to achieve goals in ways that didn’t address the problems that humans were trying to solve. In a life simulation game, for example, “creatures bred for jumping were evaluated on the height of the block that was originally closest to the ground. The creatures developed a long vertical pole and flipped over instead of jumping.” Another foible appeared in the video game Sims, in which “creatures exploited physics simulation bugs by twitch‐ ing, which accumulated simulator errors and allowed them to travel at unrealistic speeds.” Our absolute favorite, however, involves the cannibalistic parents of another life simulation. As Krakovna describes: In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children). These comical examples—edible children?—point to the sometimes weird conclu‐ sions that machine learning systems make when looking at data that humans would 4 | Chapter 1: Machine Learning and Deep Learning Models in the Cloud

Page 18

find opaque. But we can’t blame these machines for their will to “win.” After all, that’s all they’re programmed to do! Machine learning algorithms are very diverse, having many different models that we can use for solving problems like prediction of outcomes and estimation (in which some value fluctuates by some natural causes) and classification of elements (e.g., examining satellite pictures to determine which areas are urban, which are forests, and which are bodies of water). Examples of machine learning algorithms include the following: Anomaly detection This can identify rare items, events, or observations that differ significantly from the majority of the data. It can be used to detect bank fraud, for example. Classification This is good for predicting the classes of given data points. It’s useful, for exam‐ ple, in spam detection or identifying tumors. Clustering This groups a set of objects in such a way that objects in the same group are more similar in some sense to one another than to those in other groups. It is useful in pattern recognition, image analysis, data compression, biological classification, and insurance, for example. Recommendation Using a dataset of user-item-rating triples, this model can generate recommenda‐ tions and find related items. It is used in media streaming services to recommend movies or music as well as online shops to recommend items to customers. Regression This is used to infer the expected quantity for an input related to a set of data points. Assuming that there is a linear, polinomic, logistic, or any other mathe‐ matical relation between inputs and outputs, the best coefficients possible of this math relation are inferred. Any type of experiment that records data uses this type of analysis to prove mathematical relation or correlation. In business, it’s used to forecast revenue, and insurance companies rely on regression analysis to estimate credit standing of policy holders and possible number of claims in a given time period. Statistical functions Machine learning can compute mathematical operations over a large set of data. It can also calculate correlation and probability scores and compute z-scores, as well as statistical distributions such as Weibull, gamma, and beta. Statistics has many applications to understand and model systems with a large number of ele‐ ments. Governments and other organizations use it to understand data pertain‐ An Introduction to Machine Learning | 5

Page 19

ing to wealth, income, crimes, and so on. Businesses use it to know what to produce and when. Social and natural scientists use it to study the demographic characteristics of a population. Text analytics This extracts information from text, such as most likely language used or key phrases. Sentiment analysis is an example application. Computer vision This is used to read text in images or handwritten notes, recognize human faces or landmarks, and analyze video in real time. What these machine learning techniques all have in common is that they rely on learning automatically from a very large set of data. The programmer can define the algorithm architecture and some initial parameters, and then the program learns itself by studying the data. If you want to know about what you can achieve with these techniques, here are some great examples of success stories for machine learning projects: JFK Files In 2017, more than 34,000 pages related to the assassination of John F. Kennedy were released by the United States government, consisting of a mixture of typed and handwritten documents. Thanks to Azure Search and Cognitive Services, a set of composable cognitive skills were applied to this data to extract knowledge and organize it into an index. Not only can this system answer many interesting questions, but you also see the answers and relationships in context with the original documents. Snip Insights This is an open source screen-capture desktop application that uses Azure Cog‐ nitive Services to gain instant insight into the image just captured. It can convert images to text and display information about the subject (for example, identify‐ ing celebrities and landmarks). If the image is of a product, the application will automatically search for similar products, providing information about how much each one costs and where to buy it. Pix2Story This is a web application that uses natural language processing (NLP) to teach an AI system, inspired by a picture, to write a machine-generated story. Captions obtained from the uploaded picture are fed to a recurrent neural network model to generate the narrative based on the genre and the contents of the picture. Sketch2Code This is an AI solution that converts hand-drawn renderings on paper or white‐ board to working HTML prototypes. After training the model with images of 6 | Chapter 1: Machine Learning and Deep Learning Models in the Cloud

Page 20

hand-drawn design elements like text boxes, buttons, or combo boxes, the Cus‐ tom Vision Service performs object detection to generate the HTML snippets of each element. To learn more about these and other examples, visit the Microsoft AI Lab website. An Introduction to Deep Learning Deep learning is a class of machine learning algorithms that does the following: • Uses a cascade of multiple layers of nonlinear processing units for feature extrac‐ tion and transformation. Each successive layer uses the output from the previous layer as input. • Learns in supervised (e.g., classification) and/or unsupervised (e.g., pattern anal‐ ysis) ways. • Learns multiple levels of representations that correspond to different levels of abstraction. The levels form a hierarchy of concepts. Deep learning’s use of this hierarchy of concepts and classifications contrasts with the often-brutish machine learning methods popular in the lab. When we have many hidden layers in a model, we say that the network is a deep learning system because deep within the layers of the model it retains the knowledge it is gaining from the examples. We can use these kinds of models in a lot of ground‐ breaking applications thanks to the great processing power available in modern com‐ puters. But with each new layer of complexity, even more power is required to achieve new goals. Neural Networks The next frontier in machine learning is the neural network. The idea of the computer-based neural network has been around for decades, but only recently has hardware arrived with sufficient horsepower to make it widely useful. The neural net‐ work is built on a model of how a human brain works. We begin with a simple data processor called a perceptron, which works as a base neuron does in our own brains. Like a neuron, each perceptron takes an input, checks it against a gate function, and produces an output that can, at the same time, control other perceptrons. This net of perceptrons is called an artificial neural network, which you can see in Figure 1-1. An Introduction to Deep Learning | 7