Serverless Applications with Node.js Using AWS Lambda and Claudia.js (Slobodan Stojanovic, Aleksandar Simovic) (z-library.sk, 1lib.sk, z-lib.sk)

Stojanovic ● Simovic T he benefi ts of cloud-hosted serverless web apps are undeniable: lower complexity, quicker time to market, and easier scalability than traditional, server-dependent designs. And thanks to JavaScript support in AWS Lambda and powerful new serverless API tools like the Claudia.js library, you can build and deploy serverless apps end to end without learning a new language. Serverless Applications with Node.js teaches you to design and build serverless web apps on AWS using JavaScript, Node, and Claudia.js. You’ll master the basics of writing AWS Lambda functions, along with core serverless patterns like API Gate- way. Along the way, you’ll practice your new skills by building a working chatbot and a voice assistant with Amazon Alexa. You’ll also discover techniques for migrating existing apps to a serverless platform. What’s Inside ● Authentication and database storage ● Asynchronous functions ● Interesting real-world examples ● Developing serverless microservices For web developers comfortable with JavaScript and Node.js. Slobodan Stojanovic and Aleksandar Simovic are AWS Serverless Heroes and core contributors to the Claudia.js project. They are also coauthors of Desolé, an open source serverless error- tracking tool, and the lead developers of Claudia Bot Builder. To download their free eBook in PDF, ePub, and Kindle formats, owners of this book should visit manning.com/books/serverless-applications-with-node-js $44.99 / Can $59.99 [INCLUDING eBOOK] Serverless Applications with Node.js WEB DEVELOPMENT/CLOUD Stojanovic Simovic M A N N I N G Serverless Applications with Node.js MANN I N G “Teaches you how to get simple stuff done in AWS Lambda quickly, without trying to change the way you structure or run projects.” —From the Foreword by Gojko Adžić, Neuri Consulting “A top-notch and hands-on resource written by world- renowned experts who will get you up to speed in no time with AWS Lambda using Claudia.js.” —Valentin Crettaz, Consulthys “One of the most comprehensive books published on the topic; contains a wealth of resources that you will not fi nd online.”—Damian Esteban, BetterPT M A N N I N G See first page ´´ Using AWS Lambda and Claudia.js´ ´ ´´ ´ Slobodan Stojanovic Aleksandar Simovic Foreword by Gojko Adžic ´ ´

MANN I NG Shelter ISland Serverless Applications with Node.js SLOBODAN STOJANOVIĆ ALEKSANDAR SIMOVIĆ

For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Email: orders@manning.com ©2019 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid- free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. ∞ Manning Publications Co. 20 Baldwin Road PO Box 761 Shelter Island, NY 11964 Development editor: Toni Arritola Review editor: Ivan Martinović Project manager: Vincent Nordhaus Copy editor: Darren Meiss Proofreader: Sarah Boyer Technical proofreader: Valentin Crettaz Typesetter: Happenstance Type-O-Rama Cover designer: Marija Tudor ISBN 9781617294723 Printed in the United States of America 1 2 3 4 5 6 7 8 9 10 – SP – 24 23 22 21 20 19

iii contents foreword ix preface xi acknowledgments xiii about this book xv about the authors xix about the cover illustration xx Part 1 Serverless pizzeria ......................................1 1 Introduction to serverless with Claudia 3 1.1 Servers and washing machines 4 1.2 The core concepts 5 1.3 How does serverless work? 6 1.4 Serverless in practice 6 Aunt Maria’s serverless pizzeria 7 ■ A common approach 7 ■ Serverless approach 9 1.5 Serverless infrastructure — AWS 9 1.6 What is Claudia, and how does it fit? 14 1.7 When and where you should use it 16

iv contents 2 Building your first serverless API 19 2.1 Assembling pizza ingredients: building the API 19 Which pizza can I GET? 22 ■ Structuring your API 25 ■ POSTing an order 30 2.2 How Claudia deploys your API 34 2.3 Traffic control: How API Gateway works 35 2.4 When a serverless API is not the solution 36 2.5 Taste it! 37 Exercise 37 ■ Solution 37 3 Asynchronous work is easy, we Promise() 42 3.1 Storing the orders 43 3.2 Promising to deliver in less than 30 minutes! 48 3.3 Trying out your API 51 3.4 Getting orders from the database 55 3.5 Taste it! 57 Exercise 57 ■ Solution 58 4 Pizza delivery: Connecting an external service 62 4.1 Connecting to an external service 63 4.2 Connecting to the delivery API 64 The Some Like It Hot Delivery API 64 ■ Creating your first delivery request 65 4.3 Potential issues with async communication 71 Forgetting to return a promise 72 ■ Not passing the value from the promise 73 ■ Not wrapping the external service in a promise 73 ■ Timeout issues with long async operations 74 4.4 Taste it! 76 Exercise 76 ■ Solution 77 5 Houston, we have a problem! 80 5.1 Debugging a serverless app 81 5.2 Debugging your Lambda function 82 5.3 X-Ray your app 85 5.4 Taste it! 88 Exercise 88 ■ Solution 88

vcontents 6 Level up your API 90 6.1 Serverless authentication and authorization 91 6.2 Creating user and identity pools 94 Controlling API access with Cognito 97 6.3 Taste it! 101 Exercise 101 ■ Solution 101 7 Working with files 104 7.1 Storing static files in a serverless application 104 7.2 Generating thumbnails 108 Deploying your S3 function 114 7.3 Taste it! 116 Exercise 116 ■ Solution 116 7.4 End of part 1: Special exercise 118 Advanced task 118 Part 2 Let’s talk ................................................... 121 8 When pizza is one message away: Chatbots 123 8.1 Pizza ordering beyond the browser 123 8.2 Hello from Facebook Messenger 124 8.3 What kinds of pizzas do you have? 128 8.4 Speeding up the deployment 130 8.5 Messenger templates 131 8.6 How does Claudia Bot Builder work? 134 8.7 Taste it! 136 Exercise 136 ■ Solution 136 9 Typing... Async and delayed responses 138 9.1 Making chatbots interactive 139 Tap to order: answering a postback 139 9.2 Making the chatbot structure more scalable 145 9.3 Connecting your chatbot to the DynamoDB database 149 9.4 Getting the user’s location from the chatbot 153

vi contents 9.5 Scheduling a delivery 156 9.6 Small talk: Integrating simple NLP 161 9.7 Taste it! 163 Exercise 163 ■ Solution 163 ■ Advanced exercise 165 10 Jarvis, I mean Alexa, order me a pizza 166 10.1 Can’t talk right now: sending an SMS with Twilio 167 An SMS pizza list 169 ■ Ordering a pizza 170 10.2 Hey Alexa! 175 Preparing the skill 179 ■ Ordering pizza with Alexa 183 10.3 Taste it! 187 Exercise 187 ■ Solution 187 10.4 End of part 2: special exercise 188 Part 3 Next steps .................................................. 189 11 Testing, Testing, 1, 2, 3 191 11.1 Testing server-hosted and serverless applications 191 11.2 How to approach testing serverless applications 193 11.3 Preparation 194 11.4 Unit tests 197 11.5 Mocking your serverless functions 201 11.6 Integration tests 207 11.7 Other types of automated testing 211 11.8 A step beyond: Writing testable serverless functions 212 11.9 Taste it! 216 Exercise 216 ■ Solution 217 12 Paying for pizza 219 12.1 Payment transactions 220 Implementing an online payment 221 12.2 Implementing your payment service 225 12.3 Could someone hack your payment service? 231 Standards 231 ■ Competence 232 12.4 Taste it! 232 Exercise 232 ■ Solution 233

viicontents 13 Migrating your existing Express.js app to AWS Lambda 235 13.1 Uncle Roberto’s taxi application 236 13.2 Running an Express.js application in AWS Lambda 236 Proxy integration 239 ■ How serverless-express works 239 13.3 Serving static content 240 13.4 Connecting to MongoDB 242 Using a managed MongoDB database with your serverless Express. js app 243 13.5 Limitations of serverless Express.js applications 248 13.6 Taste it! 249 Exercise 249 ■ Solution 249 14 Migrating to serverless 251 14.1 Analyzing your current serverless application 252 14.2 Migrating your existing application to serverless 252 14.3 Embrace the platform 256 Serving static files 256 ■ Storing state 256 ■ Logs 257 ■ Continuous integration 258 ■ Managing environments: production and development 259 ■ Sharing secrets 260 ■ VPC (Virtual Private Cloud) 262 14.4 Optimizing your application 263 Bundled or single-purpose functions 263 ■ Choosing the right memory size for your Lambda function 263 14.5 Facing the challenges 264 Handling timeouts 264 ■ Cold starts 265 ■ DDoS attacks 266 ■ Vendor lock-in 267 14.6 Taste it! 268 15 Real-world case studies 269 15.1 CodePen 270 Before serverless 270 ■ Serverless migration 271 ■ Cost of the infrastructure 273 ■ Testing and challenges 274 15.2 MindMup 275 Before serverless 275 ■ Serverless migration 276 ■ Cost of the infrastructure 278 ■ Testing, logs, and challenges 280

viii contents appendix A Installation and configuration 283 appendix B Facebook Messenger, Twilio, and Alexa configuration 289 appendix C Stripe and MongoDB setup 311 appendix D The pizza recipe 321 index 323

ix foreword Amazon forever changed IT infrastructure by making it easy to provision virtual machines back in 2007. Since then, the architectural improvements for modern appli- cations have mostly been incremental. A decade later, by making it easy to provision functions, Amazon’s Lambda platform started another tectonic wave of change. This “serverless” ecosystem is revolutionizing how we design, develop, and operate internet applications. As an early adopter of this platform, I've had the privilege of working with Slobodan and Aleksandar and seeing first-hand the huge impact on time-to-market and on the cost of operation that “serverless” thinking brings. At the same time, the platform evolves so quickly that it's easy to get lost. To truly get the benefits of the new way of operations, developers have to rethink authentication, session management, storage, capacity planning, and distribution strategies. In Serverless Applications with Node.js, Slo- bodan and Aleksandar provide a front seat report about this revolution and an invalu- able guide for JavaScript developers who want to benefit from the new generation of platforms. What I love about this book is how it helps people get simple stuff done in AWS Lambda quickly, without trying to change the way we structure or run projects. Many serverless application frameworks abstract away AWS services, making framework lock-in a big risk because the ecosystem is still evolving rapidly. The authors don't force us to take a bet on their choice of frameworks, but explain how to use all the related services easily. For people new to AWS, this book introduces not just AWS Lambda, but also a whole host of related services such as DynamoDB (storage), Cognito (authenti- cation), API Gateway (running web services), and Cloudwatch (event processing and

x foreword scheduling). Even if you outgrow the authors' choice of tools later, you'll be able to keep all the code and just deploy it in a different way. Another great reason to read this book is how it introduces several important real-world use cases for serverless platforms, including web APIs, chat-bots, payment processing, and order management. By incrementally building an online store for a fictional pizzeria, the authors provide almost ready-made components that most people will need to launch modern business scenarios in the cloud. This way of building up knowledge lets the book explore deeper development process topics, such as organiz- ing automated testing and designing applications so they are easy to maintain. The last part of the book deals with migration strategies and answers some of the most common questions from people who already have applications partially running in some other cloud platform, and who want to get some quick wins and reduce time-to-market or cost of operation. I hope you'll have as much fun with this book as I did and that you’ll start finding better ways to deliver value quickly with your software in the cloud. Gojko Adzic, Partner, Neuri Consulting LLP

xi preface We’ve both been developers for over 10 years. We started with our first computers in the 90s, developing our first Pascal and BASIC functions and even went to program- ming competitions. But everything changed when the Web appeared. We immediately started building our first web applications and web pages playing with static HTML and CSS. When JavaScript and jQuery became the new standard, we switched almost immediately (even though, one was still playing with Flash and ActionScript). With the appearance of Node.js, it was natural to switch from languages we used such as Python and C#. Even though we're still sometimes writing a few functions in those languages, our switch to Node.js was permanent. Approximately three years ago, we turned our attention to serverless. Gojko Adzic introduced AWS Lambda to us with his initial work on Claudia.js as a deployment tool. We were amazed how fast and how easy was it to develop and deploy serverless applica- tions, and how easy was it to scale them, and we began working together with him on creating Claudia Bot Builder. From one day to another, our whole perspective on building and maintaining web applications completely changed with serverless. Backend services got replaced by serverless functions and instead of writing bash scripts, logging into our servers, and planning our capacity, we stopped caring about those issues and focused more on busi- ness logic and application value. We published our first serverless web applications into production, and developed hundreds of chat bots. Our production increased almost five-fold. It was incredible. The months spent learning how to configure and maintain application servers with bash, ssh, rsync, and so on were no longer important. Everything changed. From our starting point, the serverless ecosystem went a long way -- the serverless providers are

xii preface easier to use, and there are more and more serverless app components available each year (with Amazon re:Invent). It went so fast and a huge number of things happened -- we’ve made serverless our career. We started giving talks about serverless, holding workshops, and giving serverless consultations. We tried to gather our experience and knowledge, combined with mul- tiple other sources and bring it together in an easy-to-learn and easy-to-follow format.

xiii acknowledgments It was hard writing this book, as it was our first one. Some of the chapters have been rewritten over five times to get to the point that you, as a reader, would easily grasp it and learn the most. During the process, we had lots of support from our friends and families, but we’d like to thank a few people specifically that helped us tremendously along the way. First and foremost, we’d like to thank Gojko Adzic. He introduced the world of serverless to us several years ago. Especially for his reviews during the development of our book and comments such as “this page is worthless, delete it”, “don’t lie to your readers about the steps,” and so on. We loved them. Next we’d like to mention our editor at Manning, Toni Arritola. Thank you for work- ing with us, when we struggled in the first few chapters, for being patient when we got behind schedule and supporting us with everything we needed. You always pushed for quality and made the book better for everyone who reads it. Also we’d like to thank Michael Stephens and Bert Bates, who helped us better explain all the serverless details and focus on important topics. We’d also like to thank the folks at Manning who worked on the production and promotion of the book, it was a team effort. We’d also like to thank Valentin Crettaz, our technical proofreader, and Kostas Passadis, our technical development editor, for their careful reviews of the code. Thanks, too, to the Manning reviewers who took the time to read our manuscript at various stages and gave amazing feedback, including Arnaud Bailly, Barnaby Norman, Claudio Bernardo Rodríguez Rodríguez, Damian Esteban, Dane Balia, Deepak Bhaskaran, Jasba Simpson, Jeremy Lange, Kaj Ström, Kathleen R. Estrada, Kumar Unnikrishnan, Luca Mezzalira, Martin Dehnert, Rami Abdelwahed, Surjeet Manhas, Thomas Peklak, Umur Yilmaz, and Yvon Vieville.

xiv acknowledgments Our thanks also go to Amazon and AWS teams for creating such an amazing comput- ing service: AWS Lambda. You are changing the world. Lastly we’d like to thank Aunt Maria and all other imaginary characters from the book!

xv about this book Serverless Applications with Node.js is a book whose primary goal is to teach about and help you build serverless Node.js applications. It features a pragmatic approach, where you start with a story of your fictional Aunt Maria’s Pizzeria, whose problems you’re try- ing to solve by going serverless. The book begins by explaining serverless, tackling each problem Aunt Maria encountered by a separate serverless concept, which slowly start to form a clear picture how to build effective and clean serverless Node.js applications. Who should read this book Serverless Apps with Node.js is for JavaScript web developers seeking to learn how to build serverless applications and trying to understand how to properly organize, architect, and test them. Even though lots of Node.js content is already available online, as well as lots of tutorials on building basic serverless applications, this book introduces a step- by-step process for combining all those serverless topics and concepts to help you build big serverless applications and become a serverless Node.js developer. How this book is organized The book is organized in 3 parts with 15 chapters. Part 1 explains the basics of serverless and how to build a serverless app with a data- base, how to connect to third-party services, how to debug it, how to add authorization and authentication, and how to work with files ¡ Chapter 1 introduces you to serverless on Amazon Web Services platform and explains serverless with simple analogies. It also introduces you to Aunt Maria, her pizzeria, and the problem she is facing. Finally, you’ll learn what a common

xvi about this book serverless Node.js app looks like and find out what Claudia.js is and how it helps you to deploy Node.js apps to AWS Lambda. ¡ Chapter 2 shows you how to develop a simple Pizzeria API using AWS Lambda, API Gateway, and Claudia API Builder. It also teaches you how to deploy your API with a single command using Claudia. ¡ Chapter 3 teaches you how databases work in serverless architecture, and it teaches you how to connect your Pizzeria API with the DynamoDB, a serverless database offered by AWS. ¡ Chapter 4 teaches you how to connect Pizzeria API with third-party services, such as Some Like It Hot delivery API. It also shows you some common issues you might face when using promises with Claudia API Builder. ¡ Chapter 5 helps you learn how to find errors in your serverless applications, how to debug them, and what debugging tools you have at your disposal. ¡ Chapter 6 shows you how to implement authentication and authorization in your serverless application. You’ll learn the difference between authentication and authorization in a serverless environment, how to implement a web authoriza- tion mechanism using AWS Cognito, and how to identify your users using a social provider. ¡ Chapter 7 takes a dive into serverless file storage possibilities and examines how to create a separate file processing function that uses the storage and provides requested files to your other Lambda -- your serverless API. Part 2 covers how to create additional serverless applications that work with the same resources, how to create chatbots, voice assistants, SMS chatbots, how to add NLP, and how you should organize all those serverless applications together. ¡ Chapter 8 shows how to develop your first Facebook Messenger chatbot and how Claudia Bot Builder helps you do that in just several lines. ¡ Chapter 9 shows how to add simple NLP (natural language processing) to your chat bot, connect your chatbot to your DynamoDB database, and send delayed responses when a delivery is in progress (an asynchronous event). ¡ Chapter 10 shows how to develop your first Alexa skill and a Twilio SMS chatbot, and how with Claudia Bot Builder you can do that incredibly fast. Part 3 covers the more advanced topics on how to test, architect your serverless apps, and migrate your existing applications to serverless. It also gives recommendations, general patterns, and solutions to common issues and frequent questions. It also show- cases two medium scaled companies that went serverless. ¡ Chapter 11 teaches you about testing serverless applications, writing testable serverless functions, and running automated tests locally. Along with that, it explains Hexagonal Architecture and how to refactor your serverless applica- tions to make them easier to test and to remove potential risks.

xviiabout this book ¡ Chapter 12 covers processing payments with serverless applications, implement- ing payments to your serverless API, and understanding the PCI compliance in payment processing. ¡ Chapter 13 makes sure you know all about running Express.js applications in AWS Lambda and the serverless ecosystem, serving static content from an Express.js application, connecting to MongoDB from a serverless Express.js application, and understanding the limitations and risks of Express.js apps in a serverless ecosystem. ¡ Chapter 14 covers how to approach migrating to serverless, structuring your app according to serverless provider characteristics, organizing your applica- tion architecture so it's business-oriented and able to grow, and dealing with the architectural differences between serverless and traditional server-hosted applications. ¡ Chapter 15 teaches you how CodePen uses serverless for its preprocessors ensur- ing hundreds of millions of requests, and how MindMup serves 400,000 active users with a two-person team and serverless. About the code This book contains many examples of source code both in numbered listings and inline with normal text. In both cases, source code is formatted in a fixed-width font `like this` to separate it from ordinary text. Sometimes it is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code. In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts. Source code for the examples in this book is available for download from the publish- er’s website at https://manning.com/books/serverless-apps-with-node-and-claudiajs Book forum Your purchase of Serverless Applications with Node.js includes free access to a private web forum section run by Manning Publications where you make comments about the book, ask technical questions, and receive help from the authors and other users. To access the forum, point your web browser to https://forums.manning.com/forums/ serverless-apps-with-node-and-claudiajs. You can also learn more about Manning’s forums and the rules of conduct at https://forums.manning.com/forums/about. Manning’s commitment to our readers is to provide a venue where a meaningful dia- log between individual readers and between readers and the authors can take place. It is not a commitment to any specific amount of participation on the part of the authors,

xviii about this book whose contributions to the forum remain voluntary (and unpaid). We suggest you ask the authors challenging questions, lest their interest stray. Online resources If you need additional help, you can: ¡ Jump over to Claudia.js Gitter https://gitter.im/claudiajs/claudia, where the authors usually respond to technical questions regarding Claudia.js, Claudia API Builder, and Claudia Bot Builder ¡ See the claudiajs tag at Stack Overflow (http://stackoverflow.com/questions/ tagged/claudiajs), where you can post problems and questions you have about developing serverless applications with Node.js and Claudia.js. You can also help someone else stuck on an issue, too.