Designing Microservices Using Django Structuring, Deploying and Managing the Microservices Architecture with Django (English… (Jain, Shayank) (Z-Library)

Designing Microservices Using Django Structuring, Deploying and Managing the Microservices Architecture with Django by Shayank Jain

ii  FIRST EDITION 2020 Copyright © BPB Publications, India ISBN: 978-93-89328-790 All Rights Reserved. No part of this publication may be reproduced or distributed in any form or by any means or stored in a database or retrieval system, without the prior written permission of the publisher with the exception to the program listings which may be entered, stored and executed in a computer system, but they can not be reproduced by the means of publication. LIMITS OF LIABILITY AND DISCLAIMER OF WARRANTY The information contained in this book is true to correct and the best of author’s & publisher’s knowledge. The author has made every effort to ensure the accuracy of these publications, but cannot be held responsible for any loss or damage arising from any information in this book. All trademarks referred to in the book are acknowledged as properties of their respective owners. Distributors: BPB PUBLICATIONS 20, Ansari Road, Darya Ganj New Delhi-110002 Ph: 23254990/23254991 MICRO MEDIA Shop No. 5, Mahendra Chambers, 150 DN Rd. Next to Capital Cinema, V.T. (C.S.T.) Station, MUMBAI-400 001 Ph: 22078296/22078297 DECCAN AGENCIES 4-3-329, Bank Street, Hyderabad-500195 Ph: 24756967/24756400 BPB BOOK CENTRE 376 Old Lajpat Rai Market, Delhi-110006 Ph: 23861747 Published by Manish Jain for BPB Publications, 20 Ansari Road, Darya Ganj, New Delhi-110002 and Printed by him at Repro India Ltd, Mumbai

 iii Dedicated to Kratika Jain My Wife, she is a gift of god.

iv  About the Author Shayank Jain is a software developer and data analyst. He is strongly passionate about coding and architectural design. He has more than 6 years of professional experience in developing scalable software solutions for various organizations. He has been programming since the age of 16 and has developed softwares for mobile, web, hardware gaming and standalone applications. After getting his hands dirty with programming, he found many new ways to debug and deploy the code successfully, with minimal time constraints. After reading and implementation, he found out that many critical concepts can be implemented easily in programming with correct and focused thinking. His research interests include information security, cryptography, analysis, design, and implementation of algorithms. He has extensively worked with python and implemented new ideas on various projects in his free time. He is also active in the computer science and education community. Through this book, he wants to share these methodologies and tricks with the beginners. Apart from work, Shayank spends his spare time helping, coaching, and mentoring young people in taking up careers in technology.

 v Acknowledgements There are a few people I want to thank for the continued and ongoing support that they have given me during the course of writing this book. First and foremost, I would like to thank my wife and my family for putting up with me while I was spending my weekends and evenings on writing - I could not have completed this book without their support. I would like to especially thank my friend Krishna Vishwakarma, who guided and motivated me throughout the writing of this book. This book wouldn’t have materialized if I did not have his support. Finally, I would like to thank BPB Publications, for giving me the opportunity to write my first book for them.

vi  Preface Microservice architectures solve one of the most important questions of software engineering evolution. Companies are wondering how to fight the obsolescence of their products. During the first few months, a start-up must convince the investors to believe in them. Time is the key to everything. Investors are interested in maximizing the ROI of a project. The speed of development that the startup needs, and is required by investors to maximize their ROI, tends to completely ignore the evolution of the software. Software developed with microservices divides the software into many services - small and cohesive, to be used and “assembled” together. It is very common today, and very convenient, to develop mobile and desktop applications through web technologies, using server services. Internet is full of “Get started” tutorials that help the developer to make the first steps with Python, the first steps to creating an app with Django, steps to secure API with JWT tokens, first steps with the Asynchronous tasks, with RabbitMQ, with databases, with AWS, with the Cloud. Django is a full-stack development framework written in Python. It already includes everything that is necessary for the development of a web application, from the user views to the information storage - model, persistence, relationships, controllers, forms, validations, rest API, and a very useful back office. Building production-ready microservices will also be covered in this book. We will learn how to create restful APIs. We will learn about Redis and Celery, as well as how to use the cache framework. You will also learn how to secure these services and deploy these microservices using Django. Finally, we will learn how to scale our services as well. The primary goal of this book is to provide the information and skills that are necessary to deploy the microservice architecture with Django. This book contains real-life examples that will show you how to install, configure, and manage the Django application, as well as how to integrate it into other third-party solutions for deployment. Over the 15 chapters in this book, you will learn the following: Chapter1: It covers the introduction to Python and explains everything that a programmer needs to get started with Python. It also covers all the basic concepts of python with syntax, sample code, and examples. Chapter 2: It discusses the major pillars of OOPs, and how it is used in python - like object, class, inheritance, abstraction, encapsulation, and polymorphism.

 vii Chapter 3: It contains the introduction of Django, its architecture, its working flow, the basic functionality of Django. Chapter 4: It covers how we can create an API with Django and what are the steps for deployment. Chapter 5: It covers database modeling with Django ORM. We use Postgresql, MySQL, SQLite as the database, and Django Shell for connecting the database. Chapter 6: This will cover all the Django web deployment processes and the basic idea of Uwsgi, Gunicorn, Supervisor, Nginx, and Apache. We use Django logger for production debugging. Chapter 7: It covers the Django API deployment on the production server, with different web servers and services like Apache, Nginx, Gunicorn, supervisor, and UWSGI. Chapter 8: It will give you a basic introduction to microservice and the comparison of the old and new methodology of architecture. It will also cover the benefits and losses of microservices. Chapter 9: It will give you a basic idea of microservices, designing patterns, parameters of microservice development, service designing, and independent deploy-ability. Chapter 10: An application that is monolithic, or a microservice, needs to authorize the users to use it, so you need a service that allows you to authenticate a user, recognize it and verify its permissions. Django already has its own way of determining the permissions and users. We will see how this can be useful in microservice architecture. We are deploying microservices with Django. Chapter 11: We are deploying microservices with Django. This chapter will cover how we can connect multiple databases into a single project, with flexible working. It will cover how the data flows between multiple services at the same time. Chapter 12: It covers the introduction of Json Web Token and how we can make our API more secure with the JWT token. It has examples and sample code. Chapter 13: It covers the introduction of the asynchronous task and the best fit technology for Django like RabbitMQ, Redis, and Celery.

viii  Chapter 14: AWS offers a complete and very interesting stack for microservice deployment. AWS Lambda allows the execution of the Django code, (in almost all the main languages) on request. AWS offers a gateway API, an Authentication API, database, storage, NoSQL, and asynchronous tasks, with business models related to their use and not related to the infrastructure. So, it covers all of them. Chapter 15: It covers the common questions and answers regarding microservices.

 ix Downloading the code bundle and colored images: Please follow the link to download the Code Bundle and the Colored Images of the book: https://rebrand.ly/06cc6 Errata We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors if any, occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at : errata@bpbonline.com Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family.

x  Table of Contents 1. Basics of Python ...........................................................................................................1 Structure .........................................................................................................................1 Objective.........................................................................................................................2 Introduction of Python programming language .....................................................2 Definition of variables, expressions, and statements ..............................................4 Statements ......................................................................................................................6 Operations on strings ...................................................................................................6 Functions ........................................................................................................................7 Python type conversion functions ..............................................................................7 User-defined functions .................................................................................................8 Execution flow of program..........................................................................................9 Scope of the variable ....................................................................................................9 “if” conditional statement with nested conditions ................................................11 Loops .........................................................................................................................14 The while loop .....................................................................................................14 The range function ..............................................................................................15 The for loop ..........................................................................................................16 Advanced data types of Python ...............................................................................17 List .........................................................................................................................17 List built-in functions ......................................................................................19 Tuple .....................................................................................................................23 Dictionaries ..........................................................................................................24 Dictionaries built-in functions ........................................................................26 Perform looping on the dictionary ...................................................................30 Set ..........................................................................................................................32 Built-in method of Set ......................................................................................33 Additional topic – PEP8 style guide for Python code ...........................................35 Naming conventions ..........................................................................................35 Python indentation .............................................................................................36 Maximum line length .........................................................................................37 Block and inline comments ...............................................................................37 Document strings ................................................................................................38

 xi Imports .................................................................................................................39 Connection closing .............................................................................................39 List comprehension ....................................................................................................39 Iterators ........................................................................................................................40 Generators ....................................................................................................................41 Conclusion ...................................................................................................................44 Questions .....................................................................................................................44 2. Major Pillars of OOPS with Python ......................................................................45 Structure .......................................................................................................................45 Objective.......................................................................................................................46 Introduction to object-oriented programming (OOP) language .........................46 Basic idea of object ......................................................................................................46 What is class? ..............................................................................................................47 Decorators in Python..................................................................................................50 Introduction of class method and static method in Python .................................55 Inheritance ...................................................................................................................57 Single inheritance ................................................................................................57 Multiple inheritance ...........................................................................................58 Multilevel inheritance ........................................................................................60 Use of super() function .......................................................................................62 Encapsulation ..............................................................................................................63 Polymorphism .............................................................................................................64 Method overloading ...........................................................................................64 Method overriding ..............................................................................................64 Abstraction ..................................................................................................................65 Abstract class .......................................................................................................65 Conclusion ...........................................................................................................66 Questions .....................................................................................................................67 3. Getting Started with Django ...................................................................................69 Structure .......................................................................................................................69 Objective.......................................................................................................................70 Basics of Django ..........................................................................................................70 Django architecture ....................................................................................................70 Django execution and basic commands ..................................................................73

xii  Django program execution flow ...............................................................................75 How Django handles requests ..................................................................................75 HTTP request .......................................................................................................77 Create the view file and configure user defined URL ...........................................78 URL understanding ............................................................................................80 Basics of Loggers and available logger types .........................................................80 Loggers .........................................................................................................................80 Handler ........................................................................................................................80 Filters ............................................................................................................................80 Formatters ....................................................................................................................81 Types of Loggers .........................................................................................................81 Logger Implementation .............................................................................................81 Define template in Django .........................................................................................83 Conclusion ...................................................................................................................95 Questions .....................................................................................................................95 4. API Development with Django ..............................................................................97 Structure .......................................................................................................................97 Objective.......................................................................................................................97 The basic idea of API ..................................................................................................98 Create an app in Django ............................................................................................98 Connect Django with database ...............................................................................101 Function-based and class-based views ..................................................................101 Function-based views .......................................................................................101 Class-based views .............................................................................................103 Postman tool description .........................................................................................104 Create function-based views ...................................................................................108 Create class-based views .........................................................................................112 Conclusion .................................................................................................................115 Questions ...................................................................................................................115 5. Database Modeling with Django .........................................................................117 Structure .....................................................................................................................117 Objective.....................................................................................................................118 The Django ORM ......................................................................................................118

 xiii Django shell ...............................................................................................................119 Django database modeling ......................................................................................119 Connect Django with various databases ...............................................................121 Django connectivity with PostgreSQL ...........................................................121 Django connectivity with SQLite ....................................................................123 Django connectivity with MySQL ..................................................................123 Create app with model ............................................................................................125 Access data through shell ........................................................................................127 Insert query ........................................................................................................130 Select query ........................................................................................................131 Update query .....................................................................................................134 Delete query .......................................................................................................134 Conclusion 135 Questions 135 6. First Django API Deployment on Web ................................................................137 Structure .....................................................................................................................137 Objective.....................................................................................................................137 The introduction of web technologies ...................................................................138 uWSGI ................................................................................................................138 NGINX ................................................................................................................139 Apache ................................................................................................................140 Gunicorn ............................................................................................................141 Supervisor ..........................................................................................................142 Use case of Supervisor ...................................................................................142 API deployment on web servers with Apache .....................................................142 Django loggers configuration and use case ..........................................................150 Conclusion .................................................................................................................154 Questions ...................................................................................................................154 7. Django Project Deployment on Various Web Servers ......................................155 Structure .....................................................................................................................155 Objective.....................................................................................................................155 New Django project creation and its overview ....................................................156

xiv  PostgreSQL database connectivity .................................................................157 Model file creation through Django ORM ....................................................158 Get and post request creation with ORM queries ........................................159 Loggers implementation in the project ..........................................................162 Deploy Django with NGINX, PostgreSQL, and uWSGI .....................................164 Deploy Django with NGINX, PostgreSQL, Gunicorn, and Supervisor ............168 Conclusion .................................................................................................................173 Questions ......................................................................................................................... 8. What are Microservices ..........................................................................................175 Structure .....................................................................................................................175 Objective.....................................................................................................................175 The introduction of the microservices ...................................................................176 Monolithic vs. Microservices ..................................................................................178 Monolithic ..........................................................................................................178 Microservices .....................................................................................................180 Some important characteristics of microservice ..................................................184 The microservices way and its benefits .................................................................187 In the microservices speed of changes ...........................................................187 In the microservices safety of changes ...........................................................187 The way of microservices ................................................................................188 Scalability in microservices .............................................................................188 Microservices pitfalls ...............................................................................................188 Conclusion .................................................................................................................189 Questions ...................................................................................................................190 9. Designing Microservice Systems .........................................................................191 Structure .....................................................................................................................191 Objective.....................................................................................................................191 Approach to microservices ......................................................................................192 Service .................................................................................................................192 Solution ..............................................................................................................193 Process ................................................................................................................193 Organization ......................................................................................................193 Culture ................................................................................................................193 When working on the all designing element together ...................................193

 xv Follow the standardized process or set a standard for process .................194 The designing process of microservices ................................................................194 Setting up the optimization goals ...................................................................194 Principles of development ...............................................................................195 Sketch the system design .................................................................................195 The designer of the microservices system .....................................................196 Important points for establishing a foundation ...................................................196 Goals and principles .........................................................................................196 Goals for the microservices way .....................................................................196 Reduce the system cost ..................................................................................197 Increase release speed .....................................................................................197 Improve the system resilience ........................................................................198 Enable the visibility .......................................................................................198 Operating the principles ..................................................................................198 Shared capabilities ............................................................................................199 Hardware services ..........................................................................................199 Management of the code, it’s testing and deployment ..................................199 Data stores .....................................................................................................199 Coordination between the services .................................................................200 Security and identity .....................................................................................200 Way to host ................................................................................................................200 Service designing parameters and boundaries ....................................................201 Domain-driven design .....................................................................................202 Context should be bounded ............................................................................203 Smaller is better .................................................................................................203 API design for microservices ..........................................................................203 Message-oriented ...........................................................................................204 Hypermedia-driven ........................................................................................204 Data and microservices ............................................................................................204 Asynchronous message-passing and microservices ....................................205 Independent deployability ......................................................................................206 Microservice architecture is a product of its time ........................................207 Security ...............................................................................................................207 Monitoring and alerting ...................................................................................207 Conclusion .................................................................................................................208 Questions ...................................................................................................................208

xvi  10. Service Authentication ............................................................................................209 Structure .....................................................................................................................209 Objective.....................................................................................................................209 Authentication ..........................................................................................................210 Authorization ............................................................................................................210 Authentication and authorization with Django ...................................................211 Table configuration with Django admin ...............................................................220 Authenticate the services .........................................................................................222 HTTP or HTTPs based .....................................................................................223 Session and cookies based ...............................................................................223 Token based .......................................................................................................224 Conclusion .................................................................................................................224 Questions ...................................................................................................................224 11. Microservices Deployment with Django ............................................................225 Structure .....................................................................................................................225 Objective.....................................................................................................................225 Flow figure for microservice architecture .............................................................226 Data flow and description of used API .................................................................227 Service name and its purpose .........................................................................227 User service ........................................................................................................228 Folder architecture and code ..........................................................................228 Database connection code ..............................................................................230 API code .........................................................................................................232 API request and response description ...........................................................239 userregistartion API ......................................................................................239 userlogin API .................................................................................................240 userinfo API ...................................................................................................241 Product service ..................................................................................................241 Folder architecture and code ..........................................................................242 Database connection code ..............................................................................243 API code .........................................................................................................244 Shipment service ...............................................................................................247 Folder architecture and code ..........................................................................247 Database connection code ..............................................................................248 API code .........................................................................................................250

 xvii API request and response description ............................................................253 Payment service ................................................................................................254 Folder architecture and code ..........................................................................255 Database connection code ..............................................................................256 API code .........................................................................................................258 API request and response description ............................................................263 Microservices deployment with Django ...............................................................265 Microservice architecture flow figure ....................................................................265 Conclusion .................................................................................................................266 Questions ...................................................................................................................266 12. JWT Auth Service .....................................................................................................267 Structure .....................................................................................................................267 Objective.....................................................................................................................267 Introduction of JSON Web Tokens (JWT) .............................................................268 Applicable scenarios for JWT implementation ....................................................268 Structure of JWT token ............................................................................................268 Header ................................................................................................................269 Payload ...............................................................................................................269 Signature ............................................................................................................270 JWT working flow ....................................................................................................271 JWT deployment with Django ................................................................................272 APIs request and response ..............................................................................275 Conclusion .................................................................................................................278 Questions ...................................................................................................................278 13. Asynchronous Tasks ...............................................................................................279 Structure .....................................................................................................................279 Objective.....................................................................................................................280 Introduction of asynchronous task ........................................................................280 Why we need asynchronous task? .................................................................280 Overview of Celery, RabbitMQ, and Redis ...........................................................281 Celery ..................................................................................................................281 Flow case ........................................................................................................281 Use case ..........................................................................................................282 RabbitMQ ...........................................................................................................282

xviii  Flow case ........................................................................................................283 Redis ...................................................................................................................284 Use case of Redis ............................................................................................285 Installation and basic configuration of Celery with Django...............................285 Deploy Django with Celery and RabbitMQ .................................................285 Deploy Django with Celery and Redis ..........................................................287 Tasks distribution and scheduling .........................................................................289 Task scheduling .................................................................................................289 Use case ..........................................................................................................289 Schedule the tasks in Ubuntu OS..................................................................290 Conclusion .................................................................................................................291 Questions ...................................................................................................................291 14. AWS Serverless ........................................................................................................293 Structure .....................................................................................................................293 Objective.....................................................................................................................293 Introduction of AWS Serverless ..............................................................................294 AWS API Gateway ............................................................................................294 AWS Lambda .....................................................................................................295 RDS .....................................................................................................................295 AWS DynamoDB ..............................................................................................295 Sample code with API Gateway and Lambda ......................................................296 Serverless microservice architecture on AWS ......................................................303 Django microservice architecture on AWS ...........................................................305 AWS EC2 service ...............................................................................................305 Django microservices architecture on AWS ..................................................305 Conclusion .................................................................................................................307 Questions ...................................................................................................................307 15. How to Adopt Microservices in Practice .............................................................309 Structure .....................................................................................................................309 Objective.....................................................................................................................310 Guidelines for creating the solution architecture ................................................310 How to introduce microservice in the organization? ..................................310 What are the best practices and processes for microservices? ...................311

 xix What should be the count of new features or bug fixes in the single release? .............................................................................................311 How can we track that our project is completely transformed into microservice? .............................................................................................311 Is it necessary to write all microservices code in the same programming language? .................................................................................312 What technologies are available for microservices? ....................................312 Conclusion .................................................................................................................312