Hands-on TinyML Harness the power of Machine Learning on the edge devices (Rohan Banerjee) (z-library.sk, 1lib.sk, z-lib.sk)

📄 File Format: PDF

💾 File Size: 32.3 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

(This page has no text content)

📄 Page 2

 i

📄 Page 3

Rohan Banerjee www.bpbonline.com Hands-on TinyML Harness the power of Machine Learning on the edge devices

📄 Page 4

ii  Copyright © 2023 BPB Online All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor BPB Online or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. BPB Online has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, BPB Online cannot guarantee the accuracy of this information. First published: 2023 Published by BPB Online WeWork 119 Marylebone Road London NW1 5PU UK | UAE | INDIA | SINGAPORE ISBN 978-93-55518-446 www.bpbonline.com

📄 Page 5

 iii Dedicated to My Beloved Little Niece: Arjama

📄 Page 6

iv  About the Author Rohan Banerjee is a practicing data scientist having more than 12 years of relevant industry experience. He completed his M.Tech from IIT Kharagpur in 2011. His areas of interest include advanced data science, machine learning, embedded machine learning, digital signal and image processing. Rohan is currently associated with Baker Hughes Company. Before that, he was with TCS Research, Tata Consultancy Services where he published more than 40 technical papers in international conferences, journals, and also contributed in enhancing their intellectual property portfolio. Rohan is an avid reader of contemporary literature, a traveler, and a quiz enthusiast.

📄 Page 7

 v About the Reviewers • Tushar Chugh is a machine learning engineer at Google, focusing on search ranking. With a background in robotics from Carnegie Mellon University, he previously contributed to GM’s self-driving car perception systems. Passionate about applied machine learning, deep learning, and computer vision, Tushar has experience developing innovative technologies at Qualcomm and Microsoft in ML and tech domains. • Yogesh M Iggalore has over a decade of experience in product development and is actively involved in all the disciplines of product architecture, hardware design, firmware development, testing, and cloud integration. He is currently interested in product development in TinyML.

📄 Page 8

vi  Acknowledgements There are many people I want to express my gratitude to. First and foremost, I would like to sincerely thank my family members for their unwavering support and encouragement throughout my journey — I could have never completed this book without their support. I am grateful to various online resources, blogs, and materials that enriched my learning in order to write the book. I would also like to acknowledge the valuable feedbacks of my colleagues and co-workers during many years working in the tech industry. I am particularly grateful to Mr. Avik Ghose from TCS Research for providing me the opportunity to learn and work on TinyML. I gratefully acknowledge the effort of Mr. Tushar Chugh and Mr. Yogesh M Iggalore for their technical scrutiny and suggestions for improving the quality of this book. My sincere gratitude also goes to the team at BPB Publication for being supportive enough to provide me quite a long time to finish the book and also for all the valuable editorial reviews. Finally, I would like to thank all the readers who have taken an interest in the book. Your encouragement has been invaluable.

📄 Page 9

 vii Preface TinyML is an emerging trend in machine learning, that aims at deploying complex machine learning and neural network models on low-powered tiny edge devices and microcontrollers. Modern deep learning algorithms are computationally expensive and result in large model size. They are often hosted on dedicated servers having enormous computing resources. As users, we generate the data at our end and send them via the internet to process remotely. Owing to the limitations in network bandwidth, roughly 10% of all our data can be sent over the internet. Processing of the data on the edge can revolutionize the current paradigm. Thanks to TinyML, large machine learning models can be shrunk in order to effectively deploy on smaller devices having few hundred kilobytes of RAM and few megabytes of flash memory. Such devices can operate 24x7 with a minimum power consumption. Moreover, being entirely offline, the applications not only consumes zero network bandwidth, but also preserves user privacy. TinyML is going to be the next big thing in machine learning. Major tech giants are heavily investing in standardizing the hardware and software stack. In this book, we cover the basic concepts of TinyML through practical coding examples to enable the readers to learn the basic concepts of TinyML and develop their own applications. Rather than discussing every single mathematical concept behind the machine learning algorithms, the book primarily focuses on end-to-end application development through coding examples. The projects covered in this book are implemented in open-source software commonly used in industry and academics. This book is divided into 10 chapters. The details are listed as follows. Chapter 1: Introduction to TinyML and its Applications – covers the basic concept of EdgeML and TinyML, their potential applications, and challenges. It briefly covers the hardware and software platforms required to create TinyML. We also discuss the process flow of creating TinyML applications. Chapter 2: Crash Course on Python and TensorFlow Basics – covers the basics of Python which is now the de facto programming language in machine learning for both research and creating production ready software. We start with the basic concepts of Python along with various libraries such as NymPy, Matplotlib. The later part of the chapter covers the key aspects of TensorFlow. TensorFlow is a free

📄 Page 10

viii  and open-source software library for machine learning and neural networks. The chapter briefly covers some of the fundamental concepts of TensorFlow through coding examples. Chapter 3: Gearing with Deep Learning – briefly talks about neural networks. We begin with the concept of a simple Artificial Neural Network (ANN), various activation functions, and backpropagation to learn the weights. Later, we talk about Convolutional Neural Network (CNN), a popular deep neural network architecture used in modern image processing and computer vision applications. Chapter 4: Experiencing TensorFlow – guides us to develop our first neural network using TensorFlow and Keras. Keras is a set of deep learning APIs in Python, running on top of TensorFlow, providing high level of abstraction in developing large neural networks. We begin with implementing a simple ANN for classification of handwritten digit images. Later, we implement our first CNN architecture. Chapter 5: Model Optimization Using TensorFlow – talks about how a large TensorFlow model can be effectively compressed in order to deploy on smaller edge devices using TensorFlow Lite. We create a base CNN model using TensorFlow and convert it into the lighter TFLite model. The chapter also covers TensorFlow Model Optimization Toolkit, a software library for optimizing large neural networks for easy deployment and execution. We learn about different model optimization techniques, such as quantization, weight pruning and weight clustering through coding examples using the APIs provided by TensorFlow Model Optimization Toolkit. Finally, we summarize the impact of various optimization techniques on the base CNN in terms of model size and accuracy. Chapter 6: Deploying My First TinyML Application – guides us to create the first real TinyML application on Raspberry Pi, a commercially available low-powered edge device. We create a neural network for classification of offline images on Raspberry Pi. The chapter covers two important topics, MobileNet and transfer learning. MobileNet is an optimized neural network architecture specially designed for low-powered mobile edge devices. Transfer learning is another interesting concept in machine learning, where we can reuse a pre-trained model on a new

📄 Page 11

 ix problem. Transfer learning is particularly useful when we do not have sufficient training data to create a model from scratch. Chapter 7: Deep Dive into Application Deployment – guides us to implement a more practical TinyML application of real-time on-device person identification from live video stream recorded by a camera. The application is again deployed on Raspberry Pi using various open-source software. Chapter 8: TensorFlow Lite for Microcontrollers – covers the basics of TensorFlow Lite for Microcontrollers, a highly optimized software tool for porting TensorFlow models on low-powered microcontrollers. We implement a simple neural network that modulates the voltage output of a linear potentiometer and successfully deploy it on Arduino Nano 33 BLE Sense, the recommended microcontroller board for creating TinyML applications. Chapter 9: Keyword Spotting on Microcontrollers – guides us implementing an on-device speech recognition application. Keyword spotting is an important requirement in modern voice assistant services, such as Amazon’s Alexa or Apple’s Siri. In this chapter, we implement a simple keyword spotting application on Arduino. We first implement a basic keyword detection system using TensorFlow to understand the key concepts of audio processing. Later, we implement a real keyword spotting application using Edge Impulse, a free software platform for designing end-to-end TinyML application and deploy on an Arduino device with minimum code writing. Chapter 10: Conclusion and Further Reading – summarizes our learnings in the book and covers some recent trends in TinyML.

📄 Page 12

x  Code Bundle and Coloured Images Please follow the link to download the Code Bundle and the Coloured Images of the book: https://rebrand.ly/q35pmfm The code bundle for the book is also hosted on GitHub at https://github.com/ bpbpublications/Hands-on-TinyML. In case there’s an update to the code, it will be updated on the existing GitHub repository. We have code bundles from our rich catalogue of books and videos available at https://github.com/bpbpublications. Check them out! Errata We take immense pride in our work at BPB Publications and follow best practices to ensure the accuracy of our content to provide with an indulging reading experience to our subscribers. Our readers are our mirrors, and we use their inputs to reflect and improve upon human errors, if any, that may have occurred during the publishing processes involved. To let us maintain the quality and help us reach out to any readers who might be having difficulties due to any unforeseen errors, please write to us at : errata@bpbonline.com Your support, suggestions and feedbacks are highly appreciated by the BPB Publications’ Family. Did you know that BPB offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.bpbonline.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at : business@bpbonline.com for more details. At www.bpbonline.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on BPB books and eBooks.

📄 Page 13

 xi Piracy If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at business@bpbonline.com with a link to the material. If you are interested in becoming an author If there is a topic that you have expertise in, and you are interested in either writing or contributing to a book, please visit www.bpbonline.com. We have worked with thousands of developers and tech professionals, just like you, to help them share their insights with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea. Reviews Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions. We at BPB can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about BPB, please visit www.bpbonline.com. Join our book's Discord space Join the book's Discord Workspace for Latest updates, Offers, Tech happenings around the world, New Release and Sessions with the Authors: https://discord.bpbonline.com

📄 Page 14

xii  Table of Contents 1. Introduction to TinyML and its Applications ............................................... 1 Introduction .............................................................................................................. 1 Structure .................................................................................................................... 3 Objectives .................................................................................................................. 3 Brief overview of Machine Learning ..................................................................... 4 Supervised Machine Learning ..........................................................................5 Unsupervised Machine Learning ......................................................................6 Machine Learning and Deep Learning ................................................................. 6 Edge computing and TinyML ................................................................................. 7 Applications of TinyML .......................................................................................... 9 Hardware for deploying TinyML ......................................................................... 10 Software for TinyML .............................................................................................. 13 Process flow of creating TinyML applications .................................................... 13 Prerequisites—hardware and software ................................................................ 16 Conclusion .............................................................................................................. 17 Key facts................................................................................................................... 17 2. Crash Course on Python and TensorFlow Basics .........................................19 Introduction ............................................................................................................ 19 Structure .................................................................................................................. 21 Objectives ................................................................................................................ 21 Colab Notebook ..................................................................................................... 22 Python variables ..................................................................................................... 24 Python strings ..................................................................................................25 Lists ..................................................................................................................26 Tuple ................................................................................................................27 Dictionary........................................................................................................28 Conditional and logical operations...................................................................... 28 Loops in Python ..................................................................................................... 30 Functions in Python .............................................................................................. 30 Python libraries ...................................................................................................... 32

📄 Page 15

 xiii NumPy library .................................................................................................32 Random number generation ......................................................................37 Matplotlib library ............................................................................................39 Pandas library .................................................................................................41 Introduction to TensorFlow .................................................................................. 42 Tensors and datatypes .....................................................................................44 Differentiation in TensorFlow .........................................................................46 Graphs and functions in TensorFlow .............................................................47 End-to-end Machine Learning algorithm using TensorFlow ........................48 Conclusion .............................................................................................................. 54 Key facts................................................................................................................... 55 Further reading ....................................................................................................... 55 3. Gearing with Deep Learning ........................................................................57 Introduction ............................................................................................................ 57 Structure .................................................................................................................. 58 Objectives ................................................................................................................ 59 Theory of artificial neural networks .................................................................... 59 Binary cross entropy loss function ..................................................................62 Neural network activation functions ..............................................................63 Sigmoid activation function ......................................................................64 Tanh activation function ...........................................................................65 ReLU activation function ...........................................................................65 Softmax function ........................................................................................66 Learning the neural network weights—the backpropagation algorithm ......66 Introduction to Convolutional Neural Network ................................................ 69 Architecture of a CNN ....................................................................................71 Input layer ..................................................................................................71 Convolutional layer ....................................................................................71 Pooling layer ...............................................................................................78 Fully connected layer or dense layer ..........................................................79 Output layer ...............................................................................................79 Putting them all together ................................................................................79 Neural network hyperparameters ........................................................................ 80 Number of layers .............................................................................................81

📄 Page 16

xiv  Learning rate ...................................................................................................81 Dropout ...........................................................................................................81 Regularization .................................................................................................82 Choice of optimization algorithm ...................................................................82 Mini-batch size ................................................................................................82 Conclusion .............................................................................................................. 83 Key facts................................................................................................................... 83 Further reading ....................................................................................................... 84 4. Experiencing TensorFlow .............................................................................85 Introduction ............................................................................................................ 85 Structure .................................................................................................................. 86 Objectives ................................................................................................................ 86 Keras and TensorFlow ........................................................................................... 87 Classification of handwritten digits using a feedforward neural network ...... 90 Data processing ...............................................................................................92 Model implementation ....................................................................................94 Implementation of a Convolutional Neural Network ....................................... 97 Evaluation metrics in classification models ......................................................105 Conclusion ............................................................................................................107 Key facts.................................................................................................................108 5. Model Optimization Using TensorFlow .....................................................109 Introduction ..........................................................................................................109 Structure ................................................................................................................110 Objectives ..............................................................................................................111 Experiencing TensorFlow Lite ............................................................................111 TensorFlow Model Optimization Toolkit .........................................................120 Quantization .................................................................................................121 Weight pruning ..............................................................................................128 Weight clustering ...........................................................................................134 Collaborative optimization .................................................................................138 Conclusion ............................................................................................................143 Key facts.................................................................................................................144

📄 Page 17

 xv 6. Deploying My First TinyML Application ...................................................145 Introduction ..........................................................................................................145 Structure ................................................................................................................146 Objectives ..............................................................................................................147 The MobileNet architecture ................................................................................147 Depthwise separable convolution .................................................................148 Image classification using MobileNet ................................................................148 Brief introduction to transfer learning .........................................................152 Implementing MobileNet using transfer learning ........................................153 Creating an optimized model for a smaller target device ............................154 Evaluation of the model on the test set .........................................................157 Introduction to Raspberry Pi..............................................................................158 Getting started with the Pi ..................................................................................160 Installing the operating system ......................................................................161 Setting up the Pi ............................................................................................162 Remotely accessing the Pi ..............................................................................164 Deploying the model on Raspberry Pi to make inference ..............................165 Conclusion ............................................................................................................173 Key facts.................................................................................................................173 7. Deep Dive into Application Deployment ...................................................175 Introduction ..........................................................................................................175 Structure ................................................................................................................177 Objectives ..............................................................................................................177 System requirement .............................................................................................178 The face recognition pipeline..............................................................................179 Setting up the Raspberry Pi for face recognition .............................................180 The Raspberry Pi camera module.................................................................180 Installing the necessary libraries ...................................................................184 Implementation of the project ............................................................................185 Data collection for training ...........................................................................185 Model training ...............................................................................................189 Real-time face recognition ............................................................................192

📄 Page 18

xvi  Conclusion ............................................................................................................196 Key facts.................................................................................................................197 8. TensorFlow Lite for Microcontrollers ........................................................199 Introduction ..........................................................................................................199 Structure ................................................................................................................201 Objectives ..............................................................................................................202 Arduino Nano 33 BLE Sense ..............................................................................202 Setting up the Arduino Nano ........................................................................204 First TinyML project on the microcontroller—modulating the potentiometer 209 Required components ....................................................................................210 Connecting the circuit ...................................................................................211 Read potentiometer to control the brightness of the LED ............................212 Creating a TensorFlow model to modulate the potentiometer reading .......215 Inference on Arduino Nano using TensorFlow Lite for Microcontrollers ....222 Conclusion ............................................................................................................228 Key facts.................................................................................................................229 9. Keyword Spotting on Microcontrollers ......................................................231 Introduction ..........................................................................................................231 Structure ................................................................................................................233 Objectives ..............................................................................................................233 Working principles of a voice assistant .............................................................234 Implementation of a keyword spotting algorithm in Python ........................235 Audio spectrogram ........................................................................................241 Designing a Convolutional Neural Network model for keyword spotting ..247 Introduction to Edge Impulse ............................................................................251 Implementing keyword spotting in Edge Impulse ...........................................253 Model deployment ...............................................................................................264 Conclusion ............................................................................................................266 Key facts.................................................................................................................267

📄 Page 19

 xvii 10. Conclusion and Further Reading ...............................................................269 Introduction ..........................................................................................................269 Structure ................................................................................................................270 Objectives ..............................................................................................................270 Brief learning summary.......................................................................................271 TinyML best practices .........................................................................................273 AutoML and TinyML ..........................................................................................275 Edge ML on smartphones ...................................................................................277 Future of TinyML .................................................................................................277 Further reading .....................................................................................................278 Appendix ...........................................................................................................281 Index ..................................................................................................................283

📄 Page 20

(This page has no text content)