Learning PyTorch 2.0 Experiment Deep Learning from basics to complex models using every potential capability (Matthew Rosch) (Z-Library)

Preface   This book is a comprehensive guide to understanding and utilizing PyTorch 2.0 for deep learning applications. It starts with an introduction to PyTorch, its various advantages over other deep learning frameworks, and its blend with CUDA for GPU acceleration. We delve into the heart of PyTorch – tensors, learning their different types, properties, and operations. Through step-by-step examples, the reader learns to perform basic arithmetic operations on tensors, manipulate them, and understand errors related to tensor shapes.   A substantial portion of the book is dedicated to illustrating how to build simple PyTorch models. This includes uploading and preparing datasets, defining the architecture, training, and predicting. It provides hands- on exercises with a real-world dataset. The book then segues into exploring PyTorch's nn module and gives a detailed comparison of different types of networks like Feedforward, RNN, GRU, CNN, and their combination.   Further, the book delves into understanding the training process and PyTorch's optim module. It explores the overview of optimization algorithms like Gradient Descent, SGD, Mini-batch Gradient Descent, Momentum, Adagrad, and Adam. A separate chapter focuses on

advanced concepts in PyTorch 2.0, like model serialization, optimization, distributed training, and PyTorch Quantization API.   In the final chapters, the book discusses the differences between TensorFlow 2.0 and PyTorch 2.0 and the step- by-step process of migrating a TensorFlow model to PyTorch 2.0 using ONNX. It provides an overview of common issues encountered during this process and how to resolve them.   In this book you will learn:   A comprehensive introduction to PyTorch and CUDA for deep learning. Detailed understanding and operations on PyTorch tensors. Step-by-step guide to building simple PyTorch models. Insight into PyTorch's nn module and comparison of various network types. Overview of the training process and exploration of PyTorch's optim module. Understanding advanced concepts in PyTorch like model serialization and optimization. Knowledge on distributed training in PyTorch. Practical guide to using PyTorch's Quantization API.

Differences between TensorFlow 2.0 and PyTorch 2.0. Guidance on migrating TensorFlow models to PyTorch using ONNX.   The book equips the reader with comprehensive knowledge of PyTorch 2.0 and provides practical examples to apply this knowledge, making it an ideal read for aspiring network administrators.  

  Prologue   Dear reader, we are excited to take you on an exciting adventure into the fascinating world of PyTorch 2.0. This book has been painstakingly developed as a thorough reference, with the intention of illuminating the complex terrain of deep learning. Its goal is to change you from a curious enthusiast into a proficient practitioner by providing you with the theoretical understanding as well as the practical expertise necessary to properly use PyTorch. While we embark on this journey of discovery, we want you to participate completely by testing the validity of your previously held beliefs and broadening your perspective on the fascinating field of deep learning.   The first part of the book lays a strong basis in the fundamental principles that underpin PyTorch. Tensors, the fundamental building elements of PyTorch, are the subject of substantial discussion in the first few chapters. These first few chapters will walk you through the steps of comprehending tensors, as well as the process of building and manipulating them. This book will ensure that you have a solid foundation in the

fundamentals before going on to more advanced ideas by gradually increasing the difficulty of the topics as you progress through it.   As we continue to delve further into the workings of PyTorch, the book begins to introduce the development of basic as well as more complex models. We break down the many parts that make up these models with the help of some examples to illustrate our points. You will gain knowledge on how to create and train a variety of neural networks, including feed-forward and recurrent neural networks, as well as GRUs and CNNs, which are more advanced models. You will be able to apply what you learn in the book to real-world situations since it combines theoretical explanations with real-world examples, which helps to drive home the many concepts and methods that are covered.   The knowledge of the training process comes next in the book, after an initial look at neural networks. Unraveling optimization algorithms, sometimes known as the "engines that drive learning in neural networks," is one of the most important aspects of this endeavor. The book explains each algorithm in a manner that is both comprehensive and easy to understand, beginning with the fundamental Gradient Descent and progressing all

the way up to the more complex Adaptive Moment Estimation. You will now have the ability to construct and train robust models using PyTorch thanks to the knowledge and the activities you have completed.   In the latter chapters of this book, we delve into the more sophisticated principles of the PyTorch programming language. You will acquire knowledge regarding the serialization and optimization of models, as well as the use of distributed training and the Quantization API provided by PyTorch. In addition, we investigate the connection between TensorFlow 2.0 and PyTorch 2.0, comparing and contrasting the advantages and disadvantages of both programs. In doing so, the book arms you with the knowledge necessary to select the framework that caters to your requirements in the most optimal manner.   As we embark on this adventure, we want to urge you to approach each chapter with an open heart and a mind that is full of curiosity. This book offers something to offer everyone, from someone who is just beginning out in deep learning to someone who is an experienced professional looking to develop their skills. We have high hopes that by the time you finish reading this book, you will not only have a comprehensive understanding of

PyTorch 2.0 but also the self-assurance to put what you have learned into practice in the real world. Let's not waste any more time and go right into this thrilling adventure together, shall we? PyTorch 2.0 is where you want to be, so welcome!

  Learning PyTorch 2.0   Experiment deep learning from basics to complex models using every potential capability of Pythonic PyTorch        Matthew Rosch   Copyright © 2023 by GitforGits.   All rights reserved. This book is protected under copyright laws and no part of it may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without the prior written permission of the publisher. Any unauthorized reproduction, distribution, or transmission of this work may result in civil and criminal penalties and will be dealt with in the respective jurisdiction at

anywhere in India, in accordance with the applicable copyright laws.   Published by: GitforGits Publisher: Sonal Dhandre www.gitforgits.com support@gitforgits.com   Printed in India   First Printing: July 2023   ISBN: 978-8196288372   Cover Design by: Kitten Publishing   For permission to use material from this book, please contact GitforGits at support@gitforgits.com.  

Content        Preface        Chapter 1: Introduction To PyTorch 2.0 and CUDA 11.8        Essentials of Neural Networks        Anatomy of a Neural Network        Learning in a Neural Network        Neural Network and Deep Learning        Introduction to PyTorch        PyTorch Overview        How PyTorch supports Deep Learning        PyTorch 2.0 and CUDA 11.8  

     PyTorch 2.0 Key Features        Enhanced TorchScript        Model Deployment and Serving        Improved Mobile Support        Enhanced Quantization Support        Named Tensors        Extended Ecosystem        Expanded ONNX Support        CUDA for Accelerated Deep Learning        A Primer on CUDA        Why is CUDA Necessary for Deep Learning?        Blend of CUDA and PyTorch  

     Setting up PyTorch 2.0 and CUDA 11.8 on Linux        Update Your System        Install Necessary Dependencies        Verify CUDA-capable GPU        Download and Install CUDA 11.8        Setup Environment Variables        Verify the Installation        Install PyTorch 2.0        Verify PyTorch and CUDA Integration        Summary        Chapter 2: Getting Started with Tensors        Diving into Tensors

       Understanding Tensors        Different Tensors        Tensor Terminologies and Concepts        Scalar        Vector        Matrix        Tensor Operations        Broadcasting        Device        Sample Program to Create Tensors        Tensor Data Types        Standard Arithmetic Operations

       Addition        Subtraction        Multiplication        Division        Tensor Manipulation        Reshaping Tensors        Slicing Tensors        Joining Tensors        Matrix Multiplication        Manage Tensor Shape Errors        Matrix Multiplication  

     Element-wise Operations        Reshaping Tensors        Aggregation Operations        Sum        Mean        Max        Min        Summary        Chapter 3: Advanced Tensor Operations        Advanced Tensor Operations Overview        Reshaping        Viewing  

     Stacking        Changing Shape of Tensor        Viewing Tensor Dimensions        Stacking Sequence of Tensors        Squeezing a Tensor        Un-squeezing a Tensor        Permuting a Tensor        Indexing and Selecting Data from Tensors        Indexing a Tensor        Slicing a Tensor        Selecting Specific Indices        Switching Access Between GPU and CPU  

     Necessity of Access of Choice        Summary        Chapter 4: Building Neural Networks with PyTorch 2.0        PyTorch NN Module Overview        Understanding Feedforward Networks        Overview        Advantages of Feedforward Networks        Build Multi-layer Perceptron using PyTorch        Understanding Recurrent Neural Networks        Overview        Creating RNNs in PyTorch        Exploring Gated Recurrent Units (GRUs)

       Overview        Advantages of GRUs        Creating Gated Recurrent Units (GRUs)        Understanding Convolutional Neural Networks        Overview        Advantages of CNNs        Creating Convolutional Neural Networks (CNNs)        Combining RNN and CNN Together        Need of Designing Custom Layers        Design Custom Layers        Summary        Chapter 5. Training Neural Networks in PyTorch 2.0

       Neural Network Training - Overview        PyTorch optim Module        Using Optimizer        Optimization Algorithms        Gradient Descent        Using Gradient Descent        Stochastic Gradient Descent        Using Stochastic Gradient Descent        Mini-batch Gradient Descent        Using Mini-batch Gradient Descent        Momentum