Page
1
(This page has no text content)
Page
2
Applied Machine Learning Explainability Techniques Copyright © 2022 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Publishing Product Manager: Dinesh Chaudhary Senior Editor: Tazeen Shaikh Content Development Editor: Manikandan Kurup Technical Editor: Devanshi Ayare Copy Editor: Safis Editing Project Coordinator: Farheen Fathima Proofreader: Safis Editing Indexer: Sejal Dsilva Production Designer: Jyoti Chauhan Marketing Coordinator: Shifa Ansari and Abeer Riyaz Dawe First published: July 2022 Production reference: 2010722 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-80324-615-4 www.packt.com
Page
3
Contributors About the author Aditya Bhattacharya is an explainable AI researcher at KU Leuven with 7 years of experience in data science, machine learning, IoT, and software engineering. Prior to his current role, Aditya worked in various roles in organizations such as West Pharma, Microsoft, and Intel to democratize AI adoption for industrial solutions. As the AI lead at West Pharma, he contributed to forming the AI Center of Excellence, managing and leading a global team of 10+ members focused on building AI products. He also holds a master's degree from Georgia Tech in computer science with machine learning and a bachelor's degree from VIT University in ECE. Aditya is passionate about bringing AI closer to end users through his various initiatives for the AI community. I am immensely grateful to all who have been close to me and have supported me throughout the journey of writing this book, especially my wife Shreya, parents, sister, and all my aunts. A special thanks to my colleagues at the Augment research group of KU Leuven. I am grateful to Dr. Katrien Verbert for giving me the opportunity to pursue my journey as an XAI researcher. A big shoutout to all the reviewers for helping me throughout the writing process. Last, but not least, thanks to Abhijit Jana, who has always inspired me to step outside my comfort zone and pursue bigger challenges in life.
Page
4
About the reviewers Sumedh Vilas Datar is a machine learning engineer with 6 years of work experience in the field of deep learning, machine learning, and software engineering. He has a proven track record of single-handedly delivering end-to-end engineering solutions to real-world problems. He works at the intersection of engineering and products and has developed deep learning products from scratch that have been used by a lot of customers. Currently, Sumedh works in R&D on applied deep learning and has several granted patents and several more applied for. Sumedh studied biomedical engineering focused on computer vision and then went on to pursue a master's in computer science focused on AI. Abhijit Jana is a trusted technology leader and advisor with 15 years of experience in the IT industry, with expertise in development, architecting, engineering, consulting, service delivery, and leadership. He is currently associated with West Pharmaceutical Services as a director of software engineering and is responsible for building and leading the software engineering team. Previously, he worked at Microsoft and is a former Microsoft MVP and Code Project MVP, and has been a speaker at various technology conferences. He is the author of the book Kinect for Windows SDK Programming Guide and coauthored the book HoloLens Blueprint. He is also the founder of Daily .NET Tips, a well-known website for developers, architects, and consultants. Sk Nishan Ali works as a full stack data scientist at UnitedHealth Group. He has 6 years of experience across diverse areas of AI and machine learning, which include computer vision, classical machine learning, and natural language processing, and touches upon several business domains: healthcare, CRM, and sourcing. He has been instrumental in building high-performance end-to-end AI/ML products.
Page
5
(This page has no text content)
Page
6
Table of Contents Preface Section 1 – Conceptual Exposure 1 Foundational Concepts of Explainability Techniques Introduction to XAI 4 Understanding the key terms 4 Consequences of poor predictions 6 Summarizing the need for model explainability 8 Defining explanation methods and approaches 11 Dimensions of explainability 12 Addressing key questions of explainability 13 Understanding different types of explanation methods 15 Understanding the accuracy interpretability trade-off 16 Evaluating the quality of explainability methods 18 Criteria for good explainable ML systems 19 Auxiliary criteria of XAI for ML systems 22 Taxonomy of evaluation levels for explainable ML systems 23 Summary 26 References 26 2 Model Explainability Methods Technical requirements 28 Types of model explainability methods 28 Knowledge extraction methods 31 EDA 31 Result visualization methods 36 Using comparison analysis 36 Using Surrogate Explainer methods 37 Influence-based methods 39 Feature importance 40 Sensitivity analysis 41
Page
7
viii Table of Contents PDPs 43 LRP 44 Representation-based explanation 49 VAMs 50 Example-based methods 51 CFEs in structured data 52 CFEs in unstructured data 54 Summary 56 References 56 3 Data-Centric Approaches Technical requirements 60 Introduction to data-centric XAI 61 Analyzing data volume 61 Analyzing data consistency 62 Analyzing data purity 63 Thorough data analysis and profiling process 64 The need for data analysis and profiling processes 65 Data analysis as a precautionary step 65 Building robust data profiles 66 Monitoring and anticipating drifts 67 Detecting drifts 68 Selection of statistical measures 68 Checking adversarial robustness 71 Impact of adversarial attacks 72 Methods to increase adversarial robustness 74 Evaluating adversarial robustness 75 Measuring data forecastability 76 Estimating data forecastability 77 Summary 79 References 80 Section 2 – Practical Problem Solving 4 LIME for Model Interpretability Technical requirements 84 Intuitive understanding of LIME 84 Learning interpretable data representations 85 Maintaining a balance in the fidelity-interpretability trade-off 86 Searching for local explorations 86 What makes LIME a good model explainer? 88 SP-LIME 89 A practical example of using LIME for classification problems 90 Potential pitfalls 97 Summary 98 References 98
Page
8
Table of Contents ix 5 Practical Exposure to Using LIME in ML Technical requirements 100 Using LIME on tabular data 100 Setting up LIME 100 Discussion about the dataset 101 Discussions about the model 102 Application of LIME 102 Explaining image classifiers with LIME 104 Setting up the required Python modules 104 Using a pre-trained TensorFlow model as our black-box model 105 Application of LIME Image Explainers 105 Using LIME on text data 107 Installing the required Python modules 108 Discussions about the dataset used for training the model 108 Discussions about the text classification model 109 Applying LIME Text Explainers 109 LIME for production-level systems 111 Summary 112 References 112 6 Model Interpretability Using SHAP Technical requirements 114 An intuitive understanding of the SHAP and Shapley values 114 Introduction to SHAP and Shapley values 115 What are Shapley values? 115 Shapley values in ML 119 The SHAP framework 120 Model explainability approaches using SHAP 121 Visualizations in SHAP 121 Explainers in SHAP 130 Using SHAP to explain regression models 133 Setting up SHAP 134 Inspecting the dataset 134 Training the model 137 Application of SHAP 138 Advantages and limitations of SHAP 139 Advantages 139 Limitations 140 Summary 141 References 141
Page
9
x Table of Contents 7 Practical Exposure to Using SHAP in ML Technical requirements 144 Applying TreeExplainers to tree ensemble models 144 Installing the required Python modules 145 Discussion about the dataset 145 Training the model 148 Application of TreeExplainer in SHAP 149 Explaining deep learning models using DeepExplainer and GradientExplainer 152 GradientExplainer 153 Discussion on the dataset used for training the model 153 Using a pre-trained CNN model for this example 154 Application of GradientExplainer in SHAP 155 Exploring DeepExplainers 156 Application of DeepExplainer in SHAP 157 Model-agnostic explainability using KernelExplainer 158 Application of KernelExplainer in SHAP 159 Exploring LinearExplainer in SHAP 160 Application of LinearExplainer in SHAP 161 Explaining transformers using SHAP 162 Explaining transformer-based sentiment analysis models 162 Explaining a multi-class prediction transformer model using SHAP 164 Explaining zero-shot learning models using SHAP 166 Summary 169 References 169 8 Human-Friendly Explanations with TCAV Technical requirements 172 Understanding TCAV intuitively 172 What is TCAV? 173 Explaining with abstract concepts 174 Goals of TCAV 174 Approach of TCAV 175 Exploring the practical applications of TCAV 176 Getting started 177 About the data 178 Discussions about the deep learning model used 179 Model explainability using TCAV 180 Advantages and limitations 181 Advantages 182 Limitations 183
Page
10
Table of Contents xi Potential applications of concept-based explanations 184 Summary 186 References 187 9 Other Popular XAI Frameworks Technical requirements 190 DALEX 190 Setting up DALEX for model explainability 192 Discussions about the dataset 192 Training the model 193 Model explainability using DALEX 194 Model-level explanations 195 Prediction-level explanations 198 Evaluating model fairness 201 Interactive dashboards using ARENA 203 Explainerdashboard 205 Setting up Explainerdashboard 205 Model explainability with Explainerdashboard 206 InterpretML 210 Supported explanation methods 211 Setting up InterpretML 211 Discussions about the dataset 212 Training the model 213 Explainability with InterpretML 213 ALIBI 220 Setting up ALIBI 220 Discussion about the dataset 221 Training the model 221 Model explainability with ALIBI 222 DiCE 226 CFE methods supported in DiCE 226 Model explainability with DiCE 227 ELI5 230 Setting up ELI5 230 Model explainability using ELI5 230 H2O AutoML explainers 232 Explainability with H2O explainers 233 Quick comparison guide 235 Summary 237 References 237 Section 3 – Taking XAI to the Next Level 10 XAI Industry Best Practices Open challenges of XAI 243 Guidelines for designing explainable ML systems 246 Adopting a data-first approach for explainability 249 Emphasizing IML for explainability 250
Page
11
xii Table of Contents Emphasizing prescriptive insights for explainability 252 Summary 254 References 254 11 End User-Centered Artificial Intelligence User-centered XAI/ML systems 258 Different aspects of end user-centric XAI 259 Rapid XAI prototyping using EUCA 263 Efforts toward increasing user acceptance of AI/ML systems using XAI 265 Providing a delightful UX 266 Summary 266 References 267 Index Other Books You May Enjoy
Page
12
Preface Explainable AI (XAI) is an emerging field for bringing artificial intelligence (AI) closer to non-technical end-users. XAI promises to make machine learning (ML) models transparent, and trustworthy and promote AI adoption for industrial and research use-cases. This book is designed with a unique blend of industrial and academic research perspectives for gaining practical skills in XAI. ML/AI experts working with data science, ML, deep learning, and AI will be able to put their knowledge to work with this practical guide to XAI for bridging the gap between AI and the end-user. The book provides a hands-on approach for implementation and associated methodologies of XAI that will have you up-and-running, and productive in no time. Initially, you will get a conceptual understanding of XAI and why it's needed. Then, you will get the necessary practical experience of utilizing XAI in the AI/ML problem-solving process by making use of state-of-the-art methods and frameworks. Finally, you will get the necessary guidelines to take XAI to the next step and bridge the existing gaps between AI and end-users. By the end of this book, you will be able to implement XAI methods and approaches using Python to solve industrial problems, address the key pain points encountered, and follow the best practices in the AI/ML life cycle. Who this book is for This book is designed for scientists, researchers, engineers, architects, and managers who are actively engaged in the field of ML and related areas. In general, anyone who is interested in problem-solving using AI would benefit from this book. You are recommended to have a foundational knowledge of Python, ML, deep learning, and data science. This book is ideal for readers who are working in the following roles: • Data and AI scientists • AI/ML engineers • AI/ML product managers
Page
13
xii Preface • AI product owners • AI/ML researchers • User experience and HCI researchers In general, any ML enthusiast with a foundational knowledge of Python will be able to read, understand and apply knowledge gained from this book. What this book covers Chapter 1, Foundational Concepts of Explainability Techniques, gives the necessary exposure to Explainable AI and help you understand it's importance. This chapter covers various terminology and concepts related to explainability techniques, which is frequently used throughout this book. This chapter also covers the key criteria of human- friendly explainable ML systems and different approaches to evaluating the quality of the explainability techniques. Chapter 2, Model Explainability Methods, discusses the various model explainability methods used for explaining black-box models. Some of these are model agnostic, some are model specific. Some of these methods provide global interpretability while others provide local interpretability. This chapter will introduce you to a variety of techniques that can be used for explaining ML models and provides recommendation for the right choice of explainability method. Chapter 3, Data-Centric Approaches, introduces the concept of data-centric XAI. This chapter covers various techniques to explain the working of ML systems in terms of the properties of the data, data volume, data consistency, data purity and actionable insights generated from the underlying training dataset. Chapter 4, LIME for Model Interpretability, covers the application of one of the most popular XAI frameworks, called LIME. This chapter discusses about the intuition behind the working of the LIME algorithm and some important properties of the algorithm which makes the generated explanations human-friendly. Certain advantages and limitations of the LIME algorithm are also discussed in this chapter, along with a code tutorial for applying LIME for a classification problem. Chapter 5, Practical Exposure to Using LIME in ML is an extension of the previous chapter, but more focused towards the practical applications of the LIME Python framework on different types of datasets like images, texts along with structured tabular data. Practical code examples are also covered in this chapter for providing exposure to on-hand knowledge using Python LIME framework. This chapter also covers if LIME is a good fit for production-level ML systems.
Page
14
Preface xiii Chapter 6, Model Interpretability Using SHAP focuses on understanding the importance of the SHAP Python framework for model explainability. It covers the intuitive understanding of Shapley values and SHAP. This chapter also discusses how to use SHAP for model explainability through a variety of visualization and explainer methods. A code walkthrough for using SHAP to explain regression models is also covered in this chapter. Finally, we will discuss the key advantages and limitations of SHAP. Chapter 7, Practical Exposure to Using SHAP in ML provides the necessary practical exposure of using SHAP with tabular structured data as well unstructured data like images and texts. We have discussed about the different explainers available in SHAP for both model-specific and model agnostic explainability. We have applied SHAP for explaining linear models, tree ensemble models, convolution neural network models and even transformer models in this chapter. Necessary code tutorials are also covered in this chapter for providing exposure to hands-on knowledge using Python SHAP framework. Chapter 8, Human-Friendly Explanations with TCAV covers the concepts of TCAV, a framework developed by Google AI. This chapter provides both conceptual understanding of TCAV and practical exposure to applying the Python TCAV framework. The key advantages and limitations of TCAV are discussed along with interesting ideas about potential research problems that can be solved using concept-based explanations are discussed in the chapter. Chapter 9, Other Popular XAI Frameworks covers about seven popular XAI frameworks available in Python – DALEX, Explainerdashboard, InterpretML, ALIBI, DiCE, ELI5, and H2O AutoML explainers. We have discussed about the supported explanation methods for each of the framework, practical application, and the various pros and cons of each framework. This chapter also provides a quick comparison guide for helping you decide which framework you should go for considering your own use-case. Chapter 10, XAI Industry Best Practices focuses on the best practices for designing explainable AI systems for industrial problems. In this chapter, we have discussed about the open challenges of XAI and necessary design guidelines for explainable ML systems, considering the open challenges. We have also highlighted the importance of considering data-centric approaches of explainability, interactive machine learning and prescriptive insights for designing explainable AI/ML systems. Chapter 11, End User-Centered Artificial Intelligence introduces the ideology of end user centered artificial intelligence (ENDURANCE) for the design and development of explainable AI/ML Systems. We have discussed about the importance of using XAI to steer towards the main goals of the end user for building explainable AI/ML systems. Using some of principles and recommended best practices presented in the chapter, we can bridge the gap between AI and the end user to a great extent!
Page
15
xiv Preface To get the most out of this book To run the code tutorials provided in this book, you will need a Jupyter environment with Python 3.6+. This can be achieved in either of the following ways: • Install one on your machine locally via Anaconda Navigator or from scratch with pip. • Use a cloud-based environment such as Google Colaboratory, Kaggle notebooks, Azure notebooks, or Amazon SageMaker. You can take a look at the supplementary information provided at the code repository if you are new to Jupyter notebooks: https://github.com/PacktPublishing/ Applied-Machine-Learning-Explainability-Techniques/blob/main/ SupplementaryInfo/CodeSetup.md. You can also take a look at https://github.com/PacktPublishing/ Applied-Machine-Learning-Explainability-Techniques/blob/main/ SupplementaryInfo/PythonPackageInfo.md and https://github. com/PacktPublishing/Applied-Machine-Learning-Explainability- Techniques/blob/main/SupplementaryInfo/DatasetInfo.md for getting the supplementary information about the Python packages and datasets used in the tutorial notebooks. For instructions on installing the Python packages used throughout the book, please refer the specific notebook provided in the code repository. For any additional help needed, please refer the original project repository of the specific package. You can use PyPi (https://pypi.org/) and search for the specific package and navigate to the code repository of the project. It is expected that installation or execution instructions of these packages can change from time to time, given how often packages change. We also tested the code with specific versions detailed in the Python package information README file under the supplementary information provided at the code repository. So, if anything doesn't work as expected with the later versions, please install the specific version mentioned in the README instead. If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book's GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.
Page
16
Preface xv For beginners without any exposure to ML or data science, it is recommended to read the book sequentially as many important concepts are explained in sufficient detail in the earlier chapters. Seasoned ML or data science experts who are relatively new to the field of XAI can skim through the first three chapters to get clear conceptual understanding of various terminology used. For chapters four to nine, any order should be fine for seasoned experts. For all level of practitioners, it is recommended that you read chapter 10 and 11 only after covering all the nine chapters. Regarding the code provided, it is recommended that you either read each chapter and then run the corresponding code, or you can run the code simultaneously while reading the specific chapters. Sufficient theory is also added in the Jupyter notebooks to help you understand the overall flow of the notebook. When you are reading the book, it is recommended that you take notes of the important terminologies covered and try to think of ways in which you could apply the concept or the framework learned. After reading the book and going through all the Jupyter notebooks, hopefully, you will be inspired to apply the newly gained knowledge into action! Download the example code files You can download the example code files for this book from GitHub at https:// github.com/PacktPublishing/Applied-Machine-Learning- Explainability-Techniques. If there's an update to the code, it will be updated in the GitHub repository. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out! Download the color images We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/DF7lG.
Page
17
xvi Preface Conventions used There are a number of text conventions used throughout this book. Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "For this example, we will use the RegressionExplainer and ExplainerDashboard submodules." A block of code is set as follows: pdp = PartialDependence( predict_fn=model.predict_proba, data=x_train.astype('float').values, feature_names=list(x_train.columns), feature_types=feature_types) pdp_global=pdp.explain_global(name='Partial Dependence') When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold: explainer = shap.Explainer(model, x_test) shap_values = explainer(x_test) shap.plots.waterfall(shap_values[0], max_display = 12, show=False) Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "Due to these known drawbacks, the search for a robust Explainable AI (XAI) framework is still on." Tips or important notes Appear like this. Get in touch Feedback from our readers is always welcome. General feedback: If you have questions about any aspect of this book, email us at customercare@packtpub.com and mention the book title in the subject of your message.
Page
18
Preface xvii Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form. Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at copyright@packt.com with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com. Share Your Thoughts Once you've read Applied Machine Learning Explainability Techniques, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback. Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.
Page
19
(This page has no text content)
Page
20
Section 1 – Conceptual Exposure This section will give you the necessary conceptual exposure to explainability techniques for machine learning (ML) models with practical examples. You will learn about the foundational concepts, different dimensions of explainability, various model explainability methods, and even data-centric approaches to explainability. Knowledge of the foundational concepts will help you understand the guidelines for designing robust explainable ML systems like those covered in this book. This section comprises the following chapters: • Chapter 1, Foundational Concepts of Explainability Techniques • Chapter 2, Model Explainability Methods • Chapter 3, Data-Centric Approaches