Behavioral Data Analysis with R and Python Customer-Driven Data for Real Business Results (Florent Buisson) (Z-Library)

📄 File Format: PDF

💾 File Size: 12.6 MB

📖 Read Online ⬇️ Download

Registered users can read the full content for free

Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.

📄 Page 1

Florent Buisson Behavioral Data Analysis with R & Python Customer-Driven Data for Real Business Results

📄 Page 2

(This page has no text content)

📄 Page 3

Praise for Behavioral Data Analysis with R and Python “Whereas some data science books seek to teach their readers a list of new techniques, Florent’s goal is different and more profound. He seeks to teach us data-centric wisdom: how to build a detailed and nuanced understanding of the data footprints of human behavior.” —Steve Wendel, Head of Behavioral Science, Morningstar "Behavioral Data Analysis will help you make sense of data even when you can’t run controlled experiments.” —Colin McFarland, Dir. of Experimentation Platform, Netflix “We are awash with data and this is a long-needed resource to guide practitioners in using that data for building valid causal models to predict and explain real-world behaviors.” —David Lewis, President, BEworks Research Institute at BEworks “For anyone looking to apply behavioral science to guide business decision making, this book provides a valuable in-depth introduction on how to leverage causal diagrams in experimentation and behavioral analysis.” —Matt Wright, Dir. of Behavioral Science, WiderFunnel “Part of what makes behavioral science work is seamlessly blending quantitative and qualitative insights to support our understanding of why people do what they do. This book can help anyone with some basic data skills to meaningfully participate in that behavioral science process.” —Matt Wallaert, Head of Behavioral Science at frog, Author of Start at the End: How to Build Products That Create Change

📄 Page 4

(This page has no text content)

📄 Page 5

Florent Buisson Behavioral Data Analysis with R and Python Customer-Driven Data for Real Business Results Boston Farnham Sebastopol TokyoBeijing

📄 Page 6

978-1-492-06137-3 [LSI] Behavioral Data Analysis with R and Python by Florent Buisson Copyright © 2021 Florent Buisson. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Michelle Smith Development Editor: Gary O’Brien Production Editor: Deborah Baker Copyeditor: Charles Roumeliotis Proofreader: Kim Sandoval Indexer: Sue Klefstad Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea June 2021: First Edition Revision History for the First Edition 2021-06-15: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492061373 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Behavioral Data Analysis with R and Python, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

📄 Page 7

Table of Contents Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Part I. Understanding Behaviors 1. The Causal-Behavioral Framework for Data Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Why We Need Causal Analytics to Explain Human Behavior 4 The Different Types of Analytics 4 Human Beings Are Complicated 5 Confound It! The Hidden Dangers of Letting Regression Sort It Out 8 Data 9 Why Correlation Is Not Causation: A Confounder in Action 9 Too Many Variables Can Spoil the Broth 11 Conclusion 17 2. Understanding Behavioral Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 A Basic Model of Human Behavior 20 Personal Characteristics 21 Cognition and Emotions 23 Intentions 24 Actions 25 Business Behaviors 26 How to Connect Behaviors and Data 28 Develop a Behavioral Integrity Mindset 28 Distrust and Verify 29 Identify the Category 30 Refine Behavioral Variables 32 Understand the Context 33 v

📄 Page 8

Conclusion 35 Part II. Causal Diagrams and Deconfounding 3. Introduction to Causal Diagrams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Causal Diagrams and the Causal-Behavioral Framework 40 Causal Diagrams Represent Behaviors 41 Causal Diagrams Represent Data 42 Fundamental Structures of Causal Diagrams 46 Chains 46 Forks 50 Colliders 52 Common Transformations of Causal Diagrams 53 Slicing/Disaggregating Variables 54 Aggregating Variables 55 What About Cycles? 57 Paths 60 Conclusion 61 4. Building Causal Diagrams from Scratch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Business Problem and Data Setup 64 Data and Packages 64 Understanding the Relationship of Interest 65 Identify Candidate Variables to Include 67 Actions 69 Intentions 70 Cognition and Emotions 71 Personal Characteristics 72 Business Behaviors 75 Time Trends 75 Validate Observable Variables to Include Based on Data 77 Relationships Between Numeric Variables 78 Relationships Between Categorical Variables 81 Relationships Between Numeric and Categorical Variables 84 Expand Causal Diagram Iteratively 86 Identify Proxies for Unobserved Variables 86 Identify Further Causes 87 Iterate 88 Simplify Causal Diagram 88 Conclusion 90 vi | Table of Contents

📄 Page 9

5. Using Causal Diagrams to Deconfound Data Analyses. . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Business Problem: Ice Cream and Bottled Water Sales 92 The Disjunctive Cause Criterion 94 Definition 94 First Block 95 Second Block 97 The Backdoor Criterion 97 Definitions 98 First Block 100 Second Block 101 Conclusion 103 Part III. Robust Data Analysis 6. Handling Missing Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Data and Packages 109 Visualizing Missing Data 110 Amount of Missing Data 113 Correlation of Missingness 115 Diagnosing Missing Data 121 Causes of Missingness: Rubin’s Classification 124 Diagnosing MCAR Variables 126 Diagnosing MAR Variables 128 Diagnosing MNAR Variables 130 Missingness as a Spectrum 132 Handling Missing Data 136 Introduction to Multiple Imputation (MI) 137 Default Imputation Method: Predictive Mean Matching 140 From PMM to Normal Imputation (R Only) 141 Adding Auxiliary Variables 143 Scaling Up the Number of Imputed Data Sets 145 Conclusion 146 7. Measuring Uncertainty with the Bootstrap. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 Intro to the Bootstrap: “Polling” Oneself Up 148 Packages 148 The Business Problem: Small Data with an Outlier 148 Bootstrap Confidence Interval for the Sample Mean 150 Bootstrap Confidence Intervals for Ad Hoc Statistics 155 The Bootstrap for Regression Analysis 157 When to Use the Bootstrap 160 Table of Contents | vii

📄 Page 10

Conditions for the Traditional Central Estimate to Be Sufficient 161 Conditions for the Traditional CI to Be Sufficient 161 Determining the Number of Bootstrap Samples 164 Optimizing the Bootstrap in R and Python 165 R: The boot Package 165 Python Optimization 168 Conclusion 169 Part IV. Designing and Analyzing Experiments 8. Experimental Design: The Basics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 Planning the Experiment: Theory of Change 174 Business Goal and Target Metric 175 Intervention 177 Behavioral Logic 179 Data and Packages 181 Determining Random Assignment and Sample Size/Power 182 Random Assignment 182 Sample Size and Power Analysis 185 Analyzing and Interpreting Experimental Results 199 Conclusion 202 9. Stratified Randomization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203 Planning the Experiment 205 Business Goal and Target Metric 205 Definition of the Intervention 207 Behavioral Logic 208 Data and Packages 208 Determining Random Assignment and Sample Size/Power 209 Random Assignment 209 Power Analysis with Bootstrap Simulations 217 Analyzing and Interpreting Experimental Results 224 Intention-to-Treat Estimate for Encouragement Intervention 225 Complier Average Causal Estimate for Mandatory Intervention 226 Conclusion 232 10. Cluster Randomization and Hierarchical Modeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Planning the Experiment 234 Business Goal and Target Metric 234 Definition of the Intervention 234 Behavioral Logic 236 viii | Table of Contents

📄 Page 11

Data and Packages 236 Introduction to Hierarchical Modeling 237 R Code 238 Python Code 240 Determining Random Assignment and Sample Size/Power 242 Random Assignment 242 Power Analysis 244 Analyzing the Experiment 252 Conclusion 252 Part V. Advanced Tools in Behavioral Data Analysis 11. Introduction to Moderation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257 Data and Packages 258 Behavioral Varieties of Moderation 258 Segmentation 259 Interactions 265 Nonlinearities 266 How to Apply Moderation 269 When to Look for Moderation? 270 Multiple Moderators 281 Validating Moderation with Bootstrap 287 Interpreting Individual Coefficients 289 Conclusion 295 12. Mediation and Instrumental Variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297 Mediation 298 Understanding Causal Mechanisms 298 Causal Biases 299 Identifying Mediation 301 Measuring Mediation 302 Instrumental Variables 307 Data 307 Packages 308 Understanding and Applying IVs 308 Measurement 311 Applying IVs: Frequently Asked Questions 314 Conclusion 315 Table of Contents | ix

📄 Page 12

Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 x | Table of Contents

📄 Page 13

Preface Statistics is a subject of amazingly many uses and surprisingly few effective practitioners. —Bradley Efron and R. J. Tibshirani, An Introduction to the Bootstrap (1993) Welcome to Behavioral Data Analysis with R and Python! That we live in the age of data has become a platitude. Engineers now routinely use data from sensors on machines and turbines to predict when these will fail and do preventive maintenance. Similarly, marketers use troves of data, from your demographic information to your past purchases, to determine which ad to serve you and when. As the phrase goes, “Data is the new oil,” and algorithms are the new combustion engine powering our economy forward. Most books on analytics, machine learning, and data science implicitly presume that the problems that engineers and marketers are trying to solve can be handled with the same approaches and tools. Sure, the variables have different names and there is some domain-specific knowledge to acquire, but k-means clustering is k-means clus‐ tering, whether you’re clustering data about turbines or posts on social media. By adopting machine learning tools wholesale this way, businesses have often been able to accurately predict behaviors, but at the expense of a deeper and richer understand‐ ing of what’s actually going on. This has fed into the criticism of data science models as “black boxes.” Instead of aiming for accurate but opaque predictions, this book strives to answer the question, “What drives behavior?” If we decide to send an email to prospective cus‐ tomers, will they buy a subscription to our service as a result of the email? And which groups of customers should receive the email? Do older customers tend to purchase different products because they’re older? What is the impact of customer experience on loyalty and retention? By shifting our perspective from predicting behaviors to explaining them and measuring their causes, we’ll be able to break the curse of “cor‐ relation is not causation,” which has prevented generations of analysts from being confident in the results of their models. xi

📄 Page 14

This shift won’t come from the introduction of new analytical tools: we’ll use only two data analysis tools: good old linear regression and its logistic sibling. These two are intrinsically more readable than other types of models. Certainly, this often comes at the cost of a lower predictive accuracy (i.e., they make more and larger errors in pre‐ diction), but that doesn’t matter for our purpose here of measuring relationships between variables. Instead we’ll spend a lot of time learning to make sense of data. In my role as a data science interviewer, I have seen many candidates who can use sophisticated machine learning algorithms but haven’t developed a strong sense for data: they have little intuition for what’s going on in their data apart from what their algorithms tell them. I believe that you can develop that intuition, and along the way increase the value and outcomes of your analytics projects—often dramatically—by adopting the following: • A behavioral science mindset, which sees data not as an end in itself but as a lens into the psychology and behaviors of human beings • A causal analytics toolkit, which allows us to confidently say that one thing causes another and to determine how strong that relationship is While each of these can provide great benefits on its own, I believe that they are natu‐ ral complements that are best used together. Given that “a behavioral science mindset using a causal analytics toolkit” is a bit of a mouthful, I’ll call it instead a causal- behavioral approach or framework. This framework has an added benefit: it applies equally to experimental and historical data while leveraging the differences between them. This contrasts with traditional analytics, which handles them with completely different tools (e.g., ANOVA and T-test for experimental data) and data science, which doesn’t treat experimental data differently from historical data. Who This Book Is For If you’re analyzing data in a business with R or Python, this book is for you. I use the word “business” loosely to mean any for-profit, nonprofit, or governmental organiza‐ tion where correct insights and actionable conclusions driving action are what matters. In terms of math and stats background, it doesn’t matter whether you are a business analyst building monthly forecasts, a UX researcher looking at click-through behav‐ iors, or a data scientist building machine learning models. This book has one funda‐ mental prerequisite: you need to be at least somewhat familiar with linear and logistic regression. If you understand regression, you can follow the argument of this book and reap great benefits from it. On the other side of the spectrum, I believe even expert data scientists with PhDs in statistics or computer science will find the xii | Preface

📄 Page 15

material new and useful, provided they are not already specialists in behavioral or causal analytics. In terms of programming background, you need to be able to read and write code in R or Python, ideally both. I will not show you how to define a function or how to manipulate data structures such as data frames or pandas. There are already excellent books doing a better job of it than I would (e.g., Python for Data Analysis by Wes McKinney (O’Reilly) and R for Data Science by Garrett Grolemund and Hadley Wick‐ ham (O’Reilly)). If you’ve read any of these books, taken an introductory class, or used at least one of the two languages at work, then you’ll be equipped for the mate‐ rial here. Similarly, I will usually not present and discuss the code used to create the numerous figures in the book, although it will be in the book’s GitHub. Who This Book Is Not For If you’re in academia or a field that requires you to follow academic norms (e.g., pharmaceutical trials), this book might still be of interest to you—but the recipes I’m describing might get you in trouble with your advisor/editor/manager. This book is not an overview of conventional behavioral data analysis methods, such as T-test or ANOVA. I have yet to encounter a situation where regression was less effective than these methods for providing an answer to a business question, which is why I’m deliberately restraining this book to linear and logistic regression. If you want to learn other methods, you’ll have to look elsewhere (e.g., Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (O’Reilly) by Aurélien Géron for machine learning algorithms). Understanding and changing behaviors in applied settings requires both data analysis and qualitative skills. This book focuses squarely on the former, primarily for reasons of space. In addition, there are already excellent books that cover the latter, such as Nudge: Improving Decisions About Health, Wealth, and Happiness (Penguin) by Richard Thaler and Cass Sunstein and Designing for Behavior Change: Applying Psy‐ chology and Behavioral Economics (O’Reilly) by Stephen Wendel. Nonetheless, I’ll provide an introduction to behavioral science concepts so that you can apply the tools from this book even if you’re new to the field. Finally, if you’re completely new to data analysis in R or Python, this is not the book for you. I recommend starting with some of the excellent introductions out there, such as the ones mentioned in this section. Preface | xiii

📄 Page 16

R and Python Code Why R and Python? Why not whichever of the two is superior? The “R versus Python” debate is still alive and kicking on the Internet. It is also, in my humble opin‐ ion, mostly irrelevant. The reality is that you’ll have to use whatever language is used in your organization, period. I once worked in a healthcare company where, for his‐ torical and regulatory reasons, SAS was the dominant language. I regularly used R and Python for my own analyses, but since I couldn’t avoid dealing with the legacy SAS code, I taught myself as much SAS as I needed during my first month there. Unless you spend your entire career in a company that doesn’t use R or Python, you’ll most likely end up picking up some basics in both anyway, so you might as well embrace bilingualism. I have yet to encounter anyone who stated that “learning to read code in [the other language] was a waste of my time.” Assuming that you have the good luck of being in an organization that uses both, which language should you work with? I think it really depends on your context and the tasks you have to do. For example, I personally prefer doing exploratory data analysis (EDA) in R, but I find that Python is much easier to use for webscraping. I advise choosing based on the specifics of your job and relying on up-to-date informa‐ tion: both languages are constantly improving, and what was true for a previous ver‐ sion of R or Python may not be true for the current version. For example, Python is becoming a much friendlier environment for EDA than it once was. Your energy is better spent on learning both than on scouring forums to pick the best of the two. Code Environments At the beginning of each chapter, I’ll call out the R and Python packages that need to be loaded specifically for that chapter. In addition, I’ll also be using a few standard packages across the entire book; to avoid repetition, these are called out only here (they are already included in all the scripts on GitHub). You should always start your code with them as well as with a few parameter settings: ## R library(tidyverse) library(boot) #Required for Bootstrap simulations library(rstudioapi) #To load data from local folder library(ggpubr) #To generate multi-plots # Setting the random seed will ensure reproducibility of random numbers set.seed(1234) # I personally find the default scientific number notation (i.e. with # exponents) less readable in results, so I cancel it options(scipen=10) ## Python import pandas as pd import numpy as np xiv | Preface

📄 Page 17

import statsmodels.formula.api as smf from statsmodels.formula.api import ols import matplotlib.pyplot as plt # For graphics import seaborn as sns # For graphics Code Conventions I use R in RStudio. R 4.0 was launched while I was writing this book and I have adop‐ ted it to keep the book as current as possible. R code is written in a code font with a comment indicating the language used, like this: ## R > x <- 3 > x [1] 3 I use Python in Anaconda’s Spyder. The “Python 2.0 vs. 3.0” discussion is hopefully behind us (at least for new code; legacy code can be a different story), and I’ll be using Python 3.7. The convention for Python code is somewhat similar to the one for R: ## Python In [1]: x = 3 In [2]: x Out[2]: 3 We’ll often look at the output of regressions. It can be quite verbose, with a lot of diagnostics that are not relevant to the arguments of this book. You shouldn’t disre‐ gard them in real life, but that’s a matter better covered in other books. Therefore, I’ll abbreviate the output like this: ## R > model1 <- lm(icecream_sales ~ temps, data=stand_dat) > summary(model1) ... Coefficients: Estimate Std. Error t value Pr(>|t|) (Intercept) -4519.055 454.566 -9.941 <2e-16 *** temps 1145.320 7.826 146.348 <2e-16 *** ... ## Python model1 = ols("icecream_sales ~ temps", data=stand_data_df) print(model1.fit().summary()) ... coef std err t P>|t| [0.025 0.975] ------------------------------------------------------------------------------ Intercept -4519.0554 454.566 -9.941 0.000 -5410.439 -3627.672 Preface | xv

📄 Page 18

temps 1145.3197 7.826 146.348 0.000 1129.973 1160.666 ... Functional-Style Programming 101 One of the steps of going from beginner to intermediate level as a programmer is to stop writing scripts in which your code is just a long succession of instructions and to structure your code into functions instead. In this book, we’ll write and reuse func‐ tions across chapters, such as the following ones to build Bootstrap confidence intervals: ## R boot_CI_fun <- function(dat, metric_fun, B=20, conf.level=0.9){ boot_vec <- sapply(1:B, function(x){ cat("bootstrap iteration ", x, "\n") metric_fun(slice_sample(dat, n = nrow(dat), replace = TRUE))}) boot_vec <- sort(boot_vec, decreasing = FALSE) offset = round(B * (1 - conf.level) / 2) CI <- c(boot_vec[offset], boot_vec[B+1-offset]) return(CI) } ## Python def boot_CI_fun(dat_df, metric_fun, B = 20, conf_level = 9/10): coeff_boot = [] # Calculate coeff of interest for each simulation for b in range(B): print("beginning iteration number " + str(b) + "\n") boot_df = dat_df.groupby("rep_ID").sample(n=1200, replace=True) coeff = metric_fun(boot_df) coeff_boot.append(coeff) # Extract confidence interval coeff_boot.sort() offset = round(B * (1 - conf_level) / 2) CI = [coeff_boot[offset], coeff_boot[-(offset+1)]] return CI Functions also have the added advantage of limiting incomprehension spillovers: even if you don’t understand how the preceding functions work, you can still take for granted that they return confidence intervals and follow the rest of the reasoning, postponing a deeper dive into their code until later. Using Code Examples Supplemental material (code examples, etc.) is available for download at https:// oreil.ly/BehavioralDataAnalysis. xvi | Preface

📄 Page 19

If you have a technical question or a problem using the code examples, please send email to bookquestions@oreilly.com. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission. We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Behavioral Data Analysis with R and Python, by Florent Buisson (O’Reilly). Copyright 2021 Florent Buisson, 978-1-492-06137-3.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at permissions@oreilly.com. Navigating This Book The core intuition of the book is the idea that effective data analysis relies on a con‐ stant back and forth between three things: • Actual behaviors in the real world and related psychological phenomena, such as intentions, thoughts, and emotions • Causal analytics and especially causal diagrams • Data The book is split into five parts: Part I, Understanding Behaviors This part sets the stage with the causal-behavioral framework and the connec‐ tions between behaviors, causal reasoning, and data. Part II, Causal Diagrams and Deconfounding This part introduces the concept of confounding and explains how causal dia‐ grams allow us to deconfound our data analyses. Part III, Robust Data Analysis Here we explore tools for missing data and introduce Bootstrap simulations, as we’ll rely extensively on Bootstrap confidence intervals in the rest of the book. Preface | xvii

📄 Page 20

Data that is small, incomplete, or irregularly shaped (e.g., with multiple peaks or outliers) is not a new problem, but it can be especially acute with behavioral data. Part IV, Designing and Analyzing Experiments In this part, we’ll discuss how to design and analyze experiments. Part V, Advanced Tools in Behavioral Data Analysis Finally, we bring everything together to explore moderation, mediation, and instrumental variables. The various parts of the book build somewhat on each other, and consequently I rec‐ ommend reading them in order, at least for your first pass. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold Shows commands or other text that should be typed literally by the user. Constant width italic Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion. This element signifies a general note. xviii | Preface