UX for AI A Framework for Designing AI-Driven Products (Greg Nudelman Daria Kempka) (Z-Library)

TESTIMONIALS Absolutely brilliant! —Linda Lane, BFA, MSIM, UX Designer, Researcher, Writer/editor, Technical Product Manager at Microsoft & Infosys Insightful perspectives on designing user experiences tailored for AI applications. —Alex Faundez (www.linkedin.com/posts/faundez_uxlx2024- ux- design- activity- 7201518211496861696- taCZ) Forward- thinking ideas for reimagining our UX process, encouraging a shift from repetitive UI tasks to strategic UX design strategy. —Laura Graham (www.linkedin.com/posts/laura- graham- 765b406_uxlx- userexperience- designcommunity- activity- 7201192500655529984- OiJv) Thank you … Great workshop [material] on UX for AI. —Katrin Ellice Heintze (www.linkedin.com/posts/activity- 7201154293511376896- 0U76) Another favorite … Showcasing how to storyboard, identify AI use cases, and create digital twins. —Shahrukh Khan (www.linkedin.com/posts/shahrukhkhan07_just- came- home- from- lisbon- and- what- activity- 7200969521086504960- 98Fp) Hands- on, practical, engaging … Equipped us with innovative tools. Loved it! —Sabrina S. (www.linkedin.com/posts/activity- 7200781769015537664- Ym_e) D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

UX FOR AI D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

UX FOR AI A FRAMEWORK FOR DESIGNING AI- DRIVEN PRODUCTS BY GREG NUDELMAN WITH DARIA KEMPKA, CONTRIBUTING EDITOR D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

Copyright © 2025 by John Wiley & Sons, Inc. All rights reserved, including rights for text and data mining and training of artificial intelligence technologies or similar technologies. Published by John Wiley & Sons, Inc., Hoboken, New Jersey. Published simultaneously in Canada and the United Kingdom. ISBNs: 9781394345922 (paperback), 9781394345946 (ePDF), 9781394345939 (ePub) No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Sec- tion 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Pub- lisher, or authorization through payment of the appropriate per- copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750- 8400, fax (978) 750- 4470, or on the web at www.copyright .com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748- 6011, fax (201) 748- 6008, or online at http://www .wiley.com/go/permission. The manufacturer’s authorized representative according to the EU General Product Safety Regulation is Wiley- VCH GmbH, Boschstr. 12, 69469 Weinheim, Germany, e- mail: Product_Safety@wiley.com. Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/ or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book. Limit of Liability/Disclaimer of Warranty: While the publisher and authors have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762- 2974, outside the United States at (317) 572- 3993 or fax (317) 572- 4002. For product technical support, you can find answers to frequently asked questions or reach us via live chat at https://support.wiley.com. If you believe you’ve found a mistake in this book, please bring it to our attention by emailing our Reader Support team at wileysupport@wiley.com with the subject line “Possible Book Errata Submission.” Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be avail- able in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com. Library of Congress Control Number: 2025934398 Cover image: © CreativeIMGIdeas/stock.adobe.com Cover design: Wiley D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

To all those eager to evolve. — Greg To the believing mirrors and coconspirators. — Daria D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

ix ACKNOWLEDGMENTS Although for simplicity I used the singular “I” as a first- person voice throughout the book, it really does take a village to bring a book like this to life. First of all, I’d like to thank my contrib- uting editor, Daria Kempka, without whose gentle help the content would have been much less friendly and useful. She is the reason you don’t see some of the really bad jokes that my mind kept conjuring up on a regular basis. Daria is also the author of the key section on AI ethics in Part 4 and many important sections and edits throughout the book. If you enjoyed the book, please tell her “thank you!” I also wish to thank the many coauthors who generously contributed their wisdom, insights, skills, and time to deliver the perspectives that make this book so much better. I am truly in your debt. My dear coauthors (in alphabetical order) are: Casey Hudetz Chris Noessel Daria Kempka David Andrzejewski Greg Aper Jakob Nielsen Josh Clark Kathryn Campbell Michael Oren Paul Bryan Ranjeet Tayi Thomas Wilson I also wish to thank the people who do not appear in these pages, including Tej Redkar, my manager at Sumo Logic who came up with some amazing AI- driven projects for me to work on, Itai Kranz, Saurabh Suman, and other members of the UX team at Sumo Logic who provided their generous support for the last two years of my work, and my long- time friend, Jim Mora- bitto, who helped review the book in its early stages. I wish to thank all of the members of my wonderful editorial team for accepting my third book for publication with John Wiley & Sons and for helping me every step of the way— particularly James Minatel, associate publisher, Christine O’Connor, the book’s managing editor, and Elizabeth Welch, without whose generous input and open mind this book would not have been possible. Finally, I want to thank Shannon, Sky, and Juliette for dealing with a very distracted hus- band and father for far longer than it ever seemed possible. I love you. D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

xi ABOUT THE AUTHOR Greg is currently a Distinguished Designer/UX Architect at Sumo Logic, creating innovative AI/ML solutions for security, network, and cloud monitoring. A veteran of 35 AI projects, he led UX teams at Cisco Cloud Analytics, LogicMonitor, and GE Oil & Gas, where he was a Senior Director of UX and founded and led UX research for GE’s cross- product efforts on industrial AI applications, industrial Internet of Things (IoT), Smart Oil Wells, Smart Helmets, AI Corro- sion Manager, and more. Prior to leading UX teams in the corporate world, Greg led a suc- cessful boutique consultancy, DesignCaffeine, Inc., helping Fortune 100 clients like Cisco, IBM, and Intuit to create millions of loyal customers and generate hundreds of millions of dol- lars in additional valuation. An ardent inventor, Greg helped his clients and employers author 24 patents. Greg is the author/co- author of six popular UX design books (Designing Search, Android Design Patterns, The Mobile Book, The $1 Prototype, Frontiers of Web Design, and the UX for AI book you hold in your hands). He wrote over 200 popular articles about UX for AI, mobile design, DesignOps, and design strategy. Greg is a consistently top- rated speaker at conferences and companies around the world, with 100+ keynotes and workshops in 18 countries. He taught thousands of designers to use lightweight lean RITE methodology through hands- on workshops and graduate and under- graduate UX design courses at Marquette University, Hult Business School, and San Francisco State University. In 2023, Greg founded UXforAI.com, a newsletter dedicated to “leading with UX in the age of singularity” and perfecting the art of communication with our Robot Overlords. He is also working on a TEDx talk. D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

xiii ABOUT THE CONTRIBUTING EDITOR Currently Daria is Director of UX at LogicMonitor, where she leads a dynamic global team of designers and researchers who are on a mission to deliver the best, most useful, most usable AI- powered hybrid observ- ability platform in the world. She can be reached at www.linkedin .com/in/dariakempka. D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CONTENTS AT A GLANCE INTRODUCTION XXV HOW TO USE THIS BOOK XXXI PART 1—Framing the Problem 1 CHAPTER 1 Case Study: How to Completely F*ck Up Your AI Project 3 CHAPTER 2 The Importance of Picking the Right Use Case 11 CHAPTER 3 Storyboarding for AI Projects 21 CHAPTER 4 Digital Twin—Digital Representation of the Physical Components of Your System 39 CHAPTER 5 Value Matrix—AI Accuracy Is Bullshit. Here’s What UX Must Do About It 51 PART 2—AI Design Patterns 67 CHAPTER 6 Case Study: What Made Sumo Copilot Successful? 69 CHAPTER 7 UX Best Practices for SaaS Copilot Design 75 CHAPTER 8 Reporting—One of the Most Important Copilot Use Cases 89 CHAPTER 9 LLM Design Patterns 103 CHAPTER 10 Search UX Revolution: LLM AI in Search UIs 121 CHAPTER 11 AI-Search Part 2: “Eye Meat” and DOI Sort Algorithms 133 CHAPTER 12 Modern Information Architecture for AI-First Applications 145 CHAPTER 13 Forecasting with Line Graphs 165 xv D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CHAPTER 14 Designing for Anomaly Detection 179 CHAPTER 15 UX for Agentic AI 203 PART 3—Research for AI Projects 213 CHAPTER 16 Case Study: MUSE/Disciplined Brainstorming 215 CHAPTER 17 The New Normal: AI-Inclusive User-Centered Design Process 223 CHAPTER 18 AI and UX Research 235 CHAPTER 19 RITE, the Cornerstone of Your AI Research 247 PART 4—Bias and Ethics 263 CHAPTER 20 Case Study: Asking Tough Questions Through Vision Prototyping 265 CHAPTER 21 All AI Is Biased 275 CHAPTER 22 AI Ethics 283 CHAPTER 23 UX Is Dead. Long Live UX for AI! 299 INDEX 303  xvi Contents at a GlanCe D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CONTENTS INTRODUCTION XXV HOW TO USE THIS BOOK XXXI PART 1—Framing the Problem 1 CHAPTER 1 Case Study: How to Completely F*ck Up Your AI Project 3 A Boiling Pot of Spaghetti 3 Fail #1: Try to Replace a Trained Expert with AI 4 Fail #2: Forget About Cost vs. Benefit 4 Fail #3: No ML Training Data? No Problem! 5 Fail #4: It Makes No Difference What Question Your AI Model Is Answering 6 Fail #5: Don’t Worry About User Research—You Have an SME! 6 Final Thoughts 8 Reference 9 CHAPTER 2 The Importance of Picking the Right Use Case 11 Presuming That AI Will Be Telling Experts How to Do Their Job Is a Red Flag 11 Ask a Better Question 12 Reference 14 CHAPTER 3 Storyboarding for AI Projects 21 Why Bother with a Storyboard? 22 How to Create a Storyboard 24 Establishing Shot 24 Things 25 People 26 Faces 26 Transitions 27 Storyboard Conclusion 31 Storyboarding for AI 34 Final Thoughts 36 Design Exercise: Create Your Own Storyboard 36 Storyboarding Exercise Example: Death Clock 37 References 38 xvii D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CHAPTER 4 Digital Twin—Digital Representation of the Physical Components of Your System 39 Digital Twin of a Wind Turbine Motor 39 The Digital Twin Is an Essential Modeling Exercise for Designing AI-Driven Products 43 How to Build a Digital Twin: An Example 44 Wait, There’s More! 46 Design Exercise: Create Your Own Digital Twin 48 Reflect 49 Design Exercise Example: Life Clock Digital Twin 49 CHAPTER 5 Value Matrix—AI Accuracy Is Bullshit. Here’s What UX Must Do About It 51 The Big Secret 51 Confusion Matrix: How Can Accurate AI Be Wrong? 53 Value Matrix: The AI Tool for the Real World 55 Training AI on Real-Life Outcomes to “Think” Like a Human 56 One More Example 58 Final Thoughts: The Importance of Human Cost/Benefit 58 Design Exercise: Create Your Own Value Matrix 59 Design Exercise Example: Life Clock Value Matrix 60 References 61 PART 2—AI Design Patterns 67 CHAPTER 6 Case Study: What Made Sumo Copilot Successful? 69 Strong Use Case 69 Clear Vision 70 Dedicated Full-Screen UI 71 AI-Driven Autocomplete 71 Next-Steps Suggestions 73 Final Words 74 References 74 CHAPTER 7 UX Best Practices for SaaS Copilot Design 75 The More Important the Task, the More Real Estate Is Required 75 Side Panel 75 Large Overlay 76 Full Page 77 xviii CONTENTS D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CONTENTS xix SaaS Copilot Is Stateful 78 Specialized Fine-Tuned ChatGPT Model 78 Plug-Ins: Integrated Continuous Learning About Your Specific System 79 The IA of the AI Is Straightforward, Focused on Chat 80 Promptbooks: No Need to Twist into Pretzels to Write Prompts 82 Final Thoughts 82 Design Exercise: Create Your Own Mobile Copilot 83 Design Exercise Example: Life Clock Copilot 84 References 87 CHAPTER 8 Reporting—One of the Most Important Copilot Use Cases 89 Zoom AI Companion 89 Meeting Summary 89 Answer Questions About the Meeting 89 Set It and Forget It 90 UI Modality Switch 90 Microsoft Security Copilot 91 Executive Summary: A General Report 93 Pinboard: A Specialized Report Focused Only on Selected Key Details 93 Info for Report: Ignore Automatically vs. Pick Manually? 94 Security and Privacy 97 Design Exercise: Create Your Own Copilot Report 97 Design Exercise Example: Life Clock Copilot Report 98 Daily Report 98 Weekly Report 99 CHAPTER 9 LLM Design Patterns 103 Restating 105 Auto-Complete 106 Talk-Back 108 Initial Suggestions 108 Next Steps 111 Regen Tweaks 113 Guardrails 113 Design Exercise: Try Out the LLM Patterns 116 Design Exercise Example: “Life Copilot Plus” 117 D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

xx CONTENTS CHAPTER 10 Search UX Revolution: LLM AI in Search UIs 121 The Current State of Search 121 Google Search 121 Amazon Search 122 The “Mysteries That Are Not Scary” Problem 122 Enter LLMs 126 Design Exercise: Design Your Own LLM Search UI 130 Design Exercise Example: Life Copilot LLM Search 130 CHAPTER 11 AI-Search Part 2: “Eye Meat” and DOI Sort Algorithms 133 What Are Dynamic Dashboards? 133 Beware of Bias in AI Recommendations 137 DOI: Degree of Interest/Sort Algorithms 138 Design Exercise: Create Your Own Dynamic Dashboards and Sort UI 143 References 143 CHAPTER 12 Modern Information Architecture for AI-First Applications 145 Design Pattern du Jour: The Canvas 145 Is Information Architecture Dead? 146 Amazon.com: Conventional Approach 148 “AI-Minus”? Homepage 148 Conventional Search Results Page 149 AI-Plus Item Detail Page 150 Conventional Maintenance Pages 151 AI-First Amazon.com Redesign 151 AI-First Analysis Overview Page 151 AI-First Category Analysis Pages 152 AI-First LLM Search 155 AI-First Item Detail 156 AI-First Maintenance Pages 159 Long Live Information Architecture! 159 CHAPTER 13 Forecasting with Line Graphs 165 Linear Regression 166 R-Squared 167 R vs. R-Squared 169 Forecasting with AI 169 Nonlinear Regression 170 Seasonality 172 D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CONTENTS xxi Forecasting an Aggregate Variable 173 Final Words 175 Design Exercise: Design Your Own Forecasting UI 175 Design Exercise Example: Life Clock Forecasting 176 References 177 CHAPTER 14 Designing for Anomaly Detection 179 Why Is Detecting Anomalies Important? 179 Four Main Anomaly Types 180 Point Anomaly 181 Change Point Anomaly 185 Contextual Anomaly 188 Curve Shape Anomalies 192 Design Exercise Example: Life Clock Anomaly Detection UI 195 References 196 Getting Ready for AI-pocalypse: Shorthand UX Design Notation as AI Prompt 197 Shorthand UX Design Notation 197 Shorthand Notation as AI Prompt to Go Directly to Working Code 198 What Is Human Work? 202 CHAPTER 15 UX for Agentic AI 203 What Are AI Agents? 203 How Do AI Agents Work? 204 Use Case: CloudWatch Investigation with AI Agents 205 Final Thoughts 209 References 211 PART 3—Research for AI Projects 213 CHAPTER 16 Case Study: MUSE/Disciplined Brainstorming 215 Design Idea #1 215 Design Idea #2 217 Design Idea #3 217 Design Idea #4 218 Design Idea #5 218 But Wait, Did You Catch That? 219 Design Exercise: Create Your Novel Designs Using Bookending 221 Design Exercise Example: Novel Design Ideas for Life Clock 222 References 222 D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

xxii CONTENTS CHAPTER 17 The New Normal: AI-Inclusive User-Centered Design Process 223 In the Beginning … 223 The Monkey or the Pedestal? 225 A New Way of User-Centered Thinking 226 What the Heck Is a Spike? 226 What Is the Role of Data? 227 Where Is the Customer in All This? 227 Why Is This Change Necessary? 227 How Does This Affect the Role of UX? 228 Does This Mean I Have to Learn About AI So That I Can Ask My Data Science Teammates Good Questions? 229 Final Handoff to Dev 229 Many More Changes to Come 230 Reference 230 CHAPTER 18 AI and UX Research 235 UX Techniques That Will Likely See Full Automation 235 Routine Usability Studies 235 Routine NPS Studies and Surveys 236 Collecting and Organizing the Research Data 236 Triangulation of Quantitative and Qualitative Insights 236 UX Techniques That Will Be Radically Augmented 236 Competitive Analysis 237 Identification of Novel Use Cases 237 RITE Studies 237 UX Techniques That Will Become Increasingly Valuable 237 Core Skills 237 Workshop Facilitation 238 Formative Research, Field Studies, Ethnography, and Direct Observation 238 Vision Prototyping 238 Augmenting the Executive Strategy 239 AI Bullshit 239 AI Strategic Analysis Tools That Replace Humans in Coming Up with Novel Ideas and Business Use Cases 239 AI Heuristics Analysis Replacing User Research and Design 240 AI Acting as “Synthetic Users” for the Purposes of Usability Research 240 Build Your Persona Using AI 240 Final Words 241 References 242 D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

CONTENTS xxiii CHAPTER 19 RITE, the Cornerstone of Your AI Research 247 RITE Study vs. Usability Test 247 #1: RITE Studies Form the Core of the Design Process. Usability Tests Are Often Treated as QA 248 #2: RITE Studies Demand the Simplest Appropriate Prototypes That Change Rapidly. Usability Tests Often Mean Fancy Rigid Prototypes 249 #3: RITE Studies Produce Solutions. Usability Tests Produce Reports 250 A Fringe Benefit of RITE Studies 251 How to Conduct a RITE Study 252 A Few More RITE Rounds 255 The RITE Design Evolution 257 Dear Future: AI-Assisted RITE Methodology 257 Design Exercise: Run Your Own RITE Study 259 References 259 PART 4—Bias and Ethics 263 CHAPTER 20 Case Study: Asking Tough Questions Through Vision Prototyping 265 References 272 CHAPTER 21 All AI Is Biased 275 What Do You Expect When You Ask for “Biologist”? 275 How About “Basketball Player”? 275 Third Time’s the Charm: “Depressed Person” 277 References 281 CHAPTER 22 AI Ethics 283 CHAPTER 23 UX Is Dead. Long Live UX for AI! 299 AI Is Happening for Us, Not to Us 299 Staying on the Rollercoaster Is Optional 299 “UX Elitism” Is Over 300 Designers Are “Ambassadors of Innovation” 300 Core Skills Are in Demand 301 Combine Low-Fi UX Tools and Sophisticated AI Models 301 AI Is a “Wicked Problem” 301 AI Is Just Too Important to Be Left to Data Scientists 302 The Best AI is Augmented Intelligence 302 References 302 INDEX 303  D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

xxv INTRODUCTION For Captain Bhavye Suneja and his co pilot Harvino, this 6:20 a.m. flight started just like any other. The skies were clear, and the Jakarta Soekarno- Hatta International Airport was not par- ticularly busy. Yet just two minutes after takeoff, the nearly new jet started behaving erratically. The plane warned pilots it was in a stall and began to dive in response. (A “stall” is when the airflow over a plane’s wings is too weak to generate lift and keep the plane flying.) The captain fought the controls, trying to get the plane to climb, but the AI, still incor- rectly sensing a stall, continued to push the nose down using the plane’s trim system. For the next nine minutes, AI and humans fought each other for control as the massive jet continued to buck, losing altitude and airspeed. The chart below dispassionately documents the desperate struggle between human pilots and the Maneuvering Characteristics Augmentation System (MCAS) AI, a real- life HAL- 9000 from Arthur C. Clarke’s 2001: A Space Odyssey, determined to act according to its programming (see Figure I.1). Figure I.1 MCAS AI forcing the crash of Lion Air Flight JT 610 Source: Adapted from Komite Nasional Keselamatan Transportasi D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense

xxvi INTroduCTIoN The “trim manual” line shows the pilot’s efforts to redirect the plane; the “trim automatic” line below shows the MCAS actions (1). In just 12 minutes after takeoff, the AI won the battle, and the ill- fated Lion Air flight JT 610 hit the water, killing all 189 people on board. As I write this introduction in the hot Cali- fornia summer of 2024, it is nearly the sixth anniversary of that fatal crash. Six years later, most people still do not realize that the crash was not, at its core, a hardware or software problem. It was a UX for AI design problem. According to TAC, 737 Max had six “checklists” (predefined procedures pilots have to fol- low to solve a problem) that pertained to the situation faced by JT 610’s pilots: ■ Unreliable airspeed ■ Unreliable altitude ■ Angle of attack (AoA) disagree ■ Speed trim failure ■ Stabilizer out of trim ■ Runaway stabilizer trim Can you guess which one concerned checking the runaway AI’s actions? If you guessed “Runaway stabilizer trim,” you are better at guessing than the pilots of JT 610 were on that tragic day. Greg Bowen, Southwest Airline Pilots Association (SWAPA) Training & Standards Chair, pointed out that the cutout switches (which would have turned off the hallucinating MCAS AI) are the “fourth or fifth” item on the runaway stabilizer trim checklist. In addition to being down on the list of Boeing’s recommended procedures, the runaway stabilizer trim checklist was a “memory item,” which means the pilots had to memorize it. “So one of the things we’re looking at is redesigning that checklist so that it follows the conscript of what people would normally be expected to remember or not,” Greg Bowen said. Not surprisingly, the official regulatory guidance from the FAA has discouraged the use of memory items as part of procedures. “Memory items should be avoided whenever possible,” according to a 2017 Advisory Circular from the US aviation regulator. “If the procedure must include memory items, they should be clearly identified, emphasized in training, less than three items, and should not contain conditional decision steps.” Designers reading this will recognize the problem as a classic “recognition vs. recall” dilemma. It’s much easier to recognize something you see on the screen than to remember something (particularly something you do only rarely) in a stressful situation. In the single case of pilots recovering from an incorrect MCAS activation in the real world, which occurred on a Lion Air flight that took place the day before the crash, it took 3 minutes and 40 seconds for the pilots to figure out what was wrong— with the help of a third pilot who happened to be present to dig through the three separate checklists it took to resolve the prob- lem. Unfortunately, these pilots did not pass on to the next crew all of the information about the problems they encountered. In contrast, pilots on JT 610 could not recall the right checklist to turn off the trim switch. As the captain tried in vain to find the right procedure in the hand- book, the first officer was unable to control the bucking plane. D ow nloaded from https://onlinelibrary.w iley.com /doi/ by ibrahim ragab - O regon H ealth & Science U niversity , W iley O nline L ibrary on [20/05/2025]. See the T erm s and C onditions (https://onlinelibrary.w iley.com /term s-and-conditions) on W iley O nline L ibrary for rules of use; O A articles are governed by the applicable C reative C om m ons L icense