Programming Scala Scalability = Functional Programming + Objects (Dean Wampler) (Z-Library)

Dean Wampler Foreword by Seth Tisue 3rd Edition Covers Scala 3 Programming Scala Scalability = Functional Programming + Objects

Praise for Programming Scala, Third Edition “Whether you’re new to entirely Scala or making the two to three transition, Dean Wampler is the ideal traveling companion. Some Scala books make you feel like you’re back in a classroom. This one makes you feel like you’re pair-programming with a helpful expert by your side.” —Seth Tisue, Senior Software Engineer, Scala Compiler Team, Lightbend Inc. “Dean leaves no question unanswered. Rather than telling you only what you need to know to produce working code, he takes an extra step and explains exactly: How is this feature implemented? Is there a more general idea behind it that can provide extra context? Reading this book will enable you to make new connections between concepts you couldn’t connect before. Which is to say, you’ll learn something.” —Lutz Huehnken, Chief Architect, Hamburg Süd, A Maersk Company “Dean has succeeded in giving a complete and comprehensive overview of the third major release of the Scala language by not only describing all the new features of the language, but also covering what’s changed from Scala 2. Highly recommended for both newbies and experienced Scala 2 programmers!” —Eric Loots, CTO, Lunatech “At his many Strata Data + AI talks and tutorials, Dean made the case for using Scala for data engineering, especially with tools such as Spark and Kafka. He captures his Scala expertise and practical advice here." —Ben Lorica, Gradient Flow

“I’ve had the great pleasure of working with Dean in a few different roles over the past several years. He is ever the strong advocate for pragmatic, effective approaches for data engineering–especially using Scala as the ideal programming language in that work. This book guides you through why Scala is so compelling and how to use it effectively.” —Paco Nathan, Managing Partner at Derwen, Inc. “An excellent update to the earlier edition that will help developers understand how to harness the power of Scala 3 in a pragmatic and practical way.” —Ramnivas Laddad, cofounder, Paya Labs, Inc.

Dean Wampler Programming Scala Scalability = Functional Programming + Objects THIRD EDITION Boston Farnham Sebastopol TokyoBeijing

978-1-492-07789-3 [LSI] Programming Scala by Dean Wampler Copyright © 2021 Dean Wampler. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com. Acquisitions Editor: Suzanne McQuade Development Editor: Michele Cronin Production Editor: Katherine Tozer Copyeditor: Sonia Saruba Proofreader: Athena Lakri Indexer: WordCo Indexing Services, Inc. Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: Kate Dullea June 2021: Third Edition Revision History for the Third Edition 2021-05-26: First Release See http://oreilly.com/catalog/errata.csp?isbn=9781492077893 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Programming Scala, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

To Peggy Williams Hammond, September 10, 1933–May 11, 2018. —Dean

Table of Contents Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi 1. Zero to Sixty: Introducing Scala. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Why Scala? 1 The Appeal of Scala 2 Why Scala 3? 3 Migrating to Scala 3 3 Installing the Scala Tools You Need 3 Building the Code Examples 4 More Tips 4 Using sbt 5 Running the Scala Command-Line Tools Using sbt 7 A Taste of Scala 9 A Sample Application 20 Recap and What’s Next 30 2. Type Less, Do More. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 New Scala 3 Syntax—Optional Braces 31 Semicolons 33 Variable Declarations 33 Ranges 35 Partial Functions 36 Method Declarations 39 Method Default and Named Parameters 39 Methods with Multiple Parameter Lists 40 Nesting Method Definitions and Recursion 45 vii

Inferring Type Information 48 Repeated Parameter Lists 49 Language Keywords 51 Literal Values 54 Numeric Literals 54 Boolean Literals 55 Character Literals 56 String Literals 56 Symbol Literals 58 Function Literals 58 Tuples 58 Option, Some, and None: Avoiding Nulls 60 When You Really Can’t Avoid Nulls 61 Sealed Class Hierarchies and Enumerations 62 Organizing Code in Files and Namespaces 63 Importing Types and Their Members 64 Package Imports and Package Objects 65 Parameterized Types Versus Abstract Type Members 66 Recap and What’s Next 69 3. Rounding Out the Basics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Defining Operators 71 Allowed Characters in Identifiers 76 Methods with Empty Parameter Lists 77 Operator Precedence Rules 78 Enumerations and Algebraic Data Types 79 Interpolated Strings 82 Scala Conditional Expressions 83 Conditional and Comparison Operators 85 for Comprehensions 86 for Loops 86 Generators 87 Guards: Filtering Values 87 Yielding New Values 87 Expanded Scope and Value Definitions 88 Scala while Loops 90 Using try, catch, and finally Clauses 90 Call by Name, Call by Value 94 Lazy Values 97 Traits: Interfaces and Mixins in Scala 99 When new Is Optional 102 Recap and What’s Next 103 viii | Table of Contents

4. Pattern Matching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Safer Pattern Matching with Matchable 105 Values, Variables, and Types in Matches 107 Matching on Sequences 111 Pattern Matching on Repeated Parameters 114 Matching on Tuples 114 Parameter Untupling 116 Guards in Case Clauses 117 Matching on Case Classes and Enums 117 Matching on Regular Expressions 119 Matching on Interpolated Strings 121 Sealed Hierarchies and Exhaustive Matches 121 Chaining Match Expressions 122 Pattern Matching Outside Match Expressions 122 Problems in Pattern Bindings 124 Pattern Matching as Filtering in for Comprehensions 125 Pattern Matching and Erasure 126 Extractors 126 unapply Method 127 Alternatives to Option Return Values 129 unapplySeq Method 131 Implementing unapplySeq 133 Recap and What’s Next 134 5. Abstracting Over Context: Type Classes and Extension Methods. . . . . . . . . . . . . . . . . . 137 Four Changes 138 Extension Methods 139 Build Your Own String Interpolator 142 Type Classes 142 Scala 3 Type Classes 143 Alias Givens 150 Scala 2 Type Classes 152 Scala 3 Implicit Conversions 154 Type Class Derivation 158 Givens and Imports 159 Givens Scoping and Pattern Matching 161 Resolution Rules for Givens and Extension Methods 162 The Expression Problem 163 Recap and What’s Next 163 6. Abstracting Over Context: Using Clauses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165 Using Clauses 165 Table of Contents | ix

Context Bounds 167 Other Context Parameters 171 Context Functions 172 Constraining Allowed Instances 175 Implicit Evidence 178 Working Around Type Erasure with Using Clauses 179 Rules for Using Clauses 180 Improving Error Messages 182 Recap and What’s Next 183 7. Functional Programming in Scala. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 What Is Functional Programming? 185 Functions in Mathematics 186 Variables That Aren’t 187 Functional Programming in Scala 189 Anonymous Functions, Lambdas, and Closures 190 Purity Inside Versus Outside 193 Recursion 193 Tail Calls and Tail-Call Optimization 194 Partially Applied Functions Versus Partial Functions 195 Currying and Uncurrying Functions 196 Tupled and Untupled Functions 197 Partial Functions Versus Functions Returning Options 198 Functional Data Structures 199 Sequences 200 Maps 202 Sets 203 Traversing, Mapping, Filtering, Folding, and Reducing 204 Traversing 204 Mapping 205 Flat Mapping 206 Filtering 208 Folding and Reducing 210 Left Versus Right Folding 215 Combinators: Software’s Best Component Abstractions 219 What About Making Copies? 222 Recap and What’s Next 224 8. for Comprehensions in Depth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Recap: The Elements of for Comprehensions 225 for Comprehensions: Under the Hood 227 Translation Rules of for Comprehensions 230 x | Table of Contents

Options and Container Types 232 Option as a Container? 232 Either: An Alternative to Option 236 Try: When There Is No Do 239 Validated from the Cats Library 241 Recap and What’s Next 243 9. Object-Oriented Programming in Scala. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Class and Object Basics: Review 246 Open Versus Closed Types 247 Classes Open for Extension 250 Overriding Methods? The Template Method Pattern 251 Reference Versus Value Types 252 Opaque Types and Value Classes 253 Opaque Type Aliases 255 Value Classes 258 Supertypes 261 Constructors in Scala 262 Calling Supertype Constructors 263 Export Clauses 263 Good Object-Oriented Design: A Digression 265 Fields in Types 267 The Uniform Access Principle 268 Unary Methods 270 Recap and What’s Next 270 10. Traits. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271 Traits as Mixins 271 Stackable Traits 275 Union and Intersection Types 279 Transparent Traits 281 Using Commas Instead of with 282 Trait Parameters 282 Should That Type Be a Class or Trait? 284 Recap and What’s Next 284 11. Variance Behavior and Equality. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285 Parameterized Types: Variance Under Inheritance 285 Functions Under the Hood 286 Variance of Mutable Types 290 Improper Variance of Java Arrays 292 Equality of Instances 292 Table of Contents | xi

The equals Method 293 The == and != Methods 293 The eq and ne Methods 294 Array Equality and the sameElements Method 294 Equality and Inheritance 295 Multiversal Equality 296 Case Objects and hashCode 299 Recap and What’s Next 300 12. Instance Initialization and Method Resolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301 Linearization of a Type Hierarchy 301 Initializing Abstract Fields 305 Overriding Concrete Fields 307 Abstract Type Members and Concrete Type Aliases 308 Recap and What’s Next 309 13. The Scala Type Hierarchy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 Much Ado About Nothing (and Null) 312 The scala Package 315 Products, Case Classes, Tuples, and Functions 316 Tuples and the Tuple Trait 317 The Predef Object 319 Implicit Conversions 319 Type Definitions 321 Condition Checking Methods 321 Input and Output Methods 322 Miscellaneous Methods 322 Recap and What’s Next 323 14. The Scala Collections Library. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 Different Groups of Collections 326 Abstractions with Multiple Implementations 326 The scala.collection.immutable Package 327 The scala.collection.mutable Package 329 The scala.collection Package 331 The scala.collection.concurrent Package 332 The scala.collection.convert Package 333 The scala.collection.generic Package 333 Construction of Instances 333 The Iterable Abstraction 334 Polymorphic Methods 336 Equality for Collections 337 xii | Table of Contents

Nonstrict Collections: Views 338 Recap and What’s Next 339 15. Visibility Rules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341 Public Visibility: The Default 342 Visibility Keywords 342 Protected Visibility 343 Private Visibility 343 Scoped Private and Protected Visibility 343 Recap and What’s Next 345 16. Scala’s Type System, Part I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 347 Parameterized Types 347 Abstract Type Members and Concrete Type Aliases 348 Comparing Abstract Type Members Versus Parameterized Types 348 Type Bounds 349 Upper Type Bounds 349 Lower Type Bounds 350 Context Bounds 352 View Bounds 352 Intersection and Union Types 353 Intersection Types 353 Union Types 356 Phantom Types 359 Structural Types 362 Refined Types 367 Existential Types (Obsolete) 367 Recap and What’s Next 368 17. Scala’s Type System, Part II. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 Match Types 369 Dependently Typed Methods 371 Dependent Method and Dependent Function Types 373 Dependent Typing 374 Path-Dependent Types 380 Using this 380 Using super 381 Stable Paths 381 Self-Type Declarations 382 Type Projections 385 More on Singleton Types 386 Self-Recursive Types: F-Bounded Polymorphism 387 Table of Contents | xiii

Higher-Kinded Types 388 Type Lambdas 391 Polymorphic Functions 393 Type Wildcard Versus Placeholder 394 Recap and What’s Next 395 18. Advanced Functional Programming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397 Algebraic Data Types 397 Sum Types Versus Product Types 397 Properties of Algebraic Data Types 399 Final Thoughts on Algebraic Data Types 400 Category Theory 400 What Is a Category? 401 Functor 402 The Monad Endofunctor 404 The Semigroup and Monoid Categories 408 Recap and What’s Next 409 19. Tools for Concurrency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 The scala.sys.process Package 411 Futures 413 Robust, Scalable Concurrency with Actors 416 Akka: Actors for Scala 417 Actors: Final Thoughts 425 Stream Processing 426 Recap and What’s Next 427 20. Dynamic Invocation in Scala. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 Structural Types Revisited 429 A Motivating Example: ActiveRecord in Ruby on Rails 432 Dynamic Invocation with the Dynamic Trait 433 DSL Considerations 437 Recap and What’s Next 437 21. Domain-Specific Languages in Scala. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 Internal DSLs 440 External DSLs with Parser Combinators 446 About Parser Combinators 447 A Payroll External DSL 447 Internal Versus External DSLs: Final Thoughts 449 Recap and What’s Next 450 xiv | Table of Contents

22. Scala Tools and Libraries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451 Scala 3 Versions 451 Command-Line Interface Tools 452 Coursier 453 Managing Java JDKs with Coursier 453 The scalac Command-Line Tool 454 The scala Command-Line Tool 455 The scaladoc Command-Line Tool 456 Other Scala Command-Line Tools 456 Build Tools 457 Integration with IDEs and Text Editors 457 Using Notebook Environments with Scala 457 Testing Tools 458 Scala for Big Data: Apache Spark 459 Typelevel Libraries 461 Li Haoyi Libraries 461 Java and Scala Interoperability 462 Using Java Identifiers in Scala Code 462 Scala Identifiers in Java Code 462 Java Generics and Scala Parameterized Types 463 Conversions Between Scala and Java Collections 463 Java Lambdas Versus Scala Functions 464 Annotations for JavaBean Properties and Other Purposes 464 Recap and What’s Next 465 23. Application Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467 Recap of What We Already Know 467 Annotations 468 Using @main Entry Points 469 Design Patterns 470 Creational Patterns 470 Structural Patterns 471 Behavioral Patterns 472 Better Design with Design by Contract 474 The Parthenon Architecture 477 Recap and What’s Next 483 24. Metaprogramming: Macros and Reflection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 485 Scala Compile Time Reflection 487 Java Runtime Reflection 488 Scala Reflect API 490 Type Class Derivation: Implementation Details 491 Table of Contents | xv

Scala 3 Metaprogramming 491 Inline 491 Macros 496 Staging 500 Wrapping Up and Looking Ahead 502 A. Significant Indentation Versus Braces Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 Bibliography. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 507 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 xvi | Table of Contents

Foreword Foreword, Third Edition Forward-looking programming languages don’t always make it. Yet Scala is not only surviving but thriving. Some languages never get commercial adoption at all. Those first few companies brave enough to bet their business on your language are hard to find. Other languages get their time in the commercial sun but don’t manage to hang on, like Common Lisp and Smalltalk. They live on as influences, their genes still dis‐ cernable in contemporary languages. That’s success of a kind, but not what the crea‐ tors wanted. Scala has been defying these trends for well over a decade now. Circa 2008, compa‐ nies such as Twitter and Foursquare brought Scala out of academia and into the com‐ mercial world. Since then, the Scala job market and ecosystem have been sustained not only by independent enthusiasts but by superstar open source projects, such as Spark and Kafka, and companies like those on the Scala Center’s advisory board, who collectively employ impressive numbers of Scala programmers. Can Scala continue to pull it off? Its creator, Martin Odersky, thinks it can and I agree. Scala 3, launching in 2021, is a bold leap into the future of programming. Other languages will be playing catch-up for years to come. And not for the first time, either. In the years since Scala’s initial success, Java emerged from its long torpor with a parade of Scala-inspired language features. Swift and Rust also show Scala’s influence. Direct competitors have appeared too. Kotlin remains a contender, while others, such as Ceylon, have already fallen by the wayside. How much innovation is too much? How much is too little? Explorers must be bold but not foolhardy. Dangers lurk in new seas and on new lands, but you’ll never dis‐ cover anything if you just stay home. Scala’s bet is that being a better Java isn’t enough to meet programmers’ needs—not even if that better Java is Java itself. For one thing, competing with Java isn’t enough xvii

anymore. If Scala’s growth has leveled off somewhat in recent years, perhaps it’s because Java’s has too, and because we’ve already converted all the Java programmers we can hope to convert. We need to show that Scala is also a more-than-worthy alter‐ native to now-mainstream languages, like Python and TypeScript, and insurgent lan‐ guages, like Rust and Haskell. The big reason Scala still matters and is worth fighting for is that it fully embraces functional programming. Yes, it’s wonderful that Java has added lambdas and pattern matching, features that come from the functional tradition. But functional program‐ ming isn’t just a bag of disconnected individual features. It’s a paradigm shift, a fresh way of thinking. Learning Scala, or any functional language, makes you wonder how you ever programmed any other way. Learning Scala doesn’t mean forgetting everything you already know. Scala fuses the object-oriented and functional programming traditions into a single language you’ll never grow out of. And though Scala offers its own vibrant ecosystem of libraries, Scala programmers are also free to leverage vast worlds of Java and JavaScript. The design of Scala 3 retains the same pragmatism that has been crucial to its success all along. My teammates and I at Lightbend, along with our colleagues at the Scala Center and in Martin’s lab, work hard to make migration to new versions smooth, even as we bring you a Christmas-morning’s worth of new toys to play with. It’s truly remarkable how much of the Scala ecosystem has already made the leap. Scala 3 only just came out this month, but a rich array of libraries and tooling is already available for it. Whether you’re entirely new to Scala or making the 2 to 3 transition, Dean Wampler is the ideal traveling companion. Some Scala books make you feel like you’re back in a classroom. This one makes you feel like you’re pair-programming with a helpful expert. The text is bristling with practical know-how, with all of the nuances and need-to-knows for when you’re actually at the keyboard, trying to make something run. Dean inspires with how programming in Scala ought to be and is candid about what it is actually like. He gives you tomorrow, and today. Whatever the future holds for Scala, it will always be known as the language that took functional programming from a daring experiment to a practical, everyday reality. My fondest wish for this book is that it will find its way into the hands of a new gen‐ eration of Scala programmers. This new crew will be younger and more diverse than the old guard, and less encumbered by programming’s past. Professors: teach your students Scala! I can’t wait to see what they’ll build. — Seth Tisue Senior Software Engineer, Scala Compiler Team, Lightbend, Inc. Reno, Nevada, May 2021 xviii | Foreword