Name: Data-Oriented Programming in Java (Meap 1-8 chapters) (Chris Kiehl)（Z-Library）
Author: Unknown Author

Page 1

(This page has no text content)

Page 2

(This page has no text content)

Page 3

Data-Oriented Programming in Java 1. welcome 2. 1_Data_Oriented_Programming 3. 2_Data,_Identity,_and_Values 4. 3_Data_and_Meaning 5. 4_Representation_is_the_Essence_of_Programming 6. 5_Modelling_Domain_Behaviors 7. 6_Implementing_the_Domain_Model 8. 7_Guiding_the_design_with_properties

Page 4

welcome Thanks for purchasing the MEAP for Data-Oriented Programming in Java! This book is a distillation of everything I’ve learned about what effective development looks like in Java. It’s what’s left over after years of experimenting, getting things wrong (often catastrophically), and slowly having anything resembling “devotion to a single paradigm” beat out of me by the great humbling filter that is reality. Data Orientation is not some new paradigm here to beat up all other paradigms and take their lunch. If you like object orientation, functional programming, or any other paradigm, a little touch of data-orientation will make you better at those styles of programming! Data Orientation is about the data, not the specific tools. There are, of course, some patterns and approaches that naturally emerge when you focus on the data, but all of those can be applied readily to whatever paradigm is your preferred one. This is because data-orientation is born from a very simple idea, and one that people have been rediscovering over and over again since the dawn of computing: “representation is essence of programming”. Programs that are organized around the data they manage tend to be simpler, smaller, and significantly easier understand. When we do a really good job of capturing the data in our domain, the rest of the system tends to fall into place in a way which can feel like it’s writing itself. So, this book is about data. What it is, how to think about it, how to understand its semantics, how to model it, how to represent it in our code, and, somewhat surprisingly, how to listen to its feedback. The act of trying to capture what data is can often reveal how much we don’t understand about the domain we’re supposed to be modeling. To quote Bertrand Russel, “everything is vague to a degree you don’t realize until you try to make it precise.” Data-Orientation gives us the tools for making things precise. All you’ll need to follow along is a basic working knowledge of Java. As

Page 5

long as you know what a class is, and how to define an interface, and have at superficial understanding of generics (i.e. you’ve used a type like List<String> before), you’ve got everything you need to follow along. Thanks again for purchasing this book! Please share your thoughts and questions in the liveBook Discussion forum. Tell your friends and coworkers to buy a copy, too. I need the royalties to buy a yacht. —Chris Kiehl In this book welcome 1 Data Oriented Programming 2 Data, Identity, and Values 3 Data and Meaning 4 Representation is the Essence of Programming 5 Modelling Domain Behaviors 6 Implementing the Domain Model 7 Guiding the design with properties

Page 6

1 Data Oriented Programming This chapter covers Introducing Data-Oriented programming Data as Data How representation effects on our programs This book is about data. What it is, how to think about it, how to model it, how to represent it in our code, and all the good things that happen when we do. Programs that are organized around the data that they manage tend to be simpler, smaller, and, most importantly, significantly easier to understand. We’re going to learn how to model data “as data” using Java. Meaning data on its own, as ordinary values, independent of any class, operation, or behavior. We’ll still use those things throughout our programs, but at the heart of everything will be data, and its representation independent of any other code. We lift data up to this lofty place because it means something within its domain and to the people that use it. Data is more than just a collection of values that gets shoveled around our programs. It’s more than that stuff inside of our objects. Data has a semantics which differentiates it from all the other data it could be. An integer can represent an infinite number of "somethings", but it's only within a particular domain that it becomes something meaningful like an age or population count. It’s by breaking out data on its own that we can focus on this semantic meaning. When we understand our data at a deep level, we unlock new expressive power in our programs. At its core, data-oriented programming is ultimately just learning to be really, really precise about what we mean. That's pretty much the whole trick. Studying the data on its own enables us to move away from ambiguous generalities and towards explicit representations that capture the essence of what we're modeling. Before we get to the question of "what does it do?", we try to capture something much more fundamental, bordering on

Page 7

philosophical, the question of "What is it?". If we can answer that question and express the data’s meaning in our code, some amazing properties start to emerge which can reshape our programs. 1.1 Objects in a Data Oriented World Since we’re all Java programmers, it’s worth clearing this up before we go any further: Data-orientation doesn’t mean giving up objects! Data orientation is not some new paradigm here to replace all the others and shame you for ever having used them. Objects are invaluable tools. I’ve tried programming without them in other languages and I always come crawling back. It’s tough to beat the object’s ability to effortlessly manage stateful runtime resources (thread pools, socket connections, resource lifecycles, etc.). Since objects aren’t the enemy, the only mental shift required in the data- oriented approach is to view objects not as “The One True Way” to build software, but as something much more humble: another tool available for our use. Objects are great at some things and less good at others. This is true of all things that are tools. A hammer is not good at delicately tightening a watch screw. It is very good at being a hammer, though. It excels at all hammer related tasks. So, the main shift in data-oriented programming is about where and how much we use objects. We put them where they excel: up at the top of our applications where they can enforce boundaries (encapsulation boundaries, maintenance boundaries, “past this line, I don’t want to know or care about X” boundaries). One of the object’s strengths is that it naturally pushes us towards creating these strong isolation boundaries in our code. It puts a subtle design pressure on us that nudges us towards higher level abstractions. This is great when we’re drawing up the borders in our application as a whole. The pressure nudges us in the right direction. However, this design pressure can get in the way when we’re dealing with certain kinds of business logic. Sometimes what our programs really need is not another layer of abstraction or indirection, but to drop down a level of power, and work directly with the concrete, plain, easy to understand data sitting at the bottom of everything. Listing 1.1 A class modeling a piece of data

Page 8

class Point { private final double x; #A private final double y; #A #B } This can feel a bit funny at first. Maybe even bordering on "wrong." We generally don’t approach modeling data this way in Java (though we happily use it when it exists). The atomic unit of object orientation is data and its behaviors together, coordinating as one. Further, good object-oriented design encourages us to abstract “above” the data as much as possible, and focus on the interfaces through which our objects interact. The object-oriented design process is largely the act of figuring out where you draw borders, and who calls which interface, and which piece is responsible for what (along with other concerns of and making sure each object has enough to do, but also not too much). For all the guff Java gets for being the “kingdom of nouns,” we actually spend the bulk of our effort stressing about how the verbs attached to those nouns interact with one another. Figure 1.1 Objects and how they communicate is our focus during object-oriented design

Page 9

The process looks much different when we focus on “data as data” during our design phase. The representation of that data becomes the primary point of interest (it also becomes the main point where we can improve a design). We get to take a brief step back from the complexities of objects and how they interact, and instead focus on the much simpler question of “what is this stuff?”. We spend a lot of our time exploring what the information in this

Page 10

domain is and understanding the semantics which govern it at a fundamental level. Figure 1.2 The representation of our data is the primary focus during data oriented design

Page 11

So where do we put the objects? When do they come back into play? My sell to you is that if we do a really good job of representing our data, the objects we need to support it tend to naturally emerge in the right spot (often in a way that can feel almost inevitable). They seem to fall naturally into place because their role, vending a piece of data we’ve already designed, gets figured out during our modeling phase. All that’s left is gluing everything together atop our strong foundation of data. It will definitely take some getting used to. There will be some challenges ahead of us. However, if we can quiet that little voice of discomfort and skepticism long enough to get over the initial hump, there’s an exciting style of programming to explore. Focusing on modeling data as data puts a very different kind of design pressure on us from the one we usually feel under object orientation. It nudges in a direction that makes us view our programs under a new light. When data is out there on its own, it suddenly needs to do something that it never needs to do when encapsulated behind an object: describe itself. Without the encapsulation of a class to give it context though behaviors, it has to communicate what it is through other mechanisms. We're forced to consider what the data really means and, most importantly, how it should be represented. It is our data’s "representation", to quote Fred Brooks, that "is the essence of programming." When we get it right, everything else tends to fall into place. 1.2 The soul of data-oriented programming in a single line: To start to explore the outsized effect that data’s representation has on our programs as a whole, we only need a single line. We'll give ourselves a sole, individual piece of data. An identifier of some kind. It'll be out there on its own. Unadorned and without any kind of containing object. Listing 1.2 One line. A vague identifier of some kind String id; #A So, the question is, without a class to contextualize it, just what is this thing? What does the current representation of this data communicate to us as

Page 12

readers of the code? The answer I’d give is “not much.” When we encounter this code, all we know is that it involves a variable called "id," and that it's a String, and… that’s it. The representation doesn’t tell us anything about what the id is supposed to be. The information that’s in the code barely narrows it down at all. A string can represent just about anything. So, the thing we have to answer is, of that just about anything that it could be, what should it be? What is a meaningful ID for our domain? Figuring this out isn't always straight forward. More often than not, this domain information doesn’t live anywhere in the code itself. Instead, we have to leave the code entirely to chase down the information elsewhere -- either a more tenured coworker, external docs or wikis, or (on (frustratingly many) projects I’ve worked on) watching production traffic to see what is flowing across the wires. And this is the core of the problem: communication. The current representation doesn't communicate anything to us about what this piece of data is supposed to be. Even if we give a better name than id, or add a bunch of comments and java doc, the problem remains that the code itself -- the stuff we program with -- doesn't communicate anything about the semantics of the data. It doesn't tell us what kind of thing this identifier is meant to be. The current representation is a little speed bump in the code that slows down each person's understanding of what's going on. It's small and trivial in this example, but small ambiguities grow into incomprehensible ones over the course of a project. This is an ambiguity that people will have to spend time resolving. If they do it right, the only cost is some time. If they do it wrong, the cost is usually a bug. So, here's the data-oriented view of this: when we're designing our data type, which, in this case, represents an identifier of some kind, we'd look at that ID and go: "Ok. What does it really mean to be an ID in our domain?" It's probably something with more specific semantics than that String is conveying. Another way of thinking about it would be to imagine if we were to drop someone fresh into this part of the code. What is it that we'd want them to know about this id? How can we communicate that knowledge to them in the code directly?

Page 13

As for our identifier, it could be anything, but let's say, for ease of example, that IDs are actually UUIDs in our particular domain. Now we know what it is, we can ask if the code captures the semantics of "being" a UUID. Our String, while it can technically hold a UUID in string form (and is extremely common to do so), can also hold all kinds of things that aren't a UUID. Listing 1.3 One line. Many wrong answers. The wrong ways to create IDs. String id = "not-a-valid-uuid"; #A String id = "Hello World!"; #A String id = "2024-05-04"; #A String id = "172.16.24.105"; #A String id = “1010011001011011”; #A Our one-line example here, while about as trivial as it gets, embodies the problems that an imprecise representation can cause in our programs. There's a mismatch between what we "know" something means ("This ID is a UUID") and what the code says it is ("literally any String is A-OK"). This creates an interesting dynamic in the code. If you just think about what's allowed to be expressed -- what kind of values could you create with this code, it’s actually kind of dire. There's an infinite number of things that aren’t UUIDs! And our representation allows any of them to be incorrectly applied to our id! We usually don’t try to quantify how many “wrong” states our program can enter for this reason. Facing down something like “an infinite number of wrong ways to do something” is a heavy burden for a programmer to bear. The "standard" approach to this problem might be to try to wrangle some of those illegal states under control by sprinkling in some defensive programming throughout the code wherever this data gets used. A precondition here, an if check there, maybe a full Anti-Corruption Validation Layer at the front door. Regardless of how we do it, we have to do it somewhere, because the representation allows those wrong states to be expressed, which means they could be entered (and on a long enough time scale: will). And mounting those defenses means writing more code. And then tests of that code. And then more code after that. All to work around the fact that the representation we've picked for something that's supposed to only be a UUID allows things that aren't UUIDs.

Page 14

That brings us back to the data-oriented view. All of the problems, like the infinite number of wrong ways of creating an ID, and the need for defending against those not UUIDs, stem directly from how our data is represented in the code. A String is not a good way to represent a UUID. It allows too many things that aren’t UUIDs. We know what our data is supposed to be, so we need to bring the representation of the data in closer alignment with it. In the case of our UUID, this is super simple, because Java has a ready-made type we can use. We can swap out the ambiguous String for the concrete UUID. Listing 1.4 Changing how we represent that one line String id UUID id #A Which is an obvious change, right? (“obvious” things that aren’t common currency are going to be a very big theme of this book). However, what's interesting about changing this one single line, as "obvious" as it might be, is that it fundamentally changes what the code communicates to us as readers. We’ve made the code describe itself. There’s no ambiguity. We don’t have to chase down coworkers or dig through databases. What it means to be an id in our domain (A UUID) is expressed directly in the code. This subtle shift in representation, despite being a single line, has a massive impact on our program as a whole. We've moved the semantics of what it means to be an ID out of our heads and into where it belongs: the code. As a result, something kind of magical happened: all of the illegal “not-UUID” states disappeared! Actually something even more profound than that happened. Our program can now only construct correct states. There is literally no way to create anything that isn't a valid UUID. As such, there are no bad states to defend against, so there’s no need for additional code. And no need for tests of that additional code. It can be hard to quantify because it’s invisible, but picking a better representation made our entire code base smaller and simpler because of all the other code we now don’t have to write. There’s one more thing that’s really interesting about this little “obvious” example: how infrequently we collectively take this incremental step towards

Page 15

making our code express itself. At the time of this writing, if you search for "String id" on Github, you'll find 4.5 million instances of this exact ambiguity in Java alone. Some of those might very well actually mean “an ID is literally any conceivable String,” however, I suspect that most of those represent a gap in the developer’s modelling -- an instance where a just a little bit of unnecessary friction gets placed on the programmer's ability to understand the code. Data Orientation is largely just taking that tiny incremental extra step towards the "obvious" representation. And that's the basic idea in a one-line nutshell. Good representation ripples outward throughout our codebase. It has a simplifying effect because of all the other code we don’t have to write. It eliminates illegal states and, in doing so, makes our programs as a whole smaller, simpler, and easier to reason about. Let's keep going with a bigger example. Data is everywhere in our programs, but it’s often hidden just below the surface due to the flexibility that encapsulation allows. A big part of data-oriented programming is training ourselves to notice that data – to lift it up to the surface so that we can make it explicit. The more we clarify the data in our domain, the more understandable our programs become. 1.3 Show me your data, and the rest will be obvious Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious Fred Brooks One of the central ambiguities that pops up in most mainline code stems from being unclear about what the state inside of our objects means. We assume our readers will just “get it” while reading our code in the same way we do while writing it. However, so often, there’s a mismatch between what we think we’ve expressed in the code, versus what we’ve actually expressed in the code.

Page 16

This presents as explanations that go "well, if this one field is set, then it means this, but if this other field is set then it means..." (often times this keeps going "if they're both not set then it means... " (and going “but when we set…”)). The state inside our objects is being used to model… something, but that “something” is only alluded to and hinted at. It’s never expressed directly. It’s something that works fine when we’re the people writing the code and have the implicit meaning tacitly in our head. However, it becomes impenetrable when we’re the ones piecing together why that one method sets that one field some of the time. Here's an example of what I'm talking about. It's a small slice of a larger set of classes that deals with running scheduled tasks asynchronously. We're going to give ourselves a simple job: understand what the `reschedule` method does. We're purposefully going in blind and ignoring anything else that might be in the class for now so that we can really focus on what the code communicates on its own, exactly for what’s written on the page. Listing 1.5 if attempts is not set then it means..." class ScheduledTask { private LocalDateTime scheduledAt; #A private int attempts; #A void reschedule() { #B if (this.someSuperComplexCondition()) { this.setScheduledAt(now().plusSeconds(this.delay())); this.setAttempts(this.attempts() + 1); } else if (this.someOtherComplexCondition()) { this.setScheduledAt(this.standardInterval()); this.setAttempts(0); } else { this.setScheduledAt(null); this.setAttempts(0); } } } The code itself, we could all probably agree, is trivial. The statements are all easy to follow, and there aren't too many of them. A few conditions and a few variable assignments. However, easy to read doesn’t always mean easy to

Page 17

understand! If you actually try to reason about what this code does, you'll find that something fundamental is actually left completely unsaid in the code: what does all of this actually do? I don't mean what fields does it set, that's obvious. What’s not obvious here what it means when those fields are set. The big challenge with understanding code like this is that you have to take off your engineering hat and swap it out for something more like a psychologist hat. We have to peer inside the dark recesses of the original developer's mind to attempt to piece together what they were thinking. Those fields mean something when they take on (or don't take on!) certain values. The author of this code had a model in their head, and they were using these particular values assigned to these particular fields to represent it. However, they did so only partially. They encoded just enough of their world to make their intentions clear to the computer, but we humans are left guessing. Of course, it's not totally grim. Dealing with this type of ambiguity might as well be what programming is most of the time. It’s what we do every day. If you stare at this code long enough you could start to make some decent guesses as to what the various branches mean. For instance, in the first branch, where someComplexCondition is true, attempts gets incremented. However, that same attempts variable is reset in all other branches. Pair those together and you have interesting clues that there’s some kind of lifecycle going on here. It’s unstated in the code, but we can kind of see it in between the lines if we squint. Listing 1.6 Looking closer at the reschedule method void reschedule() { #A if (this.someSuperComplexCondition()) { this.setScheduledAt(now().plusSeconds(this.delay())); this.setAttempts(this.getAttempts() + 1); #A } else if (this.someOtherComplexCondition()) { this.setScheduledAt(this.standardInterval()); this.setAttempts(0); #B } else { this.setScheduledAt(null); this.setAttempts(0); #B } }

Page 18

When it comes to understanding what these variable assignments mean, the absolute best we can do within the confines of this method is make a guess. All we have are some variable assignments. There's just not enough information to build up a working theory of why these values are set or what it means when they are. We’ve reached an informational dead end because the code doesn’t communicate anything to us. So, we have to leave the method and go foraging around in the wider world. The code doesn’t tell us what these variable assignments mean, so we have to look at where and how they’re used. It's only by studying everything that we can inductively start to build up our own mental model that maps meaning onto what these individual variable assignments denote when they hold certain values. If you're lucky, you'll find usages elsewhere that give context as to what a particular assignment means. Here’s an example we might find in another class that gives some meaning to one of the possible states. Listing 1.7 finding clues throughout the code class Scheduler { private void pruneTasks() { this.getTasks().removeIf((task) -> task.getScheduledAt() == null #A } } The process is to just keep going like this, over and over, poking around the codebase, reading all of the implementations, until we’ve found a large enough set of examples to construct our own model of what’s going on in the code – what the code means. Table 1.1 the meaning we’ve worked out for various states assigned in the reschedule method When these are set Then it seems to mean… attempts+=1; scheduledAt = small delta Go for an immediate retry

Page 19

attempts=0; scheduledAt=large delta; Give up for now and try again later attempts=0; scheduledAt=null Give up on this job entirely This process always represents hard won knowledge. It could be a few minutes, hours, or even days depending on how complex the code is. The unsatisfying part of all of this is that you can never really be sure that your analysis and model of the world lines up with the one originally envisioned. We build up a theory by plugging each example we find into our own evolving inductive model of the world. However, there could be another usage lurking in the code that invalidates all of our assumptions. It's a similar problem to the old observation about the limits of software testing: you can only prove the existence of bugs, never their absence. Understanding an existing codebase written in this style presents us with the same limitations. It's a lossy process. Without a direct line to the original developer, we can only reason about their intention from the examples we see in the code. We can reason inductively about the examples we’ve seen, but we can never really be sure we’re correct, because the code itself doesn’t tell us. This lack of explicit and clear modeling presents a very real problem. Beyond just being plain hard to understand, I’d argue that it’s one of the most common sources of bugs in our programs. To use some lingo from the world of databases, the code in our example lacks “semantic integrity.” Those variables mean something when they’re set, and the interpretation of that meaning is critical to the correct functioning of the software, but that meaning isn’t enforced or encoded anywhere. It’s left entirely implicit, which means each and every part of the code has to reinterpret that implicit meaning over and over again. This dooms us to semantic drift as time goes on. All it takes is for one method to misinterpret what a particular state means in order for the whole thing to collapse in on itself. Which all brings us back to the data-oriented view. There’s some data in there that’s floating just below the surface. It’s the ideas behind what these individual variable assignments mean. It’s not currently expressed, but we can bring it to light if we focus on it. This is where representing data “as data” comes in. Focusing on just the data

Page 20

by itself lets us take a step back from the details of the code as a whole and instead just think about, ok, what is it that I'm really talking about? What does it mean when I set those fields? I've got some mental model floating around in my head, and it has a semantics that I'm implicitly honoring. How do I get that out of my head and into the code? So, what are we really talking about in this case? When a ScheduledTask fails, our system has to make a decision about what to do next. What we’ve been piecing together as we explored this code is that it’s not just any arbitrary decision, it selects from a fixed set of next possible decisions. That’s what’s ultimately going on with those variable assignments. It’s just hidden by the current modeling. Figure 1.3 Being explicit about what a task can transition to after failing

Tip the Site

Data-Oriented Programming in Java (Meap 1-8 chapters) (Chris Kiehl)（Z-Library）

Data-Oriented Programming in Java (Meap 1-8 chapters) (Chris Kiehl)（Z-Library）

Registered users can read the full content for free

AI Reading Assistant

Passage locations

Recommended for You

Tip the Site