PawCast with GeePaw Hill

Here's another technique card from my seminar, "Leading Technical Change". We first get into midwifing change precisely because we want it to be smoother, easier, and faster. But sometimes, a coach needs not to rush a decision through, but to slow it down.

---

If we're going to enable and support change, we're going to fail, more often than we succeed, and we want to bake tht idea in, early on, lest we fail both more often, and potentially more disastrously. 

---

Can we be honest? If we're going to be successful change midwives, honesty is very important. In this technique card from Leading Technical Change, I talk about some of the ins & outs of this complicated topic.

---

The people who hire me ask me to help their teams make changes. Most of the time, my first step is to see how I can slash those teams' load. To me, this is dreadfully obvious, but for a lot of folks seeking change, it comes as a completely shocking idea. The weirdest part, though, is that it not only improves our ability to change, it usually also increases our actual productivity. Let's talk about the change effect of that first, then the productivity effect.

---

Joy project: Today's a comparatively light work day, so I'm gonna lay out what I'm actually trying to do with this project. The source, btw, is at: https://github.com/GeePawHill/fgno-plotter. 

TDD Pro-Tip: I advocate against automated macro-tests -- those whose base is entire running programs --, as their cost is high and their benefit is doubtful. I very rarely write them. There is a bewildering variety of terminology out there around what I'm calling macro-tests, so let's poke around a little.

I spoze the historic and ongoing inability/unwillingness of the software trade to grasp and adopt test-driven development (TDD) is one of the most frustrating & demoralizing events of my forty-two years as a professional geek.

Let's talk for a minute about "over-coding". Over-coding, when you're a TDD'ist, is writing more code than you (intended to) have test to cover. But I will offer a few thoughts on this to *non* TDD'ers and TDD'ers alike.

Here's ten I-Statements about change, in the geek trades, and beyond. My hope is that it will give you a richer sense of where I'm coming from in my blogs, talks, videos, and courses. 

I've oft mentioned how the twin cost-revolutions in geekery warped & nearly destroyed our trade. Then wondered if we'll get to a place where it's no longer profitable for most companies to write bad software poorly. This morning I wonder if I'm seeing the beginning of it.

I was a good programmer because I was a *terrific* memorist: I could learn things by heart, and I could organize them in my mind in such a fashion that I could get to them whenever I needed them. It is the nature of humans that whatever they have had so far, they assume they will have forever. There's a default assumption that whatever's going on will continue to go on, ad infinitum. This applies to the good things they have, and also the bad things, of course, and is a defining property of humans, in my view.

On the cover of Hofstadter's famous _Godel, Escher, and Bach_, there's a photo of an artifact he made, called a "trip-let". The trip-let, when lit from three different angles, produces shadows that spell out "G", "E", and "B".

Let's talk about software design.

True story: Eighteen or so years ago, I had a gig rolling code at an engineering company. We were writing a windows app using Microsoft Foundation Classes to drive a TTY interface to a box of various radio hardware junk.

In our efforts to optimize the Many More Much Smaller Steps (MMMSS) path, we've tried and rejected the "shortest-distance" floptimization. Today, let's take up the "pin-making" floptimization, in which we create specialists, stations, and hand-offs.

We've built ourselves a positive case for "Many More Much Smaller Steps" (MMMSS). There's a counter-case, tho, based in a trio of proposed optimizations. Sadly, those optimizations usually flop. Today, let's take up the "Shortest Distance" floptimization.

We've covered the MMMSS idea a couple of times now. This time we want to consider the key element of the case in favor: the remarkable intrinsic value we receive from Many More Much Smaller Steps. It's not immediately obvious, so let's take our time.

As a person who has been successfully coaching software development teams for twenty years, let me throw out a few ideas to chew over. With luck, maybe one of them will jiggle the frame enough for you to find a next step.

Earlier, we sketched out MMMSS, Many More Much Smaller Steps, laying out the bare bones. Cool, cool, but now we need to thicken our sense of the idea. Today, let's close in a little more on what "step" means to us.

The first plank of my take on fixing the trade is MMMSS: If you want more value faster, take Many More Much Smaller Steps. Today I want to start laying this out for folks. This isn't gonna happen in one thread, but let's get started.

In the spirit of my Ten I-Statements about TDD, here's ten more, this time about refactoring. I'm not covering everything, just hitting some of the high points of my refactoring activity here.

Programmers will ask me how they can become stronger at programming. A very good way, one I use currently, is to get together once a week with a handful of geek friends of varying talents, interests, and projects, for a casual geekery-sharing session.

Folks, I see a lot of ideas and opinions about TDD fly around, passed off as holy writ. By way of counter, I offer you Ten I-Statements About TDD.

Coaching Pro-Tip: Don't fall -- or get pushed -- into the trap of being responsible for a team's targets. Your mission as a coach is to care for the health of your team, it's not to hit a deadline.

Coaching Pro-Tip #1: Everything good about agility is rooted in relationship, so everything good about coaching is, too. As coaches, we usually start from negative trust, and our central priority has to be reversing that position. Coaching Pro-Tip #2: When the developers act like testing the code is a time-wasting checkbox that costs a great deal and rewards very little, they are usually right, and one has to find out why, and tackle that, before pressing them to test more. Coaching Pro-Tip: Sometimes it's just chemistry. One advanced coaching skill is understanding when you're not some team's cup of tea, and seeing when the right thing to do is help them find someone they can relate to.

Two recurring phrases in my work are 1) It is like this because we built it to be like this. 2) The code works for you, you don't work for the code. Two sides of one page, phrased on the front as negative critique, and on the verso as positive encouragement.

"Path-focused design", of stories, architecture, code, is design that understands that we can only reach a distant City on the Hill by taking one stride-limited shipping step at a time. 

Big-batch releases, coordinated and controlled by a central intelligence, fail, and fail frequently. Several aspects of this are fascinating, because of the interplay of hard mathematical reality with human frailty. Let's take a swing.

You can put all your configuration in a shared library and eliminate just about every mis-configuration in your multi-process application. It's not free, but it's cheap, and it kills a lot of minor pain. Let's take a gander.

When large teams struggle with trunk-based development (TBD) or continuous integration/deployment (CI/CD), a good strategy is to re-orient how the teams face "backsteps", moments in our workflow where a discovery forces us to re-open a stage we thought was closed. Large teams & projects often pre-date the use of modern synthesis techniques like TBD or CD, and during adoption, intermixing their Before with this new After can produce quite a bit of discomfort. If it gets bad enough, it can even kill off the improvement effort. What to do? 

---

A couple of days back, I tweeted about SAFe. It created some stir on the timeline, which was great, as I got to see a lot of perspectives. I want to use that tweet as an excuse to talk about something much larger. This will be a long one. :) 

---

The standup is a short recurring meeting used to (re-)focus the team-mind on effectively moving stories through our workflow. Here's my recommended approach to having standups be useful and brief. The general sequence is 1) address team-wide emergency issues, 2) work story-by-story, 3) distribute new work, 4) address team-wide non-emergency issues. Note that, quite often, there is no part 1, and no part 4. Sometimes there's not even a part 3. Some general tips, then:

---

The ultimate making app for a shipping multi-service system is actually a one-machine monolith with a UI. If your team is experiencing the most common pains from working in a large SOA environment, the productivity payback will be enormous. We've talked a lot about the idea of having a shipping app for our customers and a making app for us. We can use the same source base to make multiple binaries. We target customer needs with one of those, and we target developer needs with the rest. The economics of this approach are straightforward: As long as a making app's benefit — improved productivity on the shipping app — is less than its cost — the time spent working on something we don’t ship — we get a net gain in productivity.

---

My rice'n'garlic advice, "take many more much smaller steps," can be said another way: reject any proposed path that requires a step size larger than the limit you've set for that particular domain of activity. "Rice'n'garlic advice" is blind advice, for when people ask you what to do, but you're not there & can't see. You have to guess. A professional chef I know, when asked to give blind advice, always says this: 1) That's too much rice. 2) That's not enough garlic. This kind of advice isn't expressing a universal law. (It is difficult to do, but you *can* have too much garlic, after all.) Nevertheless, for most people doing the asking, most of the time, that's a pretty good hipshot, applicable to most of her actual audience. My blind advice, in that same spirit, is this: Take many more much smaller steps. It's not a universal law. It's just a shot from the hip, and I find it's generally good advice, for most of the people asking, most of the time. Here's how it works.

---

Once armed with the idea of a shipping app and a making app, a whole range of possibilities open up. Among the most powerful: give your making app a UI just for making. A "making app" is when we take the same sourcecode from the program we're shipping, and use it for another program at the same time. That program is one we develop and tailor expressly to enable us to work more effectively on the shipping app. We tend to overlook it, but when we run our unit tests, for instance, we're actually running a making app. Whole great swathes of the code are exactly the same code we ship, just put together in a different way. But there's no reason to have just one making app, and no reason to limit its capability to only running our microtests. We can do all sorts of cool things in a making app.

---

In a data-rich environment, we can use the Builder concept to make DSL's for our Making application. This often makes testing the hard business core of our code both faster and easier. We've spoken in the past about using our codebase to do more than one thing. We always use it to create our shipping app. But we can and do use it for an app that improves our *making* process. We call that the making app. When you're running your microtests, it doesn't necessarily look or feel like you're running an app, but you are. And that app uses much of the same source-code that what you ship does. It just uses it in a very different way. Some applications manipulate a whole lot of intricately connected data. In the world of healthcare, for instance, an app might have literally dozens of classes, all rooted to and connected with a Patient. And the business logic is quite thick, with many cases and variants.

---

Approaches in software development -- or anything else -- that don't take ordinray human failings as their starting point are prone to dramatic failure. "The Impossible Human" is, well, noticeably uncommon. Let's dig in on that. Some years back, I made content for a CMS that had a whole lot of overlapping parts, each with its own special language. I found it very difficult to express myself quickly & cleanly, and, it being me, I complained about it a lot. And very often, the responses I received were in the form of "It's easy, just remember X." It still being me, I became sullen and resentful, and was largely unproductive for long stretches. "It's easy, just remember these 371 simple rules" is a formula for disaster, because it's an approach based on "The Impossible Human". Ordinary humans don't remember 371 simple rules about anything they haven't already mastered.

---

"It puts the database on its browser skin, or else it gets the hose again." This task occupies the daily life of a great many programmers. Today, I want to throw out some random sparks of advice for people working in that setting. In enterprise IT, it is commonplace for backend folks to work on problems shaped like this: there's a web endpoint controller on the top, a database on the bottom, and some simple or complicated business logic in the middle. I watch people working in this kind of environment quite a bit, and I have to tell you, most of what I see is them having a lousy time doing boring work in a tedious and time-consuming fashion. These ideas I want to toss at you are something of a grab-bag. They're just little seeds, not worked out answers, but my goal in offering them is to jiggle a mind or two: to get folks thinking about how they could do this kind of work in a way that *wasn't* a boring drag. And I spoze that's the meta-seed: "You don't have to live like this."

---

When we talk about transitioning to microtest TDD, we have to figure out how to provide the right experiences in the right order. That's why I propose we start by getting the experience of changing a well-microtested graceful class. "Create Experiences, Not Arguments" is one of the core habits of change-harvesters. We want to take that slogan very seriously when we approach any significant change to our practice. And microtest TDD, believe me, is a significant change. A would-be TDD coach or instructor nearly always carries an implicit belief that her victims do not have, drawn from lived experience that they don't have. It is this: "What we are doing now is much more unpleasant and unproductive than what we'll be doing when we TDD."

---

Microtest TDD is a "way of coding", not an after-market bolt-on to old-school approaches, and as a result, we have to constantly intertwine our conversation about technique with our conversation about transition. Technique, skills, philosophy, theory, all of these are tremendous delights to me. I love to muse & mull & illustrate & analyze & argue & point, and I greatly enjoy doing it on the topic of the modern synthesis in software development. But all of this explication & analysis is about a framework of related ideas. And that body of knowledge is a "there", a point on the time horizon we can orient towards. We, the trade, well, we're "here". How do we move from here to there? Because it doesn't matter what color the walls are painted in the City on the Hill if we can't find a path that takes us toward it.

---

In microtest TDD, we describe collaborations as "awkward" or "graceful". The distinction is critical to understanding how the Steering premise and the Pieces premise work together to make TDD a productivity tool. Let's dig in. We talked the other day about understanding & manipulating dependency in microtest TDD. The awkward/graceful distinction is at the heart of this. It can be a long journey to get it all, but it *starts* as soon as you take TDD for a spin in your day job. To get into this, let's take two classes most of us are familiar with: String and File. (I'm thinking in Java, but I suspect much of what we see will apply in most languages.) String is graceful. File is awkward. Let's see what that means.

---

Successful microtest TDD depends on our ability to solve a goldilocks challenge: when we test our code pieces, do we isolate them too much, too little, or just right? Finding the sweet spot will mean letting go of rulesets & purity. The five premises we've been over: value, judgment, correlation, steering, and pieces, guide us in a complicated dance of software design. At the center of that dance is our awareness of -- and our manipulation of -- "dependency", how one part of the code uses other parts of it. From the moment you wrote your first program, you produced a high-structure high-detail text that *depends* on the output of other high-structure high-detail text.

---

Because microtest TDD is more a "way of geeking" than a technique or a received body of knowledge, building one's faculties is a long and sometimes perilous effort. Let's talk about learning. I want to approach the question(s) of learning TDD in some ways that are a little different from others I've seen. I've written two TDD curricula myself, and am involved currently in helping with two more. I see all of them, including the current ones, as "interesting failures". At first, learning/teaching TDD seems like a road, then over time it seems to become a tree, and ultimately it feels much more like a spreading activation network, including lots of external triggers, plateau periods, and transformative experiences.

(I'm open to the possibility that this is what *any* judgment-centric skill looks like, but for now I just want to talk about TDD.)

---

Microtest TDD is an effective change strategy because it dramatically improves our performance at comprehension, confirmation, and regression detection, all critical factors in handling change quickly & safely. We've covered a lot of ground in considering TDD recently. The five premises, the need for a change strategy, the increasing collaboration, complication, and continuity of modern software, and some high-level less-obvious aspects of the practice itself. I want to put some of these pieces together, and show you how TDD adds up to an effective change strategy.
It doesn't "boil down" very well: as with most conversations around complex systems, the effect is multi-sourced and multi-layered.

This may take us a little while. :)

---

I got one of those messages I sometimes get from a reader, telling me that including my politics in my muses/blogs is off-putting.

As a general rule, I don't bother to respond to these. I gain and lose followers all the time, everyoen who makes content does, for all sorts of reasons, and that's just one more.

Today, though, surrounded by all of this, I feel like I want to give a more full statement on the matter.

---

Before we can make the case that microtest TDD is an effective change strategy, there's a few high-level aspects of it that we need to highlight. We tend to take these for granted in our case, but newcomers won't already know them. Today, I'm writing for the newcomer. It's going to be a little rambly, because the ideas aren't partitioning a single conceptual level into its component parts. Think of me as going over a text with a yellow highlighter, bringing out some points newcomers might not readily see. Twinning: When we do TDD, we are working in a single textual codebase that we use to produce two entirely separate applications, each serving a different purpose. The one app will be the one we ship. The other app will be a primary tool for producing the first app more quickly.

---

Microtest Test-Driven Development is a strategy for *change*. To understand the case, we need to answer two questions: 1) Why have a strategy for change? 2) How does TDD provide one? Let's take up the first question today. Why is having a change strategy so urgent?

The TL;DR is dead simple:

Because continuous ongoing change is at the base of all software development, both before and after our software is in the field.

---

Today it's microtest TDD's Value Premise: TDD ships more value faster when that value depends on changing our branching logic safely & quickly. Let's dig in. I am frequently presented with declarations that TDD is fine for plebs like me, but useless for software of type X. I've heard every variety of app type or coding specialty or domain substituted for that X. In other parts of the forest, I hear that the purpose of TDD is high quality, and if you don't care about that, you don't need TDD. Or that it's for satisfying personal or professional ethics, and if you don't care about that you don't need TDD and you're a bad person. The Value premise says that TDD is an instrument for productivity, and its effectiveness depends on how much of our work requires us to change our branching logic, which it lets us do with greater speed and lower risk than non-TDD methods.

---

Today, let’s take on microtest TDD’s Correlation Premise: Internal software quality (ISQ) and productivity are directly correlated. They go up together, and they go down together. The correlation premise lies in direct opposition to the widespread but grossly over-simple analysis that suggests we can “go faster” by shorting the internal quality of our code. It says, on the contrary, the lower that internal quality, the lower our productivity. Start by noting that we’re talking about internal software quality (ISQ), not external software quality (ESQ). What’s the difference? ISQ is the attributes of the program that we can only see when we have the code in front of us. ESQ is those attributes a user of the program can see.

---

Today, let’s talk about microtest TDD’s Judgment Premise: “We are absolutely and permanently reliant on individual humans using their individual judgment in TDD.” The judgment premise emphasizes the human in test-driven development. There are no known non-humans practicing TDD, so it may seem a odd that we have to talk about this, and yet, we do. As software geeks, we work with the most rigid deterministic systems conceivable, and we do much of that work in abstract mental space. To say that our target systems are machine-like is to say too little, really: they’re more machine-like than any real-world machine. The uber-mechanical material of our job can lead us to seeing every aspect of the job as if it were the same as that material, a phenomenon the French call “deformation professionelle”. Put simply: working with uber-machines all day, we imagine uber-machines everywhere.

---

Microtest TDD's Steering Premise is quite simple, which may be why it sometimes meets furious opposition. It says "Tests and testability are first-class citizens in design." Let's talk that over a little. There are, for any software problem, an infinite number of functionally correct designs. If implemented, they will work. But we don't *implement* an infinite number of designs. Why not? Because though they may be functionally correct, they still don't fit our context. We reject designs -- more often we reject individual choices in designs -- for a variety of reasons: reliability, cost of hardware, poor fit to our toolset, and so on. Those reasons are the "citizens" of the design process. And the steering premise says that one of the citizens -- not a secondary or minor or ex post facto consideration -- is whether that design can be tested. To probe at it: If you brought us a functionally correct design that required our app to run on a million AWS instances, we'd casually say "no, that's not valid." So obvious is this conclusion, that no one ever does it. Designs don't even get that far with such an issue.

---