Managers and Individual Contributors in Code

2014-04-21

manager-individualContributorI’m lucky to work for a company that makes it possible to have a good technical career, so that I don’t have to become a manager just to get more money.

I like being an individual contributor because it gives me a sense of accomplishment when I get stuff done myself.

Anyway, I believe the distinction between manager and individual contributor is useful when writing code as well.

Let me show that using the example of Conway’s Game of Life.

Baby Steps

At code retreats, you’ll find people who start working on the problem from the bottom (cell) and those who start at the top (universe, world, grid, game, or whatever they call it).

Interestingly, both groups tend to run into a similar problem.

baby-stepsStarting at the bottom makes it easy to quickly implement the rules in a cell. But then the problem becomes how to connect the cells into a universe.

This often looks like too big of a step to properly test, e.g. with a new green test every two minutes.

Starting at the top makes sure we are not doing anything unnecessary or building any parts that would not connect properly. But here too it’s difficult to keep chipping away in baby steps.

The Single Responsibility Principle

To evolve a whole universe, at least these things need to happen:

  1. Iterate over all cells
  2. Determine each cell’s neighbors
  3. Count the number of alive neighbors
  4. Determine the cell’s next state based on its current state and the number of alive neighbors

Depending on some implementation choices (like keeping the universe immutable), there may be more work.

Yet the Single Responsibility Principle tells us that a class should have only one reason to change. You may count items #2 and #3 as one responsibility (or #3 and #4), but clearly there is more than one involved.

Manager Classes and Individual Contributor Classes and How to Test Them

Now remember the manager/individual contributor (IC) distinction. Managers manage ICs; that’s their whole job. We should count managing other classes as a responsibility too.

That would mean that a class either implements some logic (IC), or coordinates with other classes (manager), but not both.

mockI think this also ties into the state-based versus interaction-based testing debate: we should use state-based testing for IC classes and interaction-based testing (using mocks) for manager classes.

That would explain why both code retreat groups seem to get stuck: they have to switch from state-based to interaction-based testing and it seems that many developers don’t have a lot of experience with the latter.

Test-Driving Manager Classes

So let’s try interaction-based testing of a manager class. I’ll keep with the example of the Game of Life.

I’d like to begin with an architectural choice: I want the universe to be immutable, so that it would be easy to introduce multi-threading later. As the game evolves, I’ll have to create a new universe for every generation:

public class GameTest {
  @Test
  public void clonesImmutableUniverseWhenEvolving() {
    Universe universe = mock(Universe.class, "old");
    Universe nextGeneration = mock(Universe.class, "new");
    when(universe.clone()).thenReturn(nextGeneration);

    Game game = new Game(universe);
    assertSame("Initial", universe, game.universe());

    game.evolve();
    assertSame("Next generation", nextGeneration, 
      game.universe());
  }
}

I’m using Mockito to define how the Game interacts with the Universe. Mocking works best with interfaces, so Universe is an interface.

That means the game doesn’t know the type of universe and therefore can’t create a new instance of it. The universe provides a factory method clone() for this purpose.

If the universe is to be immutable, then its state should be provided in its constructor. This constructor is called by clone(), so that method needs the new state as its input.

What does that state look like? At this point we don’t know anything about the universe other than that it contains cells. Let’s keep it simple and provide the cells as input:

public class GameTest {
  @SuppressWarnings("unchecked")
  @Test
  public void clonesImmutableUniverseWhenEvolving() {
    Universe universe = mock(Universe.class, "old");
    Universe nextGeneration = mock(Universe.class, "new");
    when(universe.clone(any(Iterable.class)))
        .thenReturn(nextGeneration);

    Game game = new Game(universe);
    assertSame("Initial", universe, game.universe());

    game.evolve();
    assertSame("Next generation", nextGeneration, 
        game.universe());
  }
}

OK, so who is going to supply those cells to clone()? The game is a manager, so it should only manage, not provide logic. The universe is an IC that is responsible for the cells and their interconnections, so it already has a responsibility.

So it seems like we need a new type. The new type should be managed by the game, since that’s the only manager class we have. Thus we should add a new test in GameTest for this interaction.

The new type should be responsible for determining the new state of the universe from the old one. This is where the game’s rules come into play, so let’s call the new type Rules.

Let’s start with testing that the game manages the rules:

public class GameTest {
  @Test
  public void consultsRulesWhenEvolving() {
    Rules rules = mock(Rules.class);

    Game game = new Game(null, rules);
    assertSame("Rules", rules, game.rules());
  }
}

Now we want to test that evolve() consults the rules. Since the game is a manager, it can only request output from a collaborator and provide that as input to another collaborator. It shouldn’t contain any logic itself, since that would make it an IC.

The only other collaborator that the game currently has is the universe and that seems the correct candidate to provide the cells. But we need more than the cells to implement the rules: we also need to number of alive neighbors of each cell.

The universe is clearly the correct place to determine a cell’s neighbors. Should it also count the number of alive ones?

I guess it could, but I don’t particularly like that: the rules for Life use the number of alive neighbors, but it’s not hard to imagine a different set of rules that also depend on the number of dead neighbors. So I feel that the rules should do the counting.

This means that the input to the rules is a cell and its neighbors. Since Java doesn’t allow returning two pieces of information, we need to combine them. Let’s call the combination a neighborhood:

public class NeighborhoodTest {
  @Test
  public void holdsACellAndItsNeighbors() {
    Cell cell = mock(Cell.class, "cell");
    List<Cell> neighbors = Arrays.asList(
        mock(Cell.class, "neighbor1"), 
        mock(Cell.class, "neighbor2"));

    Neighborhood neighborhood = new Neighborhood(cell, 
        neighbors);

    assertSame("Cell", cell, neighborhood.cell());
    assertEquals("Neighbors", neighbors, 
        neighborhood.neighbors());
  }
}

Now we can make the universe return a neighborhood for each of the cells it contains and verify that those neighborhoods are used as input for the rules:

public class GameTest {
  @SuppressWarnings("unchecked")
  @Test
  public void clonesImmutableUniverseWhenEvolving() {
    Universe universe = mock(Universe.class, "old");
    Universe nextGeneration = mock(Universe.class, "new");
    when(universe.clone(any(Iterable.class)))
        .thenReturn(nextGeneration);
    when(universe.neighborhoods()).thenReturn(
        new ArrayList<Neighborhood>());

    Game game = new Game(universe, mock(Rules.class));
    assertSame("Initial", universe, game.universe());

    game.evolve();
    assertSame("Next generation", nextGeneration, 
        game.universe());
  }

  @Test
  public void consultsRulesWhenEvolving() {
    Universe universe = mock(Universe.class);
    Neighborhood neighborhood1 = new Neighborhood(
        mock(Cell.class), 
        Arrays.asList(mock(Cell.class)));
    Neighborhood neighborhood2 = new Neighborhood(
        mock(Cell.class), 
        Arrays.asList(mock(Cell.class)));
    when(universe.neighborhoods()).thenReturn(
        Arrays.asList(neighborhood1, neighborhood2));
    Rules rules = mock(Rules.class);

    Game game = new Game(universe, rules);
    assertSame("Rules", rules, game.rules());

    game.evolve();
    verify(rules).nextGeneration(neighborhood1);
    verify(rules).nextGeneration(neighborhood2);
  }
}

The next step is to make sure that the output from the rules is used to construct the new universe:

public class GameTest {
  @Test
  public void consultsRulesWhenEvolving() {
    Universe universe = mock(Universe.class);
    Neighborhood neighborhood1 = new Neighborhood(
        mock(Cell.class), 
        Arrays.asList(mock(Cell.class)));
    Neighborhood neighborhood2 = new Neighborhood(
        mock(Cell.class), 
        Arrays.asList(mock(Cell.class)));
    when(universe.neighborhoods()).thenReturn(
        Arrays.asList(neighborhood1, neighborhood2));
    Rules rules = mock(Rules.class);
    Cell cell1 = mock(Cell.class, "cell1");
    Cell cell2 = mock(Cell.class, "cell2");
    when(rules.nextGeneration(neighborhood1))
        .thenReturn(cell1);
    when(rules.nextGeneration(neighborhood2))
        .thenReturn(cell2);

    Game game = new Game(universe, rules);
    assertSame("Rules", rules, game.rules());

    game.evolve();
    verify(universe).clone(eq(
        Arrays.asList(cell1, cell2)));
  }
}

At this point we’re done with the Game class. All that is left to do, is create implementations for the three interfaces we introduced: Universe, Cell, and Rules. Each of these is an IC class, and thus fairly straightforward to test-drive using state-based testing.

Conclusion

I find that the distinction between manager and individual contributor classes helps me in deciding what the next test should be.

What do you think? Could this be part of the missing piece for a proper theory of Test-Driven Development?


Too Many Open Files, Or Too Few Bounded Contexts?

2014-02-18

A Business Men Climbing a Pile of PapersServer software needs to run unsupervised for long periods of time to be practical.

Release It! is full of horror stories about programming mistakes that get in the way of that lofty goal.

One example is opening files and forgetting to close them.

On some operating systems this will eventually lead to a Too many open files error when the number of open files passes a certain limit.

Of course we want to make sure that doesn’t happen to us. IDEs like Eclipse can give you warnings in certain cases to help with that, but if you’re test infected like me, you will want to write a test to make sure.

This is one of those situations where people less committed to (unit) testing tend to give up. There is nothing in the Java file system API that tells you how many files are open, so it simply cannot be done, right?

Not so fast, my friend!

First, with some proprietary code you can, in fact, count the number of open files in Java.

Second, we can do even better if we step back for a moment and take a look at the bigger picture. Chances are that our application isn’t really about files; that using files is simply a convenient implementation choice.

In the lingo of Eric Evans’ classic Domain-Driven Design, we have (at least) two bounded contexts: your core domain (what people buy/use your application for) and the file system.

A bounded context delimits the applicability of a particular model so that team members have a clear and shared understanding of what has to be consistent and how it relates to other contexts.

Evans describes a number of strategies for dealing with different bounded contexts. In our example, the file system API is not under our control, which rules out the majority of them. The remaining strategies are:

  • Separate Ways, i.e. use something other than the file system. That probably doesn’t make sense in this example
  • Conformist, i.e. follow the Java model slavishly. This is what most developers do without giving it much thought
  • Anti-Corruption Layer, i.e. create an isolating layer to provide clients with functionality in terms of their own domain model. This layer translates in both directions as necessary between the two models

This last strategy gives us more than “just” the opportunity to keep our models clear and to the point. By introducing interfaces that we control in our anti-corruption layer, we also gain the opportunity to mock those interfaces in our tests, making it very easy to verify that we indeed close all the files we open.

This is yet another example where difficulty in unit testing a piece of code points to an opportunity to improve the design. If we consistently act on such opportunities, we will end up with a clean architecture that is a joy to work with.


Adventures in JavaScript: Getting Started

2013-05-13

Node.jsOne of the high potentials for a Frictionless Development Environment (FDE) is Cloud9.

It is one of a growing number of web applications that uses JavaScript as the programming language for both front-end and back-end. The latter brought to you by Node.js.

So I thought it was time to start playing around with JavaScript and Node. Here is an account of my very first adventure into this Brave New World.

Preparations: Adding JavaScript Support to Eclipse

To keep the number of changes low, I wanted to keep my trusted old Eclipse. So the first step was to install Nodeclipse and jshint-eclipse.

To support documentation in the Markdown format that Node uses, I installed the Markdown Editor plugin for Eclipse.

This left me with nothing for unit tests. So I installed the JavaScript tools from Eclipse. That gave me some JS support, but nothing for creating unit tests.

Some googling told me there is such a thing as JsUnit, the JS port of my beloved JUnit. Unfortunately it doesn’t seem to come with Eclipse support, even though this thread indicates it does (or did).

JsTestDriverMaybe I’m just doing it wrong. I’d appreciate any hints in the comments.

Some more googling informed me that Orion is using JsTestDriver.

This introduction to JsTestDriver explains in detail how it works.

First Exercise: Roman Numerals

Now that I’m all set up, it’s time to do a little exercise to get my feet wet. For this I picked the Roman Numerals kata.

I started out by following this JsTestDriver example. I created a new JavaScript project in Eclipse, added src/main/js and src/test/js folders, and created the JsTestDriver configuration file:

server: http://localhost:9876

load:
  - src/main/js/*.js
  - src/test/js/*.js

Next, I opened the JsTestDriver window using Window|Show View|Other|JavaScript|JsTestDriver and started the JsTestDriver server. I then opened the client in FireFox at http://127.0.0.1:42442/capture.

The next step was to create a new run configuration: Run|Run Configurations|JsTestDriver Test. I selected the project and the JsTestDriver configuration within the project, and checked Run on Every Save.

Now everything is set up to start the TDD cycle. First a test:

RomanNumeralsTest = TestCase("RomanNumeralsTest");

RomanNumeralsTest.prototype.testArabicToRoman
    = function() {
  var romanNumerals = new TestApp.RomanNumerals();
  
  assertEquals("i", romanNumerals.arabicToRoman(1));
};

And then the implementation:

TestApp = { };

TestApp.RomanNumerals = function() { };


TestApp.RomanNumerals.prototype.arabicToRoman
    = function (arabic) {
  return null;
};

I completed the rest of the kata as usual.

Reflections

The cool thing about JsTestDriver is that it automatically runs all the tests every time you change something. This shortens the feedback cycle and keeps you in the flow. For Java, InfiniTest does the same.

The problem with my current tool chain is that support for renaming is extremely limited. I got Operation unavailable on the current selection. Select a JavaScript project, source folder, resource, or a JavaScript file, or a non-readonly type, var, function, parameter, local variable, or type variable.

Other refactorings do exist, like Extract Local Variable and Extract Method, but they mess up the formatting. They also give errors, but then work when trying again.

All in all I feel satisfied with the first steps I’ve taken on this journey. I’m a little worried about the stability of the tools. I also realize I have a more to learn about JavaScript prototypes.


Five Essential Components of a Frictionless Development Environment

2013-02-04

One of the challenges of maintaining a consistent programming style in a team is for everyone to have the same workspace settings, especially in the area of compiler warnings.

Every time a new member joins the team, an existing member sets up a new environment, or a new version of the compiler comes along, you have to synchronize settings.

My team recently started using Workspace Mechanic, an Eclipse plug-in that allows you to save those settings in an XML file that you put under source control.

The plug-in periodically compares the workspace settings with the contents of that file. It notifies you in case of differences, and allows you to update your environment with a couple of clicks.

Towards a Frictionless Development Environment

Workspace Mechanic is a good example of a lubricant, a tool that lubricates the development process to reduce friction.

LubricationMy ideal is to take this to the extreme with a Frictionless Development Environment (FDE) in which all software development activities go very smoothly.

Let’s see what we would likely need to make such an FDE a reality.

In this post, I will look at a very small example that uncovers some of the basic components of an FDE.

Example: Creating the Class Under Test

In Test-Driven Development, we start out with a test and there is no class under test yet. Eclipse has a Quick Fix to create the class, but we still have to manually invoke it and select a source folder to store it in (assuming you have different source folders for main and test code).

It would be nicer if the IDE would understand what you’re trying to do and automatically create the skeleton for the class under test for you and save it in the right place.

Big DataThe crux is for the tool to understand what you are doing, or else it could easily draw the wrong conclusion and create all kinds of artifacts that you don’t want.

This kind of knowledge is highly user and potentially even project specific. It is therefore imperative that the tool collects usage data and uses that to optimize its assistance. We’re likely talking about big data here.

Given the fact that it’s expensive in terms of storage and computing power to collect and analyze these statistics, it makes sense to do this in a cloud environment.

That will also allow for quicker learning of usage patterns when working on different machines, like in the office and at home. More importantly, it allows building on usage patterns of other people.

What this example also shows, is that we’ll need many small, very focused lubricants. This makes it unlikely for one organization to provide all lubricants for an FDE that suits everybody, even for a specific language.

Open Source SoftwareThe only practical way of assembling an FDE is through a plug-in architecture for lubricants.

Building an FDE will be a huge effort. To realize it on the short term, we’ll probably need an open source model. No one company could put in the resource required to pull this off in even a couple of years.

The Essential Components of a Frictionless Development Environment

This small example uncovered the following building blocks for a Frictionless Development Environment:

  1. Cloud Computing will provide economies of scale and access from anywhere
  2. Big Data Analytics will discern usage patterns
  3. Recommendation Engines will convert usage patterns into context-aware lubricants
  4. A Plug-in architecture will allow different parties to contribute lubricants and usage analysis tools
  5. An Open Source model will allow many organizations and individuals to collaborate

What do you think?

Do you agree with the proposed components of an FDE? Did I miss something?

Please share your thoughts in the comments.


Towards a Theory of Test-Driven Development

2013-01-07

Red, Green, RefactorThis post examines how well we really understand the practice of Test-Driven Development (TDD).

Red, Green, Refactor

By now we all know that Test-Driven Development (TDD) follows a simple cycle consisting of these steps:

  1. Start by writing a test. Since there is no code, it will fail (Red)
  2. Write just enough code to make the test pass (Green)
  3. Clean up the code (Refactor)

The beauty of this division is that we can focus on one thing at a time.

Specify, Transform, Refactor

Although simple, TDD isn’t easy. To execute the TDD cycle well, we need a deeper understanding that we can only get from experience.

For instance, after doing TDD for a while we may look at the steps as:

  1. Specify new required functionality
  2. Improve the functionality while keeping the design constant
  3. Improve the design while keeping the functionality constant

When we look at the TDD cycle in this light, we see that the Green and Refactor phases are each others opposite.

Refactorings and Transformations

In the Refactor phase, we use Martin Fowler‘s refactorings to clean up the code.

TransformationRefactorings are standard alterations of the code that change its internal structure without changing its external behavior.

Now, if the Green and Refactor phases are each others opposite, then you might think that there are “opposite refactorings” as well. You would be right.

Robert Martin‘s transformations are standard alterations of the code that change its external behavior without changing its internal structure.

Automated Transformations?

Most of us use powerful IDEs to write our code. These IDEs support refactorings, which means that they can do the code alteration for you in a manner that is guaranteed to be safe.

So do we need something similar for transformations? I think not.

Some transformations are so simple in terms of the changes to code, that it wouldn’t actually save any effort to automate them. I don’t see a lot of room for improving the change from if to while, for instance.

Other transformations simply have an unspecified effect. For example, how would you automate the statement->statements transformation?

RefactoringThe crux is that refactorings keep the external behavior the same, and the tools depend on that to properly implement the refactorings. However, transformations don’t share that property.

Standardized Work

In the Specify/Transform/Refactor view of TDD, we write our programs by alternating between adding tests, applying transformations, and applying refactorings.

In other words, if we look at the evolution of our non-test code through a series of diffs, then each diff shows either a transformation or a refactoring.

It seems we are getting closer to the Lean principle of Standardized Work.

What’s still missing, however, is a deeper insight into the Red/Specify phase.

How to Write Tests

The essential part of the Red/Specify phase is obviously to write a test. But how do we do that?

For starters, how do we select the next test to implement?

Unit test failureThere is almost always more than one test to write for a given requirement.

And the order in which you introduce tests makes a difference for the implementation.

But there is very little advice on how to pick the next test, and this is sorely needed.

Kent Beck has a kata for experimenting with test order, which helps in gaining understanding. But that’s a far cry from a well-developed theory like we have for refactorings.

So what do you think? If we understood this phase better, could we come up with the test writing equivalent of transformations and refactorings?

Please share your thoughts in the comments.


TDD and the Transformation Priority Premise

2012-12-31

TransformationLast time, we looked at the Red/Green/Refactor phases of Test-Driven Development (TDD).

This time we’ll take a detailed look at the transformations applied in the Green phase.

The Transformation Priority Premise

Most of you will have heard of the refactorings we apply in the last TDD phase, but there are corresponding standardized code changes in the Green phase as well. Uncle Bob Martin named them transformations.

The Transformation Priority Premise (TPP) claims that these transformations have an inherent order, and that picking transformation that are higher on the list leads to better algorithms.

Anecdotal evidence is provided by the example of sorting, where violating the order leads to bubble sort, while the correct order leads to quicksort.

After some modifications based on posts by other people, Uncle Bob arrived at the following ordered list of transformations:

Transformation Description
{}–>nil no code at all->code that employs nil
nil->constant
constant->constant+ a simple constant to a more complex constant
constant->scalar replacing a constant with a variable or an argument
statement->statements adding more unconditional statements
unconditional->if splitting the execution path
scalar->array
array->container ??? this one is never used nor explained
statement->tail-recursion
if->while
statement->recursion
expression->function replacing an expression with a function or algorithm
variable->assignment replacing the value of a variable
case adding a case (or else) to an existing switch or if

Applying the TPP to the Roman Numerals Kata

Roman numeral symbolsReading about something gives only shallow knowledge, so let’s try out the TPP on a small, familiar problem: the Roman Numerals kata.

For those of you who are unfamiliar with it: the objective is to translate numbers into Roman. See the table at the left for an overview of the Roman symbols and their values.

As always in TDD, we start off with the simplest case:

public class RomanNumeralsTest {

  @Test
  public void arabicToRoman() {
    Assert.assertEquals("i", "i", RomanNumerals.arabicToRoman(1));
  }

}

We get this to compile with:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    return null;
  }

}

Note that we’ve already applied the first transformation on the list: {}->nil. We apply the second transformation, nil->constant, to get to green:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    return "i";
  }

}

Now we can add our second test:

public class RomanNumeralsTest {

  @Test
  public void arabicToRoman() {
    assertRoman("i", 1);
    assertRoman("ii", 2);
  }

  private void assertRoman(String roman, int arabic) {
    Assert.assertEquals(roman, roman, 
        RomanNumerals.arabicToRoman(arabic));
  }

}

The only way to make this test pass, is to introduce some conditional (unconditional->if):

  public static String arabicToRoman(int arabic) {
    if (arabic == 2) {
      return "ii";
    }
    return "i";
  }

However, this leads to duplication between the number 2 and the number of is returned. So let’s try a different sequence of transformations. Warning: I’m going into baby steps mode now.

First, do constant->scalar:

public static String arabicToRoman(int arabic) {
  String result = "i";
  return result;
}

Next, statement->statements:

public static String arabicToRoman(int arabic) {
  StringBuilder result = new StringBuilder();
  result.append("i");
  return result.toString();
}

Now we can introduce the if without duplication:

public static String arabicToRoman(int arabic) {
  StringBuilder result = new StringBuilder();
  if (arabic >= 1) {
    result.append("i");
  }
  return result.toString();
}

And then another statement->statements:

public static String arabicToRoman(int arabic) {
  StringBuilder result = new StringBuilder();
  if (arabic >= 1) {
    result.append("i");
    arabic -= 1;
  }
  return result.toString();
}

And finally we do if->while:

public static String arabicToRoman(int arabic) {
  StringBuilder result = new StringBuilder();
  while (arabic >= 1) {
    result.append("i");
    arabic -= 1;
  }
  return result.toString();
}

Our test now passes. And so does the test for 3, by the way.

With our refactoring hat on, we spot some more subtle duplication: between the number 1 and the string i. They both express the same concept (the number 1), but are different versions of it: one Arabic and one Roman.

We should introduce a class to capture this concept:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    StringBuilder result = new StringBuilder();
    RomanNumeral numeral = new RomanNumeral("i", 1);
    while (arabic >= numeral.getValue()) {
      result.append(numeral.getSymbol());
      arabic -= numeral.getValue();
    }
    return result.toString();
  }

}

public class RomanNumeral {

  private final String symbol;
  private final int value;

  public RomanNumeral(String symbol, int value) {
    this.symbol = symbol;
    this.value = value;
  }

  public int getValue() {
    return value;
  }

  public String getSymbol() {
    return symbol;
  }

}

Now it turns out that we have a case of feature envy. We can make that more obvious by extracting out a method:

public static String arabicToRoman(int arabic) {
  StringBuilder result = new StringBuilder();
  RomanNumeral numeral = new RomanNumeral("i", 1);
  arabic = append(arabic, result, numeral);
  return result.toString();
}

private static int append(int arabic, StringBuilder builder,
    RomanNumeral numeral) {
  while (arabic >= numeral.getValue()) {
    builder.append(numeral.getSymbol());
    arabic -= numeral.getValue();
  }
 return arabic;
}

Now we can move the append() method to RomanNumeral:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    StringBuilder result = new StringBuilder();
    RomanNumeral numeral = new RomanNumeral("i", 1);
    arabic = numeral.append(arabic, result);
    return result.toString();
  }

}

public class RomanNumeral {

  private final String symbol;
  private final int value;

  public RomanNumeral(String symbol, int value) {
    this.symbol = symbol;
    this.value = value;
  }

  public int getValue() {
    return value;
  }

  public String getSymbol() {
    return symbol;
  }

  public int append(int arabic, StringBuilder builder) {
    while (arabic >= getValue()) {
      builder.append(getSymbol());
      arabic -= getValue();
    }
    return arabic;
  }

}

We can further clean up by inlining the getters that are now only used in the RomanNumeral class:

public class RomanNumeral {

  private final String symbol;
  private final int value;

  public RomanNumeral(String symbol, int value) {
    this.symbol = symbol;
    this.value = value;
  }

  public int append(int arabic, StringBuilder builder) {
    while (arabic >= value) {
      builder.append(symbol);
      arabic -= value;
    }
    return arabic;
  }

}

There is one other problem with this code: we pass in arabic and builder as two separate parameters, but they are not independent. The former represents the part of the arabic number not yet processed, while the latter represents the part that is processed. So we should introduce another class to capture the shared concept:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    ArabicToRomanConversion conversion
        = new ArabicToRomanConversion(arabic);
    RomanNumeral numeral = new RomanNumeral("i", 1);
    numeral.append(conversion);
    return conversion.getResult();
  }

}

public class RomanNumeral {

  private final String symbol;
  private final int value;

  public RomanNumeral(String symbol, int value) {
    this.symbol = symbol;
    this.value = value;
  }

  public void append(ArabicToRomanConversion conversion) {
    while (conversion.getRemainder() >= value) {
      conversion.append(symbol, value);
    }
  }

}

public class ArabicToRomanConversion {

  private int remainder;
  private final StringBuilder result;

  public ArabicToRomanConversion(int arabic) {
    this.remainder = arabic;
    this.result = new StringBuilder();
  }

  public String getResult() {
    return result.toString();
  }

  public int getRemainder() {
    return remainder;
  }

  public void append(String symbol, int value) {
    result.append(symbol);
    remainder -= value;
  }

}

Unfortunately, we now have a slight case feature envy in RomanNumeral. We use conversion twice and our own members three times, so it’s not too bad, but let’s think about this for a moment.

Does it make sense to let the roman numeral know about an conversion process from Arabic to Roman? I think not, so let’s move the code to the proper place:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    ArabicToRomanConversion conversion
        = new ArabicToRomanConversion(arabic);
    RomanNumeral numeral = new RomanNumeral("i", 1);
    conversion.process(numeral);
    return conversion.getResult();
  }

}

public class RomanNumeral {

  private final String symbol;
  private final int value;

  public RomanNumeral(String symbol, int value) {
    this.symbol = symbol;
    this.value = value;
  }

  public String getSymbol() {
    return symbol;
  }

  public int getValue() {
    return value;
  }

}

public class ArabicToRomanConversion {

  private int remainder;
  private final StringBuilder result;

  public ArabicToRomanConversion(int arabic) {
    this.remainder = arabic;
    this.result = new StringBuilder();
  }

  public String getResult() {
    return result.toString();
  }

  public void process(RomanNumeral numeral) {
    while (remainder >= numeral.getValue()) {
      append(numeral.getSymbol(), numeral.getValue());
    }
  }

  private void append(String symbol, int value) {
    result.append(symbol);
    remainder -= value;
  }

}

We had to re-introduce the getters for RomanNumeral‘s fields to get this to compile. We could have avoided that rework by introducing the ArabicToRomanConversion class first. Hmm, maybe refactorings have an inherent order too!

OK, on to our next test: 4. We can make that pass with another series of transformations. First, scalar->array:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("i", 1)
  };
  conversion.process(numerals[0]);
  return conversion.getResult();
}

Next, constant->scalar:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("i", 1)
  };
  int index = 0;
  conversion.process(numerals[index]);
  return conversion.getResult();
}

Now we need an if:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("i", 1)
  };
  int index = 0;
  if (index < 1) {
    conversion.process(numerals[index]);
  }
  return conversion.getResult();
}

And another constant->scalar:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("i", 1)
  };
  int index = 0;
  if (index < numerals.length) {
    conversion.process(numerals[index]);
  }
  return conversion.getResult();
}

You can probably see where this is going. Next is statement->statements:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("i", 1)
  };
  int index = 0;
  if (index < numerals.length) {
    conversion.process(numerals[index]);
    index++;
  }
  return conversion.getResult();
}

Then if->while:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("i", 1)
  };
  for (RomanNumeral numeral : numerals) {
    conversion.process(numeral);
  }
  return conversion.getResult();
}

And finally constant->constant+:

public static String arabicToRoman(int arabic) {
  ArabicToRomanConversion conversion
      = new ArabicToRomanConversion(arabic);
  RomanNumeral[] numerals = new RomanNumeral[] {
    new RomanNumeral("iv", 4),
    new RomanNumeral("i", 1)
  };
  for (RomanNumeral numeral : numerals) {
    conversion.process(numeral);
  }
  return conversion.getResult();
}

Now we have our algorithm complete and all we need to do is add to the numerals array. BTW, this should be a constant:

public class RomanNumerals {

  private static final RomanNumeral[] ROMAN_NUMERALS 
      = new RomanNumeral[] {
    new RomanNumeral("iv", 4),
    new RomanNumeral("i", 1)
  };

  public static String arabicToRoman(int arabic) {
    ArabicToRomanConversion conversion
        = new ArabicToRomanConversion(arabic);
    for (RomanNumeral romanNumeral : ROMAN_NUMERALS) {
      conversion.process(romanNumeral);
    }
    return conversion.getResult();
  }

}

Also, it looks like we have another case of feature envy here that we could resolve as follows:

public class RomanNumerals {

  public static String arabicToRoman(int arabic) {
    return new ArabicToRomanConversion(arabic).getResult();
  }

}

public class ArabicToRomanConversion {

  private static final RomanNumeral[] ROMAN_NUMERALS 
      = new RomanNumeral[] {
    new RomanNumeral("iv", 4),
    new RomanNumeral("i", 1)
  };

  private int remainder;
  private final StringBuilder result;

  public ArabicToRomanConversion(int arabic) {
    this.remainder = arabic;
    this.result = new StringBuilder();
  }

  public String getResult() {
    for (RomanNumeral romanNumeral : ROMAN_NUMERALS) {
      process(romanNumeral);
    }
    return result.toString();
  }

  private void process(RomanNumeral numeral) {
    while (remainder >= numeral.getValue()) {
      append(numeral.getSymbol(), numeral.getValue());
    }
  }

  private void append(String symbol, int value) {
    result.append(symbol);
    remainder -= value;
  }

}

Retrospective

Magnifying glassThe first thing I noticed, is that following the TPP led me to discover the basic algorithm a lot quicker than in some of my earlier attempts at this kata.

The next interesting thing is that there seems to be an interplay between transformations and refactorings.

You can either perform a transformation and then clean up with refactorings, or prevent the need to refactor by using only transformations that don’t introduce duplication. Doing the latter is more efficient and also seems to speed up discovery of the required algorithm.

Certainly food for thought. It seems like some more experimentation is in order.

Update: Here is a screencast of a slightly better version of the kata:


The Differences Between Test-First Programming and Test-Driven Development

2012-12-24

Red, Green, RefactorThere seems to be some confusion between Test-First Programming and Test-Driven Development (TDD).

This post explains that merely writing the tests before the code doesn’t necessarily make it TDD.

Similarities Between Test-First Programming and Test-Driven Development

It’s not hard to see why people would confuse the two, since they have many things in common.

My classification of tests distinguishes six dimensions: who, what, when, where, why, and how.

Test-First programming and Test-Driven Development score the same in five of those six dimensions: they are both automated (how) functional (what) programmer (who) tests at the unit level (where) written before the code (when).

The only difference is in why they are written.

Differences Between Test-First Programming and Test-Driven Development

Test-First Programming mandates that tests be written before the code, so that the code will always be testable. This is more efficient than having to change already written code to make it testable.

Test-First Programming doesn’t say anything about other activities in the development cycle, like requirements analysis and design.

This is a big difference with Test-Driven Development (TDD), since in TDD, the tests drive the design. Let’s take a detailed look at the TDD process of Red/Green/Refactor, to find out exactly how that differs from Test-First Programming.

Red

Unit test failureIn the first TDD phase we write a test. Since there is no code yet to make the test pass, this test will fail.

Unit testing frameworks like JUnit will show the result in red to indicate failure.

In both Test-First Programming and Test-Driven Development, we use this phase to record a requirement as a test.

TDD, however, goes a step further: we also explicitly design the client API. Test-First Programming is silent on how and when we should do that.

Green

In the next phase, we write code to make the test pass. Unit testing frameworks show passing tests in green.

In Test-Driven Development, we always write the simplest possible code that makes the test pass. This allows us to keep our options open and evolve the design.

JUnit passing testWe may evolve our code using simple transformations to increase the complexity of the code enough to satisfy the requirements that are expressed in the tests.

Test-First Programming is silent on what sort of code you write in this phase and how you do it, as long as the test will pass.

Refactor

In the final TDD phase, the code is refactored to improve the design of the implementation.

This phase is completely absent in Test-First Programming.

Summary of Differences

So we’ve uncovered two differences that distinguish Test-First Programming from Test-Driven Development:

  1. Test-Driven Development uses the Red phase to design the client API. Test-First Programming is silent on when and how you arrive at a good client API.
  2. Test-Driven Development splits the coding phase into two compared to Test-First Programming. In the first sub-phase (Green), the focus is on meeting the requirements. In the second sub-phase (Refactor), the focus is on creating a good design.

I think there is a lot of value in the second point. Many developers focus too much on getting the requirements implemented and forget to clean up their code. The result is an accumulation of technical debt that will slow development down over time.

TDD also splits the design activity into two. First we design the external face of the code, i.e. the API. Then we design the internal organization of the code.

This is a useful distinction as well, because the heuristics you would use to tell a good API from a bad one are different from those for good internal design.

Try Before You Buy

KataAll in all I think Test-Driven Development provides sufficient value over Test-First Programming to give it a try.

All new things are hard, however, so be sure to practice TDD before you start applying it in the wild.

There are numerous katas that can help you with that, like the Roman Numerals Kata.


Building Both Security and Quality In

2012-10-29

One of the important things in a Security Development Lifecycle (SDL) is to feed back information about vulnerabilities to developers.

This post relates that practice to the Agile practice of No Bugs.

The Security Incident Response

Even though we work hard to ship our software without security vulnerabilities, we never succeed 100%.

When an incident is reported (hopefully responsibly), we execute our security response plan. We must be careful to fix the issue without introducing new problems.

Next, we should also look for similar issues to the one reported. It’s not unlikely that there are issues in other parts of the application that are similar to the reported one. We should find and fix those as part of the same security update.

Finally, we should do a root cause analysis to determine why this weakness slipped through the cracks in the first place. Armed with that knowledge, we can adapt our process to make sure that similar issues will not occur in the future.

From Security To Quality

The process outlined above works well for making our software ever more secure.

But security weaknesses are essentially just bugs. Security issues may have more severe consequences than regular bugs, but most regular bugs are expensive to fix once the software is deployed as well.

So it actually makes sense to treat all bugs, security or otherwise, the same way.

As the saying goes, an ounce of prevention is worth a pound of cure. Just as we need to build security in, we also need to build quality in general in.

Building Quality In Using Agile Methods

This has been known in the Agile and Lean communities for a long time. For instance, James Shore wrote about it in his excellent book The Art Of Agile Development and Elisabeth Hendrickson thinks that there should be so little bugs that they don’t need triaging.

Some people object to the Zero Defects mentality, claiming that it’s unrealistic.

There is, however, clear evidence of much lower defect rates for Agile development teams. Many Lean implementations also report successes in their quest for Zero Defects.

So there is at least anecdotal evidence that a very significant reduction of defects is possible.

This will require change, of course. Testers need to change and so do developers. And then everybody on the team needs to speak the same language and work together as a single team instead of in silos.

If we do this well, we’ll become bug exterminators that delight our customers with software that actually works.


Software Development and Lifelong Learning

2012-09-10

The main constraint in software development is learning. This means that learning is a core skill for developers and we should not think we’re done learning after graduation. This post explores some different ways in which to learn.

Go To Conferences

Conferences are a great place to learn new things, but also to meet new people. New people can provide new ways of looking at things, which helps with learning as well.

You can either go to big and broad conferences, like Java One or the RSA conference, or you can attend a smaller, more focused event. Some of these smaller events may not be as well-known, but there are some real gems nonetheless.

Take XML Amsterdam, for example, a small conference here in the Netherlands with excellent international speakers and attendees (even some famous ones).

Attend Workshops

Learning is as much about doing as it is about hearing and watching. Some conferences may have hands-on sessions or labs, but they’re in the minority. So just going to conferences isn’t good enough.

A more practical variant are workshops. They are mostly organized by specific communities, like Java User Groups.

One particularly useful form for developers is the code retreat. Workshops are much more focused than conferences and still provide some of the same networking opportunities.

Get Formal Training

Lots of courses are being offered, many of them conveniently online. One great (and free) example is Cryptography from Coursera.

Some of these course lead to certifications. The world is sharply divided into those who think certifications are a must and those that feel they are evil. I’ll keep my opinion on this subject to myself for once ;) but whatever you do, focus on the learning, not on the piece of paper.

Learn On The Job

There is a lot to be learned during regular work activities as well.

You can organize that a bit better by doing something like job rotation. Good forms of job rotation for developers are collective code ownership and swarming.

Pair programming is an excellent way to learn all kinds of things, from IDE shortcuts to design patterns.

Practice in Private

Work has many distractions, though, like Getting a Story Done.

Open source is an alternative, in the sense that it takes things like deadlines away, which can help with learning.

However, that still doesn’t provide the systematic exploration that is required for the best learning. So practicing on “toy problems” is much more effective.

There are many katas that do just that, like the Roman Numerals Kata. They usually target a specific skill, like Test-Driven Development (TDD).


A Classification of Tests

2012-09-03

There are many ways of testing software. This post uses the five Ws to classify the different types of tests and shows how to use this classification.

Programmer vs Customer (Who)

Tests exist to give confidence that the software works as expected.

But whose expectations are we talking about? Developers have different types of expectations about their code than users have about the application. Each audience deserves its own set of tests to remain confident enough to keep going.

Functionality vs Performance vs Load vs Security (What)

When not specified, it’s assumed that what is being tested is whether the application functions the way it’s supposed to. However, we can also test non-functional aspects of an application, like security.

Before Writing Code vs After (When)

Tests can be written after the code is complete to verify that it works (test-last), or they can be written first to specify how the code should work (test-first). Writing the test first may seem counter-intuitive or unnatural, but there are some advantages:

  • When you write the tests first, you’ll guarantee that the code you later write will be testable (duh). Anybody who’s written tests for legacy code will surely acknowledge that that’s not a given if you write the code first
  • Writing the tests first can prevent defects from entering the code and that is more efficient than introducing, finding, and then fixing bugs
  • Writing the tests first makes it possible for the tests to drive the design. By formulating your test, in code, in a way that looks natural, you design an API that is convenient to use. You can even design the implementation

Unit vs Integration vs System (Where)


Tests can be written at different levels of abstraction. Unit tests test a single unit (e.g. class) in isolation.

Integration tests focus on how the units work together. System tests look at the application as a whole.

As you move up the abstraction level from unit to system, you require fewer tests.

Verification vs Specification vs Design (Why)

There can be different reasons for writing tests. All tests verify that the code works as expected, but some tests can start their lives as specifications of how yet-to-be-written code should work. In the latter situation, the tests can be an important tool for communicating how the application should behave.

We can even go a step further and let the tests also drive how the code should be organized. This is called Test-Driven Design (TDD).

Manual vs Automated Tests (How)


Tests can be performed by a human or by a computer program. Manual testing is most useful in the form of exploratory testing.

When you ship the same application multiple times, like with releases of a product or sprints of an Agile project, you should automate your tests to catch regressions. The amount of software you ship will continue to grow as you add features and your testing effort will do so as well. If you don’t automate your tests, you will eventually run out of time to perform all of them.

Specifying Tests Using the Classification

With the above classifications we can be very specific about our tests. For instance:

  • Tests in TDD are automated (how) programmer (who) tests that design (why) functionality (what) at the unit or integration level (where) before the code is written (when)
  • BDD scenarios are automated (how) customer (who) tests that specify (why) functionality (what) at the system level (where) before the code is written (when)
  • Exploratory tests are manual (how) customer (who) tests that verify (why) functionality (what) at the system level (where) after the code is written (when)
  • Security tests are automated (how) customer (who) tests that verify (why) security (what) at the system level (where) after the code is written (when)

By being specific, we can avoid semantic diffusion, like when people claim that “tests in TDD do not necessarily need to be written before the code”.

Reducing Risk Using the Classification

Sometimes you can select a single alternative along a dimension. For instance, you could perform all your testing manually, or you could use tests exclusively to verify.

For other dimensions, you really need to cover all the options. For instance, you need tests at the unit and integration and system level and you need to test for functionality and performance and security. If you don’t, you are at risk of not knowing that your application is flawed.

Proper risk management, therefore, mandates that you shouldn’t exclusively rely on one type of tests. For instance, TDD is great, but it doesn’t give the customer any confidence. You should carefully select a range of test types to cover all aspects that are relevant for your situation.


Follow

Get every new post delivered to your Inbox.

Join 272 other followers