implements Elegance {

// Elwyn Malethan's musings on software development, mountain biking and general navel–gazing...

Articles tagged with 'software'

Practical TDD: It's iterative

I employ TDD/BDD as my software development methodology. I have found over the years that it is the best way of producing good quality, well-designed code. Not only that, when employed with a customer willing to work iteratively, I have found it to be one of the most reliable ways of delivering software that fulfills the requirements the customer actually has and not what they thought the requirements were at the beginning of the project. So, as far as I‘m concerned, all the evidence I‘ve seen and experienced so far suggests to me that TDD/BDD produces the best results.

Not everyone agrees.

There are developers I respect that disagree, people who I‘ve spoken to, people who‘s writings I read and people I‘ve worked with who just do not see the benefits of TDD. And some that are cynical about software craftsmanship in general. None have made arguments that have convinced me that they‘re right to doubt the benefits of TDD. However, I‘m not a fundamentalist, I‘ll listen to arguments and enjoy having my views challenged.

One thing I‘ve noticed about many (not all) of the people that are skeptical of TDD is that they don‘t really understand what it is. A common misconception is that tests are written in advance of production code; that test code for a component is completed before the implementation. It is perhaps not surprising given TDD is referred to as a test-first methodology.

This is not at all how it has worked for me and not my understanding of what the literature on the matter describes. It is certainly not an approach for TDD I would advocate or advise.

As a result of the misunderstanding, some developers find a real barrier to being productive when trying to implement new code employing TDD. This article isn‘t written with the intention of making an argument in favour of TDD or to envangelise in anyway. Hopefully, it will go some way to address the barrier that some experience.

TDD is iterative

To illustrate the iterative nature of TDD I‘m going to write some code. The process whereby this problem will be solved will illustrate how TDD helps us understand the problem and how our solution will evolve. It will also illustrate how the resulting solution can more easily be refactored to provide a cleaner solution by relying on the tests to verify the correctness of the refactored code.

The simple problem in question is – bizarrely – based on an actual requirement I had to fulfil not so long ago. It is to place a sequence of integers from 1 to n , where n is an odd number, in a collection ordered such that 1 is placed in the middle with each successive number placed either side of 1, alternating from left to right.

Getting started

I know it‘s meant to be test–first, but I usually start with a class and a method representing the implementation. I‘m not too worried about the name of either the class or the method at the moment, nor the signature of the method. I can always change these as I go along and further clarify the requirements, usually multiple times.

class MiddleOutNumberSorter {
  def sort(count: Int): List[Int] = {
    return Nil
  }
}

Oh, and by the way, this is going to be Scala. I develop in Java for a living, Scala I develop to keep me sane. Writing software in Scala reminds me why I got into software development in the first place. The requirements are represented by Specs specifications.

So, now we get to the TDD bit, the very next thing we do is create a testcase (or a specification) with stubbed tests (or examples) for the requirements we know of or can easily deduce.

object MiddleOutNumberSorterSpec extends Specification with ScalaCheck {

  "MiddleOutNumberSorter" should {
    "reject even number counts and throw an exception" in { }
    "return a list of the same length as the count specified" in { }
    "produce 3 numbers in the correct order" in { }
    "produce 5 numbers in the correct order" in { }
    "produce 15 numbers in the correct order" in { }
  }  
}

One of the things I like about Specs is the fact that empty examples (or more specifically examples without assertions) will manifest as skipped tests and not passing ones.

One thing you will find when producing these test stubs for business requirements is that it will prompt you to think about what the requirements you have actually mean. If, in fact, you can‘t think of any tests to write it is probably an indication that you either don‘t understand the requirements or they are not well defined. Either way it means that you need to go back to the customer to clarify the requirements.

Fail, fix, repeat

Because they are stubs, none of these examples actually break our code. We‘re going to change that. So let‘s write a test that represents a real requirement. This will cause our (currently minimal) implementation to fail.

object MiddleOutNumberSorterSpec extends Specification with ScalaCheck {

  var sorter: MiddleOutNumberSorter = _

  "MiddleOutNumberSorter" should {
    doBefore {
      sorter = new MiddleOutNumberSorter
    }
    "reject even number counts and throw an exception" in {
      Array(0,2,4,10,212).foreach {=>
        sorter.sort(i) must throwA[IllegalArgumentException]
      }
    }
    "return a list of the same length as the count specified" in { }
    "produce 3 numbers in the correct order" in { }
    "produce 5 numbers in the correct order" in { }
    "produce 15 numbers in the correct order" in { }
  }  
}

Ok,now the production code fails the test criteria so we turn our attention to the implementation to address this. We do the bare minimum possible to achieve this. So the implementation now looks like this:

class MiddleOutNumberSorter {
  def sort(count: Int): List[Int] = {
    if(count % 2 == 0) {
      throw new IllegalArgumentException("sort only accepts odd numbers")
    } else {
      return Nil
    }
  }
}

“Hang on“, I hear you say “that‘s not going to sort our numbers, not even close”. You‘re right. All this does is satisfy the one requirement for which we have an example.

This is obviously no more correct than it was before in terms of satisfying the end requirements. This is a bit of a difficult concept to get over for non TDDers. Someone on the #java IRC channel once sarcastically said words to the effect:

TDD is wasting time writing code I know is wrong all day.

What it does mean is we can quickly get back to specifying more of our requirements by filling out another stub. So, on to the next example.

object MiddleOutNumberSorterSpec extends Specification with ScalaCheck {

  var sorter: MiddleOutNumberSorter = _

  "MiddleOutNumberSorter" should {
    doBefore {
      sorter = new MiddleOutNumberSorter
    }
    "reject even number counts and throw an exception" in {
      Array(0,2,4,10,212).foreach {=>
        sorter.sort(i) must throwA[IllegalArgumentException]
      }
    }
    "return a list of the same length as the count specified" in {
      Array(1,3,9,15,133).foreach {=>
        sorter.sort(i).size must_== i
      }
    }
    "produce 3 numbers in the correct order" in { }
    "produce 5 numbers in the correct order" in { }
    "produce 15 numbers in the correct order" in { }
  }  
}

And back to the implementation, again doing the bare-minimum we can to satisfy the requirement.

class MiddleOutNumberSorter {
  def sort(count: Int): List[Int] = {
    if(count % 2 == 0) {
      throw new IllegalArgumentException("sort only accepts odd numbers")
    } else {
      return (0 until count).toList
    }
  }
}

And so we continue until we have an implementation…

class MiddleOutNumberSorter {
  def sort(count: Int): List[Int] = {
    if (count % 2 == 0) {
      throw new IllegalArgumentException("sort only accepts odd numbers")
    } else {
      val numbers = new Array[Int](count);
      for (number <- 1 to count) {
        val index = if (number == 1) {
          count/2
        } else if(number % 2 == 0) {
          (count/2) + (number/2)
        } else {
          (count/2) - (number/2)
        }

        numbers(index) = number
      }
      return numbers.toList
    }
  }
}

...that satisfies the entire specification:

object MiddleOutNumberSorterSpec extends Specification with ScalaCheck {

  var sorter: MiddleOutNumberSorter = _

  "MiddleOutNumberSorter" should {
    doBefore {
      sorter = new MiddleOutNumberSorter
    }
    "reject even number counts and throw an exception" in {
      Array(0,2,4,10,212).foreach {=>
        sorter.sort(i) must throwA[IllegalArgumentException]
      }
    }
    "return a list of the same length as the count specified" in {
      Array(1,3,9,15,133).foreach {=>
        sorter.sort(i).size must_== i
      }
    }
    "produce 3 numbers in the correct order" in {
      sorter.sort(3) must containInOrder(List(3,1,2))
    }
    "produce 5 numbers in the correct order" in {
      sorter.sort(5) must containInOrder(List(5,3,1,2,4))
    }
    "produce 15 numbers in the correct order" in {
      sorter.sort(15) must containInOrder(List(15,13,11,9,7,5,3,1,2,4,6,8,10,12,14))
    }
  }  
}

Now I commit this code to version control. It‘s complete, and I can be confident that it fulfills the requirements because they are codified in unit tests. Not only that, those tests provide an invaluable resource to other developers working on the project. They now have code that when run generates a report that tells them in plain English what it should and should not do.

Being a an aspiring software craftsman I want all the code I write to be maintainable, with minimal technical debt. I want it to be sustainable in the longer term, so I'm not entirely happy with the implementation. In general, I want to reduce the number of WTFs per minute to a minimum.

The benefit

Also, I‘m becoming less keen on the imperative style and Programming in Scala promotes a more functional style using immutable objects. The thing is, having been mainly a developer of imperative languages, I'm still learning to code in a functional style – particularly in Scala – and so it doesn't quite come naturally to me yet. Without test coverage, this would be a risky approach. In my attempt to produce more functional code, I might cause it to produce incorrect results. Because I‘m new to FP, I might not really understand why. However, because I produced the test coverage as I wrote the production implementation I have a safety net which tells me instantly if I broke anything.

So after a bit of experimentation and copious runs of the specifications, I end up with the following, far more elegant solution. Although elegance is somewhat in the eye of the beholder, so I don‘t expect everyone to agree :)

class MiddleOutNumberSorter {
  def sort(count: Int): List[Int] = {
    if (isOddNumber(count)) (0 until count).map(=> numberAtIndex(i, count)).toList
    else throw new IllegalArgumentException("sort only accepts odd numbers")
  }

  private def isOddNumber(count: Int): Boolean = {
    count % 2 != 0
  }

  private def numberAtIndex(index: Int, count: Int) = {
    val middleIndex: Int = count / 2
    if (index > middleIndex) {
      2 * (index - middleIndex)
    } else {
      2 * (middleIndex - index) + 1
    }
  }
}

As I mentioned, I'm currently learning Scala and FP, so I'd love it anyone reading this could suggest alternative implementations.

First published on Dec 10, 2010. Last updated on: Dec 10, 2010.

Human fallibility: A simple argument in favour of unit testing

I was asked about TDD recently. The discussion was in the context of the polarised views with regard TDD and even unit testing. The conversation reminded me a little of the response to a post I wrote a while back.

One argument against TDD and (possibly unit testing in general) that I had in a comment to my Why TDD: Freedom to refactor post, in particular response to the assertion that unit tests evaluate the correctness of features, was the following:

You can do that with your brain and your own powers of reasoning…

The essence of this argument, as I understand it, was expressed to me recently. Essentially that – as a ‘good developer‘ – you can evaluate your code and your software yourself without the need for unit tests.

Obviously I don‘t agree.

This kind of evaluation of the correctness of your code requires constant or repeated human attention. My argument against this point of view is simple:

Human beings are fallible, they get tired, they get lazy, they get ill and they get run over by buses. Tests that rely on human beings to attend to those tests or require a human's expertise to verify the correctness of those tests are therefore not as reliable, reusable or portable as tests that can run unattended and verify their own success.

First published on Oct 28, 2010. Last updated on: Nov 16, 2010.

Zed Shaw: There Are No Famous Programmers

From There Are No Famous Programmers.


[T]oday I was offered a system administrator job, again. It was very humbling to say the least. It kind of knocked me out to have someone think through all the things their company needs and the only thing they could think I'd be good at was system administration.

Yep, just a system administrator. Still.

I still have to do programmer interviews like everyone else. No matter how much code I put out, I still have to solve stupid puzzles about coconuts and manholes. No matter how many web servers or email frameworks or database servers or chat servers or assemblers I write I still have to prove I can code. No matter how many copies of my software get deployed I still have to prove I can make reliable software..

If, with his obvious achievements. Zed Shaw gets mistaken for someone to do sysadmin work, maybe much less accomplished guys like me should just give up.

Having said that, what‘s the alternative?

First published on Jun 9, 2010. Last updated on: Jun 9, 2010.

Comments have no place in code

The view I have developed about code comments is informed by years at the coal face of software development and re-enforced by what @unclebobmartin has to say about them in Clean Code, which I read last year and continue to refer back to.

Stale comments lie.

In my opinion, commenting code is the laziest way of expressing intention or semantics in code. Why? Because comments are not code. They are not an integral part of the running program. They are not bound by syntax or verified by unit testing.

This wouldn‘t be so bad if that code were never subsequently to be changed. Code is always changing. As agilists we‘re always tweaking and refactoring our code to reflect new or changing requirements. It‘s all part of embracing change. There is therefore nothing (beyond attentive human inspection) to prevent comments from becoming stale, detached or irrelevant. Stale, detached or irrelevant comments are worse than no comments. Opaque code with no comments merely obfuscates, opaque code with stale comments lies.

So what‘s the alternative? It is is more–or–less accepted wisdom now that if it is unclear what some code does (or at least what is intended) by reading it then it should be refactored so that it is clear, elegant and expressive. Rename methods, extract unclear code into new methods and classes, reduce responsibilities.

This is all expressed more elegantly and in far more detail by Bob Martin in Clean Code and countless other books and articles that predate it.

What about the ‘why‘

Jeff Atwood has already contributed his tuppence worth on this subject and I agree with a lot of he says. Apart one from one thing, he says that comments have a legitimate role in describing to the reader the reasoning behind decisions to implement a solution in a given way. I agree with him that it is difficult to express this information in code alone and that an additional enrichment of the code is required. I think he‘s wrong to say that comments are the best way to achieve this.

Version history and meaningful commit messages can better and more reliably express the reasons behind decisions made. I do this all the time in my day to day coding. If I see some code that I fail to understand the reasoning behind I look up the history of the file and read the commit messages. The code base I‘m currently working with has undergone a lot of change and few of the many comments that litter the code are of any use. Luckily, there is a wealth of information in the Subversion commits, some with references to issues in JIRA.

The commit messages are persistent and reliably associated with lines of code. They don‘t become stale and they don‘t become detached from their context.

Ugly

So there it is. Code comments are bad, they‘re ugly and they create more trouble than their worth.

More than anything, it‘s the ugliness I take most issue with.

If you‘re into software craftsmanship, then you‘re interested in elegant and (dare I say it) beautiful code. You‘re interested in code that tells a story, that expresses to the reader your intention. Essentially, this is code that documents itself. This sort of code does not need comments.

First published on May 19, 2010. Last updated on: May 19, 2010.

If the users aren't motivated to help with software development...

If the users aren't motivated to help with software development by answering questions or reviewing the results of a sprint, then the software isn't creating any value. Stop work now and find something the users really want. — Stephen Lott

The number of times I‘ve wanted to do that! Some customers I‘ve worked with are really averse to engaging in the development process. However, it makes those customers that do eagerly engage in the process all the more a delight to work with. Guess which customers had most success with their projects…

First published on Feb 24, 2010. Last updated on: Feb 24, 2010.

Why TDD: Freedom to refactor

Dave and Jen work for software companies that produce exciting and clever software products. Both Dave and Jen understand design patterns. They both know about encapsulation, inheritance, composition and all that jazz. They‘re both good software developers. The only difference is that Jen has taught herself to develop test-driven. Dave thinks his time is better spent writing production code.

Dave and Jen started their current project at the same time. Dave spent some time up–front to carefully design his architecture to the requirements and implemented all the features. Dave added some unit tests, but only as an afterthought. Dave is confident that his architecture is well suited to providing a maintainable solution to the requirements specified. Dave is OK with the fact he has less than 25% test coverage.

Jen used TDD, she started off by adding some tests to represent some of the requirements and implementing production code to pass those tests. She did this iteratively and in small digestible chunks until she had satisfied all of the requirements. Jen changed the architecture of the project frequently. She refactored as she went, renaming classes and methods so that they better represented their function, moving code and responsibilities between classes to continually improve the design of the software. She did this safe in the knowledge that her tests would pick up any errors introduced. Jen builds up test coverage of over 80%.

As always happens in a software project, the requirements change. In Dave‘s case, management decided that – for political reasons – Dave needed to completely change the way his product worked. Jen‘s customer‘s priorities changed. To stay competitive, they needed to have different features. In both cases – as ever – time was short and deadlines were tight.

Both Jen and Dave found that their existing design wasn‘t really suited to the new requirements. Dave was concerned about the deadline and the time it would take to evaluate the correctness of any change he made in the design of the software. He has no timely or repeatable means of doing this. So Dave added the feature without changing the design. He knew it wasn‘t ideal and that the he would need to change the architecture at some point in the future. Given the time constraints, Dave believed he was being pragmatic.

Jen – on the other hand – had been refactoring the code extensively throughout the development of the project. Jen recognises an alternative design that would elegantly accommodate this new requirement and refactors. She runs the tests to see if she broke something. A few of them fail. So she fixes them and runs the tests again, safe in the knowledge that her tests represent the requirements.

Over the next few years a number of new features are requested by both Dave‘s and Jen‘s customer(s). Each time, Dave opts for the change that least affects the existing code-base. Each time Dave‘ project becomes more like a ball of mud. Dave becomes more and more resistant to new features. He started asking management for some “engineering time“ to refactor the code and have the time to evaluate it‘s correctness. This – he knew – would take some time.

In the same time, Jen‘s team have been doing nothing but implementing new features. The design of the software has changed frequently in that time. She doesn‘t need any “engineering time”, her tests evaluate the correctness of every feature she adds.

Dave has become scared of changing the code, he doesn‘t know what he might break.

Jen breaks the code every day. She‘ll know in seconds what she‘s broken and what she needs to do to fix it.

The design patterns link above is an affiliate link. Any and every software developer should have a well–thumbed copy of this book.

First published on Jan 10, 2010. Last updated on: Oct 28, 2010.

Software Engineering is not dead

Software Engineering is not dead, it has never lived! There has been a lot of controversy about that, some people arguing it's more of an art, others saying it's science, yet others that it's more like craftsmanship. At the end of the day, the only fact that there is a controversy shows that it doesn't fit in the hole we're trying to stick it into. And yes, this blog is called Software Artist, but I have to tell you that it's more as a reaction to those who are fiercely trying to 'industrialize IT processes'. Oh man, that makes me angry! In the end, I have to say that the more I think about it, the more I like the idea of craftsmanship.

Because craftsmanship is all about finding the right balance between the techniques that you've learnt and the intuition that you have about what you're doing. And finding this right balance requires a lot of practice.

First published on Jul 23, 2009. Last updated on: Dec 29, 2009.