Bonny Code

Just Say 'No' To Boring Code

Thanks to Casey for inspiring this entry and for fighting the good fight against boring code.

I have extreme feelings of dissatisfaction with boring code. Boring code is monotonous and has a low signal to noise ratio. It takes a long time to communicate anything of value in boring code because most lines are dedicated to boilerplate. For example, according to a recent statistic I just made up, 95% of Java code consists of the following:

int getBlah() {
   return blah;
}

void setBlah(int blah) {
   this.blah = blah;
}

String getSoBored() {
   return soBored;
}

void setSoBored(String soBored) {
  this.soBored = soBored;
}

In Ruby, this looks like:

attr_accessor :blah, :soBored

If you are using Java, a certain amount of this type of verbose code is just necessary. The far more insidious problem is that it trains developers into believing this type of pattern is a good thing. The best thing about Ruby and the functional programming communities is that the first point emphasized is that code should be concise and full of meat and if it isn't you aren't thinking hard enough. For instance, the Hello World program in Ruby:

puts "Hello, World!"

Straight to the point. In Java, you are already entering the world of boilerplate thinking (from http://java.sun.com/docs/books/tutorial/getStarted/application/index.html):

/**
 * The HelloWorldApp class implements an application that
 * simply displays "Hello World!" to the standard output.
 */
class HelloWorldApp {
    public static void main(String[] args) {
        System.out.println("Hello World!"); //Display the string.
    }
}

My knock here is not actually that Ruby or functional languages magically eliminate all boilerplate code, or that this extra overhead in these cases is really that painful. You can easily use a code generator for both of these situations and if this is the only boilerplate in your code you are doing pretty good. The problem is the Java community teaches its developers that all of their code should be at about this level of signal to noise.

This is not an argument about aesthetics. Boring code is bad for precisely three reasons:

1. Boring code is boring to read. Code is read far more than it is written. This means programmers coming in to modify your code are more likely to miss something and then make the wrong code change.
2. Boring code is boring to review. A critical quality insurance practice in professional software development is code reviews. But boring code is boring to review. If you write boring code, it is far more likely the reviewer will just go 'yadda yadda yadda looks good to me' and gloss over mistakes.
3. Boring code is boring. Come on, do you really want to be boring?

The only thing worse than boring code is magic code. Stick to the exciting, elegant, simple code, believe me you'll sleep better at night if you do.

A Timeless Way of Coding

What do the Macarena and the GoF's Design Patterns have in common? They all became wildly popular in 1995 but in the following years experienced a large backlash from overexposure. There have been many arguments against Design Patterns as practiced in software development, chiefly that they promote over complicated code and that they really just paper over what should be seen as core deficiencies in mainstream object oriented languages.

The original inspiration for the design patterns movement in software was Christopher Alexander's The Timeless Way of Building. Somewhere from Christopher Alexander's original text to the software industries adoption of design patterns, the very soul of his meaning was left behind. Envision for a moment the stereotype of a hard core believer in software design patterns. Their code is designed all up front using UML and every class is named after one design pattern or another. Often times even multiple of them, such as the FooBarStrategyAbstractFactorySingleton. They create big inorganic design, suitable to distinct job roles where one can be an architect that doesn't design, a designer who doesn't code and a coder who does as they are told.

The spirit of the timeless way of building is to evoke that quality without a name. Some buildings are just lifeless, cold and somehow wrong. Others have a cozy liveliness that just feels comfortable and right. To obtain the quality without a name, one has to look at how such designs organically arose and the micro principles and patterns they applied. The centrally planned cities, strip malls and cookie cutter suburban homes all lack this organically grown life.

This is the central problem with how design patterns are applied. They are used to dictate design as if it is something that can be rigorously planned rather than something that is adapted to. This is why so much enterprise code feels so bloated and lifeless, it lacks the natural flow and elegance of code that is designed to fit a specific problem rather than trying to fit into a wholly consistent uber architecture.

How does a developer achieve that quality without a name though? No one intends to write bad lifeless code and please don't tell me this is just a problem for bad programmers. I'd like to know what people think. Have you ever written code that was great and had that quality without a name, not only at initial conception, but actually got better and more elegant over time? How did you do it? What about code that you thought was great at first, but quickly showed its age as the requirements shifted? What went wrong?

Building Software Over Time

When choosing a sorting algorithm, we typically don't look at how fast it performs on collections of 10 items. What matters is how it scales with the size of the input data. It is very easy to get sidetracked by success on toy problems or to concentrate on optimizing the wrong thing. Sure you can spend days optimizing the inner loop of your sort, but if you're using an n squared algorithm you're not using your time efficiently.

The same lessons apply in software design. Programmers most often optimize around initial construction time. If you have a fixed set of requirements for the lifetime of the software than that is indeed a suitable area to optimize and in that case you are probably working for a government contractor of some sort. For the rest of us, you are almost certainly working in an area with changing requirements and code that will be read and reworked many times through its life. Much more time will be spent on the maintenance side, an often cited number is greater than 90% of time is spent on maintenance. People's first response to this is that it is a code quality issue, but that is largely untrue from my experience. The reason maintenance absorbs so much time is that requirements are constantly in flux. The code must be changed to meet new demands as those demands stream in.

Sounds good so far, so this means build in lots of flexibility to your code right? Not quite as most software developers are amazingly bad at predicting where they will need the flexibility. Some use their own creative imaginations to dream up ways in which new requirements may come in, this leads to the really horribly bad architectures. The more experienced developers draw from previous experience from previous projects which is sometimes useful, more often misleading. This leads to the principle the agilistas refer to as YAGNI aka you aren't gonna need it. Basically they revoke your flexible architecture card on the premise that you are going to screw it up.

YAGNI is a good principle, better than the overbuilding it seeks to prevent, but it ultimately comes from too detached a place. Don't get me wrong, for most developers out there, following YAGNI will lead to better code and better design. Most developers are over confident in themselves and their knowledge of what their software is used for, and spend very little time thinking about why they are building their software. The place where YAGNI falls short of nirvana though is that it presumes the software developer as contractor, which means the software developer is fed requirements from some unknown source which are then turned into code. Much better is software developer as active requirements gatherer and problem solver. The mindset has to change away from "God knows what the requirements will be next" to "What is the trajectory for this software and how can I provide the most value over time." Is there value in you building something now vs. later? What is the opportunity cost to building it later? What is the risk of it changing? Is the best way to get more information to just build what we know so far and get further feedback? These are the questions that need to be asked and that you need to have deep requirements knowledge to answer. This is where as a developer you will provide real value to a business though, and where a single developer close to the business can easily outperform a fleet of programmers in some far off place.

Software design is about scalability. It is an organic process where you must constantly ask yourself not what something will cost at this point in time, but what will the cost vs. value be over time through the long, long, long lifetime of the software. Software developers are bad at this and so many just give up and resort to YAGNI like principles, but ultimately this is because software developers are too detached from the requirements and are poorly trained in risk management.

The Bad Programmer

Nick makes a good argument for why go to should not be used. His closing argument is that zealous programmers will abuse go to. Pete makes a similar argument that go to has too much potential for abuse from programmers that we can't be bothered to explain the use of to and you are better off banning them.

It is a similar argument people make when arguing for gun restrictions. As the politicians will tell you, "Sure we trust you with a gun, but do you trust your neighbor or the guys from across town with them?"

The difference though is that you have a lot of choice over who you work with. If you are limiting your choice of language or features based on the fact that you work with bad programmers I think you are trading off against the wrong factor. Do you want to work with bad programmers? Removing dangerous features from a language will certainly remove possibilities from bad programmers shooting themselves in the foot. They still won't be good programmers though.

How many bad programmers will you even work with? Companies tend to have fairly regular hiring standards, at least at any departmental level and in my experience good programmers tend to work in the same place and bad programmers equally congregate. Indeed, in PeopleWare they demonstrated a strong correlation between programming ability and the layout of the office where they worked. If there is a strong correlation between office layout and ability then programmers in the same office will have a strong correlation in programming ability amongst themselves.

The boogeyman of bad programmers is a false threat to individual programmers. You are very likely only working with bad programmers if you are a bad programmer yourself. As a good programmer working with other good programmers, is go to still a bad idea?

In Defense of Go To

I continue this entry from my defense of duplicated code. If programmers were politicians, writing an article like this would surely prevent me from ever running for Programmer President. Twenty years from now someone would dig it up and say "Lucas Pierce supports the use of go to! Heresy!".

That said, let's say we have been contracted to write a program to generate Fibonacci numbers. Believe me, Fibonacci generators are a hot commodity these days so this is a totally realistic example. In this case you run the program and then type in a number (n) and it gives the first n Fibonacci numbers and then exits.

main
  integer n = console.getInt()
  integer first = 0
  integer second = 1
  integer next

  fib_loop:
    print first
    next = first + second
    first = second
    second = next
    n = n - 1
  if (n > 0) goto fib_loop
end

<sarcasm_tags_for_the_clueless>Egads! That program is totally obtuse! Here let me write a version without go to so that it makes more sense</sarcasm_tags_for_the_clueless>:

main
  integer n = console.getInt()
  integer first = 0
  integer second = 1
  integer next

  do
    print first
    next = first + second
    first = second
    second = next
    n = n - 1
  while (n > 0)
end

Now it makes sense! Although we could use a for loop to make it even better:

main
  integer first = 0
  integer second = 1
  integer next

  for (integer n = console.getInt(); n > 0; n = n - 1)
    print first
    next = first + second
    first = second
    second = next
  end
end

Why is this version so much better than the go to version though? We've all been taught go to is evil but I find all three equally readable. This is me throwing down the gauntlet, I challenge you to tell me why go to is wrong.

In Defense of Duplicated Code

One of the first principles you will hear from anyone hoping to impart software development wisdom on young disciples is DRY. Don't Repeat Yourself. If software best practices all lived in a little nation, the Honourable Commonwealth of Softwaria, its king would be DRY. A good king he was too.

Since ages long since past, DRY has helped to keep javascripters from copy and pasting themselves into another rathole. At first, King DRY used methods to remove duplicate code. Formatting integers to currency a bajillion different places? No problem make a method to do the formating. The code is better because it is more readable (formatAsCurrency) and if you ever need to change or extend the logic you can do so in one place. Huzzah!

Over the years, DRY took on further battles for the people of Softwaria. In the Thirty Year War against Visual Basic programmers, DRY brought forth the mighty powers of composition. Now large segments of functionality and data, not just individual methods, could be transported and duplicated to be used anywhere. A method to format an integer into currency? How about a currency class instead that can not only format itself to a string but also do any other logic you want to on currency all bundled nicely in a little package. Besides removing duplicate code, composition also provided encapsulation, higher cohesion and lowered coupling (the bad kind not the good kind!).

As Lord Action once said though, power tends to corrupt, and absolute power corrupts absolutely. For in his zealous fervor to ban any duplicate code or logic he was tempted into delving into powerful and dark arts. DRY called upon his most powerful sorcerers who gave him the power of implementation inheritance. Yes he did eke out yet more removal of duplicated code. But in the process he sacrificed most of the benefits that encapsulation had provided.

Yet more dissidents were found in the kingdom of Softwaria though and DRY delved into yet darker arts to combat them. This time he tapped into the power of reflection. With this power he could make mountains move (even if they were declared private!). Giddy with power, the king annihilated the duplicated code. Yes he had to sacrifice readability, but it seemed a small price to pay to eliminate those treasonous and heretical duplicate code bastards.

Finally, in the dark recesses of the kingdom, DRY found the last remaining resistance movements of duplicated code. They didn't bother people much, they were largely just simple workers tilling away at their small plot of program. By this time the good king had become fanatical though. ALL DUPLICATED CODE MUST DIE! He called on his sorcerers one final time and asked for the power to remove the remaining resistance. He offered anything in his kingdom just for this one final grant of power. And the sorcerers delivered for they offered the power of metaprogramming. Now the King was no longer bound by the traditional physics of the land, instead he could reshape it in whatever likeness he desired. In the land of Java the king created code generators that created wide areas of forbidden code, code never to be touched or even looked at. XML spread like a disease, soon used to store the most important program logic. In the land of Ruby, programs created other programs and whole new languages sprang forth from nowhere to solve the most trivial of problems. Ruby on Rails came riding high with instant magical scaffolds. And the last of the duplicated code died...

The king was satisfied with himself for he had finally fulfilled his ultimate goal. Old King DRY stepped out to view his wondrous kingdom and no longer recognized what he saw. It was incomprehensible. Nothing made sense anymore, nothing was predictable and nothing worked as it seemed. Nobody, not even King DRY who created this world, understood it anymore. The sorcerers laughed knowing the land was now ruled by magic and the good king wept.

Persistence: The long lost virtue of fixing a bug

The heights by great men reached and kept
Were not attained by sudden flight,
But they, while their companions slept,
Were toiling upward in the night.
-- Henry Wadsworth Longfellow

You're working on a difficult bug. A strange log message showing that your program fails every once in a while. Looking at your watch you see you've already been working on it for half an hour. You put the bug back on the router thinking to yourself maybe someone else will have better luck. And then a man from the future appears behind you... oh never mind I won't subject you to that torture again.

If you are like most software developers you've been in the situation above. Maybe instead you mark the bug as "Cannot Reproduce" or you instead assign it to your resident expert. Regardless of the means you use to escape from fixing the bug, you are dodging an important opportunity. This is normally when I start hearing the excuses. The bug is too hard. Someone else could fix it sooo much faster. There are too many bugs to waste time on just one. Forget the excuses, it is worth it both to you and the product you are creating. Oh let me count the ways.

1. You are actually learning something deep about the technology you work with. Sure the first time you are learning something you are slow. Really slow. But you are learning. You are dissecting the technology and learning how it really works. In the future with this deep knowledge you will be able to quickly solve problems. You are increasing your skill set as a developer which is good for the company you work for now and for your future marketability later. See Figure 1 below.
2. You are wasting the time you spent getting up to speed on the problem. You pick up the bug, spend 30 minutes and then put it back on the router. The next person picks it up, spends 30 minutes and then puts it back on the router. yadda yadda. This is a huge waste of everyone's time. The same if you mark it as cannot reproduce, but instead you are wasting the time of the customer or testers and then more developers later when they bring up the bug again in a new form.
3. Not only is the bug noise on the router, it is also noise in the logs (hopefully you do have some means of tracking errors from your system). It is a common tradition in too many software shops to have their set of ignorable errors that just seem to happen and nobody really knows why but they seem fairly harmless so nobody fixes them. Not to say people didn't try, I'm sure at least 5 people spent their 30 minutes on trying to fix it. Noise in the logs is not a good thing though. It could actually be a problem that will only manifest itself at an inopportune time. Even if it is genuinely harmless, it is still one more thing for humans to mentally parse and ignore and distracts away from seeing the real bugs.

At this point you maybe be telling yourself "Great I'm going to fix all the bugs". Look a null pointer exception, all I need is a little if statement null check and I'm good.

Hold On!

Resist the urge to cut off the investigation early. Fixing the symptoms of a bug is NOT fixing the bug. Why is that variable null? Is it supposed to be nullable even? You must find the root cause even if it takes you 10 or 20 times longer. Treat the bug as if it was a memory corruption bug. You can try treating each of the various symptoms of the memory corruption and varied they will be. Or you can hit that one spot where it is actually occurring and fix it for good.

The trade off is really this, you can either spend 2 days going deep, really understanding how something works and understanding the core of the issue, or you can spend the next month making tiny one off fixes that treat just the symptoms. In the end sticking with it and finding the real problem is 95% of the time more cost effective than just treating the symptoms. And speaking from experience, it is much more gratifying as well. Figure 2 shows a comparison between fixing the root cause and just fixing the first symptom you can patch over.

Persistence does not equal stupidity. Banging your head against a problem and making no progress is not persistence anymore than running on a treadmill is a form of transportation. Well... not normally at least. I see lack of persistence as the biggest thing that holds people back, but many people don't even know how to go about fixing the hard problems. It is a skill and I could go on about divide and conquer, how to actually use google and a million other tidbits. As an honest to goodness skill though, my best advice is to pair up with the best, most knowledgeable bug fixer you know. Who is the developer that ends up actually fixing the really hard problems when they come up? Tackle a really hard problem with them. When you get stuck on a really hard bug, don't give up, but do ask for advice explaining your progress so far. Don't ask them to do the work for you, but do ask for pointers on where to look next. Often times you'll find the act of just explaining what you've discovered will lead you to the solution on your own.

Remember this. Any script kiddie can copy and paste some code, tweak some values and get a system they barely understand to kinda work. If you want to distinguish yourself as a software developer, work to truly understand the system you are building, don't shy away from the difficult problems and don't stop until you really know what is going on.

Do not fix what you don't understand

It's monday at 2pm and you are working on another bug report. It is a null pointer exception in a class you aren't familiar with.

class Nose
...
void upYourNoseWithARubberHose(RubberHose hose) {
   if (hose.diameter() < 1 inch)
      noseHose = hose;
   else
      throw HoseTooBigForNoseException();
}
....
}

*yawn*

hose is null and a null pointer exception is thrown when accessing its diameter. At least this is an easy fix you think to yourself.

if (hose == null || hose.diameter() < 1 inch)

Awesome... bug fixed... just need to commit. "Fixed NPE in upYourNoseWithARubberHose" you type into the comment box.

Just before clicking the commit button a whooshing sound comes from behind you. Whirling around you see a huge man with a large gun.

"I've come from the future and I must stop your bug fix. You have no idea what will happen when you make this change. You didn't reproduce the problem. You didn't find the root cause. You didn't talk to someone knowledgeable in this part of the code. You didn't find the person who caused the bug and ask them about the fix. You didn't even test your fix. I'm sorry but for the sake of the future I can't let you commit that change."

The man lowers the gun and fires.

Your monitor disintegrates into a thousand shards of glass and plastic. Next he shoots your keyboard, the letter q smacks you in the forehead as it flies away from the carnage. Your brand new laser mouse is next. You begin to move your hand to shield it from the oversized rounds this future man is shooting but thinking again you pull it back just before he fires. Finally, he takes aim at your computer. At this point you've resigned yourself. This man has travelled back in time to stop you from checking in this horribly stupid bug fix and he means to finish the job. He fires and the hard drives spins for its last time.

He turns and starts to walk away. Just as he exits the door though he turns and he shoots one of the wheels off your chair. Your chair slumps towards where the now missing wheel was throwing your body to one side. Future man smiles with a satisfaction that only comes from knowing one has done a thorough job. Future man wonders why you couldn't have been as thorough in your own bug fixing in finding the root cause of the problem before carelessly throwing around fixes.

As you sit in the carnage of your computer equipment, slanted to one side in your broken chair, smelling the still fresh smoke you think over what you learned today. The future man's words echo in your brain...
"You have no idea what will happen when you make this change. You didn't reproduce the problem. You didn't find the root cause. You didn't talk to someone knowledgeable in this part of the code. You didn't find the person who caused the bug and ask them about the fix. You didn't even test your fix."

Your left to ponder what future your simple bug fix had unintentionally caused that it would induce man to invent time travel to send someone back to stop you. Truly horrible it must have been and with that you promise in the name of the future man with the big gun to never make another bug fix on a problem that you don't understand.

The Syllabus

I explained before why I thought "programology" should be taught, but I didn't go into detail on what I would teach.

To start, there would need to be at least a cursory mention of Literate Programming. Just like many English Literature classes start with Beowulf, we could start with a reading of TeX. Donald Knuth has a little sensor implanted in his brain that tells him every time someone reads his code; make the professor happy and at least try to read some of it.

I wouldn't dwell very long on Literate Programming. To do so would, sadly, lead to a place where we focus on finding the best source code and studying that. Instead the point is to find the best programs and study their source code. Now given, there should be some correlation between quality of source code and quality of program, but I want that correlation to be discovered in its true form and not the usual manifestation of neophytes arguing over number of comments or spaces vs. tabs.

So, at this point I believe we are at about day 3 of the class. The first true source code reading I would do would be of the Java JDK compared and contrasted with Apache Commons. For instance, take a gander at java.lang.String vs. org.apache.commons.lang.StringUtils. Library code like this is a good place to start because it is easy to read, self contained, and well commented. I find the java.lang classes especially to be friendly late night reading while sipping on hot cocoa next to a roaring fire.

Comparing the latest and greatest snapshots of programs is interesting in and of itself, but to truly understand a program you need to see its evolution. What were the choices the programmers made originally. What design decisions stuck around and which were abandoned. To this end I would now turn to the Ruby programming languages source code. This is a good candidate for such study because it was written mostly by one person, it is fairly small overall, and it was written over the course of many years with at least 10 years of subversion history to dig through. The reason I think it is important that it was largely the work of one person is that there will be less noise where code is rewritten simply because it is not understood which is very common in large programming teams. My inherent love of string code showing, take a look at the subversion history for string.c. Look at some of the methods in his version 2 and see how they evolve over time. In what ways does the code get more and more complex? In what ways does it actually become easier to understand?

So far, we've only had to deal with largely isolated components which is nice when trying to study source code, but eventually one has to grasp with understanding large systems. The big guns in this case would be studying the Eclipse IDE code base. It is large, modular and built by a large disjointed set of people. I wouldn't concentrate on reading code at this stage as I would on scanning it. The takeaway skill would be the ability to conceptualize the architecture of a large program and then be able to answer questions such as: if there was a bug in the string differencing, where would you go to fix it and if you were to add a feature that let you tear off subviews into their own windows where would you start.

This is of course just a taste of what the class would look like. There would have to be quizzes that prove you comprehended the code, midterms that involved reading new code on the spot as well as recalling code previously read and a major project where students would pick the source code to read and write their observations on. 

Please let me know if there is publicly viewable source code that you have read in the past that you think is of particular note.

Programology

If I were to teach a course at the University level, I would teach programology. Not the study of programming, but the study of existing computer programs. In the entire course, we would only read source code. Not small snippets of algorithms, and not trivialized polished examples of code created just for class. I'm talking about the industrial gritty always evolving code that actually runs a good part of the industrialized world now. Just as in cultural anthropology you study tribes from New Guinea to understand the various ways in which human society has evolved, we would study old programs, new programs, large programs, and small programs. We would study programs developed under a variety of different paradigms.

It would be a weeder class. Writing code is hard; reading code is harder. But that is the point, being able to take an existing code base and quickly conceptualize the architecture and its design elements is what separates the software developer who can make the trivial one line fix to a problem and the one who bolts on three hundred lines of monstrous code because they didn't quite get it.

It will also help us not repeat the same mistakes that the majority of software developers have made before us. Most new software developers start out writing spaghetti code, code without known structure, features organically tacked on and general anarchy. Most developers eventually learn from their mistake, but when given the chance to rewrite their original software they swing too far the other way. They create a huge over-designed Frankenstein creation that is fully table driven, has an intricate inheritance tree and incorporates god knows how many design patterns. If we are lucky, the developer spends enough time in the field to eventually learn from this mistake as well and finds a happy medium.

Fred Brooks described this problem as the second system effect in his book the Mythical Man-Month, published in 1975, before most software developers today were even born. Yet, the vast majority of software developers today still follow the same evolution because we don't study source code and what factors led that source code to arise why it did. We are like building architects who have never actually studied the design of any other building  besides those we personally took part in creating. Or a writer of novels who has never read any novels besides his own. To progress as a field, we must study those who came before us.

I probably will never teach this class, despite the desperate need for it. In the end though, I say to anyone that wishes to be a great software developer, the surest way is to write lots and lots of code, and to read even more.