Dunning-Kruger effect on effort estimates

This post has two parts. The first is an experiment with a poll. The second is the actual content with my thoughts.

The experiment and the poll comes first as I don’t want to infect you with my idea before you answer the questions. If you are in the mood of reading a short story and answering a couple of questions, keep reading. In case you are only concerned with my ideas, you may skip the first part.

I won’t give any discussion about the subject. I’m just throwing my ideas to the internet, be warned.

Part 1. The experiment

You have to estimate the effort needed to complete a particular task of software development. You may use any tool you’d like to do it, but you will only get as much information as I will tell you now. You will use all the technologies that you already know, so you won’t have any learning curve overhead and you will not encounter any technical difficulty when doing the task.

Our customer is bothered by missing other co-workers birthdates. He wants to know all co-workers that are cellebrating birthday or just cellebrated, so he can send a “happy birthday” message at the very morning, when he just turned on his computer. To avoid sending duplicated messages, he doesn’t want to see the same person on multiple days at the list.

Your current sofware system already have all workers of the company with birthdates and their relationship, so you can figure out pretty easily who are the co-workers of the user and when is the birthdate of everyone.

Now, stop reading further, take your time and estimate the effort of this task by answering the following poll.


Estimate your effort

Okay, now I’ll give you more information about it and ask for your estimate again.

Some religions do not celebrate birthdates and some people get really mad when receiving a message of “happy birthday”. To avoid this, you also need to check if the user wants to make its birthdate public.

By the way, the customer’s company closes at the weekend, so you need to take into account that at monday you will need to show birthdates that happened at the weekend and not only of the current day.

This also applies to holidays. The holidays are a bit harder as it depends on the city of the employee, as they may have different holidays.

Oh, and don’t forget to take into account that the user may have missed a day, so it needs to see everyone that he would on the day that he missed the job.

Now, take your time and estimate again.


Estimate your effort – II

Part 2. The Dunning-Kruger effect on estimates

I don’t know if the little story above tricked you or not, but that same story tricked me in real-life. 🙂

The Dunning-Kruger effect is stated at Wikipedia as:

“[…] a cognitive bias wherein relatively unskilled individuals suffer from illusory superiority, mistakenly assessing their ability to be much higher than is accurate. This bias is attributed to a metacognitive inability of the unskilled to accurately evaluate their own ability level. Conversely, highly skilled individuals may underestimate their relative competence, erroneously assuming that tasks that are easy for them are also easy for others.”

I’m seeing that this effect contributes to make the task of estimating effort to be completely innacurate by nature, as it always pulls to a bad outcome. If you know little about it, you will overestimate your knowledge and consequently underestimate the effort to accomplish it. If you know much, you will underestimate your knowledge and consequently overestimate the effort.

I guess one way to minimize this problem is to remove knowledge up to the point that you only have left the essential needed to complete the task. Sort of what Taleb calls “via negativa” in his Antifragile book.

What do you think? Does this makes any sense to you?

Premature optimization: it’s not just about code

When we talk about performance, we shouldn’t look only to performance of code execution. There are more types that we should care about, like time to market, development time and processes. Actually, we shouldn’t look into any of the performance measures at a single perspective. As I see, all these measures sums up to constitute a single value of business performance.

There are lots of things that can slow down your business, and if you have a slow business, you’ll have a hard time to keep pace with your rivals. As you discover that some areas of your company may be slowing you down, you may become tempted to add key performance indicators to each step trying to squeeze all possible value out of each part of your processes. This can bring up a whole new set of problems, your business can be seen as a complex system and it will adapt itself to render good results even in chaotic scenarios. [1]

So, to avoid going nuts with indicators all over the place, you decide to avoid bottlenecks from the start. To each and every new thing you need to deliver, you’ll start a precise planning routine, choosing among a plethora of providers, technologies and unicorns to avoid at all costs to have a new bottleneck in the future. This is what I like to call premature optimization in business performance.

Simply put: you can’t have a business if you don’t have a business. How can you possibly know all the events that will happen to have an impact in the decision you take today? You can’t.I’m not saying that planning is wrong or a Bad Thing™. What I really want to avoid is losing energy on stuff that won’t make a difference. You may spend a whole year choosing between apples and oranges, and in the end be crushed by some weirdo playing around with random technology. Why? Because he was not worried about future bottlenecks. He was really trying to do something cool, and he did it while you were arguing in endless and purposeless meetings.

You’re always trading performance measurements. For example, everyone is talking about service oriented architecture (SOA), but what does it really means in terms of business? It means a tradeoff between code execution performance and others benefits to the business as a whole, like decentralized grow and continuous delivery. Or, you may look at the difference between Apple’s App Store and Google’s Play Store model. There is a huge tradeoff of quality assurance and time to market between these two players: one offers their customers (smartphone owners), quality assured apps; the other offers their customers (operating system users), fast time to market for their apps.

The big question really is: what are you trying to deliver to your customer? Are you trying to deliver software over time or value over time? I bet most of you are trying to deliver value over time, so why bother with premature optimization of technologies or processes? Let the bad stuff to die on its own: failing fast is orders of magnitude better than not doing anything at all. And let the good stuff to accompany you to where you want to go.

Don’t bother guessing the future, embrace the uncertainty of life: you can’t predict how a complex system will behave, you can’t know how your systems, services or whatever you are doing will be used in the real world. Put it into the wild, take the complaints and tune it (or throw it away) afterwards, this is a constant process. [2]

Start measuring how much you are delivering over time and stop over-planning. Your end goal is to deliver stuff. The world couldn’t care less about what you can do, it only cares about what you do.

References

  1. Complex Systems and Key Performance Indicators – Márcio Geovani Jasinski
  2. Complaint-Driven Development – Jeff Atwood

Code infected with exceptions

I believe that code infected with exceptions is bad. Imagine the following harmless code:

public static Foo newFoo(int bar) {
  switch (bar) {
    case 1: 
      return new FooOne();

    case 2: 
      return new FooTwo();

    default: 
      throw new IllegalArgumentException("invalid bar code: " + bar);
  }
}

You use it in the middle of your business code with ease. The code compiles and everybody is happy:

public static List getAllBazFoo() {
  List result = new ArrayList();
  List allBaz = BazDAO.getAllBaz();
  for (Baz baz : allBaz) {
    int id = baz.getId();
    int bar = baz.getBar();
    Foo foo = Foo.newFoo(bar);
    BazFoo e = new BazFoo(id, foo);
    result.add(e);
  }
  return result;
}

The problem is that the call Foo.newFoo(bar) can throw an IllegalArgumentException and when this happens you will have no clue of which Baz was invalid.

To address this and make the message more useful, we have to remember to add a try/catch in the code:

public static List getAllBazFoo() {
  List result = new ArrayList();
  List allBaz = BazDAO.getAllBaz();
  for (Baz baz : allBaz) {
    int id = baz.getId();
    int bar = baz.getBar();
    Foo foo;
    try {
      foo = Foo.newFoo(bar);
    } catch (IllegalArgumentException e) {
      throw new IllegalArgumentException("invalid baz: " + id, e);
    }
    BazFoo e = new BazFoo(id, foo);
    result.add(e);
  }
  return result;
}

Add multiple layers of abstraction to your application and you’r ready: spaghetti with an exceptional taste. You add an exception at some point and you will have to review the entire stack of abstractions you have to ensure that the exception does no harm to anyone and has a useful message for future maintenance.

Haskell solves much of the need for exceptions using returns that symbolize failures, e.g. Maybe and Either, and the use of Monad eliminates the need to check the results at each step.

However, there are exceptions in Haskell, why? I can not understand the reason for preferring exceptions rather than special results. In which situations is it better to use exceptions? They look like a modern goto to me.