Dunning-Kruger effect on effort estimates

This post has two parts. The first is an experiment with a poll. The second is the actual content with my thoughts.

The experiment and the poll comes first as I don’t want to infect you with my idea before you answer the questions. If you are in the mood of reading a short story and answering a couple of questions, keep reading. In case you are only concerned with my ideas, you may skip the first part.

I won’t give any discussion about the subject. I’m just throwing my ideas to the internet, be warned.

Part 1. The experiment

You have to estimate the effort needed to complete a particular task of software development. You may use any tool you’d like to do it, but you will only get as much information as I will tell you now. You will use all the technologies that you already know, so you won’t have any learning curve overhead and you will not encounter any technical difficulty when doing the task.

Our customer is bothered by missing other co-workers birthdates. He wants to know all co-workers that are cellebrating birthday or just cellebrated, so he can send a “happy birthday” message at the very morning, when he just turned on his computer. To avoid sending duplicated messages, he doesn’t want to see the same person on multiple days at the list.

Your current sofware system already have all workers of the company with birthdates and their relationship, so you can figure out pretty easily who are the co-workers of the user and when is the birthdate of everyone.

Now, stop reading further, take your time and estimate the effort of this task by answering the following poll.


Estimate your effort

Okay, now I’ll give you more information about it and ask for your estimate again.

Some religions do not celebrate birthdates and some people get really mad when receiving a message of “happy birthday”. To avoid this, you also need to check if the user wants to make its birthdate public.

By the way, the customer’s company closes at the weekend, so you need to take into account that at monday you will need to show birthdates that happened at the weekend and not only of the current day.

This also applies to holidays. The holidays are a bit harder as it depends on the city of the employee, as they may have different holidays.

Oh, and don’t forget to take into account that the user may have missed a day, so it needs to see everyone that he would on the day that he missed the job.

Now, take your time and estimate again.


Estimate your effort – II

Part 2. The Dunning-Kruger effect on estimates

I don’t know if the little story above tricked you or not, but that same story tricked me in real-life. 🙂

The Dunning-Kruger effect is stated at Wikipedia as:

“[…] a cognitive bias wherein relatively unskilled individuals suffer from illusory superiority, mistakenly assessing their ability to be much higher than is accurate. This bias is attributed to a metacognitive inability of the unskilled to accurately evaluate their own ability level. Conversely, highly skilled individuals may underestimate their relative competence, erroneously assuming that tasks that are easy for them are also easy for others.”

I’m seeing that this effect contributes to make the task of estimating effort to be completely innacurate by nature, as it always pulls to a bad outcome. If you know little about it, you will overestimate your knowledge and consequently underestimate the effort to accomplish it. If you know much, you will underestimate your knowledge and consequently overestimate the effort.

I guess one way to minimize this problem is to remove knowledge up to the point that you only have left the essential needed to complete the task. Sort of what Taleb calls “via negativa” in his Antifragile book.

What do you think? Does this makes any sense to you?

Advertisements

3 thoughts on “Dunning-Kruger effect on effort estimates

  1. … and this is compounded with the fact that estimating with relative sizes, whether T-shirt sizes, easiness or story points is relative to the estimator's context. Some people would think in terms of hours of effort, some days, some with a linear scale in mind, some with an exponential…

    Thanks for this nice little experiment although I am not sure it proves your point and I am not sure I would agree with your last sentence : what is the minimal knowledge to complete the task you suggest?

    Like

  2. Thanks for reading and adding the relative-size matter. I totally agree with you. I used adjectives (“easy”/”hard”) to try to get the overall feeling of the effort for the task, instead of a precise measuring.

    I expected to see an easier task at first estimate and a harder task at the second estimate — it's turning out to be true so far. No matter the results, it does not prove anything.

    The last sentence is my 2 cents about how to minimize the effect. For example, the second part throws away a lot of “noise” requirements that could be summarized to something like “you need to check all birthdates that happened since the user logged in last time and only include those teammates that are on the job”. I think that reading just this single statement as the sole requirement seems to make it look more realistic or “easier” than reading a bunch of requirements that sums up to the same thing.

    Maybe I should put a third question with it. What do you think?

    Like

  3. I think the current two questions correctly hit the nail. The second form accurately represents the situation where one discovers the full complexity of a feature one piece at a time, something quite common.

    I must say I couldn't complete “Antifragile”: I found this book to be very repetitive, badly written and carrying the self-esteem of the author to unforeseen heights 🙂 I vaguely remember him mentioning this via negativa but not sure what he meant.

    In my experience, on the contrary developing software is a knowledge acquisition and transformation process. It is not true the more knowledge you have, the more efficient you will be, but there is a good balance to strike between being a domain expert which might lead you (as a developer) to paralysis and a total noob which will make you overconfident and lead to naive code.

    As a side note, another factor which makes estimating task's complexity a very random process is the code itself: as more features get added and so-called technical debt accumulates, code becomes more complex. More often than not I have seen cases where adding an innocuous-looking feature actually required (or made obvious the benefits of) refactoring large parts of a code-base which raised the complexity of completing the feature.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s