Holistic development and the tale of the unclear requirement

I am a big believer in holistic development, and before I go and present my opinion on this let’s start with the elephant in the room: What exactly does holistic mean ?

Holistic — characterized by the belief that the parts of something are intimately interconnected and explicable only by reference to the whole.

Oxford Dictionary

So, what does this all mean ?

Well, I admit that this might sound a wee bit woo woo to some people, but bear with me for a while. You might end up liking this holistic approach.

The word that I want to focus on is interconnected. Why ? Well most people strongly believe that in order to have a successful software project you need some requirements some software developer / coders / engineers, a manager and everything after falls into place. Starting from this assumption there are a ridiculous amount of click-baity articles on the internet that go to the trouble of telling you why software projects fail.

Here: example search

If you take the time and go through some of them, you will probably get a lot of good and useful advice, and you will mostly see that all of them include the following reasons:

• Poor communication
• Unclear requirements

Assuming that you have access to a team that is quite capable technically, you still have 50–50 chances to pull it through. Are you ready for the coin flip ?

Not so fast tiger!

Let’s just pause here for a moment and give this some thought. From this two bullet points we can deduce that this has something to do with the attitude and engagement of everybody involved.

Well, let’s pick “unclear requirements”.

What does this mean ? That they were improperly written ? That they were way to vague ? That there were items there that were contradicting themselves ? What actually was the problem, and more importantly when did this became a “problem” ? Who wrote this anyway ? Off with their heads!

Okay… I got a bit carried away there, but we have to admit that this is the way that project postmortems work, someone on the business side comes and asks for a ROOT CAUSE ANALYSIS, with the single intent to find a poor soul that can be made responsible for the current mess, point some fingers and move to the next project where the gods of coin flipping will, hopefully, favor them. But as we will further see is that, more than usually there isn’t one person that is responsible for something, it’s a team effort.

What can we do different to try to increase our chances in the future ? Well let us pick one of the aforementioned questions and have a go at it with some ridiculous, yet oddly plausible, scenarios:

When did the requirements become a problem ?

Yeah, no kidding, when? Because this is the single, most important question that we need to ask and address in order to unlock the other ones.

The point of this question is not to be snarky or look smart, is just to pin point the moment in time when somebody discovered and “communicated” that the requirements are not crystal clear. By asking this question we can get one of the following scenarios:

1. Right at the beginning

This is the most glorious scenario, someone noticed the problem, mustered the courage and told everyone that there is a problem here. Hopefully this happened before any estimations were done ( if any ) or before any code was written. Chances are that in this case, the inconsistencies were analyzed and cleared and the coin heavily favors the “Heads” outcome.

If you are in this case you have probably have an environment where people are engaged and feel safe to raise possible issues.

2. In the middle

Well, let’s say that you are ending sprint 7, and one of the epics that was started in sprint 3, and it was estimated for 2 sprints is still in progress ( I know this hardly happens in reality). People start to get antsy, the project manager sends everyone an email invitation for a status meeting. When the mail hits the inbox, you could hear a rather powerful gulp sound resonating through the fancy open space office.

What happens next ?

It’s 2 pm, everybody is waiting in the meeting room. More or less most of the team members have knowledge of this problem. Everybody has an opinion, they just don’t really want to talk about it. Finally the PM arrives and the meeting kicks off, after the usual pleasantries that are part of this kind of “escalation” meeting, everybody discusses the problem, starting from the possible misunderstanding of the developer that was working on the problem, to possible implementation issues, to the fact that the requirement itself was contradicting with an already implemented requirement ( feature ). After some productive brainstorming the team goes through a slew of workarounds, alternative implementations, different technical options, and conclude that the actual problem was with the requirement. They formulate a few alternatives to the requirement that will resolve this problem, send it to the customer which admits that the initial requirement was flawed and everybody goes their marry way. What a win!

The managers congratulates everyone for their contribution, praise their ingenuity and tells his superiors about his team capacity of solving problems.

Right ? Well it is all well when it ends well. Is it though ? I mean the coin still has a slight edge toward the “Heads” side but let’s think about how frequent this is ? In all seriousness, how many resources have been spent on a problem that could have been fixed easily in the first scenario ?

Let’s say it is a team of 5 people with sprints of 2 weeks and a 25% allocated manager. The cost breakdown would be something like this: 1 developer worked on this for 4 sprints ( 2 months ) — 40h / week 8 weeks, all the team participated in a 4 hour meeting (6 people 4 hours ). This is roughly 344h (man hours) , and this does not include the implementation of the refined requirement epic. Also apart from the cost (paychecks / rent / electricity / free drinks / free coffee ), there are also the frustration of the people involved, just imagine ( or remember ) how is it to spend 8 weeks,aka 2 months, on something that is doomed to fail. Awesome, right ?!

I know, I know … This example is a bit extreme, but think about it, this can happen on a story basis, several times a sprint, and to make matters worse, all this compounds over time, and worse, sometimes this kind of problems don’t get escalated, people start building weird kind of workarounds, dubious implementation, and in worse cases, go along with a pseudo implementation of the contrived requirement which leads to our next scenario.

3. The release disappointment

So in this scenario, the developer managed to sneak in a really clever workaround, jumped some hoops, and managed to fulfill the crocked requirement. It does what it says in the epic and stories. The release was delayed a few times, but it’s agile, you know how this works (spoiler: it doesn’t ). Software is a form of art, it takes time (quite a lot), but a few month later, the time for the great release has arrived, the last commit was made, code freeze.

The team has this awesome CI / CD pipeline, branches, unit tests, 99% code coverage on the business logic code, everything works. The code is live. Everybody pat themselves on the back, tomorrow, the client acceptance meeting will be just a formality, everything works, no crashes, just like according to the specs.

The next day, the big moment, all stakeholders are here, the company booked their nicest conference room. The highest ranking person has the awesome opening speech, everybody laughs, everybody cheers. Some QA or PM starts the presentation / walk through … starts presenting the creation. Everything goes as planned, seems the lunch time break might be on time, but and at some point, out of nowhere a resounding “Oh!…” is heard in the room.

Suddenly tension fills the room. Nobody on the development team says anything, hoping for this just to go away … right when the presenter is about to click the mouse and move on, that one guy on the clients team that you never liked, yeah the one in the smart casual sporty suit says something that hits like a ton of bricks:

This is not how we do it!

That one guy!

The aftermath

So, what now ? Well, one of your PM / Sales guys starts taking notes, asking questions and taking more notes, a few days of heated debates, things become a bit more clear, the overall system was good, apart from this one small thing in the core domain. So the sales guy drafts an offer to do the required changes on the companies money, or if the sales guy is really crafty he could argue that the software is according to spec, and in some cases he would convince the client to pay for the necessary work as a change request. Now this is a win, mo`money is always good !

Now, after the 6 month, having a 4 person refactoring project that had to redo the problematic parts of the system and also the parts that were highly dependent on them, everybody is happy!

Yet again, the example is quite naive and simplistic, but you get the point. The client could be internal or external, it does not really matter. The problem is that this was major flaw, be it in the form of a requirement. Now depending on your company’s business model this might even be considered success, but again it sure does not feel like it is.

What can we learn from this scenario ? How did we get here ? At first, the knee jerk reaction is to say that since the requirements were wrong, it’s not the development teams fault. But, unfortunately in the real world there are so many cases were people are so disengaged, afraid to speak up, or even worse, hide behind the “it’s not my job” mantra. This leads to having this type of problems occur quite frequent. They don’t have to be this dramatic, and they may occur on the sprint basis, but again if this happens once every two sprints, they will add up.

4. The everything seems fine

In this scenario, in the meeting room, that guy in the weird suit was playing candy crush on his phone and did not pay attention, nobody else had any comments on the software, everybody is happy. The company writes a new contract for further development and maintenance of the application for the next 2 years.

I believe you see where I am going with this, this is the most insidious and dangerous path possible.

There are so many way on how things can go wrong but here a just a few:

Your boss decides that your team, since you successfully delivered the project from scratch, your team should get on the next new project, and let a more junior team handle the further development of the current software project, since it is stable, and the client was really satisfied.

After a few months of “production time” between all the new functionality the client adds a new story that corrects the initial flawed requirement. Now the new team, trying to uphold the high standards that were set up by the founding team, goes in guns blazing and start implementing the changes. But this seems to be breaking a big chunk of the application. Most of the functionality since then was built on top of that specific requirement, you are in sort of a check mate position.

This takes way longer than anticipated, more resources get pulled in, the overall velocity goes out the window. At some point the team thinks they did it push it live, only to have 20 bugs or so be reported in the first few hours. Now everybody is talking to everybody, escalation after escalation. Two month in, 70% of the time is used to fix bugs. Each time something changes, something somewhere breaks. Since this escalated, nobody had the time to update or add new automated tests, currently the only way to get feedback is by having some QA team do smoke tests, all the time, for each change.

Sooner or later the customer will be disappointed, after that they will decide to switch you for another shop, that will be tasked with the maintenance of this application transforming their lives in this living hell of workarounds, fixes, bugs and weird technical solutions.

Your team, probably will be burn out, demoralized, heck Jim handed his resignation, and after completing an online class opened his own handmade leather goods business. Mary still gets an empty stomach sensation went she gets and email notification from Jira.

Dramatic, I know, but just try to remember the last project that you were tasked to maintain that you were not part of the development. Did you ever wondered why you had to do it ? Why didn’t the people who initially developed it were abandoned ?

Conclusion

In this imagination exercise was just went for one question. This whole chain of events was set from only one bad requirement. Maybe no one really thought about, or really cared about its underlying implication (coders be coding) and took it at face value, maybe someone actually discovered the problem but did not communicate it. Or maybe someone did raised this problem, was discussed in the team but the project manager wasn’t informed, or the client wasn’t informed.

I will let you do this kind of exercise with the other question. I have faith in you.

TLDR; Software development is complex

This is the point of the holistic approach. Building software is a complex system problem. We need to own this, accept this and start paying more attention to details. Almost everything counts, all the small details come together and influence an outcome. In order to try and deliver a successful project, there are a lot of things that we need take into account.

For those that are not familiar with complex systems I’m planning the next post to be a very informal introduction to the concept an also provide some materials (but of course you can always look to cousin google for this).