Pair Programming – My Ignorant View

Firstly a disclaimer. I have never done pair Programming. Never read officially about it I.e. Kent Becks Extreme Programming Explained. I have only what I have read in blogs, which is limited and my own thoughts. 
I feel it is important to document my ignorant understanding before I embark on learning as it gives a base level of my understanding based on what I currently know about agile methods which I can look at in the near or distant future and see my progress. 

Now, for my ideas on pair Programming .

Say you have 2 people, A and B. A has knowledge that B needs.

First phase, A and B sit together. A works while B observes. B can ask questions, but A explains as they go along what they are dong.

Second phase, B takes over while A observes. B applies the knowledge they have learned, while A corrects B for any mistakes or answers any questions as required.

At the end of both phases, you now have 2 people converse in the knowledge.

Now it it time for the third phase. B now teaches C and simultaneously, A can teach D if required.

This I see as a good way to disseminate knowledge through a group. You get the one on one time and then apply the knowledge through teaching. 

I first saw something similar in a TV show. ER as a method for surgeons to disseminate knowledge. The saying was…

  • See one
  • Do one
  • Teach one

Now, as I mentioned earlier, this is my ignorant thoughts, but I would like to know everyone else’s on the topic. Let me know in the comments.

The Founder

I recently watched the movie ‘The Founder’, and i was watching the MacDonalds brothers talking about how they started. To me, the way they worked everything out looked like Lean to me. They way they worked out the restaurant layout, on a tennis court, with chalk outlines then get the people in to simulate their work. Try it out, when it doesn’t work, scrub out the lines, re-draw. The way the brothers automated, also how they failed but kept trying different things. They also didn’t hide anything. Eye showed everything off. Not only to Ray Kroc, but anyone interested. Just like TOyota does.
The whole system looked like lean in the making, if this story is accurate, it meant that the US had lean concepts 30+ years before it was formally introduced by Toyota in the 80’s. The movie was sad in the end, the McDonald brothers lost their legacy. Had there been collaboration instead of confrontation between the McDonald brothers and Ray Kroc, maybe lean might have been incorporated into the culture a decade or so earlier.

I guess we’ll never know.

Yellow Belt

I’ve just completed and passed the Lean Six Sigma Yellow Belt course at https://goleansixsigma.com/

The course is an introduction to Lean Six Sigma and goes through the DMAIC steps. The basic introduction is the white belt which is also free, but I didn’t take.

D – Define

M – Measure

A – Analyze

I – Implement

C – Control

The course itself is free and I recommend doing the course as it will give you another perspective to Lean. Whether or not you choose to do the certification, I leave that up to you, but it costs $99 US to take the test with unlimited retakes, which I happily paid.

At some point I’ll go through the things I found interesting in the course, but for now I just want to let you know about it.

The Agile Mindset

This is the second chapter. Lets see how we go.

The Agile Mindset

There are two parts to being Agile. The first half is the Agile mindset. This is where your thinking needs to change. Agile when implemented correctly is a paradigm shift in the thought process compared to how work is performed today.
The second half is the process. The process adds discipline and structure to your Agile thinking. Implementing the Agile mindset, is a cultural change. Even if you do not implement the process side, you can still do great things. The problem is that most people, when they start to implement Agile only implement the process. Although benefits can be gained, they are small. The process itself feels empty like you are just going through the motions, which you are. If you don’t make the leap to the mind set, then most likely your Agile journey will end and you will regress back to your previous work practices.

So what is the Agile mindset and what is the alternative mindset?
According to Linda Rising and her talk on the Agile Mindset, there are 2 types of people. The first is people that think you are born with a certain level of intelligence/talent or ability, you can learn new things, but you can’t change how intelligent/talented or able you are.
The second type of people are those that think that no matter how intelligent /talented or abled you are, you can always get better. Sometimes you can improve a lot.

The first type of people are considered the “Fixed” mindset. The second are considered the “Agile” or “Growth” mindset. Now, we all have these mindsets at some point. We flip flop depending on the situation and what we know all the time, but some people are more dominant in one mindset that the other.
People with the fixed mindset are more prone to fear failure. They will choose an easier path. The path of less resistance. The one less prone to failure.
Their thinking is that it is too difficult to do, so why try. They avoid challenge and prefer proven techniques. Techniques that have proved their success. Alternatively, when they fail, they look at the “success” in the failure and focus on that. These people are more interested in looking good.

On the other hand, those with the Growth or Agile mindset, look at a difficult situation and think – If I just work a little longer, try a bit harder, spend a bit more time on it, I will get it. If they fail, they try again.
There have been numerous psychological studies on this. In one of these studies, it found that a particular mindset can be easily induced. At least for a short period. Ever been told that something is too difficult to do, then give up? Alternatively, when you do something well, you think I’ve made it. Then stop? That is the fixed mindset.
On the other hand, if you have been told that something is too difficult and decided to try it for yourself anyway, or when you get something right, try to improve it, that is the Agile mindset. These are just a couple of examples.

The thing is that we become either mindset when we are young. If we are told when we are children that we are “Talented” in something or “You’re so smart!” or the negative “You are so stupid” or “You won’t amount to anything”, then this induces the expectation. For those that get the praise, they become praise junkies. They look to what they can do to keep getting the “You’re so talented” praise. The best way to do that is to find a challenge that isn’t too difficult. One that isn’t too far out of reach.
The problem is, that when a child with this type of thinking comes across a challenge that is too difficult, the child either avoids or ignores the challenge.
These are the children that grow up to be fixed mind set dominant.

Alternatively, children that are praised on their “effort” rather than that result, even if they fail, or for the negative, told that they “need to work harder” actually become more resilient to change. They try to tackle it head on. They are more open to ideas and willing to give them a go, because failure is part of life and you just move on.
These children grow up to have an Agile dominant mindset.

Its not only people that can have these types of mindsets. Organizations can too. It is not surprising. Organizations are made up of people.

But, all is not lost. People can change. Those with a fixed mindset can have an Agile mindset. You just need to want it.

So with Agile development in mind, the Agile mindset at its core is:

  • Thinking of your customer first. How can you make their life easier/better or what value can you give.
  • The willingness to try and learn new things.
  • Have the goal to improve yourself, your team and your colleagues and
  • Never be afraid to fail.

Thinking Of Your Customer

When I say “Thinking Of Your Customer” the question that comes to mind is “Who is your customer?”. The first and obvious answer is the people who use the product or service that you provide. These may be the general public or other companies. By thinking of the customer in this instance, you try to provide a product that is valuable to them. Value can be very subjective. It could be something simple that helps your customer accomplish a goal or it could be a complex.
For example, Facebook started out as a web site where University students (I’m Australian so I won’t say College – A College here is high school) stay in contact and share information. Google started out as a search engine because the founders didn’t like the current search engines that were out there. They thought that a good search engine would be valuable to others. Zappos wanted to see if people wanted to buy shoes online.
These companies started by adding large amounts of value to a small group of people. It wasn’t until later that they stared expanding and adding value for other customers.
The second lot of customers are your internal customers. These are other teams or other members within your team. Be they under you, your peers or your seniors. These customers, you help indirectly. For the people under you, there is the concept of “Servant Leadership”. Servant leadership is much different to the standard method of leadership which I will term “Command And Control”. With Servant Leadership, instead of telling people what to do as you would with Command and Control leadership, your job is to guide and grow your people. Give them direction and hen get out of their way.
You, as the leader have the big picture. So you give the team the goal to help accomplish the big picture. You do not tell them how to reach the goal, you leave it up to the team. If the team veers too far from the goal, you nudge them back toward the goal. You never ever tell them how to reach the goal. The reason for this is that it helps grow your team. The team becomes stronger because they make mistakes (and they will) but you guide them back. You never solve the problems for them, you need to let them solve the problems them self. Believe it or not, this makes for a happier team as the members then have full responsibility for reaching the goal. The autonomy to make decisions on how to reach the goal themselves. They also master the skills required to accomplish the goal. For those that are peers, you help make their jobs easier. Try not to send crap work. Share knowledge. For your manager, help them accomplish their goal.

By thinking of the people first instead of results, you grow your customer base, both externally meaning more profit, and internally meaning you have a more dedicated workforce.
One thing to remember though is that respect for one another can be gained very easily. It can also be lost very easily. The problem is that once it is lost, it can be very difficult to regain.

The Willingness To Try And Learn New Things

By simply having an open mind to try something new. By trying something new, you open your mind to new possibilities. A way to work better. A way to work faster. A way to improve quality. As humans, we try to find the one true or best solution. For some problems, this is the case.
These types of problems though in reality tend to be the simple problems. What font size is the best for me to read. What is the boiling point of water at sea level. These are like “Mount Fuji” type problems where the peak is the optimal solution.

photo credit: reggiepen Reaching Out via photopin (license)

For more complex type of problems, the landscape is a little different. There are peaks and valleys. One solution is at a peak, the dismal failure is in the valley. The problem is that there are many peaks. Some higher than others. The only way to reach another peak is via a different path.
These types of landscapes are like a “mountain range”. The only way to find out which peaks are the highest is to try different things. Different combinations or completely methods altogether. Even counter intuitive methods can bring surprising results.

photo credit: romanboed Sunny Mont Blanc via photopin (license)

For really complex problems. More like real life problems. We have the mountain range, but it shifts continuously. In other words, its waves. You may be high at one point, but if you stay still long enough, or short enough, you are in the valley of failure. The only way to continually remain high is to “surf” the waves. You have to continuously shift and change. What worked one day will not necessarily work the next.

photo credit: jijake1977 Rip Curl via photopin (license)

Companies like Google continuously try new things. They have great successes, Their search engine, Gmail but they also have dismal failures. Google Wave, Google+. Yet they continuously surf.

Peter Palchinsky a Russian engineer in the early 20th century summarized how to surf through his 3 principles.

  1. Seek out new ideas and try new things.
  2. When trying something new do it at a scale that is survivable.
  3. Seek Feedback and learn from your mistakes.

This leads us to our next section.

Continuous Improvement

Continues Improvement as the name suggests is well, improving continuously. Not every now and then. Not when you have the time. But always. Always looking for ways to improve. This on itself is hard. Paul Akers from 2 second Lean suggests everyone looking for an improvement of 2 seconds every day. His findings are that if everyone looks for improvements of at least 2 seconds, then every now and then, a significant improvement is found. Even if it doesn’t. Those little improvements build up over time. This in Lean terms is called Kaizen.
So instead of thinking “If it ain’t broke, don’t fix it” think “If it ain’t broke, fix it anyway”.
So the question now becomes, how do you improve? What is the definition of “improvement”. Well, its different for everyone, but for me and what I think it is in Agile is simplification. Simplify the process, simplify the program, simplify as much as you can. The best way I think to simplify something, is to subtract the complexity.

Waste

In Lean, the simplification process is done through the removal of what they call Waste. There are 3 major types of waste in Lean.

  • Muri – The waste of Overburden.
  • Mura – The waste of Unevenness and finally
  • Muda – The waste of Futility.

These wastes are further broken down into 8 more wastes.

  • Overproduction – This occurs when you create more than you currently need. When you over produce, you need to store the over produced product. This becomes
  • Inventory – Then you need to figure out how to move all your stock. This leads to
  • Transport where goods are moved from one area of the warehouse to another. Then you have to work out a process to keep track of everything. This is
  • Over processing. Your processes become more complex. Stock gets damaged. The damaged stock become
  • Defects. To fix the defects, your people need to move the stock back onto the floor, fix the defect then move it back into storage. This is a lot of
  • Motion. All the while, your customers are
  • Waiting for their goods. All the while your people are sick of redoing work. They want make things better, they want to be happy at work. But they can’t because they are not allowed to. They fell they have
  • Underutilized talent.

Notice how I put the 8 wastes into a story I have taken this queue from Paul Akers of 2 Second Lea. The idea being is that having a little story you can tell about the wastes. It helps you learn them, but also helps you understand them. its much better than just remembering an Acronym such as DOWNTIME.
Now these wastes are generally for the Manufacturing industry, but they are similar to what you will see in the IT space. Saying that, there is a list of wastes specifically for IT. I’m going to attempt a story here, but I don’t think it will be as good at the one above.

When you have so much work to be done, and its all done on the fly with no

  • Standards and Manual work, the priority of tasks keep changing. Do this now, no stop that and do this. You are constantly
  • Task Switching. This leads to
  • Partiality Done Work. Nothing really gets completed on time. Everything gets late. the customer is
  • Waiting for their software longer. Since it takes so long to get to the customer, and we are under the pump, we make mistakes, cut corners this introduces
  • Defects that need to be fixed. The buggy code goes back to development, then testing, then UAT again. Its constantly in
  • Motion. The software is so buggy, we introduce gated changes and procedures and other.
  • Extra Processess to prevent the issues.
    The deadline looms, Nothing is complete. Everyone stays back late into the night. Works weekends. Their
    -> Heroics is astounding.
    We finally ship. We planned everything up front, Did everything the customer initially specified. Oh no, they didn’t really need that
  • Extra Feature that we spent months building. It has no business value. It was just a nice to have.

Believe it or not, but over 90% of the work you do is waste of some sort or another. I know, it’s an outlandish statement, but this is part of the mindset. The hard bit is trying to remove that 90%. Most of the stuff we do we think is required. That is why we do it. But if you really really look at it, there are ways we can automate, remove or change our practice to reduce. Over time, you will find that you are doing the same amount of work quicker. In most cases significantly quicker. You will also find that you can do more in the same amount of time as you do one thing now.
It is through the reduction and or removal of these wastes which is at the heart of Agile. It is one of the methods on how you get faster. What? You thought that doing Stand-Ups, Sprints and so forth would make you faster? How would it? you are adding extra processes (a form of waste) if you apply it to the way you currently work without changing the way you work. You are actually making things longer. Just think about it.

Feedback loops

The other tool to help with Continuous Improvment is the Feedback loop.
This is a universal tool to improve. There are many feedback loops. I mentioned one earlier. Peter Palchinsky’s principles.
Another one that is synonymous with Lean is the Demming Cycle, or the PDSA/PDCA loop which stands for Plan, Do, Check/Study and Act.

You

Plan what you are going to do.
Do what you are going to do.
Check or Study the results from what you have done.
Act on the results of that Check or Study and feed that back into the next Plan.

Another type of feedback loop created by Eric Ries of Lean Startup is the Build, Measure, Learn loop.

You Build a piece of the product. Just enough to get a minimum working product.
Release to your customers.
Measure their responses to that minimum viable product.
Learn based on the measurements what the next direction will be, then Build the next iteration. Do this continuously.

Yet another feedback look is the Observe, Orient, Decide, Act.

Observe the situation.
Orient yourself based on the observation.
Decide What to do.
Act Then do it.

In Scrum, there is a feedback loop within a feedback loop. I will go through that in more detail when I cover Agile methods. Especially Scrum.

Now, there is no sure way you can eliminate waste, adding feedback loops and short cycles will help, but ultimately this is a trial and error process. The problem with trial and error, is that you have to be able to accept error. In other words, failure.

Not Being Afraid To Fail

The fear of failure is generally what holds people and companies back. They become cautious, slow down. Take the easy route. If you fail, you are seen as a pariah. Failure is stigmatized so it is avoided at all costs. Eventually you stagnate.
People and companies that embrace failure, look at it as a learning experience actually become more healthy. They learn and grow. Some companies actually embrace failure. For example, at Etsy, the person who broke the build the most times at the end of the year is awarded a 3 armed sweater. Other companies hold a celebration. The failure is celebrated as a learning experience. It is investigated and ways are found to prevent the failure from happening again in the future.
Yes you may fail again, but so what. It’s part of learning.
Those of us who are in senior roles, why are you there? Is it because you got everything right? How did you get it right? Or Is it built on a lifetime of failures where you learned?

Further along, I’ll go through a game you can play to help remove the stigma of failure from your team.

It is my belief, and thats all it can be as I’m not a psychologist or expert on human behavior, that by taking these 4 core principles of Thinking of your customer (people), Having a willingness to try new thing, continuously try to improve and finally not being afraid to fail, will give you the required foundation to have an Agile mindset.

If you find it hard to have this mindset, and it will be difficult to change, my advice is “Fake it till you make it”. The mere fact that you are trying to change will help you to change.

What Is Agile

I’m going to try to write a book. It will be the third attempt at a book. The first one I started over 8 years ago on Java CAPS testing. I got about 30 pages in and it needed up abandoned. My last attempt was at DevOps.  This was started at the end of last year. I got a couple of chapters in, but it ended up being abandoned as well as I got stuck on a chapter. No time, I kept concentrating on my blogging and working on my Six Sigma Yellow Belt. This time, i’m going to try to write the chapters as blog posts. This kills two birds with one stone. I get a blog post out and a chapter. So it may be a couple of weeks between posts,but I will try to keep at least one per week.

So, here is the first chapter in its current form. Subject to change before the book is generated.

What Is Agile

Agile

  1. Able to move quickly and easily.
  2. Able to think and understand quickly.
    Oxford Dictionary (Online)

If you do a search for “Agile” online in the Oxford Dictionary you get the following which is related to the Agile methodology

Relating to or denoting a method of project management, used especially for software development, that is characterized by the division of tasks into short phases of work and frequent reassessment and adaptation of plans.
‘agile methods replace high-level design with frequent redesign’
Contrasted with waterfall (adjective)

The contrast to Waterfall is that Waterfall does the design up front of the whole solution, then you develop the whole solution. Finally you test then deploy the whole solution.

Although the definition is correct, I don’t like the following:
The phrase software development which I have emphasized denotes that Agile is specifically for Software Development, but this is now not true. Agile methods and techniques are now no longer in the software domain. It is being used to make cars (Wikispeed), Law Firms (Lonely Planet), Education, even the US military uses a form of Agile for combat.
As for Project Management again which I have emphasized, it no longer is specific for this domain. Agile in the Software Development industry is used for day to day activities, as it is with the other domains I have mentioned above.

For me:

Agile is an umbrella term for a set of methodologies that developed originally in the software industry but are being adapted outside of this domain to formalize continuous improvement, simplification of tasks and iterative/incremental approach to work, with the goal to allow change and adaptation to happen quickly and easily with minimal disruption.

Even this sentence doesn’t quite explain Agile as I see it.

In this book, I want to go through exactly what Agile is – at least to me. How to have an agile mindset, how to implement agile – at least one of the methodologies, Scrum and some techniques on how to expand Scrum to get the most benefit.

Firstly, lets go through the history of Agile.

History Of Agile

If you are reading this book. You most likely work in the IT industry. Either a Software Developer, Architect, Project Manager, Manager or so forth and are curious about this new Agile thing.
Well, Agile isn’t that new. In fact Scrum, the most popular implementation of Agile techniques is 21 years old at the time of writing, but the methods of Agile and the thinking process are even older than that.

In fact Agile techniques have been used for a long time, it just wasn’t called Agile.
For example, the construction of the Empire State Building. The building was designed in 2 weeks. They had 1 year to construct the 80 story high building ready for opening on the 1st of May 1931. They designed the building “after” purchasing the site. Construction at its peak was at 4 and a half floors per week and over 500 trucks per day coming to the site. The trucks would come up, their load unloaded and placed where it was needed then and there. No waiting, no putting aside, just work. Remember, this was 1930, so no computers to schedule everything. It was all done by hand and worked out on the fly. Planning was done for the weeks worth of work. Not months in advance, they just didn’t have the time. Workers had a problem solving attitude. Every day over the scheduled opening day would cost the company $10,000 1930’s per day. so whatever they could do to save time, even if it cost money was implemented. For example, they had a miniature railway to bring materials.
To top it all off, the project was completed 12 days ahead of schedule. That is amazing!
These techniques were not even new at the time, it was just how buildings were constructed.
This wasn’t standard across all industries. In fact the mass manufacturing techniques developed by Henry Ford and the teachings for Frederich Winslow Taylor were the standard.
Fast forward 10 years and then you have World War II. Men would go off to fight and women would be the ones working in the factories. These factories needed to be efficient. They needed to make the armaments. weapons, bombs, planes, tanks even clothing for the soldiers as fast as possible so that they could be used for the war effort. The quality needed to be high grade. Who would want their sons fighting a war with substandard equipment, but had to be made cheaply. W. Edwards Demming was part of a 5 man team that developed the statistical methods that helped the manufacturing standards during World War II. When the war ended, the men came back and things started to follow they way there were done before the war. All the efficiencies developed with the women were abandoned.

Deming and his colleagues efforts were not unnoticed. General Douglass MacArthur asked Deming and his colleagues to help Japan with their national census with statistical knowledge. In 1950, the Japanese Union of Scientists and Engineers had been studying Walter A. Shewarts techniques for reconstruction and manufacturing and since Deming had studies under Shewart, he was asked to speak and teach. One of those students of Demings was the co-founder of Sony.

Demings work in Japan was revered. An award for Quality was given in his name, the Deming Prize. Unfortunately in his own country his name was unknown and remained so until the 1980’s.

Another company that took these teachings on board was Toyota. More specifically, a man named Taiichi Ohno. He took the teachings further and developed techniques such as Just In Time (JIT) and eliminating waste (Muda) and Continuous Improvement (Kaizen). He is also considered the father of the Toyota Production System.
Toyota had a reputation of quality cars that were manufactured cheaply, and this didn’t come to light in the west until Toyota started exporting to the USA. Toyota made cars cheaper, and of better quality that any of the US manufacturers and it had them scared. Even so, it still took them decades to take on board the learnings. When they did, the name changed. You couldn’t have General Motors saying for example that they used the “Toyota Production System”, so the name changed to “Lean” to reflect the elimination of waste. Or “Trimming the fat”.

About the same time that this was all happening, many software developers were looking at better ways to develop software. The Waterfall method where you work out the requirements up front, design the system, implement, test then finally release was starting to fray at the edges. Customers would change their minds once they saw how things were working. They were not happy with the final product as it didn’t do what they wanted or since projects lasted 6months to 2 years or longer, by the time the software was released, it no longer met the requirements of the business. Something had to change. Something lightweight was needed compared to Waterfall. Using inspiration from Lean, a lot of new techniques were developed. DSDM (Dynamic Systems Development Method), (RAD) Rapid Application Development, Crystal Clear, (XP) Extreme Programming and Scrum.
Many of these techniques are still around over 20 years later, but the most common of these is Scrum.

In February 2001, seventeen software developers met together at the Snowbird resort in Utah to discuss lightweight software development techniques. Among those were the developers of Scrum. Jeff Sutherland and Ken Schwaber. There they published the Agile Manifesto.

We are uncovering better ways of develoing software by doing it and helping others do it. Through this work, we have come to value:
Individuals and Interactions over process and tools
Working Software over comprehensive documentation
Customer Collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

This is the moment when Agile became a thing.

Since then, we have had other Agile methodologies developed such as Kanban which was developed in 2010 by David Anderson and is modeled on the Kanban system used by Toyota to track the usage of parts and components during the manufacture of cars and Lean Software Development developed by Tom and Mary Poppendieck and DevOps which started Patrick Debois, John Alspaw, and Paul Hammond.

Many of the Agile methodologies have the following in common.

  1. The elimination of waste
  2. Continuous Flow of work
  3. Continuous Improvement
  4. People

Why Go Agile

There are many reasons why people start to go agile. These usually are

  • A CEO/CIO/ or some other level manager sees another company using Agile and decides that they should use agile too.
  • Agile is the new hot thing. We should do it!
  • My manager told me that we should use Agile.
  • Agile will make us go faster, reduce costs and increase quality.
    The one I like the best…
  • I want to improve and get better. I’ve heard Agile can do this.

Whatever reason you want to go Agile, I would like you to know what it involves which is why I am writing this book.
Agile can be easily implemented incorrectly. In fact, most companies that implement Agile get it done incorrectly because they do not research what they are doing. They see practices and just follow without understanding. The beauty of Agile is that even in these cases, there is some value still to be gained. The problem is that the value is only a small fraction of what can be accomplished compared to when Agile is implemented incorrectly. I will go through the pitfalls of going Agile later in the book.

Just remember, Agile isn’t a silver bullet. It can bring you all the above, but what it does do is let you know when something isn’t working sooner. What you do with that information is up to you.

 

Overwhelmed Part 2 – Teams

In my previous blog post, I talked about multitasking and how to handle it by using pull instead of push. In this post, I would like to expand on the concept to teams and my thoughts on how to optimize.

Now for this, I’m going to assume you are wanting to use Agile methods, but this could possibly work with a staggerd waterfall approach.

The Basics

In order to increase flow, you have to start at the basics.
Lets say you have 4 stages before your piece of work goes into prod.

Each piece of work goes through each stage.

As you have multiple pieces of work, each piece will go through each stage as it is pulled through the system to completion.
Since I’m assuming that you are doing Agile, each piece of work should take no more then a couple of days to complete. So initially, Test should be waiting 2 days before it starts testing. Stage/UAT should be waiting 4 days and nothing should be going to prod until about day 6. Now I’m talking stages here – not people. This system works if you have one person doing all 4 stages or if you have separate people for each stage.
Again, this is highly speculative, things may vary from situation to situation but I’ll get to that later.

Now, does this model remind you of an an assembly line? If so, that is exactly what we are trying to model here. Once we have the assembly line running with features going and everyone is working on something, our next step is to try to achieve flow.

Flow

Flow just as the name implies is that the work flows from one stage to the next without stopping. Now I hear you say, that is impossible to achieve – well that is not the point. The point is to try to achieve that goal. If you don’t try, them you definitely will not reach that goal.

Now I hear you say, work doesn’t flow through evenly through. There are variances. One task may take 2 days. Another will take 5 days at the same stage. Another half a day. Also, Testing may only take a half a day for something that took 2 days to develop, or vice versa. I will go through that later, but I’m more interested in averages, but when there isn’t an average, the largest time will do.

Analyze The Variance

This is the next step. Analyzing the variances.
In order to do this, we need to identify the time it takes for a feature to travel within each stage, and how long it waits between stages.
The time it takes within each stage helps us identify bottlenecks.
Say for example we have the following times on average for each stage.

Here, the Testers are staved for work. The same with UAT and well, we’ll just say migration to prod is the same.
In traditional manufacturing, the way to control flow is to limit the work to the slowest point. In this instance it is development, which takes 5 days. Any project manager with half a brain will tell you this isn’t acceptable, and they are right. The throughput is too low. So how do you increase the throughput?

Increase Throughput

One way is to increase the number of developers.
Now for our example we have developers releasing a feature every 2 days.

The Testers are now overworked. A backlog occurs before the testers and will keep occurring. This in itself is not a bad thing, but it needs to be managed. How can you do this, again, with Agile, cross functional teams, if the buffer for test gets too large then Developers can move across to the Testing stage and do testing. When the buffer drops down to a more manageable level, they can move back to Development.

What if increasing the number of developers isn’t an option but you still need to increase throughput. Another way to increase throughput is to identify tasks in the bottleneck stage and try to remove any non-value added work.
This in itself is a lot more difficult to do, but it is significantly cheaper. You have to really think what the developer is doing, determine if the step they are doing is really required or if the step can be done differently or more efficiently. It will take a lot of work, but ultimately it is cheaper for the simple reason you are not paying for another resource.

Now what I have said above isn’t new. Its been talked and implemented in the manufacturing world for at least the last 40 years and is known as part of the “The Theory Of Constraints”.

I know what you are thinking. It’s only a “Theory”. It’s just made up. Well, the person who came up with the “Theory” was Eliyahu M. Goldratt. A physicist, and in physics, a Theory is backed up by empirical evidence through many experiments. It is not something you just make up. It is only called a Theory because in physics as well in the sciences, there is always the possibility that the subject can be proven or disproved by a better process, equation or “Theory” that has been proven by empirical evidence. Goldratt applied the same principles to his “Theory of Constraints”.

In the book “The Goal”, Goldratt used an analogy of boy scouts walking along a track to describe this system and I’m going to steal the same analogy here.

Imagine a line of boy scouts walking down a track. Some boys are faster, and some are slower. The fast boys if they continue walking at their own pace will end up going so far ahead, that you can’t see them.
The boys that are too slow, end up falling behind and keep falling so far behind that you cannot see them either. So what is the solution. Goldratt used a proverbial rope to tie the boys together. The boys that go too fast get tugged back by the rope.
The boys that go at normal pace also get tugged by the rope. They all get tugged by the slowest person. There is no point pulling the last boy along as all you would be doing is dragging him along. Not a nice thought. So everyone walks at the slowest persons pace.
The only way you can get the whole troop to get faster is to increase the speed of the slowest person. The bottleneck. Goldratt called this “Drum, Buffer, Rope”.
Drum – The pace is set by the slowest person/process. ie, everyone follows the same rhythm of the drum.
Buffer – The slack in the rope. There needs to be some buffer to be able to handle outages and variances. Not too much buffer, nor too little. Experimentation will help determine the best level.
Finally, the rope. The rope implies a pull system. Work is pulled along the line rather than being pushed. As I mentioned in my previous post, this is how the Kanban system works. It makes sure that no person or process is overworked.

Goldratt had 5 steps to achieve this.

  1. Identify the bottleneck. I’ve described a couple of methods to do this above, but the only way you can identify a real bottleneck is through single piece flow.
  2. Exploit the bottleneck. What does this mean. Try to make the bottleneck faster by finding ways to quicken the work. Eliminate waste or try doing something differently to see if it does it quicker. Also make sure it is never staved of work.
  3. Subordinate the bottleneck. This is where you slow everything down to the bottleneck speed.
  4. Elevate the bottleneck. In this case, if you still need to increase throughput, add more resources.
  5. Finally, repeat the process fort the next bottleneck.

The steps are simple, but implementing them can be extremely difficult. Half the time you don’t know what you are doing, but this is where experimentation comes in. Keep trying new things, but remember Peter Palchinsky’s principle:

  1. Seek out and try new things.
  2. when doing something new, so it at a scale where failure is survivable, ie. not at the cost of your productivity or at a monetary cost so great that if it doesn’t work your company goes bankrupt etc.
  3. Finally, seek feedback and learn from your mistakes as you go along.

Waiting

Now, another thing I mentioned previously was recording the time between stages. In other-words, how long a piece of work waits before it is worked on.

This diagram is a very rudimentary value stream map. A value stream map shows how long it takes for a piece of work from conception to fruition to start giving value. When does a piece of work start giving value in software? When it is being used in production.
Here we have the times a piece of work waits before it is processed by the next stage. Those of you in Agile probably don’t see times like these, but with Waterfall, you definitely do. In this made up example, for a single piece on average to get to production there is 78 days of waiting! with only 6 1/2 days of active work being performed on this piece.

That means that it takes 84 days in this instance before a piece of code is helping your company make money. The shorter we make that time, the sooner the business gets the value, but also the sooner we are able to find out any problems and address them.

Now normally I would end here, this article gives some of the building blocks to improve throughput and increase speed, but as these techniques are simple and for quite a number of people seem to make sense, I thought I would go through a trap that I’ve seen.

False Flow or Multitasking By Stealth


I’ve seen the following done with some variances, but doing something similar using a Kanban Board for a team starting out with Scrum.
Here you have a piece of work flowing through the Kanban board, but its not actually a piece of work, its a task. A development task. Here, development tasks are being batched up, but because there is one card per task – not per piece of work, there is no flow although the movement of cards gives the illusion of flow.
I’ve also made it quite obvious here by naming the tasks “DEV”, but this is not always the case.

The correct way to implement is to have the work all encompassing of all stages of the development cycle. The individual tasks for each stage linked to the piece of work. This highlights when you are doing proper flow as opposed to false flow.

Using these techniques continuously over time, you should see improvements in throughput.

Please let me know your thoughts of this subject in the comments. Am I full of it or does this make sense. I’m very interested to know.

An Agile Story…Sort Of

This article first appeared on the Scrum Alliance Community Articles.

One of the most inspirational Agile stories I have heard isn’t really an Agile story. It’s the story about the New United Motor Manufacturing, Inc. (NUMMI). It’s a Lean/Toyota Production System story about how collaboration between two enterprises can lead to great things.

In the beginning . . .

The story starts at a General Motors (GM) manufacturing plant in Fremont, California. GM had been running the plant since the 60s, and by the early 80s, it was one of the worst auto manufacturing plants in America — not only in terms of quality of the cars manufactured, but also in terms of the workforce as well. The relationship between the workforce and management was, to put it mildly, adversarial. The workforce was unionized, so it was difficult to fire anybody, and management wanted throughput increased. Over time, the relationship degraded to the point where the behavior was careless and unprofessional.

Management’s hands were tied by the unions, but they made sure that the work got done. Why did things degrade to such a point? To put it simply, the workers had no pride in their work. It was only a job.

The cost of stopping the line

What was there to take pride in? All that was drilled into the workers was that the line had to move. If it didn’t, it would cost the company $15,000 per minute while it was down. If someone was fired, then the workforce would walk off the job, causing the company significant losses. Management pushed and pushed. There are accounts of people falling into the pit, yet they would not stop the line. If someone was injured, they would not stop the line unless it was an emergency. Find a defect, mark it with a slip of paper, and send the car on its way under the mindset, “It will be fixed later.”

And, boy! Was it worth fixing problems later! The overtime was lucrative. Be it a wrong bumper bar put on the car or something wrong under the dashboard, it was worth fixing later, even if it meant ripping out the entire dash, fixing the problem, and putting it back together. The dash would never be right again, but neither the workers nor management cared at that point. It was the customer’s problem. The fixing aspect was standard practice at the time, and the car quality suffered for it. (I remember that here, in Australia, there were problems with Holden Commodores. Car doors would not sit right, and “lemon laws” were introduced.) Eventually GM had enough, and in March 1982, the plant was closed. The entire workforce was laid off.

The partnership

Meanwhile, Toyota was trying to break into the American market. It entered negotiations with GM. GM would supply the facilities, and Toyota would teach them the Toyota Production System. Toyota was making cars more cheaply then, and of better quality than any American manufacturer could. So for GM to get the “secret sauce” was considered lucrative.

In 1984, the Fremont plant was chosen as the location, and the venture was called NUMMI. As part of the deal, more than 85% of the workforce that had been laid off two years earlier had to be rehired. Charles Duhigg’s book Smarter Faster Better: The Secrets of Being Productive in Life and Business recounts the story of Rick Madrid, one of the line workers who was fired from GM and rehired by NUMMI. Madrid recalls that after he was hired, he was sent to Japan to see how things worked. When he reached the Toyota plant, he remembers seeing everything pretty much as it had been in Fremont, but more clean. He recalls thinking, “Why the hell did they send me all the way here to see how things are done when it’s almost exactly the same as it was done at home?”

And then something happened: He saw a worker put in a bolt for a car door incorrectly. Nothing unusual about that; it was a common problem. But what happened next blew his mind away. Instead of marking the defect and then carrying on to the next job, the worker raised his hand and pulled on a cord. Lights flashed and music played. The worker then removed the bolt, and the line resumed. The worker’s supervisor arrived and asked questions. The worker ignored him and carried on, while giving his supervisor orders! Had anyone done that back in Fremont, they would have been decked.

When the car had reached the end of the station, something amazing happened. The entire line stopped. Another manager came up, but instead of yelling, the manager laid out a new bolt like a nurse lays out surgical instruments for a doctor. The worker then ordered both managers around. During this time, at every other station, workers were double-checking parts they had already installed. Rick was thinking that this was costing Toyota $15,000 per minute. Once the bolt was back in place correctly, the worker raised his hand and pulled the cord again. The line then resumed, and everyone went back to work. Rick was floored.

That scenario wasn’t the only surprise Rick Madrid saw while in Japan. Halfway through a shift, he observed a worker, who had an idea for a new tool to help him with his job, tell his manager about it. The manager went off to the workshop and, 15 minutes later, came back with a prototype. The two refined the tool for the rest of the shift. The next day, everyone their own similar tool.

When Rick got back to the States, he told his fellow workers what he saw. They didn’t believe him. What Rick saw was Toyota’s philosophy that the person doing the work is the expert at what needs to be done. The individual is in charge when something goes wrong; it is his or her responsibility to fix the problem. It is management’s responsibility to support the worker in fixing the problem. It is management’s responsibility to help, guide, and support the workers in doing their job, not just to order them about. Also, Toyota hates waste, and it’s a waste to not exploit everyone’s expertise.

Applying Toyota’s philosophy back home

When the Fremont plant reopened, those who returned to work didn’t go back to their old ways. They were too scared. They needed their jobs. Therefore workers didn’t start pulling their cords when things went wrong. This was not good, as Toyota’s philosophy is to fix problems at their earliest point when they occur — that is, at the time the problem is identified.

A month after the plant opened, NUMMI’s president Tetsuro Toyoda was touring the facility when he spotted a worker who had put in a taillight incorrectly and was struggling to get it back in. He looked at the man’s uniform for his name and said, “Joe, please pull the cord.” The president, Joe’s supervisor, and his manager were all watching.

“I can fix this, sir,” said Joe.

“Joe, please pull the cord,” repeated Tetsuro. Up to now, the cord had been pulled only three times since the opening of the plant, and one of those times was by accident. Everyone was afraid to cost the company $15,000 per minute.

Joe repeated, “I can fix this, sir.”

Tetsuro took Joe’s hand, raised it to the cord, and then pulled it. Joe was shaking. The two men then fixed the taillight and pulled the cord again. From that point on, workers started pulling the cord more often. Now, you would think that this would have been bad for Toyota. However, in a typical Toyota factory in Japan, the cord is pulled more than 5,000 times per day. Toyota’s philosophy of fixing problems when they occur has been found to be significantly cheaper on average than fixing the defect later. That is why pulling the cord is encouraged.

It didn’t take long, about six months, for the Fremont plant to produce cars of the same quality as those from a Japanese plant. These same workers, who two years earlier were causing all types of mischief, were taking pride in their work. In WBEZ’s This American Life broadcast and other interviews, Rick Madrid remembers fondly his time at NUMMI. He took pride in his work.

It’s a Scrum story, after all

Given that Scrum, specifically, and Agile have their origins in the Toyota Production System, it is my opinion that a ScrumMaster and a product owner have the responsibility to support team members so that they can do the best they can. If that means being ordered around by a team member to fix a P1 issue, resolve a blockage, or even give the team some tough love to point them in the right direction — basically anything that will help them succeed — then so be it. Anything short of that, and you introduce the possibility of failure. When workers are empowered to do a good job; aren’t struggling against impediments; and are given the responsibility, the right tools, control, and support, they will take pride in the work and be more productive. This, in my opinion, is what Scrum and Agile are about.

GM had sent 16 rising stars to start the NUMMI plant. Once the plant opened successfully, the 16 were ready to spread the word throughout GM. They were motivated; they wanted to change the world. But GM put a wet blanket over that spark. They didn’t know what to do with the 16. Some became frustrated over time and eventually quit. All that knowledge was lost. It took GM more than 15 years to implement the lessons learned at NUMMI, but by then it was already too late. In 2009, in the middle of the global financial crisis, the auto industry was all but destroyed. The fallout is still being felt today, with the closure of car manufacturing here in Australia. GM became the largest industrial bankruptcy in U.S. history. The NUMMI plant closed a year later, in 2010, although it has recently reopened as a Tesla plant.

Rick Madrid retired in 1992. His hat and badge are at the Smithsonian in the American Enterprise exhibit.

References

User Stories

This post was originally published on the Scrum Alliance Community Articles

User stories are, well, stories. This should be obvious, but, unfortunately, it isn’t.

They are not statements, such as “Build a new front page.” They are a story. Stories have a hero or heroine, a goal, and a quest.

User stories are not riveting, but they tell us, from a user’s perspective, what the user is trying to accomplish and, more importantly, why. They are not absorbing novels; they take the form of only three lines.

Here I want to go through my understanding of what user stories are, how to write one, and when to use one. User stories are just like any other tool — there are times when you should use them, and times when you shouldn’t.

How it all began

In Waterfall, the first thing developed in software was the “requirements specification.” (Some of us, myself included, still write these today.)

This formal document described, in intricate detail, what was needed in the development. Who was responsible, who wrote, who changed things. If you were lucky, it also included the why. The document was used as a way to get agreement from the customer on what the development was supposed to be.

It sounds logical to think through what needs to be developed before you start the work. Making it formal makes it, effectively, a contract. In most cases it did become the contract. It also allows you to figure out up front how much effort is involved.

The problem was that, once development started, people would change their minds. Customers would have changing requirements. For a project that lasts six months or more, priorities can shift. What was business critical when the requirements were written can become irrelevant by the time feature development started. The difficulty of what needed to be done could be significantly more than what was initially envisioned. Agile came about to handle this situation from a development perspective, but something needed to be done from a requirements perspective. A requirement was still needed — developers needed to know what to build, but they didn’t need a detailed specification, just enough information to have an idea of what the customer was thinking so they could get started. The details could be worked out along the way, through talking with the users.

So some bright spark developed the idea of using a story to tell what the user was trying to achieve, and the practice has developed over the years.

Why use user stories?

One of the most obvious reasons to write user stories is that they are only three lines long (in the classic case), as opposed to being a formal document of several (or more) pages. That means there is less administration. You don’t need to update the user stories.

Now I know what some of you are thinking: A user story is only three lines long, so it cannot have much detail compared to a traditional requirements document. You’re right. They don’t have the detail, but that is the point. A user story is not a replacement for a requirements document, it’s a trigger to have a conversation with the relevant people, when required, about the requirements. In fact, the idea is to have continuous collaboration with the customer through discussions, and ongoing confirmation to keep refining the product.

Since user stories are small, they are also easier to estimate accurately.

User stories promote smaller iterations of work. With smaller iterations, you can get feedback sooner, which helps the refining process. It also allows you to change priorities as needed, without significantly affecting other pieces of work already in flow.

The INVEST principle

To explain all this a bit more clearly, we have the INVEST principle. INVEST names a list of principles that each story should adhere to so that we get the most out of each story and can write them better. INVEST is an acronym that stands for:

  • Independant. Each story needs to be independent of the other stories. This allows stories to be prioritized separately and cuts down on dependencies. For example, you cannot complete one story if you are waiting on another story to be complete. Or you cannot test a story if it is dependent on another story.
  • Negotiable. Teams need to be able to negotiate stories with the product owner. Circumstances change, and the team or the product owner needs to be able to reconsider how the story is both implemented and scoped. An example may be that the story is not technically possible. (Howevr, even though the story can be negotiated, the product owner still remains accountable for it.)
  • Valuable. A story must have some value to the business. This is to prevent additional features from being developed that end up being of no use, or that the developer thought were good ideas at one point but have since become of no value.
  • Estimable. The story needs to be written in such a way that the team can estimate how much effort is involved. This tries to avoid the problem of scheduling or making promises based on inadequate information.
  • Small. Stories are small increments of functionality. This helps helps the project be built by small bits of value at a time, rather than by large chunks that might end up not being used.
  • Testable. This helps make the requirement a known entity. It gives you something that you can verify. It is part of the Definition of Done. It is to prevent stories such as, “Write requirements” and prevents ambiguities such as, “The page must load quickly.” Instead, a measurable requirement is given, such as, “Must load within 10 seconds.”

Now that we know the principles of a story, how do we format them? There is a formula.

3 C’s Formula

The 3 C’s formula outlines how stories are put together. They are:

  1. Card: User stories are captured on a card (usually a 3×5). The description is on the front, and the acceptance criteria are on the back.
  2. Conversation: The card is a trigger for discussion of the requirements among:
    • Team members
    • Team members and the product owner
    • Team members and stakeholders\users\customers
    • All of these groups
  3. Confirmation: Finally, the user story helps the team discuss with the product owner how to determine when a feature/piece of work is complete.

Now we come to the juicy bit.

How to write a user story

So, to summarize, user stories are a description of a requirement in story form, from the user’s point of view. They are not written from the developer’s point of view, nor that of the product owners, but from the point of view from the person who is actually going to use this thing.

User stories are also short and succinct. They do not waffle on (like this article).

However, they can take one of many forms. Here are a couple of useful templates.

User story templates

The following is the classic version: As a [role]
I want to [requirement]
So that [reason/ROI]

The role is the “who,” our hero/heroine, the user. The requirement is the “what,” or quest, that our hero/heroine is trying to achieve. The reason/ROI is the “why.” Why does the user want the requirement? What is he or she trying to achieve? In other words, what is the goal?

An example based on a traditional story could be:

As a member of the Rebel Alliance,
I want to destroy the Death Star
So that I can save the galaxy.

Another variation on this template is the “In order to” template. It goes like this:

In order to reason/ROI
As a role,
I want to requirement.

Same information, but different wording. These are the more traditional ways of writing user stories, but they are not the only ways.

Comics

I attended a seminar at Elabor8 on using comics as a method of writing user stories. As they say, a picture is worth a thousand words, what better way than with a comic strip?

Can’t draw? It doesn’t matter. Use stick figures. Alternatively, if you are really worried about your drawing ability, try photos.

Movies

With the prevalence of smartphones, we almost all now have a video camera in our pocket. So, if it makes sense, why not act out the scenario of the story? It’s another way to express the requirements, and who knows, it might be fun. You might even be able to record a statement of the requirements, if the product owner is willing.

These methods can then be captured in your software-based backlog if you use something like JIRA, or you can put them on a web server, and print out links for physical cards, or insert links in electronic ones.

The whole idea is to try different things. See what works. If it’s a waste of time, you’ll know quickly; but it might boost your understanding of the story.

Impact mapping

One tool I like is Impact Mapping, created by Gojko Adzic. An impact map is a type of mind map that allows you to brainstorm and explore scenarios. “Branches” are then trimmed based on the team’s ability to implement them. Ultimately you are left with the branch you will implement. The impact map is drawn in such a way that you have who the feature is for, why the user wants the feature, and finally the “What” and “How” you need to do the feature. This is what the user wants.

An example of an impact map is below:


With an impact map, you start of with the why. Why are you doing this? In the above example, we are trying to save the galaxy. Next, we look at who is trying to achieve the goal. This would be our user, the our hero or heroine. In this case, it is a member of the Rebel Alliance. Next we have the how we are going to achieve our goal. This is where the brainstorming really starts. List as many ways as you can think of to achieve the goal. It doesn’t matter if they are realistic, just write them down. Who knows, they may lead to something.

Finally, for each how list, what needs to be done. Keep everything at a high level. We are not trying to solve every branch, only to get an idea of what is involved.

Once you have your impact map, you start trimming branches. Something may be impractical. Something may not be ready to do. Something may require resources that you do not have.

Eventually, after several iterations, you should have a single path to follow. But don’t spend too much time on this map; it is just there to give a visualization of what needs to be done, along with alternative views. You still may end up selecting the original proposal, but it will be a definitive answer with alternatives having been explored.

Now that you finally have the Why, Who, How, and What, you can write your story:

As a member of the Rebel Alliance,
I want to destroy the Death Star
By Sending two torpedos down an exhaust pipe that has been found to be a weakness in the Death Star’s defenses,
In order to save the galaxy.

Now that we’ve filled out the front of the card, it is time to look at the back.

Acceptance criteria

On the back of the card (or its software equivalent), you have the acceptance criteria. It outlines how the story will be verified as Done by the product owner. This needs to be hard criteria, not anything ambiguous. It’s best that the criteria be testable. If you can test it, you can define it.

Just like the story, the product owner is responsible for writing the acceptance criteria (with consultation from the team). For example, business analysts can help write the acceptance criteria, but it is up to the PO to accept them. Sometimes the acceptance criteria are used as part of the Definition of Done.

Acceptance criteria template

Just as there is a template for the user story, you can use a template for the acceptance criteria. An example:

A [role] should be able to:

  • [Acceptance Criteria 1]
  • [Acceptance Criteria 2]
  • [Acceptance Criteria 3]

Edge cases:

  • [Edge Case 1]
  • [Edge Case 2]

Separating out the edge cases helps your thinking process. You work on the normal cases first, then brainstorm the edge cases.

This acceptance criteria is a placeholder for further discussion to get confirmation from the PO. Then more detailed acceptance criteria can be determined.

Gerkins and Cucumbers

For acceptance criteria, I like the Gerkin language. It was initially developed for use by a program called Cucumber. Cucumber is a behavior-driven development program that allows you to specify acceptance criteria in plain English, then execute those statements as tests. Yes, execute!

Gerkin takes the form of “Given, when, then” statements. The Given section is your preconditions. It’s where you set the stage for what needs to be done. The When section is when you actually do something to the preconditions. The Then section is where the expected effect is described. For example:

Given two numbers, 1 and 3,
When I add these two numbers together,
Then I expect the result to be 4.

Nice and simple, easily understood not only by developers but by anyone who can read.

Now, I mentioned that Gerkin is used by Cucumber, but that is where it started. There are now a number of products that use this language to produce executable specifications. These include, but are not limited to:

  • JBehave
  • EasyB
  • Gwen

The beauty is that you don’t need to use these products to get the benefit. Just writing your acceptance criteria in this format means that it’s easily readable, verifiable, and future proof. So, for example, if in the future you wish to use one of the above products to make the acceptance criteria executable, the writing of the tests is done.

There are other types of programs that also allow you to write executable acceptance criteria. These are more free form and allow you to do more traditional documentation; Concordion and Fitnesse are examples.

Concordion uses HTML pages to write the acceptance criteria, and Fitnesse uses a wiki.

With most of these products, to make the specifications executable, bridging code is required to allow the interpreter to execute the tests. Usually the bridging code can be simple and reusable. But determining the complexity depends on what type of application you’re writing. I won’t go into how to write these, but I recommend investigating them anyway.

Epics and themes

Now we come to stories that are too big to fit into one sprint. These are called epics and themes.

So . .

What is an epic?

To put it simply, an epic is a very large story. Not in terms of description, but in terms of implementation. An epic is a story that can take more than one sprint to complete.

In the “Story World” (going with our Star Wars theme), an epic would be the Star Wars Trilogy. The epic in this case is made of (originally) three stories, Star Wars, The Empire Stikes Back, and Return of the Jedi. Each of those stories is watched in one sitting, just as a story will be done in one sprint.

What, then, makes a story an epic? There can be a number of reasons: A story may be at a very high level and need more information. Usually when the backlog is first created, all stories are considered epics. In other words, the epic is used to capture requirements at a very high level. Another reason could be simply that the story ended up being bigger than you initially thought, and therefore it cannot be completed in one sprint.

Epics are not the only way large stories can be described. We can also use themes.

What is a theme?

A theme is a collection of stories that have a shared attribute. For example:

As a Star Wars fan,
I want a group of pages to explain the movies
So that people can learn more about the Star Wars movies.

Here, the word group insinuate that the PO wants more than one page. So, this theme can be broken down into:

As a Star Wars fan,
I want a “New Hope” page
So that I can see details about the first Star Wars movie.

As a Star Wars fan,
I want a “The Empire Strikes Back” page
So that I can see details about the Empire Strikes Back movie.

and so on. As you can see, the two stories are identical except that they discuss different movies. The theme here is that they are both pages about Star Wars movies.

Again, just like epics, themes generally cannot be completed within one sprint, unless of course the stories are quite small. This means that the stories themselves are completed in the relevant sprint, but the theme, just like an epic, is completed over multiple sprints.

Splitting and combining stories

So now that you know what an epic and a theme is, there may be times when you want to split an epic or theme into smaller stories, and there may be times when you want to combine multiple stories into an epic or a theme. Remember, an epic or a theme is still just a story.

When to split stories

Most of the time, you will want to split a story because it cannot be completed within one sprint. The work is too large for one sprint, so you split the story into two separate stories. You complete one story in the first sprint and the other story in the second. The original epic or theme is then complete.

Another reason to split stories is to get a more accurate estimate. It might be too difficult to estimate a piece or work, so break it down into more manageable chunks, then estimate the chunks individually. Estimating smaller chunks is easier.

How to split stories

Splitting stories can be tricky, simply because there are so many possible ways to do it, and not all of them result in split stories that make sense. Here are some examples.

Does not meet performance constraints. Let’s say you are building a web page that has to load within five seconds. Building the page is easy, but let’s say that the performance of the page will prove difficult; to make it load within five seconds is extremely hard. What do you do? Split the story. The first story is the building of the page. This gives the product owner value. They have a page that the end user can in fact use. The next story is the performance criteria. This will be tackled later. Maybe the next sprint, maybe five sprints later, maybe never, if it turns out that the customers don’t care that the page takes 20 seconds to load.

Separate or mixed priority. This is when some of the acceptance criteria is of higher priority than other acceptance criteria. Split the story based on the different priorities of the acceptance criteria. Do the higher-priority stories first and leave the lower-priority ones until later.

To remove crosscutting concerns. This is when a story has acceptance criteria that applies to more than just that one story. Logging, error handling, and security are examples.

When to combine stories

There may be times when you would like to combine stories. Perhaps the stories are related and both could be completed within one iteration. For example, in transactional processing, you may want to combine the pick-up and drop-off of a transaction in one story. You may also want to combine stories when you want to prioritize them as a group. For example, two stories need to be completed together.

How to combine stories

There are a number of ways to combine stories. They are all similar.

  • Create an epic and then refer the individual stories to the epic.
  • Create a new story, so long as it can fit within a sprint.
  • Create a theme if it is a set of related stories.
  • The whole idea of splitting and combining is to refine the backlog. Combine stories so that twp people do not put the same amount of effort into one story, only to find that the other is similar enough they could have been done in one. Split stories so that larger chunks of work are broken down into more manageable sizes.

Conclusion

In this article I have tried to show how stories work and how they should be used. Take from this what you are most comfortable with, but if you do use stories, make sure you know the correct terminology, what a story is, and how to use it. That is the knowledge I have tried to give you, based on my own understanding. If your understanding of stories is different, please let me know in the comments. We are all learning, and the best way to learn is to share knowledge.

Overwhelmed With Work?

Have you ever felt you have been inundated with too much work? You have been delegated a multitude of tasks, everything has been pushed onto you and you just don’t know what you should be working on, you just keep switching between tasks and get nothing done.

The reason nothing gets done is because you keep switching between tasks, in other words multitasking.
So how does this prevent you from getting things done. Well, let’s keep this simple. Say we have 3 tasks to do. A,B and C. Each task takes 10 units to complete. A unit could be minutes, hours, days, weeks it doesn’t matter. For this example, we’ll use days.

Now let’s say we do A, B and C in sequential order. What is the lead time to complete A? Well, 10 days. What is the lead time to complete B? 10 days, C? 10 days again.
Nice and simple.

Now let’s say we do a split. We do half of each task, let’s say that the first half is development. The second half is testing. Deployment to production is negligible. So we stick with 10 days to complete a task.

So we do all our development up front.
5 days for A, 5 days for B and 5 days for C. Then we do our testing. 5 days for A, 5 days for B and 5 days for C.

Now, what is the lead time? For completion of A, it is now 20 days. We have doubled our time! For B, it is also 20 days, and the same with C. OK, the overall time is still 30 days, but let’s say there is an issue. Say, during the first 5 days of B. It takes 6 days. Now it takes 21 days to complete A. A is now over due. But not only is A over due, B and C are over due. The more the delays, the more the cascading affect.

This is a basic example, but there are other things you need to consider with multitasking, context switching. Changing between tasks means a mental dump and load for the next task. Then when you switch back, you have to remember what you had previously done, and then continue on. This context switching takes time. If the two tasks are completely different, then the switching time is less. If they are similar, then the switching time is longer. For those tasks that take minutes or hours that require a high level of concentration, and most IT work does require a high level of concentration, being “in the zone” when working is when you are at your most productive. Getting in the zone takes time and breaking that concentration drops your productivity. In most cases, quite significantly.

So in the modern workplace, why is there multitasking. Quite simply, people get assigned more than one task at a time. You know the drill. Your boss needs something done, looks to you and asks if you can do it. You say yes because you are only working on one task. It happens again and again and now you are juggling 3 or more tasks. Then you look at your colleague and you see that they are only working on one task, or worse yet, they are twiddling their thumbs.

This seems to be standard practice in many organizations,but what is the alternative? It’s simple, instead of pushing, try pulling.
You have your backlog of work. No one is assigned a particular task. If you are free, you take the next task in the line. This methodology allows you to work on one task at a time and focus. It allows even distribution of work load among people.

In Agile, this is represented by the Kanban board. Funny name Kanban, it comes from Toyota. In Japanese it literally means “signboard”. In order to limit inventory in Toyota, cards were used to indicate when someting was used. For example, say you have a tub of bolts used to attach a door to a car. There would be 2 tubs. The first where you take the bolts to attach the door, the second for when the first runs out. Attached to each tub, is a card. When you run out of bolts, you start eating from the second tub. The card on the first tub is then used as an indicator for someone to get you another tub of bolts. This tub becomes your second tub. The taking of bolts from inventory then triggers a purchase order to purchase new load of bolts. Toyota has got this down to a fine art where they get several deliveries per day of parts such as bolts to be used within that 24 hour period. Anyway, back to Agile. Since IT work is non tangible, cards are used as an avatar. The kanban board is a signboard or visual indicator of the progress of the work. People “pull” the cards from “todo” to “doing” while limiting them self to only one card in this column, then they “pull” the card to “done” which then triggers the “pulling” of the next card off the “todo” column. If something is blocked, then this “can” trigger a second card to be pulled from “todo”, but keep in mind, this should be the exception rather than the norm. During the retrospective, it should be investigated and discussed how to prevent the particular instance of a blocked card from happening again.

Now, a question. How do you limit the amount of work done if you have tasks that take 10 days as per your example? What happens when something more urgent comes in?
Well, the first mistake I did with my example was to use tasks that took 10 days. Ideally you should have tasks that take only 1/2 to 1 day. That way if something more urgent comes along, there is only a half to 1 day wait.
But what about an emergency? Such as something goes down, we can’t wait half a day to a day to get things back up and running. Well, this is an exception. In the case of an emergency, its all tools down and the emergency takes priority. Everyone who needs to be involved gets involved. Fix the issue, then once everything is back to normal then you resume the process.

Pragmatism

When there is resistance to being Agile or doing best Agile Practices, “Pragmatism” is usually used as an excuse to skip important practices or disciplines, or due to lack of knowledge.

For example, “We are going to skip the Retrospectives because we are under the pump and we need to be pragmatic”

In this example, the work is seen to be important, and the retrospective is not. So it is dropped.
Why is this bad? Retrospectives are a way to look back on the work done in the past sprint and see how it can be done better. If they are dropped, then you loose the learning part of Agile, which is one of the main parts of Agile.
It could also mean your retrospectives are not bringing value. If that is the case, another format may be in order.
Another example of pragmatism being used for skipping important practices and disciplines is skipping planning sessions.

“We already know what we are doing, so we don’t need to plan”
This can happen in a “Command and Control” type structure, the person giving the orders doesn’t need to plan as he is giving the orders to those below him. This again isn’t agile, just another excuse for cowboy practices but of a different kind.
There is minimal team involvement, which can lead to people “switching off” as they have no say or control.

Another way pragmatism can be used as an excuse is to “Brute Force” the work.
“We have 50 servers to set up, we don’t have time to automate, we have to do it manually”.
At the end of the 50 servers installed, you have no automation. Something else will come up preventing you from automating, and the next time you have to provision another 50 servers, that will be done manually too and so on and so forth. This is what I like to call the death spiral of work.

At the moment, I have no answers on how to get around the “Pragmatism” argument, especially if you have no authority. I would recommend education, but even that isn’t a guarantee as those resistant are less likely to be open enough to be educated.

Thoughts, Tutorials and Learnings specifically with regards to Agile, DevOps and SOA/EAI