Overwhelmed Part 2 – Teams

In my previous blog post, I talked about multitasking and how to handle it by using pull instead of push. In this post, I would like to expand on the concept to teams and my thoughts on how to optimize.

Now for this, I’m going to assume you are wanting to use Agile methods, but this could possibly work with a staggerd waterfall approach.

The Basics

In order to increase flow, you have to start at the basics.
Lets say you have 4 stages before your piece of work goes into prod.

Each piece of work goes through each stage.

As you have multiple pieces of work, each piece will go through each stage as it is pulled through the system to completion.
Since I’m assuming that you are doing Agile, each piece of work should take no more then a couple of days to complete. So initially, Test should be waiting 2 days before it starts testing. Stage/UAT should be waiting 4 days and nothing should be going to prod until about day 6. Now I’m talking stages here – not people. This system works if you have one person doing all 4 stages or if you have separate people for each stage.
Again, this is highly speculative, things may vary from situation to situation but I’ll get to that later.

Now, does this model remind you of an an assembly line? If so, that is exactly what we are trying to model here. Once we have the assembly line running with features going and everyone is working on something, our next step is to try to achieve flow.

Flow

Flow just as the name implies is that the work flows from one stage to the next without stopping. Now I hear you say, that is impossible to achieve – well that is not the point. The point is to try to achieve that goal. If you don’t try, them you definitely will not reach that goal.

Now I hear you say, work doesn’t flow through evenly through. There are variances. One task may take 2 days. Another will take 5 days at the same stage. Another half a day. Also, Testing may only take a half a day for something that took 2 days to develop, or vice versa. I will go through that later, but I’m more interested in averages, but when there isn’t an average, the largest time will do.

Analyze The Variance

This is the next step. Analyzing the variances.
In order to do this, we need to identify the time it takes for a feature to travel within each stage, and how long it waits between stages.
The time it takes within each stage helps us identify bottlenecks.
Say for example we have the following times on average for each stage.

Here, the Testers are staved for work. The same with UAT and well, we’ll just say migration to prod is the same.
In traditional manufacturing, the way to control flow is to limit the work to the slowest point. In this instance it is development, which takes 5 days. Any project manager with half a brain will tell you this isn’t acceptable, and they are right. The throughput is too low. So how do you increase the throughput?

Increase Throughput

One way is to increase the number of developers.
Now for our example we have developers releasing a feature every 2 days.

The Testers are now overworked. A backlog occurs before the testers and will keep occurring. This in itself is not a bad thing, but it needs to be managed. How can you do this, again, with Agile, cross functional teams, if the buffer for test gets too large then Developers can move across to the Testing stage and do testing. When the buffer drops down to a more manageable level, they can move back to Development.

What if increasing the number of developers isn’t an option but you still need to increase throughput. Another way to increase throughput is to identify tasks in the bottleneck stage and try to remove any non-value added work.
This in itself is a lot more difficult to do, but it is significantly cheaper. You have to really think what the developer is doing, determine if the step they are doing is really required or if the step can be done differently or more efficiently. It will take a lot of work, but ultimately it is cheaper for the simple reason you are not paying for another resource.

Now what I have said above isn’t new. Its been talked and implemented in the manufacturing world for at least the last 40 years and is known as part of the “The Theory Of Constraints”.

I know what you are thinking. It’s only a “Theory”. It’s just made up. Well, the person who came up with the “Theory” was Eliyahu M. Goldratt. A physicist, and in physics, a Theory is backed up by empirical evidence through many experiments. It is not something you just make up. It is only called a Theory because in physics as well in the sciences, there is always the possibility that the subject can be proven or disproved by a better process, equation or “Theory” that has been proven by empirical evidence. Goldratt applied the same principles to his “Theory of Constraints”.

In the book “The Goal”, Goldratt used an analogy of boy scouts walking along a track to describe this system and I’m going to steal the same analogy here.

Imagine a line of boy scouts walking down a track. Some boys are faster, and some are slower. The fast boys if they continue walking at their own pace will end up going so far ahead, that you can’t see them.
The boys that are too slow, end up falling behind and keep falling so far behind that you cannot see them either. So what is the solution. Goldratt used a proverbial rope to tie the boys together. The boys that go too fast get tugged back by the rope.
The boys that go at normal pace also get tugged by the rope. They all get tugged by the slowest person. There is no point pulling the last boy along as all you would be doing is dragging him along. Not a nice thought. So everyone walks at the slowest persons pace.
The only way you can get the whole troop to get faster is to increase the speed of the slowest person. The bottleneck. Goldratt called this “Drum, Buffer, Rope”.
Drum – The pace is set by the slowest person/process. ie, everyone follows the same rhythm of the drum.
Buffer – The slack in the rope. There needs to be some buffer to be able to handle outages and variances. Not too much buffer, nor too little. Experimentation will help determine the best level.
Finally, the rope. The rope implies a pull system. Work is pulled along the line rather than being pushed. As I mentioned in my previous post, this is how the Kanban system works. It makes sure that no person or process is overworked.

Goldratt had 5 steps to achieve this.

  1. Identify the bottleneck. I’ve described a couple of methods to do this above, but the only way you can identify a real bottleneck is through single piece flow.
  2. Exploit the bottleneck. What does this mean. Try to make the bottleneck faster by finding ways to quicken the work. Eliminate waste or try doing something differently to see if it does it quicker. Also make sure it is never staved of work.
  3. Subordinate the bottleneck. This is where you slow everything down to the bottleneck speed.
  4. Elevate the bottleneck. In this case, if you still need to increase throughput, add more resources.
  5. Finally, repeat the process fort the next bottleneck.

The steps are simple, but implementing them can be extremely difficult. Half the time you don’t know what you are doing, but this is where experimentation comes in. Keep trying new things, but remember Peter Palchinsky’s principle:

  1. Seek out and try new things.
  2. when doing something new, so it at a scale where failure is survivable, ie. not at the cost of your productivity or at a monetary cost so great that if it doesn’t work your company goes bankrupt etc.
  3. Finally, seek feedback and learn from your mistakes as you go along.

Waiting

Now, another thing I mentioned previously was recording the time between stages. In other-words, how long a piece of work waits before it is worked on.

This diagram is a very rudimentary value stream map. A value stream map shows how long it takes for a piece of work from conception to fruition to start giving value. When does a piece of work start giving value in software? When it is being used in production.
Here we have the times a piece of work waits before it is processed by the next stage. Those of you in Agile probably don’t see times like these, but with Waterfall, you definitely do. In this made up example, for a single piece on average to get to production there is 78 days of waiting! with only 6 1/2 days of active work being performed on this piece.

That means that it takes 84 days in this instance before a piece of code is helping your company make money. The shorter we make that time, the sooner the business gets the value, but also the sooner we are able to find out any problems and address them.

Now normally I would end here, this article gives some of the building blocks to improve throughput and increase speed, but as these techniques are simple and for quite a number of people seem to make sense, I thought I would go through a trap that I’ve seen.

False Flow or Multitasking By Stealth


I’ve seen the following done with some variances, but doing something similar using a Kanban Board for a team starting out with Scrum.
Here you have a piece of work flowing through the Kanban board, but its not actually a piece of work, its a task. A development task. Here, development tasks are being batched up, but because there is one card per task – not per piece of work, there is no flow although the movement of cards gives the illusion of flow.
I’ve also made it quite obvious here by naming the tasks “DEV”, but this is not always the case.

The correct way to implement is to have the work all encompassing of all stages of the development cycle. The individual tasks for each stage linked to the piece of work. This highlights when you are doing proper flow as opposed to false flow.

Using these techniques continuously over time, you should see improvements in throughput.

Please let me know your thoughts of this subject in the comments. Am I full of it or does this make sense. I’m very interested to know.

An Agile Story…Sort Of

This article first appeared on the Scrum Alliance Community Articles.

One of the most inspirational Agile stories I have heard isn’t really an Agile story. It’s the story about the New United Motor Manufacturing, Inc. (NUMMI). It’s a Lean/Toyota Production System story about how collaboration between two enterprises can lead to great things.

In the beginning . . .

The story starts at a General Motors (GM) manufacturing plant in Fremont, California. GM had been running the plant since the 60s, and by the early 80s, it was one of the worst auto manufacturing plants in America — not only in terms of quality of the cars manufactured, but also in terms of the workforce as well. The relationship between the workforce and management was, to put it mildly, adversarial. The workforce was unionized, so it was difficult to fire anybody, and management wanted throughput increased. Over time, the relationship degraded to the point where the behavior was careless and unprofessional.

Management’s hands were tied by the unions, but they made sure that the work got done. Why did things degrade to such a point? To put it simply, the workers had no pride in their work. It was only a job.

The cost of stopping the line

What was there to take pride in? All that was drilled into the workers was that the line had to move. If it didn’t, it would cost the company $15,000 per minute while it was down. If someone was fired, then the workforce would walk off the job, causing the company significant losses. Management pushed and pushed. There are accounts of people falling into the pit, yet they would not stop the line. If someone was injured, they would not stop the line unless it was an emergency. Find a defect, mark it with a slip of paper, and send the car on its way under the mindset, “It will be fixed later.”

And, boy! Was it worth fixing problems later! The overtime was lucrative. Be it a wrong bumper bar put on the car or something wrong under the dashboard, it was worth fixing later, even if it meant ripping out the entire dash, fixing the problem, and putting it back together. The dash would never be right again, but neither the workers nor management cared at that point. It was the customer’s problem. The fixing aspect was standard practice at the time, and the car quality suffered for it. (I remember that here, in Australia, there were problems with Holden Commodores. Car doors would not sit right, and “lemon laws” were introduced.) Eventually GM had enough, and in March 1982, the plant was closed. The entire workforce was laid off.

The partnership

Meanwhile, Toyota was trying to break into the American market. It entered negotiations with GM. GM would supply the facilities, and Toyota would teach them the Toyota Production System. Toyota was making cars more cheaply then, and of better quality than any American manufacturer could. So for GM to get the “secret sauce” was considered lucrative.

In 1984, the Fremont plant was chosen as the location, and the venture was called NUMMI. As part of the deal, more than 85% of the workforce that had been laid off two years earlier had to be rehired. Charles Duhigg’s book Smarter Faster Better: The Secrets of Being Productive in Life and Business recounts the story of Rick Madrid, one of the line workers who was fired from GM and rehired by NUMMI. Madrid recalls that after he was hired, he was sent to Japan to see how things worked. When he reached the Toyota plant, he remembers seeing everything pretty much as it had been in Fremont, but more clean. He recalls thinking, “Why the hell did they send me all the way here to see how things are done when it’s almost exactly the same as it was done at home?”

And then something happened: He saw a worker put in a bolt for a car door incorrectly. Nothing unusual about that; it was a common problem. But what happened next blew his mind away. Instead of marking the defect and then carrying on to the next job, the worker raised his hand and pulled on a cord. Lights flashed and music played. The worker then removed the bolt, and the line resumed. The worker’s supervisor arrived and asked questions. The worker ignored him and carried on, while giving his supervisor orders! Had anyone done that back in Fremont, they would have been decked.

When the car had reached the end of the station, something amazing happened. The entire line stopped. Another manager came up, but instead of yelling, the manager laid out a new bolt like a nurse lays out surgical instruments for a doctor. The worker then ordered both managers around. During this time, at every other station, workers were double-checking parts they had already installed. Rick was thinking that this was costing Toyota $15,000 per minute. Once the bolt was back in place correctly, the worker raised his hand and pulled the cord again. The line then resumed, and everyone went back to work. Rick was floored.

That scenario wasn’t the only surprise Rick Madrid saw while in Japan. Halfway through a shift, he observed a worker, who had an idea for a new tool to help him with his job, tell his manager about it. The manager went off to the workshop and, 15 minutes later, came back with a prototype. The two refined the tool for the rest of the shift. The next day, everyone their own similar tool.

When Rick got back to the States, he told his fellow workers what he saw. They didn’t believe him. What Rick saw was Toyota’s philosophy that the person doing the work is the expert at what needs to be done. The individual is in charge when something goes wrong; it is his or her responsibility to fix the problem. It is management’s responsibility to support the worker in fixing the problem. It is management’s responsibility to help, guide, and support the workers in doing their job, not just to order them about. Also, Toyota hates waste, and it’s a waste to not exploit everyone’s expertise.

Applying Toyota’s philosophy back home

When the Fremont plant reopened, those who returned to work didn’t go back to their old ways. They were too scared. They needed their jobs. Therefore workers didn’t start pulling their cords when things went wrong. This was not good, as Toyota’s philosophy is to fix problems at their earliest point when they occur — that is, at the time the problem is identified.

A month after the plant opened, NUMMI’s president Tetsuro Toyoda was touring the facility when he spotted a worker who had put in a taillight incorrectly and was struggling to get it back in. He looked at the man’s uniform for his name and said, “Joe, please pull the cord.” The president, Joe’s supervisor, and his manager were all watching.

“I can fix this, sir,” said Joe.

“Joe, please pull the cord,” repeated Tetsuro. Up to now, the cord had been pulled only three times since the opening of the plant, and one of those times was by accident. Everyone was afraid to cost the company $15,000 per minute.

Joe repeated, “I can fix this, sir.”

Tetsuro took Joe’s hand, raised it to the cord, and then pulled it. Joe was shaking. The two men then fixed the taillight and pulled the cord again. From that point on, workers started pulling the cord more often. Now, you would think that this would have been bad for Toyota. However, in a typical Toyota factory in Japan, the cord is pulled more than 5,000 times per day. Toyota’s philosophy of fixing problems when they occur has been found to be significantly cheaper on average than fixing the defect later. That is why pulling the cord is encouraged.

It didn’t take long, about six months, for the Fremont plant to produce cars of the same quality as those from a Japanese plant. These same workers, who two years earlier were causing all types of mischief, were taking pride in their work. In WBEZ’s This American Life broadcast and other interviews, Rick Madrid remembers fondly his time at NUMMI. He took pride in his work.

It’s a Scrum story, after all

Given that Scrum, specifically, and Agile have their origins in the Toyota Production System, it is my opinion that a ScrumMaster and a product owner have the responsibility to support team members so that they can do the best they can. If that means being ordered around by a team member to fix a P1 issue, resolve a blockage, or even give the team some tough love to point them in the right direction — basically anything that will help them succeed — then so be it. Anything short of that, and you introduce the possibility of failure. When workers are empowered to do a good job; aren’t struggling against impediments; and are given the responsibility, the right tools, control, and support, they will take pride in the work and be more productive. This, in my opinion, is what Scrum and Agile are about.

GM had sent 16 rising stars to start the NUMMI plant. Once the plant opened successfully, the 16 were ready to spread the word throughout GM. They were motivated; they wanted to change the world. But GM put a wet blanket over that spark. They didn’t know what to do with the 16. Some became frustrated over time and eventually quit. All that knowledge was lost. It took GM more than 15 years to implement the lessons learned at NUMMI, but by then it was already too late. In 2009, in the middle of the global financial crisis, the auto industry was all but destroyed. The fallout is still being felt today, with the closure of car manufacturing here in Australia. GM became the largest industrial bankruptcy in U.S. history. The NUMMI plant closed a year later, in 2010, although it has recently reopened as a Tesla plant.

Rick Madrid retired in 1992. His hat and badge are at the Smithsonian in the American Enterprise exhibit.

References

User Stories

This post was originally published on the Scrum Alliance Community Articles

User stories are, well, stories. This should be obvious, but, unfortunately, it isn’t.

They are not statements, such as “Build a new front page.” They are a story. Stories have a hero or heroine, a goal, and a quest.

User stories are not riveting, but they tell us, from a user’s perspective, what the user is trying to accomplish and, more importantly, why. They are not absorbing novels; they take the form of only three lines.

Here I want to go through my understanding of what user stories are, how to write one, and when to use one. User stories are just like any other tool — there are times when you should use them, and times when you shouldn’t.

How it all began

In Waterfall, the first thing developed in software was the “requirements specification.” (Some of us, myself included, still write these today.)

This formal document described, in intricate detail, what was needed in the development. Who was responsible, who wrote, who changed things. If you were lucky, it also included the why. The document was used as a way to get agreement from the customer on what the development was supposed to be.

It sounds logical to think through what needs to be developed before you start the work. Making it formal makes it, effectively, a contract. In most cases it did become the contract. It also allows you to figure out up front how much effort is involved.

The problem was that, once development started, people would change their minds. Customers would have changing requirements. For a project that lasts six months or more, priorities can shift. What was business critical when the requirements were written can become irrelevant by the time feature development started. The difficulty of what needed to be done could be significantly more than what was initially envisioned. Agile came about to handle this situation from a development perspective, but something needed to be done from a requirements perspective. A requirement was still needed — developers needed to know what to build, but they didn’t need a detailed specification, just enough information to have an idea of what the customer was thinking so they could get started. The details could be worked out along the way, through talking with the users.

So some bright spark developed the idea of using a story to tell what the user was trying to achieve, and the practice has developed over the years.

Why use user stories?

One of the most obvious reasons to write user stories is that they are only three lines long (in the classic case), as opposed to being a formal document of several (or more) pages. That means there is less administration. You don’t need to update the user stories.

Now I know what some of you are thinking: A user story is only three lines long, so it cannot have much detail compared to a traditional requirements document. You’re right. They don’t have the detail, but that is the point. A user story is not a replacement for a requirements document, it’s a trigger to have a conversation with the relevant people, when required, about the requirements. In fact, the idea is to have continuous collaboration with the customer through discussions, and ongoing confirmation to keep refining the product.

Since user stories are small, they are also easier to estimate accurately.

User stories promote smaller iterations of work. With smaller iterations, you can get feedback sooner, which helps the refining process. It also allows you to change priorities as needed, without significantly affecting other pieces of work already in flow.

The INVEST principle

To explain all this a bit more clearly, we have the INVEST principle. INVEST names a list of principles that each story should adhere to so that we get the most out of each story and can write them better. INVEST is an acronym that stands for:

  • Independant. Each story needs to be independent of the other stories. This allows stories to be prioritized separately and cuts down on dependencies. For example, you cannot complete one story if you are waiting on another story to be complete. Or you cannot test a story if it is dependent on another story.
  • Negotiable. Teams need to be able to negotiate stories with the product owner. Circumstances change, and the team or the product owner needs to be able to reconsider how the story is both implemented and scoped. An example may be that the story is not technically possible. (Howevr, even though the story can be negotiated, the product owner still remains accountable for it.)
  • Valuable. A story must have some value to the business. This is to prevent additional features from being developed that end up being of no use, or that the developer thought were good ideas at one point but have since become of no value.
  • Estimable. The story needs to be written in such a way that the team can estimate how much effort is involved. This tries to avoid the problem of scheduling or making promises based on inadequate information.
  • Small. Stories are small increments of functionality. This helps helps the project be built by small bits of value at a time, rather than by large chunks that might end up not being used.
  • Testable. This helps make the requirement a known entity. It gives you something that you can verify. It is part of the Definition of Done. It is to prevent stories such as, “Write requirements” and prevents ambiguities such as, “The page must load quickly.” Instead, a measurable requirement is given, such as, “Must load within 10 seconds.”

Now that we know the principles of a story, how do we format them? There is a formula.

3 C’s Formula

The 3 C’s formula outlines how stories are put together. They are:

  1. Card: User stories are captured on a card (usually a 3×5). The description is on the front, and the acceptance criteria are on the back.
  2. Conversation: The card is a trigger for discussion of the requirements among:
    • Team members
    • Team members and the product owner
    • Team members and stakeholders\users\customers
    • All of these groups
  3. Confirmation: Finally, the user story helps the team discuss with the product owner how to determine when a feature/piece of work is complete.

Now we come to the juicy bit.

How to write a user story

So, to summarize, user stories are a description of a requirement in story form, from the user’s point of view. They are not written from the developer’s point of view, nor that of the product owners, but from the point of view from the person who is actually going to use this thing.

User stories are also short and succinct. They do not waffle on (like this article).

However, they can take one of many forms. Here are a couple of useful templates.

User story templates

The following is the classic version: As a [role]
I want to [requirement]
So that [reason/ROI]

The role is the “who,” our hero/heroine, the user. The requirement is the “what,” or quest, that our hero/heroine is trying to achieve. The reason/ROI is the “why.” Why does the user want the requirement? What is he or she trying to achieve? In other words, what is the goal?

An example based on a traditional story could be:

As a member of the Rebel Alliance,
I want to destroy the Death Star
So that I can save the galaxy.

Another variation on this template is the “In order to” template. It goes like this:

In order to reason/ROI
As a role,
I want to requirement.

Same information, but different wording. These are the more traditional ways of writing user stories, but they are not the only ways.

Comics

I attended a seminar at Elabor8 on using comics as a method of writing user stories. As they say, a picture is worth a thousand words, what better way than with a comic strip?

Can’t draw? It doesn’t matter. Use stick figures. Alternatively, if you are really worried about your drawing ability, try photos.

Movies

With the prevalence of smartphones, we almost all now have a video camera in our pocket. So, if it makes sense, why not act out the scenario of the story? It’s another way to express the requirements, and who knows, it might be fun. You might even be able to record a statement of the requirements, if the product owner is willing.

These methods can then be captured in your software-based backlog if you use something like JIRA, or you can put them on a web server, and print out links for physical cards, or insert links in electronic ones.

The whole idea is to try different things. See what works. If it’s a waste of time, you’ll know quickly; but it might boost your understanding of the story.

Impact mapping

One tool I like is Impact Mapping, created by Gojko Adzic. An impact map is a type of mind map that allows you to brainstorm and explore scenarios. “Branches” are then trimmed based on the team’s ability to implement them. Ultimately you are left with the branch you will implement. The impact map is drawn in such a way that you have who the feature is for, why the user wants the feature, and finally the “What” and “How” you need to do the feature. This is what the user wants.

An example of an impact map is below:


With an impact map, you start of with the why. Why are you doing this? In the above example, we are trying to save the galaxy. Next, we look at who is trying to achieve the goal. This would be our user, the our hero or heroine. In this case, it is a member of the Rebel Alliance. Next we have the how we are going to achieve our goal. This is where the brainstorming really starts. List as many ways as you can think of to achieve the goal. It doesn’t matter if they are realistic, just write them down. Who knows, they may lead to something.

Finally, for each how list, what needs to be done. Keep everything at a high level. We are not trying to solve every branch, only to get an idea of what is involved.

Once you have your impact map, you start trimming branches. Something may be impractical. Something may not be ready to do. Something may require resources that you do not have.

Eventually, after several iterations, you should have a single path to follow. But don’t spend too much time on this map; it is just there to give a visualization of what needs to be done, along with alternative views. You still may end up selecting the original proposal, but it will be a definitive answer with alternatives having been explored.

Now that you finally have the Why, Who, How, and What, you can write your story:

As a member of the Rebel Alliance,
I want to destroy the Death Star
By Sending two torpedos down an exhaust pipe that has been found to be a weakness in the Death Star’s defenses,
In order to save the galaxy.

Now that we’ve filled out the front of the card, it is time to look at the back.

Acceptance criteria

On the back of the card (or its software equivalent), you have the acceptance criteria. It outlines how the story will be verified as Done by the product owner. This needs to be hard criteria, not anything ambiguous. It’s best that the criteria be testable. If you can test it, you can define it.

Just like the story, the product owner is responsible for writing the acceptance criteria (with consultation from the team). For example, business analysts can help write the acceptance criteria, but it is up to the PO to accept them. Sometimes the acceptance criteria are used as part of the Definition of Done.

Acceptance criteria template

Just as there is a template for the user story, you can use a template for the acceptance criteria. An example:

A [role] should be able to:

  • [Acceptance Criteria 1]
  • [Acceptance Criteria 2]
  • [Acceptance Criteria 3]

Edge cases:

  • [Edge Case 1]
  • [Edge Case 2]

Separating out the edge cases helps your thinking process. You work on the normal cases first, then brainstorm the edge cases.

This acceptance criteria is a placeholder for further discussion to get confirmation from the PO. Then more detailed acceptance criteria can be determined.

Gerkins and Cucumbers

For acceptance criteria, I like the Gerkin language. It was initially developed for use by a program called Cucumber. Cucumber is a behavior-driven development program that allows you to specify acceptance criteria in plain English, then execute those statements as tests. Yes, execute!

Gerkin takes the form of “Given, when, then” statements. The Given section is your preconditions. It’s where you set the stage for what needs to be done. The When section is when you actually do something to the preconditions. The Then section is where the expected effect is described. For example:

Given two numbers, 1 and 3,
When I add these two numbers together,
Then I expect the result to be 4.

Nice and simple, easily understood not only by developers but by anyone who can read.

Now, I mentioned that Gerkin is used by Cucumber, but that is where it started. There are now a number of products that use this language to produce executable specifications. These include, but are not limited to:

  • JBehave
  • EasyB
  • Gwen

The beauty is that you don’t need to use these products to get the benefit. Just writing your acceptance criteria in this format means that it’s easily readable, verifiable, and future proof. So, for example, if in the future you wish to use one of the above products to make the acceptance criteria executable, the writing of the tests is done.

There are other types of programs that also allow you to write executable acceptance criteria. These are more free form and allow you to do more traditional documentation; Concordion and Fitnesse are examples.

Concordion uses HTML pages to write the acceptance criteria, and Fitnesse uses a wiki.

With most of these products, to make the specifications executable, bridging code is required to allow the interpreter to execute the tests. Usually the bridging code can be simple and reusable. But determining the complexity depends on what type of application you’re writing. I won’t go into how to write these, but I recommend investigating them anyway.

Epics and themes

Now we come to stories that are too big to fit into one sprint. These are called epics and themes.

So . .

What is an epic?

To put it simply, an epic is a very large story. Not in terms of description, but in terms of implementation. An epic is a story that can take more than one sprint to complete.

In the “Story World” (going with our Star Wars theme), an epic would be the Star Wars Trilogy. The epic in this case is made of (originally) three stories, Star Wars, The Empire Stikes Back, and Return of the Jedi. Each of those stories is watched in one sitting, just as a story will be done in one sprint.

What, then, makes a story an epic? There can be a number of reasons: A story may be at a very high level and need more information. Usually when the backlog is first created, all stories are considered epics. In other words, the epic is used to capture requirements at a very high level. Another reason could be simply that the story ended up being bigger than you initially thought, and therefore it cannot be completed in one sprint.

Epics are not the only way large stories can be described. We can also use themes.

What is a theme?

A theme is a collection of stories that have a shared attribute. For example:

As a Star Wars fan,
I want a group of pages to explain the movies
So that people can learn more about the Star Wars movies.

Here, the word group insinuate that the PO wants more than one page. So, this theme can be broken down into:

As a Star Wars fan,
I want a “New Hope” page
So that I can see details about the first Star Wars movie.

As a Star Wars fan,
I want a “The Empire Strikes Back” page
So that I can see details about the Empire Strikes Back movie.

and so on. As you can see, the two stories are identical except that they discuss different movies. The theme here is that they are both pages about Star Wars movies.

Again, just like epics, themes generally cannot be completed within one sprint, unless of course the stories are quite small. This means that the stories themselves are completed in the relevant sprint, but the theme, just like an epic, is completed over multiple sprints.

Splitting and combining stories

So now that you know what an epic and a theme is, there may be times when you want to split an epic or theme into smaller stories, and there may be times when you want to combine multiple stories into an epic or a theme. Remember, an epic or a theme is still just a story.

When to split stories

Most of the time, you will want to split a story because it cannot be completed within one sprint. The work is too large for one sprint, so you split the story into two separate stories. You complete one story in the first sprint and the other story in the second. The original epic or theme is then complete.

Another reason to split stories is to get a more accurate estimate. It might be too difficult to estimate a piece or work, so break it down into more manageable chunks, then estimate the chunks individually. Estimating smaller chunks is easier.

How to split stories

Splitting stories can be tricky, simply because there are so many possible ways to do it, and not all of them result in split stories that make sense. Here are some examples.

Does not meet performance constraints. Let’s say you are building a web page that has to load within five seconds. Building the page is easy, but let’s say that the performance of the page will prove difficult; to make it load within five seconds is extremely hard. What do you do? Split the story. The first story is the building of the page. This gives the product owner value. They have a page that the end user can in fact use. The next story is the performance criteria. This will be tackled later. Maybe the next sprint, maybe five sprints later, maybe never, if it turns out that the customers don’t care that the page takes 20 seconds to load.

Separate or mixed priority. This is when some of the acceptance criteria is of higher priority than other acceptance criteria. Split the story based on the different priorities of the acceptance criteria. Do the higher-priority stories first and leave the lower-priority ones until later.

To remove crosscutting concerns. This is when a story has acceptance criteria that applies to more than just that one story. Logging, error handling, and security are examples.

When to combine stories

There may be times when you would like to combine stories. Perhaps the stories are related and both could be completed within one iteration. For example, in transactional processing, you may want to combine the pick-up and drop-off of a transaction in one story. You may also want to combine stories when you want to prioritize them as a group. For example, two stories need to be completed together.

How to combine stories

There are a number of ways to combine stories. They are all similar.

  • Create an epic and then refer the individual stories to the epic.
  • Create a new story, so long as it can fit within a sprint.
  • Create a theme if it is a set of related stories.
  • The whole idea of splitting and combining is to refine the backlog. Combine stories so that twp people do not put the same amount of effort into one story, only to find that the other is similar enough they could have been done in one. Split stories so that larger chunks of work are broken down into more manageable sizes.

Conclusion

In this article I have tried to show how stories work and how they should be used. Take from this what you are most comfortable with, but if you do use stories, make sure you know the correct terminology, what a story is, and how to use it. That is the knowledge I have tried to give you, based on my own understanding. If your understanding of stories is different, please let me know in the comments. We are all learning, and the best way to learn is to share knowledge.

Overwhelmed With Work?

Have you ever felt you have been inundated with too much work? You have been delegated a multitude of tasks, everything has been pushed onto you and you just don’t know what you should be working on, you just keep switching between tasks and get nothing done.

The reason nothing gets done is because you keep switching between tasks, in other words multitasking.
So how does this prevent you from getting things done. Well, let’s keep this simple. Say we have 3 tasks to do. A,B and C. Each task takes 10 units to complete. A unit could be minutes, hours, days, weeks it doesn’t matter. For this example, we’ll use days.

Now let’s say we do A, B and C in sequential order. What is the lead time to complete A? Well, 10 days. What is the lead time to complete B? 10 days, C? 10 days again.
Nice and simple.

Now let’s say we do a split. We do half of each task, let’s say that the first half is development. The second half is testing. Deployment to production is negligible. So we stick with 10 days to complete a task.

So we do all our development up front.
5 days for A, 5 days for B and 5 days for C. Then we do our testing. 5 days for A, 5 days for B and 5 days for C.

Now, what is the lead time? For completion of A, it is now 20 days. We have doubled our time! For B, it is also 20 days, and the same with C. OK, the overall time is still 30 days, but let’s say there is an issue. Say, during the first 5 days of B. It takes 6 days. Now it takes 21 days to complete A. A is now over due. But not only is A over due, B and C are over due. The more the delays, the more the cascading affect.

This is a basic example, but there are other things you need to consider with multitasking, context switching. Changing between tasks means a mental dump and load for the next task. Then when you switch back, you have to remember what you had previously done, and then continue on. This context switching takes time. If the two tasks are completely different, then the switching time is less. If they are similar, then the switching time is longer. For those tasks that take minutes or hours that require a high level of concentration, and most IT work does require a high level of concentration, being “in the zone” when working is when you are at your most productive. Getting in the zone takes time and breaking that concentration drops your productivity. In most cases, quite significantly.

So in the modern workplace, why is there multitasking. Quite simply, people get assigned more than one task at a time. You know the drill. Your boss needs something done, looks to you and asks if you can do it. You say yes because you are only working on one task. It happens again and again and now you are juggling 3 or more tasks. Then you look at your colleague and you see that they are only working on one task, or worse yet, they are twiddling their thumbs.

This seems to be standard practice in many organizations,but what is the alternative? It’s simple, instead of pushing, try pulling.
You have your backlog of work. No one is assigned a particular task. If you are free, you take the next task in the line. This methodology allows you to work on one task at a time and focus. It allows even distribution of work load among people.

In Agile, this is represented by the Kanban board. Funny name Kanban, it comes from Toyota. In Japanese it literally means “signboard”. In order to limit inventory in Toyota, cards were used to indicate when someting was used. For example, say you have a tub of bolts used to attach a door to a car. There would be 2 tubs. The first where you take the bolts to attach the door, the second for when the first runs out. Attached to each tub, is a card. When you run out of bolts, you start eating from the second tub. The card on the first tub is then used as an indicator for someone to get you another tub of bolts. This tub becomes your second tub. The taking of bolts from inventory then triggers a purchase order to purchase new load of bolts. Toyota has got this down to a fine art where they get several deliveries per day of parts such as bolts to be used within that 24 hour period. Anyway, back to Agile. Since IT work is non tangible, cards are used as an avatar. The kanban board is a signboard or visual indicator of the progress of the work. People “pull” the cards from “todo” to “doing” while limiting them self to only one card in this column, then they “pull” the card to “done” which then triggers the “pulling” of the next card off the “todo” column. If something is blocked, then this “can” trigger a second card to be pulled from “todo”, but keep in mind, this should be the exception rather than the norm. During the retrospective, it should be investigated and discussed how to prevent the particular instance of a blocked card from happening again.

Now, a question. How do you limit the amount of work done if you have tasks that take 10 days as per your example? What happens when something more urgent comes in?
Well, the first mistake I did with my example was to use tasks that took 10 days. Ideally you should have tasks that take only 1/2 to 1 day. That way if something more urgent comes along, there is only a half to 1 day wait.
But what about an emergency? Such as something goes down, we can’t wait half a day to a day to get things back up and running. Well, this is an exception. In the case of an emergency, its all tools down and the emergency takes priority. Everyone who needs to be involved gets involved. Fix the issue, then once everything is back to normal then you resume the process.

Pragmatism

When there is resistance to being Agile or doing best Agile Practices, “Pragmatism” is usually used as an excuse to skip important practices or disciplines, or due to lack of knowledge.

For example, “We are going to skip the Retrospectives because we are under the pump and we need to be pragmatic”

In this example, the work is seen to be important, and the retrospective is not. So it is dropped.
Why is this bad? Retrospectives are a way to look back on the work done in the past sprint and see how it can be done better. If they are dropped, then you loose the learning part of Agile, which is one of the main parts of Agile.
It could also mean your retrospectives are not bringing value. If that is the case, another format may be in order.
Another example of pragmatism being used for skipping important practices and disciplines is skipping planning sessions.

“We already know what we are doing, so we don’t need to plan”
This can happen in a “Command and Control” type structure, the person giving the orders doesn’t need to plan as he is giving the orders to those below him. This again isn’t agile, just another excuse for cowboy practices but of a different kind.
There is minimal team involvement, which can lead to people “switching off” as they have no say or control.

Another way pragmatism can be used as an excuse is to “Brute Force” the work.
“We have 50 servers to set up, we don’t have time to automate, we have to do it manually”.
At the end of the 50 servers installed, you have no automation. Something else will come up preventing you from automating, and the next time you have to provision another 50 servers, that will be done manually too and so on and so forth. This is what I like to call the death spiral of work.

At the moment, I have no answers on how to get around the “Pragmatism” argument, especially if you have no authority. I would recommend education, but even that isn’t a guarantee as those resistant are less likely to be open enough to be educated.

Responsibility in Command And Control vs Self Management

As a child, most of us are taught to respect authority whether directly or indirectly in school. We obey our parents, we obey teachers. If we don’t, out comes the wooden spoon.

This learning carries on throughout life, well for most people. We respect police, our managers etc. It gets so ingrained, that all it takes is for someone to be dressed in a manner that garners respect. A uniform, a suit, a lab coat, that we blindly follow.

In the 60’s, when ethical standards were more lax, Stanley Milgrim conducted an experiment. He wanted to know, after World War II, how people could do atrocities to other people. Was there something different about the Germans that made them more callus than everyone else? To test his theory, he conducted an experiment.

The experiment was simple, there were 2 subjects. A Teacher and a Learner and the technician in a lab coat who would conduct the experiment. The teacher and the technician were in one room, and the learner was in another room, strapped to a chair so they could not escape and hooked up with wires. The experiment was simple. The teacher would ask a question to the learner, specifically about word pairs. If the learner answered the question incorrectly, the teacher would press a button, administering an electric shock to the learner.  The premise being that the learner will learn the word pairs more quickly with a pain incentive. Did I mention that ethical standards were more lax in the 60’s.
Well, as the learner kept getting answers wrong, the electric shocks grew in intensity.  15 volt increments up to 450 volts. 30 incorrect answers.
At those voltages, the learner would scream, plead for the teacher to stop. Beg for the teacher to stop. The pain was so intense.
Now, if the teacher started refusing to participate, the lab technician would say “Please Continue”. If the teacher refused a second time, they were simply told “The experiment requires you to continue”. A third time, they were told “It is absolutely essential that you continue” and finally if a fourth time “You have no other choice, you must go on”. There was no coercion,  the teacher was just told in a calm voice.  If the Teacher refused to comply a fifth time, the experiment ended. How do you think you would go in this experiment?
Well, 65% of teachers participants reached the maximum of 450 Volts. Many teachers were uncomfortable doing so, but still continued eventually when asked to. It got to the point where the learner was in so much pain, they either passed out, or worse. This experiment wasn’t conducted in Nazi Germany, it was in Yale University United States.

If you are absolutely horrified by now, take comfort, the Learner wasn’t really under pain. They were an actor. There were no shocks given. The Teacher was the only subject of the experiment.

So why did the Teachers continue, well simply put, psychologically, they put the responsibility of their actions onto the Lab Technician. Any consequences of the teachers actions were considered borne by the Lab Technician. That is how that 65% of teachers administered a fatal electric shock to a learner.

So, what does this have to do with Agile? Well, with traditional management, you have a heavy Command And Control mentality. A person is in charge, the manager, direct their underlings towards what needs to be done, and the underlings blindly follow. The underlings may have some say, but ultimately responsibility lies with the manager. Whether in actuality, or psychologically. The underling relies on their direction.

Yes, they may speak up if they think something is wrong, but if told, “That is the way we are going to do it” will go along if told enough times. It takes a special person to give it their all and dissent when they feel they are right. Most people won’t speak up at all. They will just “switch off” and do what they are told.

In many Agile methodologies, especially Scrum, there is no real leader. With Scrum, there are 3 roles. The team, The Product Owner, who isn’t in charge of the team. They are only in charge of the what is made, the Scrum Master who is there to help and guide the team. They do not direct the team. They do not command the team. There are no leaders in the traditional Command and Control sense. The team needs to be self organized. Giving the team this responsibility means that there is no one to shirk responsibility to. You cannot say “Something went wrong because we followed the manager”. Everyone in the team has a say, everyone in the team is responsible. Everyone in the team needs to participate in the decision making process.  Everyone in the team needs to be at the same level. No hierarchy. The team itself is able to choose its own destiny on how it accomplishes the task. Not a “leader” within the team. In other words, they have created their own purpose.

I know what you are thinking, this can get into a Lord Of The Flies situation. The team goes off on a tangent, All hell breaks loose and you have complete Anarchy.  Nothing gets done.
Well, I doubt this will happen. Most people try to do the right thing. Even those in unskilled jobs try to do the right thing. Try to do an honest days work. If you still don’t think this will work, In scrum, there is the scrum master. The scrum master is there to guide the team back on target. In Toyota, from what I have read, there are traditional manager roles, but the managers there don’t manage in the traditional sense. They guide their workers rather than command their workers. Still not convinced, look up NUMMI.

Working this way can become very powerful for a company. Your workers are now no longer there to just complete tasks assigned. They are there to accomplish a goal in a manner that they chose. They are there to learn and grow. They have purpose and a level of dedication that you would not normally see.

Just remember,  don’t abuse it. This is a very delicate balance. Start bringing in command and control policies and people become disillusioned again. The whole thing can fall into a heap and you might be worse off.

Let me know what you think in the comments. If you work in a Command Control situation, do you “switch off” and do what you are told or are you actively involved?

If you work in a team that has more control of their destiny, do you find you are more involved in reaching the goal or has things degraded so much it feels like a “Lord Of The Flies” situation.

The Cube of Rubik

Its been a while since I did my last major post. Actually, before Christmas. So what happened. I got myself a Rubik’s cube (Well, a cheap knock off from eBay) and learned how to solve it.

My Solved Cube

I watched YouTube videos and learned the beginners methods. I can now after 30 years of getting my first cube, solve it.

I got lots of practice at solving. I get my 5 yo son to mix it up. He loves doing that, and now the quickest I can solve the cube is about 1 minute 37 seconds. Most of the time its around 2-3 minutes.

Sounds impressive, but its actually pretty pathetic. The world record for a solve is under 5 seconds. Yes, 4.73 seconds to be exact.

Now, the method that I’m using, if I didn’t know any better could get me down to around a minute if I get more practice and get better. If I didn’t know any better about the world record, I would think that that would be impressive. But, in order to get faster, I need to change the way I solve the cube.

So how is this related to Agile, well, a lot of people think that getting a task done quicker means you just need to work quicker. Cracking the whip, giving carrots etc will get things done quicker. In my above example with the Rubik’s cube, that will only reduce my time by about 33%. Still an impressive number, but sometimes, if you try to do things differently, if I learn the expert method of solving the cube, which by the way is a lot harder to learn than the beginners method, I might, just might get faster. If I learn the expert method of solving the cube, which by the way is a lot harder to learn than the beginners method. In fact, when I start trying the expert method, my solving time will get significantly worse before it gets better.  I have read that I can reduce my time comfortably down to 30 seconds. That is a 60% reduction of my current best. The same thing can be done potentially with any task up to a limit.

So, if you are trying to increase your personal or team velocity, but remember you are trying to keep quality at a high level, not skimp on any value, then think about trying something differently. If the method has been tried and tested by someone else. Stick with it. Your velocity might suffer in the short term as you get use to the new way of doing things. That’s OK. You’re learning. Stick with it and you may find that as you get better, you will get significantly faster.

Failure Is Not An Option, It is a Requirement

Hope you had a good Christmas and Happy New Year!

This post is for an idea I have for a retrospective. It is not something I have tried yet. Hoping to do so at some point.

Here is the gist of it.

At the end of the previous retrospective, outline the goals for the next sprint for the next retrospective. The goal is… to fail.

I know, insane, but I am insane, so lets move on and let me explain myself.

Each time something does something wrong during the sprint, record the following.

  • The failure.
  • What caused the failure
  • What can be done to fix the failure

Points are awarded based on the the following.

For the failure – 1 point.

For What caused the failure, for immediate cause, 1 point. For root cause 3 points.

For What can be done to fix the failure, 1 point for a work around. 1 point for a manual process. 2 points for an automated process or documenting a way to remove it permanently . 3 points for making the failure never happen again. Now, for everyone involved in the solution, they all get the points. Not just the person who came up with the solution, nor the person who failed in the first place. This is to try to promote co-operation in solving problems.

Here is the kicker, everyone must have at least one failure. The way we do that is that everyone must fail at least once on purpose. That is right, I want everyone to have one failure on purpose. The more exotic the better. (For a really good one, award 2 points for the failure). Hopefully killing the system isn’t their on purpose failure.

Points can be placed on a board so that throughout the sprint, everyone knows the tally, the scrum master awards points and their say is final.

So, why do this?

Well, firstly, as I mentioned previously, I’m insane. So again, accept that and move on.

Secondly, I work in a place where failure has a stigma against it. There is finger pointing within the team. By making failure as part of the game, hopefully this removes the finger pointing. The focus then goes onto solving the problem rather than finger pointing.

Thirdly, by making failure open, you get to see where your weaknesses are, and ways to solve those weaknesses. This is the opposite of current culture where successes are focused on and celebrated, failures are hidden so you never know when problems occur.

Finally, by making failure a requirement, it removes the fear of failure. Hopefully you get to the point where people fight to take ownership of a failure.

Now, at the end of the sprint, during the retrospective. Go through the results.  For those results that have workarounds, or manual processes, try to improve them. Tease out what lessons have been learned during the sprint from failing. Any permanent solutions documented, decide whether to add them to the backlog.

Now, for the person who gets the most points, award them with a token prize. For example a block of chocolate. For second and/or third, a chocolate bar.

Finally, see if the team enjoyed this method of Retrospective, if it was valuable and if they want to do it at another point in time.

Please let me know in the comments if you tried this and how it worked. If I ever get a chance to try this out, I’ll be creating a new post.

Is Going Agile Too Expensive?

I have heard the argument that agile is a con. A way to get the certification industry money and a case to get high paid consultants to fill their coffers.

I don’t think this is true for the simple reason that if you look and you don’t have to look hard, you can find resources to learn agile techniques for free. The way I see it, most agile methodologies such as scrum and Kanban are like open source. Free to use, but if you want more support you can pay for it.

For example, with scrum, you do not need to get a CSM or PSM to practice scrum. You can download the scrum guide for free. Learn from blog posts, videos, podcasts and do your own practice. If you want more support, then you can buy books. Still need more, do a course, still need more then bring in a consultant, Even if you do a training course or bring in a consultant, you still have to do the work. There are no short cuts.  The concepts of Agile simple simple, but they not easy.

As for the perceived requirement that you need a certification. Well, that started by a requirement by businesses. Not the people who developed the methodologies in the first place. A lot of businesses generally do not want to invest in their people. They are more likely to buy or rent expertise already than grow internally. They will try any shortcut whether or not it works in the long term. What better way to know if someone knows Agile or any other technology/methodology than to have them certified in such technology/methodology. People see the requirement, get certified so they can be hired. Right or wrong this becomes a vicious cycle that feeds itself and creates the requirement for certification. 

If you or your company is serious about being agile. Not for the kudos or bragging rights, but just to be better, I suggest learning the history, reasons and purposes behind the methodology.

The following resources both from a google search and my own resources should be of help.

Free Resources

Scrum

Conference Videos

Software Houses

There are software companies that want to share their methods, help people learn.

  • Mountain Goat Software
  • Atlassian (Yay and Aussie Company)
  • Most tech companies have a blog where they share their development methodology which is usually some form of agile.

Web Sites

Pod Casts

Paid Resources

Books

This set of books is specifically not Agile related, they deal with Lean and TPS and learning from mistakes. Agile/scrum have their roots in Lean. Agile is also a learning methodology, and to learn, you have to make mistakes.

Online Courses

This is only a handful of resources, there is probably a billion other resources out there that are free or cheap that will help you learn any type of agile methodology that takes your fancy.

I’m always interested in learning new sources. If you have a favorite free or cheap learning resources, please add them in the comments.

Disclaimer

I myself have gone through the Certified Scrum Master course. I had done this after spending about a year learning about Agile and DevOps using some of the above resources.  I did the course because I wanted the alignment of what I had learned myself, and unashamedly the piece of paper.

My Views On Automation

One of the things I like doing at my job is automating repetitive tasks. For example, when a particular error occurs as part of the morning health check, an email needs to be sent out to the business owners of the source application (I work in middle ware) to notify them of the situation. (I know I’m doing it wrong – Special Cause/Common Cause, Burn’t toast etc – but that’s the situation I’m in) the process was to manually, so extract the relevant data, compose and email and send it out. All in all, about 5 minutes worth of work. Not long in the grand schema of things, but very boring. So I automated it. Now an email gets sent out automatically every morning. We don’t have to worry about it anymore.

Now, many times when you want to automate a task, there is the argument of “is it worth doing or not” comes up. In the above example, we are saving a task that only took 5 minutes a day. How long should we spend automating this task. 1 hour, 4 hours, 1 day, 1 week? (For the record, it only took me about 20 min for first version, then about another 20 min over several days to get the formatting right) for something so trivial.  Then again, lets look at another scenario. Say you have a task that will take you 5 min to do. But to after automating it, the automated process itself takes an hour. Not the writing of the process, but the actual execution. Should it have been automated?

In both these cases, there are arguments against automation because the time involved to either build or run the automation take too long. In my view, you are looking at the wrong metric.

My view is that automation should be used primarily for consistency. Spending a day on writing an automated script that saves 5 min will mean that that process will be run consistently forever and a day. The secondary benefits are that you don’t have to worry about the process any more – at least until it is no longer required. You have the knowledge on how it was automated which can be applied to automating future tasks. Finally, if you are lucky, the task may run quicker.

The same goes for a task that takes 5 minutes manually, but 1 hour automatically. You have the primary benefit of having the task consistently done.  Human error has been removed.  If the task needs to be done in a quicker time, then spend the time optimizing the automation script.

Now,  there are always exceptions to this rule, there always is. You have to use good judgement as to when to and when not to apply the rule.

Now, how do you determine what priority to automate. I’m not taking here as a manager (I’m not one) but as a person doing the manual work. Well, I work on one simple question. “What annoys me the most at this point in time?”.  I then spend time outside of my normal tasks trying to make that process better. Then I repeat. For the more common tasks, they get fixed pretty quickly. If they are not fixed properly, it annoys me and I go through the process again. For tasks that have long periods between them, I start working on them, I may not finish, but I document what I have found for the next round the task occurs and then work on it a bit more then.
The benefit of doing the “What annoys me at this point in time?” is that it is something that directly affects me. It becomes personal and thus I’m more likely to fix it. That is not to say that management cannot assign improvements, but generally, they are not the ones doing the daily work, and therefore do not see the pain points.