I’m Still Here…

Its been a while since I posted an article here, that is because I have been writing articles for other blogs.

As my previous post, you can see that “How Do You Measure Productivity?” was posted under the Scrum Alliance.

I also have a couple of posts under DZone’s Agile Zone which will be added to this site in the next few days.

In other news, I passed my PSM I exam. So now I hold both the CSM and PSM I Certifications.

How Do You Measure Productivity?

The following article first appeared on the Scrum Alliance Member Articles


How do you measure productivity? It’s a simple question, but does it have an easy answer?

If I increase my productivity by 20% or even 50%, how do I know for certain? How does your manager know? Is it a subjective, finger-in-the-air guess?

I thought I would explore the process of measuring how productive a person is. I presumed that a simple question would have a simple answer. To get at the answer, I realized that I needed to ask more questions.

What was the amount of work done?

The most obvious answer to how productive someone is, is to measure how much work that person has done. For example, John has done 50 things today (a rate of 50 items/day), whereas Jane has done only 5 (a rate of 5 items/day). Obviously, John is more productive, right?

Let’s consider another analogy: John delivered 50 parcels today. They were all on the same street, whereas Jane delivered only 5 parcels today — but they were delivered in towns 100 km apart. Now who do we think was more productive?

The amount of work done does not seem to be a good measure.

How many hours of work were spent?

The next obvious answer is the more hours you work, the more productive you are. So John works 12 hours per day, whereas Jane works 7 hours per day. John, again, is more productive, right?

Well, John spent those 12 hours writing Testing Procedure Specification reports that no one reads, browsed MySpace, and twiddled his thumbs. He also complained about losing his stapler. (Why, oh why, doesn’t Netflix carry Office Space!). By contrast, Jane worked on a method to save the company $5 million per year. Who was more productive?

The amount of time someone spends at work does not seem to be a good measure.

What was the value of work?

Perhaps we should try evaluating value. The higher the value of your work to the business, the more productive you are, right?

So John works on 5 high-value, 2 medium-value, and 1 low-value piece of work. Jane worked on 1 high-value piece of work. However, John’s 5 high-value and 2 medium-values pieces of work were not very difficult, as they took only a few hours to do. The low-value work took him weeks, whereas Jane’s one high-value piece of work was difficult and required days of effort. Who was more productive?

The amount of value of the work does not seem to be a good measure, but I think we are going in the right direction. Value does need to be taken into account.

What was the effort and difficulty?

Because the value of work must be taken into account, let’s work on the high-value items first. However, I think we need to take into account how much effort something takes.

Suppose we rate the work based on how much effort each item takes by using measures such as small, medium, and large. Assume that John worked on a large item for a week, while Jane worked on three medium items for a week. Are three mediums more work than one large? Are three mediums equal to one large? How can you tell easily? Who is more productive in this case, John or Jane?

Defining units of measure

I think we are getting close. We need a common unit of measure &mash; some simple way to convert from (S)mall/(M)edium/(L)arge to a number. Let’s say that S = 1, M = 2, and L = 3.

So, we have John who worked on 3 smalls, while Jane worked on 6 smalls. This also doesn’t sound right. Some small tasks take no time at all, whereas others can take significant effort. We need something more than just three categories. Maybe we can use extra small (XS) for those items that take little to no effort; small (S) for items that take a little bit of effort, but still not much; medium (M) for those that are a quite bit more effort; large (L) for something that takes a lot of effort, and finally extra large (XL) for something that will either pull a muscle or burst a blood vessel, based on the amount of effort.

The units of measure look something like this:
XS = 1
S = 2
M = 3
L = 4
XL = 5

Wait, something doesn’t seem right. Where is the cutoff point from “I’m exhausted from the effort” (Large) to “My head hurts” (Extra Large), or from tilts hand up/down, left/right in so-so motion (Medium) to “I’m exhausted” (Large)? The differences need to be much bigger than one unit greater.

How about this measure: The next level up is the sum of the two previous units? It looks something like this:
XS = 1
S = 2
M = 3 (same as before)
L = 5
XL = 8

Hmm, a 5 to 8 jump isn’t “Ow, my head hurts!” level of effort, but if we add an XXL, we get XXL = 13. That’s a big enough jump.

This sequence looks familiar. I remember my high school math teacher, Mr. Fibonacci, mentioning something like this. This looks very interesting. I’m going to call these work units. (Hey, who said story points? What’s that?) Now we can work out how much work has been done.

Assume that John with his Large unit of measure has completed 5 work units, whereas Jane has completed 6 work units. I think we are making progress.

Let’s standardize this to a time period. They both completed the tasks in one week. The productivity rate for John was 5 work units per week and Jane’s was 6 work units per week. (Hey, who mentioned velocity!)

Hang on, we’re not quite done yet. Individual productivity rates are all well and good, but John and Jane are a team. Measuring individual productivity rates does not promote a team mentality. If Jane is doing more than John, she should be helping John, or vice versa. I should combine these to one overall rate, so the team rate is 11 work units per week. I should only compare rates week to week, per team, and not across teams, because each team sizes work differently. I also need to make sure that the team agrees on the size of the work as a whole. There’s no point in having Jane say something is small when John thinks it’s large. They will both need to discuss and agree.

If, in a few weeks, Jane and John’s combined work increases to 17 work units per week, they have gained about a 50% increase in productivity. A measurable, comparable unit of measure for productivity. I think we’ve got it!

Conclusion

This experiment on trying to determine how a manager might work out how to measure productivity is a fun way, at least for me, of trying to understand why in Agile you would use something like story points and velocity.

Using velocity for measuring productivity is not perfect and is subjective, because it is based on an individual’s or the team’s idea of the size of work. It is also open to abuse. Jane and John could say all their work is Large and increase their productivity (throughput) significantly. Therefore, this rating should not be used for any incentives, simply because it can be abused and thus the rate becomes meaningless. However, it does indicate, for an experiment or trial, whether a productivity gain has made headway.

As you can see, this also ties into estimation, which is another reason to move away from doing estimates as time units.

One more thing: Although this story is complete fiction, I still think it deserves a happy ending.

Epilogue

With the help of their manager, John and Jane were able to visualize their productivity rates. Thus, they were able to put them into effect. They could make changes to the way they worked and get quick feedback on how those changes affected their throughput for work. Six months later, their productivity rose 1000%. They did this through experimentation, eliminating wasteful practices, and adopting a framework called Scrum.

And they lived happily ever after.

Little’s Law, Multitasking and Getting Things Done

This article first appeared on the Scrum Alliance Member Articles Section

This version is the original unedited version.

Little’s law is the theorem of queuing created by John Little, a retired professor from MIT

The theory is

l = tw

Where

l = the length of the queue

t = the rate at which items arrive or are removed from a steady state system (Throughput) and

w = the wait time in the queue

So, if you want to work out the wait time in the queue (ie, how long before your job is looked at) then we rearrange to

w=l/t

We see that the wait time is proportional to the length of the queue and inversely proportional to the processing rate.

So what does this mean. Simply put, if your processing rate is slow, then the larger the queue is, the longer waiting time before a new entry into the queue is processed. In other words your ability to change – be agile reduces.

Whereas, if your processing time is faster than your length in the queue, then you will be processing the queue quite quickly – but you run the risk of your system constantly being starved for work.

So how can I demonstrate this?

Let’s say you have a processing rate of one story every 2 days. This gives you a rate of 0.5 stories per day.

If we have a queue size of 1.

Then

w = 1/0.5

= 2 days your story will wait in the queue before it is even started on when added to the queue.

Now, lets say we push to the queue 2 stories.

w = 2/0.5

Now, for any additional stories entering the queue will take 4 days before they are looked at.

Our agility is really slowing down now.

Adding More Resources (Increasing Throughput)

Now, ok. Surely if I add more resources I can processing things quicker. My throughput has increased, so things should improve.

Ok, let’s say you now have 2 people, each with the capability of doing 0.5 stories per day. So you’re processing speed has doubled to 1 story per day.

So with a queue depth of 1.

w = 1/1

So the time waiting in the queue is now 1 day. We have increased our throughput.

Let’s try increasing our queue depth to 2.

w = 2/1

So the wait time is now 2 days again. Our agility is starting to go away again.

The more we assign to the queue, the longer it is going to take before the story is looked at.

The problem here is that you cannot always add more people. People and resources cost money. So what else can we do?

Reduce Queue Size

Let’s try something else. Let’s try reducing the queue size to zero. To keep things simple, we’ll go back to one person.

w = 0/0.5

Our wait time is now Zero????

We now have complete agility – the ability to change stories up until they are worked on.

So how do we achieve Zero queue size? Well, we’re cheating a little. There is still a queue – the backlog. What we are doing is no longer assigning stories to people. Their personal “queues” (The work in progress) has been reduced to “zero” – in other words they are working on one story/task/item at a time and “pulling” new items off the backlog when they finish. This allows the changing of the backlog without affecting the work.

Multitasking

Now, one of the problems with “assigning” tasks to people is that they have a tendency to multitask. Chop and change between what has been assigned to them and think they are making progress.

This is actually not the case. Let me explain.

Say you have 3 tasks. A, B and C. Each task takes let’s say a nice round number. 10 days to complete.

Now if we do these tasks sequentially, it’s going to take 10 days to complete each task.

But, we are supermen/women err people! We can multitask! Get everything done quicker! Is that the case?

Well, lets see. Say we split up the task so we work 5 days on each task to get something done of everything. We’re keeping it simple here.

So is it quicker to finish each task? Umm, no. We have actually doubled the time it takes to complete each task.

Now, this may not matter as the overall time for all tasks is still 30 days. The same as if we did it sequentially.

Where it comes into a problem though is that switching tasks is not free. There is a penalty. We have to dump our thought process on one task, and reload everything for the next. That takes time and energy. Especially if the time between multitasking is hours or minutes and not days.

Secondly, if there is any delay in completing a “portion” of a task – For example C took a little longer than first thought and you spent 6 days instead of 5, then you not only have delayed task C, but also completion of task A and B! And potentially the whole project! This then leads to stakeholders of A and B escalating as they need their stuff done NOW, so you stop, rearrange and the whole thing becomes a chaotic mess. Business as usual? For most people it is. So much so that for most it’s a fact of life. So much so that when developers look at how long something will take to complete, we ignore elapsed time and focus on work time, but stakeholders focus on elapsed time, so as with any mismatch, tension arises.

Caveat

Now I should probably mention that these techniques are not going to shorten the delivery time for a set amount of work in a perfect system without losses such as context switching or reorganizing regardless whether or not you have a queue size of zero or fifty or if,you switch tasks every few minutes. We are not doing any magic here. Just like the conservation of energy, the amount of work being done overall in the system is maintained. What we are doing here is rearranging things so that stuff gets done sooner and if you prioritize your items to give the most value, you get more value sooner as well. We are also dealing with a system that has reached steady state. As with anything to reach steady state, there is a ramp up time where the system is loaded and variances in work from one item to the next are minimal. Something that doesn’t come naturally in any knowledge based project.

Conclusion

I know my maths is a little off – I could probably explain it a lot better 20 years ago when I actually did maths in Uni, but Little’s Law shows a mathematical model on our ability to be Agile. It shows that “Pulling” increases your ability to respond to change as opposed to “assigning” work to someone. If you do assign, it shows that reducing the amount you load onto a person actually helps get things done sooner.

Finally, if you do overload someone by assigning tasks to them, as we are human, we tend to chop and change between those tasks for whatever reason. What this means is that for any one particular task to complete – it will actually take longer to complete any one task. Not only that, but a delay in one task could affect the delivery of other tasks that may be of a higher priority.

So keep your work in progress small (Preferably 1 thing at a time) try to keep the size of the work even and small in size to regulate the throughput to a steady state, and pull new work as required. You will get things done sooner, smoother and hopefully be one step closer to gaining control of the whole system. Better yet, you are doing this all with no additional cost!

Cross Pollination

Since I received my Lean Six Sigma Yellow belt from goleansixsigma I have been receiving a lot of emails about how to do things better. How to implement Lean and Six Sigma from doing your gardening to any project. Now, a lot of these posts include techniques taken from the Agile community such as Kanban boards, Retrospectives and such. What we are seeing here is a cross pollination of techniques across different disciplines. Its not surprising. Go Lean Six Sigma, mainly used, but not limited to the manufacturing space  as I understand it and Agile for the software industry have the same parentage in the Toyota Production System and both foster continuous improvement. It stands to reason that in improvement and practice in one discipline could be used in another.

To me, it just proves that we are all on the right track, be it Agile, Lean, DevOps, Theory Of Constraints, they are all on the same path, just in different lanes to the same destination of continuous improvement through feedback loops. Each lane a variation of solving the same or similar problems, but in different contexts.

So my thinking is that regardless of the methodology you employ, or even if you make it up yourself. Provided you are willing to question everything, have an open mind, diligently try to improve and implement a feedback loop to test those improvements and adapt to be better, you will eventually find yourself on or close to one of these methodologies, or potentially a completely different lane as your “context” may be different but still going the same way.

Retrospective Idea – Celebrate Failure

Ok, after more than 2 years of studying Lean and Agile and a little over 1 year since I got my Scrum Master certification, I am finally a Scrum Master (albeit part time) in a project.  I haven't got full control of the scrum process and I wasn't Scrum Master from the start (I was on leave when the project started) but it is a start and there have been and still are a few challenges.

Last Friday I did my second retrospective and I tried to do one of my own making this time.

After a successful showcase the day before (It was the only time we could get all the stakeholders together – so we had a 9 day sprint) I didn't want to focus on the success. Unlike most projects I've worked on, when something is successful – we tend to focus on the successes and the failures and problems get shoved under the rug. This time I wanted to make them front and center.

So, with the sprint report in hand showing a big cliff after 5 days of work, and buttering everyone up with doughnuts we went to work.

The question became "How can be make a sprint more linear in execution instead of the cliff we had?" This became the guideline for the retrospective, but what I was really after was any problems.

I had 3 rules for the Retrospective.

  1. No one can have a doughnut until they bring up a problem. This was to firstly give everyone an incentive to speak up. Doughnuts can be good incentives – except for those who are looking out for what they eat – but they still got involved.
  2. No Blaming, finger pointing. I don't want anything said to be held against anyone else.
  3. No excuses. I don't care there were BAU issues during the sprint.

The first step was to get a list of problems, issues, anything that you hated and anything that you think prevented you from completing stories in a linear fashion. We did this for the first 15 minutes.

Next we identified any duplicates and prioritized the first 5 problems.

The final step was to go through each of the 5 problems and work out the "Why?" the problem occurred. "How?" we can overcome the problem and finally any "Action" that needs to be done.

The board was broken up as follows.

In the Problem section, I inserted the first of the 5 problems.

In the Why, the team then brought up reasons for the problem with me prompting to drill down. I was trying to use the 5 why's technique to get to the root causes. I probably didn't do so well, as there was many root causes, but it is still early days experience wise for me.

In the How section, we brought up contingencies and countermeasures for the causes of the problems.

Finally in the Actions section, we bought up actions for the Scrum Master, Product Owner and Team Members either collectively or individually to do something. These were not assigned, but members decided to take responsibility for the actions.

I think the whole process went fairly well with the only problem that we went overtime by about 10 min. (From 1.5 hours time-boxed for the retro because it was a 2 week sprint).

The reason I liked this type of retrospective was that it actually focused the team on individual issues and problems that "they" found and had issue with. The sprint report gave focus  as to what was wrong rather than pie in the sky thinking.  Now we see what we can do about it in the next sprint.

As a personal note – Implementing Scrum is hard. Especially when you are doing it the first time and have no idea what you are doing, even after studying it for so long. I have this mental model in my head of what a Scrum sprint should look like and we have varied wildly from that. I'm not going to take this as a sign that Scrum doesn't work – but as a sign that I'm doing something wrong, which I know I am. I feel that my job now is to reign in the bad practices and try to introduce good practices as I understand them in gradually. For those practices that I don't quite understand, try something and see if it works while still staying to the scrum framework.

Once everything is reigned in, then start taking stats more seriously and be more aggressive in our improvements.

At least that is my thinking – which is going to be hard with a 4 sprint project. But who knows, it might help with BAU work in the future after this project.

Agile Methodologies

Ok, here is another chapter in my upcoming (hopefully) book on Agile.

This post is the raw text, it still needs refinement, but I just wanted to record my unfiltered thoughts first. At some point this will be reposted as an edited version, or I’ll just release the book if it is ever done.

Agile Methodologies

Most likely if you are thinking about going agile, you are thinking Scrum. Yes, Scrum is the most popular method of Agile, but there have been many that have been developed over the years. Some have stood the test of time and are still in use today. Others have dropped off to the wayside. Some are new comers and have managed to gain a following. In this section, I want to go through some of the agile Methodologies that are out there, but I will concentrate specifically on Scrum for two reasons. Firstly, it is the most popular implementation of Agile, and secondly, it is the agile methodology that I am most familiar with.

The Toyota Production System (Lean)

The Toyota Production System was the first Agile approach although it was developed long before the term Agile. It even pre-dates software development. In fact it started in the Car manufacturing industry. The approach was simple. Elimination of waste and continuous improvement. I won’t go through details of how it was developed, but eventually it was taken up by other car manufacturers in the 1980’s and 1990’s and changed its name to Lean as you could not have car manufacturers saying they worked the same way as Toyota. It wasn’t until Tom and Mary Poppendieck published the book “Lean Software Development: An Agile Toolkit” in 2003 that the Lean software movement took off, even though a majority of Agile Methodologies developed earlier used Lean Principles as a basis.
The lean Software development approach is summarized in the following principles.

  1. Eliminate Waste.
  2. Amplify Learning.
  3. Decide as late as possible
  4. Deliver as fast as possible
  5. Empower the team
  6. Build integrity in
  7. See The Whole

Lean Software Development Wikipedia Entry
The Lean Mindset – Tom and Mary Poppendieck

DSDM (Dynamic System Development Method)

DSDM was first developed in 1994 and was thought to provide discipline to RAD (Rapid Application Development) software development.
DSDM is an iterative and incremental approach to software development that has elements of the agile development approach and encouraged s end user/customer involvement of the process. DSDM is also where the MoSCoW prioritization method was developed.
MoSCoW uses the four following priorities.
M – Must Have Requirements
S – Should Have Requirements
C – Could Have Requirements
W – Won’t Have Requirements
DSDM is owned and managed by the Agile Business Consortium, a not-for-profit vendor independent organization.

Dynamic System Development Method

Extreme Programming (XP)

Extreme Programming was created by Kent Beck during his work at Chrysler Comprehensive Compensation payroll project which he started working on in 1996. He refined his approach and released the book Extreme Programming Explained in 1999. The premise of extreme Programming was to take the best practices of Software Development at the time and take them to extreme levels. At the time Extreme Sports was very popular. Principles of Test First development and Pair Programming were formalized through the extreme Programming

The lifecycle of Extreme Programming consists of 5 phases.

  • Exploration
  • Planning
  • Iterations to Release
  • Productionising
  • Maintenance and
  • Death

The 4 basic activities for Extreme Programming are

  • coding
  • Testing
  • Listening
  • Designing

The 5 values of Extreme Programming are

  • communication
  • Simplicity
  • Feedback
  • Courage
  • respect

Extreme Programming Wikipedia Entry

Kent Beck – Extreme Programming Explained

Agile RUP (Rational Unified Process) or AUP (Agile Unified Process)

Agile RUP is a simplified version of the Rational Unified Process. The Rational Unified Process is an iterative software process originally developed by Rational Software in the 1990’s. Rational was later purchased by IBM in 2003.

AUP has seven disciplines.

  • Model
  • Implementation
  • Test
  • Deployment
  • Configuration Management
  • Project Management
  • Environment

Philosophies

  • Your staff know what they are doing
  • Simplicity
  • Agility
  • Focus on high value activities
  • Tool independence
  • You’ll want to tailor the AUP to meet your own needs

Wikipedia Entry For Agile Unified Process

Kanban

Kanban in software development is a method for managing knowledge work. It is a visual method that specifically uses “pull” to control the flow. This is in contradiction of more traditional methods of work management where work is “pushed” into process.
Kanban was developed by David Anderson n his 2010 book “ Kanban” where he used the Theory of Constraints methodology at a project at Microsoft. The name Kanban comes from the Toyota Production System where it means “signal” and is used to signal the need for something.
A simple example of a Kanban system in manufacturing would be where an assembler might have 2 tubs of screws. The first tub he takes from, the second is standby. When he runs out of screws in the first tub, a card attached to the tub is taken to signal that the assembler has run out of screws. The assembler continues to use the second tub. The card, known as a Kanban card is then taken to inventory where the screws are kept. This signals the inventory to send out another tub of screws to the assembler, which now becomes his second tub.
That is not the end of the card though. That card is then sent to the purchasing department where it triggers he purchasing of another tub of screws. As a side note, Toyota does do orders at least daily for the work to be done the next day, if not more often. Very rarely do they order in bulk. That would only be for times when there is long shipping times for parts directly from Japan, at least here in Australia. Most parts are sourced locally, delivered either that day or the next day for the cars to be manufactured that day.
Back to software development, David Anderson modeled work in a similar manner. Cards represent the work that needs to be done. There are lanes that indicate the process the “work” needs to go through. The idea is that the worker pulls the cards as it progresses from one process to the next. Work in process is limited only to a maximum of 3 items per worker. This helps alleviate some inconsistencies. If a worker has more than 3 items in progress in a process, then the person giving the work needs to step in, and help that worker get back on track.
For example, a developer develops 4 features quickly in succession, but the tester is stuck on the first feature still. The developer stops further development and then helps the tester with testing. The developer may help test 2 features. Now when the testers work in progress drops down to 2, the developer goes back to development.
The advantage of the system is that it gets things done quicker. It also allows you to see where the problems are in the system and know where to add additional resources. Kanban is also very visual. You can see at a glance the work in progress, what is remaining to be done and what has been done. Especially when a physical board is used. Some people also like virtual boards. The advantage of a virtual board is that it can help teams with remote members, automatically generate reports and not be knocked over, but you loose the visibility. You have to “open” the software to view the board and thus it can fall to the “out of sight, out of mind” problem.

The Kanban system has been so successful that is is used on many agile Methodologies as a means to manage work to be done. Many teams that use agile of varying types can be seen to have physical boards around their office as a means to manage work, be they use Kanban itself or Scrum, XP or even Lean Software development.

Wikipedia – Kanban Development>)

David Anderson – Kanban

Crystal

Crystal is a family of Agile methodologies such as Crystal Clear, Crystal Yellow, Crystal Orange and Crystal Red. The darker the colour, the more heavy the methodology. Each approach is used depending on the project size, team size, critically, project priorities etc.
Several of the key properties of crystal include teamwork
, communication and simplicity. There is an element of frequent reflection to adjust and improve the process. Like other Agile processes, Crystal promotes the frequent delivery of working software, high user involvement, adaptability and the removal of waste in the form of bureaucracy or distractions.
Crystal was started by Alistair Cockburn when he released the book “Crystal Clear: A Human-Powered Methodology for Small Teams” published 2004.

Wikipedia Entry for Crystal Clear)

DevOps

The DevOps movement was started by Patrick Debois in 2009 when he saw the live presentation streamed over the internet by Paul Hammond and John Alspaw at the 2009 Velocity Conference.
The talk, “10 Deploys Per Day, Dev and Ops Co-operation at Flickr”.
Through the twitter stream, Patrick sited how much he would have liked to be at the conference and through a joke was told to start his own conference.
Patrick did just that and in 2009 in Ghent Belgium, the first ever DevOps Days conference was held. Due to Twitters 140 character limit, the “Days” portion of the name was dropped and the hash tag #DevOps was born and has stuck since.

DevOps was created by people in operations who were frustrated at the way work was being done. They would see Agile being used by Developers, but would stop once the features were in production. There had to be a better way. And so people in operations started their own methodology which included culture, tools and philosophy. Technology such as Infrastructure as Code, Continuous Deployment and culture borrowed from other Agile processes such as Theory of Constraints, Lean. They embraced the teachings of W. Edwards Deming, Elyahu Goldratt and Taichi Ohno. Learned lessons from the Manufacturing industry from 30 years previous. They embrace Agile methodologies such as Scrum, but expand on the process to continue into operations.
DevOps promotes continuous delivery, automation of the mundane, the difficult and frequent deployments. Some companies deploy multiple times per day. Amazon, one of the most prolific users of DevOps deploys on average once every 11.6 seconds. DevOps encourages experimentation to try to improve processes and continuously strive to get better.

Some say that DevOps may be the next evolution of Agile. They may be right.

Wikipedia Entry For DevOps

Damon Edwards – The History Of DevOps

John Alspaw and Paul Hammond – 10 Deploys Per Day

Amazon Deploys Every 11.6 Seconds

Scrum

We finally come to Scrum. Scrum is a methodology and framework developed by Jeff Sutherland and Ken Schwaber and presented to the OOPSLA conference in 1995. Scrum is an open source framework. You do not have to pay any licence fees to use Scrum, but you do need to follow the required portions of Scrum, otherwise as mentioned in the Scrum Guide, although you can use portions of Scrum, if you are not following the requirements as per the guide, you are not doing Scrum.
Scrum is a framework. It has been created to form a base process. Once you have the basics in place you can develop on top. Practices such as User Stories and Kanban Boards are examples of this.
Scrum consists of 3 roles.
The Scrum Master who is in charge of making sure that the team follows the Scrum principles. Without someone driving the principles, teams tend to regress to more familiar development processes such as Waterfall, or start doing the “Easy” Parts of Scrum without doing the hard parts, thus not getting the full benefit.
The second role is the “Product Owner”. The Product Owner is the advocate for the business. They provide the requirements. What the business wants. They are also in charge of what work is to be done which is in the form of an ordered backlog. The backlog is ordered based on what gives the most value to the business. The product owner does not dictate how the work is to be done.
The third and final role is the Development Team. This is a self managed , cross functional dedicated team who performs the work. The team is in charge of how to implement the requirements. There are no leaders within the team. The whole concept is very egalitarian.

In Scrum there are 4 ceremonies.
The Sprint Planning Sessions where planning of the sprint, a short period of development usually about 2 weeks.
The Daily Standup where the Development Team members synchronize on what they are doing.
The Sprint Review where the development team shows the stake holders what they have done during the sprint and finally the
Retrospective where the team reflects on the sprint. Determine what practices went well, what didn’t and try to find out how to improve on the next sprint.
This is another form of the feedback cycle.

Finally, there are 3 artifacts in Scrum.
The Product backlog.
The Sprint Backlog and the
Product Increment. A portion of the product that should be in a state that is releasable to production after each sprint.

We will go through the details of Scrum later in more detail.

The Scrum Guide

Wikipedia Entry – Scrum Software Development)

Scrum Hybrids

Finally there are Scrum Hybrids. These are where 2 or more methodologies are combined with Scrum. Some notable hybrids are Scrumban, a combination of Scrum and Kanban and Water-Scrum-Fall where the waterfall methodology is combined with Scrum. It should be noted that Water-Scrum-Fall is not generally an accepted methodology in the Agile community, but is seen as an example of what can happen when the easy bits of scrum are combined with the Waterfall Methodology. Generally the benefits of this approach are minimal compared to properly implemented Scrum.
Not all hybrids are bad – Scrumban is considered a success, but you do need to know what you are doing to get the full benefits.

Wikipedia Entry – Scrumban

Water-Scrum-Fall

Notable Mentions

Other notable mentions are
The Theory of Constraints. Not considered an Agile Methodology, but can be adapted to Software development and in the case of DevOps has been incorporated. Released when Elyahu Goldratt released his Management teaching novel “The Goal”.

Lean Six Sigma. Another manufacturing methodology that can be adapted to Software Development that combines the elimination of Waste part of Lean with the Continuous Improvement parts of Six Sigma.
Six Sigma uses the DMAIC process for problem resolution where it stands for
D – Define
M – Measure
A – Analyze
I – Improve
C – Control

Pair Programming – My Ignorant View

Firstly a disclaimer. I have never done pair Programming. Never read officially about it I.e. Kent Becks Extreme Programming Explained. I have only what I have read in blogs, which is limited and my own thoughts. 
I feel it is important to document my ignorant understanding before I embark on learning as it gives a base level of my understanding based on what I currently know about agile methods which I can look at in the near or distant future and see my progress. 

Now, for my ideas on pair Programming .

Say you have 2 people, A and B. A has knowledge that B needs.

First phase, A and B sit together. A works while B observes. B can ask questions, but A explains as they go along what they are dong.

Second phase, B takes over while A observes. B applies the knowledge they have learned, while A corrects B for any mistakes or answers any questions as required.

At the end of both phases, you now have 2 people converse in the knowledge.

Now it it time for the third phase. B now teaches C and simultaneously, A can teach D if required.

This I see as a good way to disseminate knowledge through a group. You get the one on one time and then apply the knowledge through teaching. 

I first saw something similar in a TV show. ER as a method for surgeons to disseminate knowledge. The saying was…

  • See one
  • Do one
  • Teach one

Now, as I mentioned earlier, this is my ignorant thoughts, but I would like to know everyone else’s on the topic. Let me know in the comments.

The Founder

I recently watched the movie ‘The Founder’, and i was watching the MacDonalds brothers talking about how they started. To me, the way they worked everything out looked like Lean to me. They way they worked out the restaurant layout, on a tennis court, with chalk outlines then get the people in to simulate their work. Try it out, when it doesn’t work, scrub out the lines, re-draw. The way the brothers automated, also how they failed but kept trying different things. They also didn’t hide anything. Eye showed everything off. Not only to Ray Kroc, but anyone interested. Just like TOyota does.
The whole system looked like lean in the making, if this story is accurate, it meant that the US had lean concepts 30+ years before it was formally introduced by Toyota in the 80’s. The movie was sad in the end, the McDonald brothers lost their legacy. Had there been collaboration instead of confrontation between the McDonald brothers and Ray Kroc, maybe lean might have been incorporated into the culture a decade or so earlier.

I guess we’ll never know.

Yellow Belt

I’ve just completed and passed the Lean Six Sigma Yellow Belt course at https://goleansixsigma.com/

The course is an introduction to Lean Six Sigma and goes through the DMAIC steps. The basic introduction is the white belt which is also free, but I didn’t take.

D – Define

M – Measure

A – Analyze

I – Implement

C – Control

The course itself is free and I recommend doing the course as it will give you another perspective to Lean. Whether or not you choose to do the certification, I leave that up to you, but it costs $99 US to take the test with unlimited retakes, which I happily paid.

At some point I’ll go through the things I found interesting in the course, but for now I just want to let you know about it.

The Agile Mindset

This is the second chapter. Lets see how we go.

The Agile Mindset

There are two parts to being Agile. The first half is the Agile mindset. This is where your thinking needs to change. Agile when implemented correctly is a paradigm shift in the thought process compared to how work is performed today.
The second half is the process. The process adds discipline and structure to your Agile thinking. Implementing the Agile mindset, is a cultural change. Even if you do not implement the process side, you can still do great things. The problem is that most people, when they start to implement Agile only implement the process. Although benefits can be gained, they are small. The process itself feels empty like you are just going through the motions, which you are. If you don’t make the leap to the mind set, then most likely your Agile journey will end and you will regress back to your previous work practices.

So what is the Agile mindset and what is the alternative mindset?
According to Linda Rising and her talk on the Agile Mindset, there are 2 types of people. The first is people that think you are born with a certain level of intelligence/talent or ability, you can learn new things, but you can’t change how intelligent/talented or able you are.
The second type of people are those that think that no matter how intelligent /talented or abled you are, you can always get better. Sometimes you can improve a lot.

The first type of people are considered the “Fixed” mindset. The second are considered the “Agile” or “Growth” mindset. Now, we all have these mindsets at some point. We flip flop depending on the situation and what we know all the time, but some people are more dominant in one mindset that the other.
People with the fixed mindset are more prone to fear failure. They will choose an easier path. The path of less resistance. The one less prone to failure.
Their thinking is that it is too difficult to do, so why try. They avoid challenge and prefer proven techniques. Techniques that have proved their success. Alternatively, when they fail, they look at the “success” in the failure and focus on that. These people are more interested in looking good.

On the other hand, those with the Growth or Agile mindset, look at a difficult situation and think – If I just work a little longer, try a bit harder, spend a bit more time on it, I will get it. If they fail, they try again.
There have been numerous psychological studies on this. In one of these studies, it found that a particular mindset can be easily induced. At least for a short period. Ever been told that something is too difficult to do, then give up? Alternatively, when you do something well, you think I’ve made it. Then stop? That is the fixed mindset.
On the other hand, if you have been told that something is too difficult and decided to try it for yourself anyway, or when you get something right, try to improve it, that is the Agile mindset. These are just a couple of examples.

The thing is that we become either mindset when we are young. If we are told when we are children that we are “Talented” in something or “You’re so smart!” or the negative “You are so stupid” or “You won’t amount to anything”, then this induces the expectation. For those that get the praise, they become praise junkies. They look to what they can do to keep getting the “You’re so talented” praise. The best way to do that is to find a challenge that isn’t too difficult. One that isn’t too far out of reach.
The problem is, that when a child with this type of thinking comes across a challenge that is too difficult, the child either avoids or ignores the challenge.
These are the children that grow up to be fixed mind set dominant.

Alternatively, children that are praised on their “effort” rather than that result, even if they fail, or for the negative, told that they “need to work harder” actually become more resilient to change. They try to tackle it head on. They are more open to ideas and willing to give them a go, because failure is part of life and you just move on.
These children grow up to have an Agile dominant mindset.

Its not only people that can have these types of mindsets. Organizations can too. It is not surprising. Organizations are made up of people.

But, all is not lost. People can change. Those with a fixed mindset can have an Agile mindset. You just need to want it.

So with Agile development in mind, the Agile mindset at its core is:

  • Thinking of your customer first. How can you make their life easier/better or what value can you give.
  • The willingness to try and learn new things.
  • Have the goal to improve yourself, your team and your colleagues and
  • Never be afraid to fail.

Thinking Of Your Customer

When I say “Thinking Of Your Customer” the question that comes to mind is “Who is your customer?”. The first and obvious answer is the people who use the product or service that you provide. These may be the general public or other companies. By thinking of the customer in this instance, you try to provide a product that is valuable to them. Value can be very subjective. It could be something simple that helps your customer accomplish a goal or it could be a complex.
For example, Facebook started out as a web site where University students (I’m Australian so I won’t say College – A College here is high school) stay in contact and share information. Google started out as a search engine because the founders didn’t like the current search engines that were out there. They thought that a good search engine would be valuable to others. Zappos wanted to see if people wanted to buy shoes online.
These companies started by adding large amounts of value to a small group of people. It wasn’t until later that they stared expanding and adding value for other customers.
The second lot of customers are your internal customers. These are other teams or other members within your team. Be they under you, your peers or your seniors. These customers, you help indirectly. For the people under you, there is the concept of “Servant Leadership”. Servant leadership is much different to the standard method of leadership which I will term “Command And Control”. With Servant Leadership, instead of telling people what to do as you would with Command and Control leadership, your job is to guide and grow your people. Give them direction and hen get out of their way.
You, as the leader have the big picture. So you give the team the goal to help accomplish the big picture. You do not tell them how to reach the goal, you leave it up to the team. If the team veers too far from the goal, you nudge them back toward the goal. You never ever tell them how to reach the goal. The reason for this is that it helps grow your team. The team becomes stronger because they make mistakes (and they will) but you guide them back. You never solve the problems for them, you need to let them solve the problems them self. Believe it or not, this makes for a happier team as the members then have full responsibility for reaching the goal. The autonomy to make decisions on how to reach the goal themselves. They also master the skills required to accomplish the goal. For those that are peers, you help make their jobs easier. Try not to send crap work. Share knowledge. For your manager, help them accomplish their goal.

By thinking of the people first instead of results, you grow your customer base, both externally meaning more profit, and internally meaning you have a more dedicated workforce.
One thing to remember though is that respect for one another can be gained very easily. It can also be lost very easily. The problem is that once it is lost, it can be very difficult to regain.

The Willingness To Try And Learn New Things

By simply having an open mind to try something new. By trying something new, you open your mind to new possibilities. A way to work better. A way to work faster. A way to improve quality. As humans, we try to find the one true or best solution. For some problems, this is the case.
These types of problems though in reality tend to be the simple problems. What font size is the best for me to read. What is the boiling point of water at sea level. These are like “Mount Fuji” type problems where the peak is the optimal solution.

photo credit: reggiepen Reaching Out via photopin (license)

For more complex type of problems, the landscape is a little different. There are peaks and valleys. One solution is at a peak, the dismal failure is in the valley. The problem is that there are many peaks. Some higher than others. The only way to reach another peak is via a different path.
These types of landscapes are like a “mountain range”. The only way to find out which peaks are the highest is to try different things. Different combinations or completely methods altogether. Even counter intuitive methods can bring surprising results.

photo credit: romanboed Sunny Mont Blanc via photopin (license)

For really complex problems. More like real life problems. We have the mountain range, but it shifts continuously. In other words, its waves. You may be high at one point, but if you stay still long enough, or short enough, you are in the valley of failure. The only way to continually remain high is to “surf” the waves. You have to continuously shift and change. What worked one day will not necessarily work the next.

photo credit: jijake1977 Rip Curl via photopin (license)

Companies like Google continuously try new things. They have great successes, Their search engine, Gmail but they also have dismal failures. Google Wave, Google+. Yet they continuously surf.

Peter Palchinsky a Russian engineer in the early 20th century summarized how to surf through his 3 principles.

  1. Seek out new ideas and try new things.
  2. When trying something new do it at a scale that is survivable.
  3. Seek Feedback and learn from your mistakes.

This leads us to our next section.

Continuous Improvement

Continues Improvement as the name suggests is well, improving continuously. Not every now and then. Not when you have the time. But always. Always looking for ways to improve. This on itself is hard. Paul Akers from 2 second Lean suggests everyone looking for an improvement of 2 seconds every day. His findings are that if everyone looks for improvements of at least 2 seconds, then every now and then, a significant improvement is found. Even if it doesn’t. Those little improvements build up over time. This in Lean terms is called Kaizen.
So instead of thinking “If it ain’t broke, don’t fix it” think “If it ain’t broke, fix it anyway”.
So the question now becomes, how do you improve? What is the definition of “improvement”. Well, its different for everyone, but for me and what I think it is in Agile is simplification. Simplify the process, simplify the program, simplify as much as you can. The best way I think to simplify something, is to subtract the complexity.

Waste

In Lean, the simplification process is done through the removal of what they call Waste. There are 3 major types of waste in Lean.

  • Muri – The waste of Overburden.
  • Mura – The waste of Unevenness and finally
  • Muda – The waste of Futility.

These wastes are further broken down into 8 more wastes.

  • Overproduction – This occurs when you create more than you currently need. When you over produce, you need to store the over produced product. This becomes
  • Inventory – Then you need to figure out how to move all your stock. This leads to
  • Transport where goods are moved from one area of the warehouse to another. Then you have to work out a process to keep track of everything. This is
  • Over processing. Your processes become more complex. Stock gets damaged. The damaged stock become
  • Defects. To fix the defects, your people need to move the stock back onto the floor, fix the defect then move it back into storage. This is a lot of
  • Motion. All the while, your customers are
  • Waiting for their goods. All the while your people are sick of redoing work. They want make things better, they want to be happy at work. But they can’t because they are not allowed to. They fell they have
  • Underutilized talent.

Notice how I put the 8 wastes into a story I have taken this queue from Paul Akers of 2 Second Lea. The idea being is that having a little story you can tell about the wastes. It helps you learn them, but also helps you understand them. its much better than just remembering an Acronym such as DOWNTIME.
Now these wastes are generally for the Manufacturing industry, but they are similar to what you will see in the IT space. Saying that, there is a list of wastes specifically for IT. I’m going to attempt a story here, but I don’t think it will be as good at the one above.

When you have so much work to be done, and its all done on the fly with no

  • Standards and Manual work, the priority of tasks keep changing. Do this now, no stop that and do this. You are constantly
  • Task Switching. This leads to
  • Partiality Done Work. Nothing really gets completed on time. Everything gets late. the customer is
  • Waiting for their software longer. Since it takes so long to get to the customer, and we are under the pump, we make mistakes, cut corners this introduces
  • Defects that need to be fixed. The buggy code goes back to development, then testing, then UAT again. Its constantly in
  • Motion. The software is so buggy, we introduce gated changes and procedures and other.
  • Extra Processess to prevent the issues.
    The deadline looms, Nothing is complete. Everyone stays back late into the night. Works weekends. Their
    -> Heroics is astounding.
    We finally ship. We planned everything up front, Did everything the customer initially specified. Oh no, they didn’t really need that
  • Extra Feature that we spent months building. It has no business value. It was just a nice to have.

Believe it or not, but over 90% of the work you do is waste of some sort or another. I know, it’s an outlandish statement, but this is part of the mindset. The hard bit is trying to remove that 90%. Most of the stuff we do we think is required. That is why we do it. But if you really really look at it, there are ways we can automate, remove or change our practice to reduce. Over time, you will find that you are doing the same amount of work quicker. In most cases significantly quicker. You will also find that you can do more in the same amount of time as you do one thing now.
It is through the reduction and or removal of these wastes which is at the heart of Agile. It is one of the methods on how you get faster. What? You thought that doing Stand-Ups, Sprints and so forth would make you faster? How would it? you are adding extra processes (a form of waste) if you apply it to the way you currently work without changing the way you work. You are actually making things longer. Just think about it.

Feedback loops

The other tool to help with Continuous Improvment is the Feedback loop.
This is a universal tool to improve. There are many feedback loops. I mentioned one earlier. Peter Palchinsky’s principles.
Another one that is synonymous with Lean is the Demming Cycle, or the PDSA/PDCA loop which stands for Plan, Do, Check/Study and Act.

You

Plan what you are going to do.
Do what you are going to do.
Check or Study the results from what you have done.
Act on the results of that Check or Study and feed that back into the next Plan.

Another type of feedback loop created by Eric Ries of Lean Startup is the Build, Measure, Learn loop.

You Build a piece of the product. Just enough to get a minimum working product.
Release to your customers.
Measure their responses to that minimum viable product.
Learn based on the measurements what the next direction will be, then Build the next iteration. Do this continuously.

Yet another feedback look is the Observe, Orient, Decide, Act.

Observe the situation.
Orient yourself based on the observation.
Decide What to do.
Act Then do it.

In Scrum, there is a feedback loop within a feedback loop. I will go through that in more detail when I cover Agile methods. Especially Scrum.

Now, there is no sure way you can eliminate waste, adding feedback loops and short cycles will help, but ultimately this is a trial and error process. The problem with trial and error, is that you have to be able to accept error. In other words, failure.

Not Being Afraid To Fail

The fear of failure is generally what holds people and companies back. They become cautious, slow down. Take the easy route. If you fail, you are seen as a pariah. Failure is stigmatized so it is avoided at all costs. Eventually you stagnate.
People and companies that embrace failure, look at it as a learning experience actually become more healthy. They learn and grow. Some companies actually embrace failure. For example, at Etsy, the person who broke the build the most times at the end of the year is awarded a 3 armed sweater. Other companies hold a celebration. The failure is celebrated as a learning experience. It is investigated and ways are found to prevent the failure from happening again in the future.
Yes you may fail again, but so what. It’s part of learning.
Those of us who are in senior roles, why are you there? Is it because you got everything right? How did you get it right? Or Is it built on a lifetime of failures where you learned?

Further along, I’ll go through a game you can play to help remove the stigma of failure from your team.

It is my belief, and thats all it can be as I’m not a psychologist or expert on human behavior, that by taking these 4 core principles of Thinking of your customer (people), Having a willingness to try new thing, continuously try to improve and finally not being afraid to fail, will give you the required foundation to have an Agile mindset.

If you find it hard to have this mindset, and it will be difficult to change, my advice is “Fake it till you make it”. The mere fact that you are trying to change will help you to change.

Thoughts, Tutorials and Learnings specifically with regards to Agile, DevOps and SOA/EAI