Metrics

How a Backlog of Work is Like a Tank of Gas

Screen Shot 2019-08-30 at 3.12.18 PM.png

Work is decomposed into smaller units (stories) and retained, to create a sufficient level of healthy backlog for a team to deliver.  Poor/low backlog may result in a team being blocked by unidentified dependencies, stopping work to refine more backlog mid-sprint, or pulling in work that is not ready, just to keep moving forward.

Much like fueling up before a long trip, if you don’t have enough gas in the tank, get stuck in traffic or need to take a detour, you may find yourself stranded. Stopping for more gas often results in you reaching your destination later than originally thought.

How much backlog you’ll need will depend on the dependencies your team has, the complexity of your work, and the demands on your team to be predictable.


Knowing your metrics

velocity variance
velocity variance

Know your metrics and the behaviors they drive

Everyone at your company should understand which metrics drive the business, and what behaviors they encourage. That's what Joe Nigro, CEO of energy company Constellation, said in a 2016 Harvard Business Review article.  He went on to say, “Everyone needs to know how each metric fits into the big picture…why and how we’re measuring something, and how it’s relevant to performance.”

More directly, I would say metrics should capture the changing environment of your business so you can make informed decisions. But how do you know your metrics are any good?

Encouraging behaviors

If you ask your fellow employees which metrics drive the business, would they know?  Would they care?  I believe their jobs should depend on them caring, though measuring that would be difficult.  If they are unwilling to do their part, perhaps they should "help" some other company.  Everyone should be held personally accountable to understand what helps drive the business and how they can help. If they knew what the metrics are, how could it change their behavior (in a positive way)?

Metrics executives may be thinking about

From an executive level, I can only imagine every CEO (from Joe Nigro of Constellation to your own) start by thinking about these metrics:

  1. Topline revenue - Money made from selling goods or services
  2. Customer retention - Attracting the right customer, getting them to buy, buy again, buy in higher quantities or at higher rates.
  3. Customer acquisition cost - The total cost associated with acquiring a new customer, including all aspects of marketing and sales.
  4. Gross margin - Calculated as a company's total sales revenue minus its cost of goods sold, divided by the total sales revenue, expressed as a percentage.
  5. Overhead costs - fixed costs that are not dependent on the level of goods or services produced by the business, such as salaries or rents being paid per month.

I hope executive management will help employees understand the metrics that drive the business and why they are important. I hope the employees will internalize these metrics and consider ways to help the company increase revenues, widen margins, or control costs.

One step deeper into the weeds

From a productivity level, be it manufacturing or software development, I think the staff (and the executives) need to understand (and improve) the system of delivery. To this, if executive management doesn't know these productivity numbers, then how can they know when they are making unreasonable requests of the system? A system of delivery in a black box and executives in an ivory tower are not a good combination.  A dissatisfied staff can put your company at serious risk, while on the other hand, a satisfied and productive staff can help drive the business.

  1. Cycle time - The total time from the beginning to the end of your process.
  2. Lead time - Starts when a request is made and ends at delivery.
  3. Utilization - 100% being the maximum, it's the act of making practical and effective use of people or things
  4. Throughput - The amount of material or items passing through a system or process. What did you get done or delivered?
  5. Cost of Delay - The means to calculate and compare the cost of not completing something now, by choosing to do it later.

I hope everyone will think of ways to shortening cycle and lead times while maintaining or increasing throughput. If maximum utilization is how you make the greatest topline revenue, how do you reach that utilization level without breaking your people or machinery?

Another step deeper into the weeds

From an individual level, I believe we are truly personally accountable. Let's ask the first two questions from this post again. Which metrics drive the business?  What behaviors do they encourage?

  1. Commitment/Completion Ratio  - What have I personally committed to? Am I meeting that commitment?
  2. Throughput (Velocity) Variance - Given the things that I've recently completed, am I predictable? Can others make commitments, based on what I do?
  3. Confidence Score - How confident am I that I will actually keep the commitment I made?

I hope individuals will be honest about what they think they can do in a given period of time. I hope they will be honest with their coworkers and with management if they don't think they can keep a commitment.

What behaviors do you think metrics encourage?

Image Source: Calculated Velocity Variance via Notion [UseNotion.com]

Leading and Lagging Indicators

leading and lagging indicators
leading and lagging indicators

When attempting to attain an objective or key result, people often refer to key performance, leading and lagging indicators. Unfortunately, a lot of people don't know the difference and how to use them to their benefit. This post should provide some clarity to the differences.

What is a Key Performance Indicator (KPI)

Indicators are statistical values to measure current conditions as well as forecast trends and outcomes. A Key Performance Indicator is a measurable value that demonstrates how effectively a company is achieving key business objectives.  Examples of business objectives can range from predictability, early ROI, and innovation, to lower costs, quality, and product fit.  In basic analysis, use indicators that quantify current conditions to provide insight into the future. Lagging indicators quantify current conditions. Leading indicators provide insight into the future.

What is a "Lagging Indicator"

Lagging indicators are typically “output” oriented. They are easy to measure but hard to improve or influence.  A lagging indicator is one that usually follows an event. The importance of a lagging indicator is its ability to confirm that a pattern is occurring.  Here is an example: Many organizations have a goal to deliver some kind of scope on a release date.  Items Delivered is a clear lagging indicator that is easy to measure.  Go look at a list of items that are done and delivered.

But how do you reach your future release objective of items delivered? For delivering product predictably, there are several “leading” indicators:

What is a "Leading Indicator"

These indicators are easier to influence but hard(er) to measure. I say harder because you have to put processes and tools in place in order to measure them.  When you start building product, a lot of what you will understand and build will emerge over time. You don't know exactly what the level of effort is, until you finish. And if you are like me, given shifting priorities and dependencies, your lagging indicator is a moving target.  If you use leading indicators, you can see if you're tracking in the right direction. You can use the leading indicators to make changes to your behavior or environment while there is still time.

Diminishing ready backlog indicates we have less clarity on upcoming deliverables. An unstable delivery team indicates we don't have accountability to meet our commitment. Unstable velocity indicates we lack measurable progress that can forecast our completion by the release date.

Examples of Leading Indicators for Product Teams

Now lets imagine you are managing the product development division of your company and your goal is to meet the release commitment you made to your customers. The outcome is easy to measure: You either finished the items you committed to or not. But how do you influence the outcome? What are the activities you must undertake to achieve the desired outcome?

For example: Make sure there is enough ready backlog that the delivery team does not start working on "unready" work.  Make sure your team members are available when needed and not being shared with other teams. Ensure the team is remediating bugs as they go and not waiting until the end of the release to fix them.  Look out ahead of the delivery team and mitigate any business, organizational, or technical risks that may delay them.

These can translate into the following “leading” indicators:

  • Amount of ready backlog
  • Percentage of team availability
  • Percentage of deviation of the velocity divided by its mean
  • Amount of outstanding bugs
  • Number of known blockers

Examples of Leading Indicators for Services Teams

Now let's imagine your goal is to be compliant with SLA’s (service level agreements) you agreed to with your customers. For instance, the maximum allowed time to resolve critical priority incidents is 48 hours.  The outcome (lagging indicator) is easy to measure: You either resolve your incidents in 48 hours or not. Again, ask yourself, how do you influence the outcome?  What are the activities to achieve the desired outcome?

For example: Make sure staff start working on incidents immediately when they occur. Make sure that incidents are assigned to the right people with the right skillset and that this person isn’t already overloaded with other work.

These can translate into the following “leading” indicators:

  • Percentage of incidents not worked on for 2 hours
  • Percentage of open incidents older then 1 day
  • Average backlog of incidents per agent
  • Percentage of team availability
  • Percentage of incidents reopened more then 3 times

Begin measuring these indicators on a daily basis and focus on improving them. If you do, your organization is much more likely to reach its objectives. I have commonly seen organizations treating the leading indicators as the goal and measure of success. This is misguided. The objectives are the lagging indicators, whatever they are.  The goal for leading indicators is to improve them over time, to positively impact the lagging indicator.

Getting Clarity

clarity-1-1024x538.jpg

I believe the number one reason for failure or waste is a lack of clarity or understanding. If you getting clarity on something, it gives you the freedom to decide if you want to do it or not.  If something is ambiguous, you may agree in principle but you don't know what you're really getting yourself into.

OKRs

Firstly, what are your Objectives and Key Results (OKR)? How do you set and communicate goals and results in your organization? Because you want people to move together in the right direction, you need to get clarity.

KPIs

What are your Key Performance Indicators (KPI)? How do you want to measure value that demonstrates how effectively your company is achieving key business objectives?  Because you want your organization to evaluate its success at reaching targets, you need to get clarity.

Structure

What does the team design or structure of the organization look like on portfolio, program, product, and service layers? We need a shared understanding of which individuals or teams are responsible for what.

Governance

What does the governance of the organization look like? How do we manage our budget, dependencies, risks, or quality? What are the inputs, outputs, and artifacts?

Metrics and Tools

Because we want to manage our system of delivery, what are necessary metrics and tools of the organization?

Getting Clarity

Remember, if you expect others to commit to something, regardless if it's a process or a deliverable, we need a shared understanding.

Lean Metrics in the Real World

Today, I was faced with the unfortunate task of renewing my driver's license. It's been 10 years since the last renewal and I remember the last time I was at the Maryland MVA (Motor Vehicle Administration) office, I waited for what seemed to be an hour.  We all know how painful the experience is.  You stand in line, you get to the front of the line, they tell you to go fill out some paperwork and then to get back in line. I will use the opportunity to teach others about lean metrics.

Lead Time

Lead time is the time between the initiation and completion of a production process.  In my case, I left the office at 09:00 and arrived back at the office at 10:00.  The lead time to get a renewed driver's license was 60 minutes.  Given my goal, the shorter the lead time the better.

Cycle Time

Cycle time is the total time from the beginning to the end of a process, as defined by you and your customer. Cycle time includes process time, during which a unit is acted upon to bring it closer to an output, and delay time, during which a unit of work is spent waiting to take the next action.  The shorter your cycle times (including delays) the shorter your lead time.

My cycle times included:

  • 15 minutes - driving to MVA office
  • 5 minutes - standing in initial line to be added to the proper queue
  • 16 minutes - wait time to get the front of the line
  • 5 minutes - actual renewal processing
  • 4 minutes - wait time to be given the new driver's license
  • 15 minutes - driving back from the MVA office

Cycle time is one of the key lean metrics

My hat comes off to Maryland MVA.  On their website, they provide current wait times at the different locations.  I took a screen grab before I headed to the local MVA branch.  This feedback was very valuable.  Given the service I needed, it allowed me to provide an estimate of my time away to others I was going to be working with today.

Throughput

Throughput is the the amount of material or items (people in this case) passing through a system or process per time unit. With an average cycle time of 5 minutes for the renewal process, the throughput in 60 minutes would be 12 people.  At first glance, I didn't see any real bottlenecks or delays in their system.  Given what I saw, I believe 10 people an hour is a reasonable throughput.

Understanding Lean Metrics

I hope this brief real world example of lean metrics is valuable to you.  When I was at a session for value stream mapping at Agile 2015, the poor guy leading the session kept getting lead time and cycle time mixed up.  The people in the room heckled the hell out of him. After reading this, that should never happen to you.