Knowing your metrics

velocity variance
velocity variance

Know your metrics and the behaviors they drive

Everyone at your company should understand which metrics drive the business, and what behaviors they encourage. That's what Joe Nigro, CEO of energy company Constellation, said in a 2016 Harvard Business Review article.  He went on to say, “Everyone needs to know how each metric fits into the big picture…why and how we’re measuring something, and how it’s relevant to performance.”

More directly, I would say metrics should capture the changing environment of your business so you can make informed decisions. But how do you know your metrics are any good?

Encouraging behaviors

If you ask your fellow employees which metrics drive the business, would they know?  Would they care?  I believe their jobs should depend on them caring, though measuring that would be difficult.  If they are unwilling to do their part, perhaps they should "help" some other company.  Everyone should be held personally accountable to understand what helps drive the business and how they can help. If they knew what the metrics are, how could it change their behavior (in a positive way)?

Metrics executives may be thinking about

From an executive level, I can only imagine every CEO (from Joe Nigro of Constellation to your own) start by thinking about these metrics:

  1. Topline revenue - Money made from selling goods or services
  2. Customer retention - Attracting the right customer, getting them to buy, buy again, buy in higher quantities or at higher rates.
  3. Customer acquisition cost - The total cost associated with acquiring a new customer, including all aspects of marketing and sales.
  4. Gross margin - Calculated as a company's total sales revenue minus its cost of goods sold, divided by the total sales revenue, expressed as a percentage.
  5. Overhead costs - fixed costs that are not dependent on the level of goods or services produced by the business, such as salaries or rents being paid per month.

I hope executive management will help employees understand the metrics that drive the business and why they are important. I hope the employees will internalize these metrics and consider ways to help the company increase revenues, widen margins, or control costs.

One step deeper into the weeds

From a productivity level, be it manufacturing or software development, I think the staff (and the executives) need to understand (and improve) the system of delivery. To this, if executive management doesn't know these productivity numbers, then how can they know when they are making unreasonable requests of the system? A system of delivery in a black box and executives in an ivory tower are not a good combination.  A dissatisfied staff can put your company at serious risk, while on the other hand, a satisfied and productive staff can help drive the business.

  1. Cycle time - The total time from the beginning to the end of your process.
  2. Lead time - Starts when a request is made and ends at delivery.
  3. Utilization - 100% being the maximum, it's the act of making practical and effective use of people or things
  4. Throughput - The amount of material or items passing through a system or process. What did you get done or delivered?
  5. Cost of Delay - The means to calculate and compare the cost of not completing something now, by choosing to do it later.

I hope everyone will think of ways to shortening cycle and lead times while maintaining or increasing throughput. If maximum utilization is how you make the greatest topline revenue, how do you reach that utilization level without breaking your people or machinery?

Another step deeper into the weeds

From an individual level, I believe we are truly personally accountable. Let's ask the first two questions from this post again. Which metrics drive the business?  What behaviors do they encourage?

  1. Commitment/Completion Ratio  - What have I personally committed to? Am I meeting that commitment?
  2. Throughput (Velocity) Variance - Given the things that I've recently completed, am I predictable? Can others make commitments, based on what I do?
  3. Confidence Score - How confident am I that I will actually keep the commitment I made?

I hope individuals will be honest about what they think they can do in a given period of time. I hope they will be honest with their coworkers and with management if they don't think they can keep a commitment.

What behaviors do you think metrics encourage?

Image Source: Calculated Velocity Variance via Notion []

Leading and Lagging Indicators

leading and lagging indicators
leading and lagging indicators

When attempting to attain an objective or key result, people often refer to key performance, leading and lagging indicators. Unfortunately, a lot of people don't know the difference and how to use them to their benefit. This post should provide some clarity to the differences.

What is a Key Performance Indicator (KPI)

Indicators are statistical values to measure current conditions as well as forecast trends and outcomes. A Key Performance Indicator is a measurable value that demonstrates how effectively a company is achieving key business objectives.  Examples of business objectives can range from predictability, early ROI, and innovation, to lower costs, quality, and product fit.  In basic analysis, use indicators that quantify current conditions to provide insight into the future. Lagging indicators quantify current conditions. Leading indicators provide insight into the future.

What is a "Lagging Indicator"

Lagging indicators are typically “output” oriented. They are easy to measure but hard to improve or influence.  A lagging indicator is one that usually follows an event. The importance of a lagging indicator is its ability to confirm that a pattern is occurring.  Here is an example: Many organizations have a goal to deliver some kind of scope on a release date.  Items Delivered is a clear lagging indicator that is easy to measure.  Go look at a list of items that are done and delivered.

But how do you reach your future release objective of items delivered? For delivering product predictably, there are several “leading” indicators:

What is a "Leading Indicator"

These indicators are easier to influence but hard(er) to measure. I say harder because you have to put processes and tools in place in order to measure them.  When you start building product, a lot of what you will understand and build will emerge over time. You don't know exactly what the level of effort is, until you finish. And if you are like me, given shifting priorities and dependencies, your lagging indicator is a moving target.  If you use leading indicators, you can see if you're tracking in the right direction. You can use the leading indicators to make changes to your behavior or environment while there is still time.

Diminishing ready backlog indicates we have less clarity on upcoming deliverables. An unstable delivery team indicates we don't have accountability to meet our commitment. Unstable velocity indicates we lack measurable progress that can forecast our completion by the release date.

Examples of Leading Indicators for Product Teams

Now lets imagine you are managing the product development division of your company and your goal is to meet the release commitment you made to your customers. The outcome is easy to measure: You either finished the items you committed to or not. But how do you influence the outcome? What are the activities you must undertake to achieve the desired outcome?

For example: Make sure there is enough ready backlog that the delivery team does not start working on "unready" work.  Make sure your team members are available when needed and not being shared with other teams. Ensure the team is remediating bugs as they go and not waiting until the end of the release to fix them.  Look out ahead of the delivery team and mitigate any business, organizational, or technical risks that may delay them.

These can translate into the following “leading” indicators:

  • Amount of ready backlog
  • Percentage of team availability
  • Percentage of deviation of the velocity divided by its mean
  • Amount of outstanding bugs
  • Number of known blockers

Examples of Leading Indicators for Services Teams

Now let's imagine your goal is to be compliant with SLA’s (service level agreements) you agreed to with your customers. For instance, the maximum allowed time to resolve critical priority incidents is 48 hours.  The outcome (lagging indicator) is easy to measure: You either resolve your incidents in 48 hours or not. Again, ask yourself, how do you influence the outcome?  What are the activities to achieve the desired outcome?

For example: Make sure staff start working on incidents immediately when they occur. Make sure that incidents are assigned to the right people with the right skillset and that this person isn’t already overloaded with other work.

These can translate into the following “leading” indicators:

  • Percentage of incidents not worked on for 2 hours
  • Percentage of open incidents older then 1 day
  • Average backlog of incidents per agent
  • Percentage of team availability
  • Percentage of incidents reopened more then 3 times

Begin measuring these indicators on a daily basis and focus on improving them. If you do, your organization is much more likely to reach its objectives. I have commonly seen organizations treating the leading indicators as the goal and measure of success. This is misguided. The objectives are the lagging indicators, whatever they are.  The goal for leading indicators is to improve them over time, to positively impact the lagging indicator.

An Introduction to OKR: Objectives and Key Results

An Introduction to OKR
An Introduction to OKR

Objectives and Key Results (OKR) is a popular leadership process for setting, communicating and monitoring goals and results in organizations on a regular schedule, usually quarterly. The intent of OKRs is to link organization, team and personal objectives in a hierarchical way to measurable results or outcomes, focusing all efforts to make measurable contributions.

Why OKRs are important

In a Harvard Business Review survey, only 55% of middle managers can name one of their company’s top five priorities. When the leaders charged with explaining strategy to their people are given five chances to list their company’s strategic objectives, nearly half fail to get one right.  This isn't anything new.  Andrew Grove first wrote about Objectives and Key Results (OKR) in his book High Output Management (1983) stating “A successful MBO system needs only to answer two questions: Where do I want to go? How will I pace myself to see if I’m getting there?"

Grove was actually referring to OKR, when he referenced MBO (Management By Objectives) in his book.  That said, knowing where you want to go will provide the Objective.  How you will pace yourself to see if you are getting there gives you milestones, or Key Results.

Later, Franklin Covey arrived at a similar strategy with The 4 Disciplines of Execution. (Have wildly important goals and a single measure of success)

Qualities of Objectives

  • Ambitious
  • Qualitative
  • Actionable
  • Time Bound

Qualities of Key Results

  • Measurable and Quantitative
  • Makes the objective achievable
  • Time Bound

OKR example for a start-up company raising a funding round

Company Objective 1 Finish raising capital for growth needs within 6 months [47% complete]

Key Results 1-4

  1. Email and phone 100 venture capital and seed funds (65 VCs contacted) [65%]
  2. Get at least 30 follow-up contact meetings or conference calls (15 follow-up meetings completed) [50%]
  3. Solicit at least 3 term sheets of our minimum required terms (1 term sheet solicited) [33%]
  4. Close an investment round with a minimum $10 Million pre-money (4 million raised) [40%]

Individual Objectives for Key Result 1 The objective is to email and phone 100 venture capital and seed funds. This will be distributed across one or more individuals. Then they will own individual key results.

Individual Key Results

Note that completion of these lower key results roll up to the higher level key result.

  1. Bob Smith research and identify 100 VC and seed funds (100 VC and seed funds identified) [100%]
  2. John Doe email or phone 4 VC or seed funds every week (3 VC's contacted this week) [75%]
  3. Bob Smith research and identity 50 Angel Investors (25 Angel Investors identified) [50%]

How to implement an OKR

  1. List 3 objectives you want to strive for on each level. (company, program, individual)
  2. For each objective, list 3-4 key results to be achieved. (lower-level objectives become higher-level key results)
  3. Communicated objectives and key results to everyone within the company.
  4. Identify metrics (via GQM) that will communicate progress of completion.
  5. Update each result at a predefined cadence on a 0-100% completion scale.
  6. Consider an objective done when its results reach 70-80%.
  7. Review OKR's regularly and set new ones.

Does monitoring OKR and goal progress promote goal attainment?

A journal article* from 2016 supports the suggestion that monitoring goal progress is a crucial process between setting and attaining the goal. Also, progress monitoring had larger effects on goal attainment when the outcomes were reported or made publicly available, and when the information was physically recorded.

*Psychological Bulletin, Vol 142(2), Feb 2016, 198-229