☕ The Blend - Issue #7

👋🏻 Hi Friends,

and welcome in the Issue #7 of ☕ The Blend. This time I would like to discuss the measurement of remote engineering teams.

As you know (or not), engineering teams are hard to measure. Software engineering involves writing code, which is inherently logical and measurable. Engineers often work with exact specifications, and defined goals. They are creating solutions based on measurable criteria. Yet, when it comes to assessing the performance of individual engineers, the metrics are not always as clear.

The irony in measuring lies in the contrast between the precise and quantifiable nature of their work and the challenges in applying similar precision to evaluate their performance.

WHAT should you measure?

Unlike salespeople, whose individual contributions can be measured (like number of customer calls, number of leads acquired, revenue generated etc.). Value created by engineers is much more complex. Most of the times (unless you're a one man army) it is driven by a collective effort, synergy and is interdependent.

You can't measure amount of code produced, as sometimes much more value can be delivered when the code is deleted.

Having said that, there are couple of things you can measure, that will help you navigate through the unknown.

There are 3 groups of metrics:

• CRAWL metrics

• WALK metrics

• RUN metrics

🧟‍♂️ CRAWL metrics

At the 'CRAWL' stage, metrics are basic and foundational. They focus on establishing 'minimal viable processes' and are ensuring stability. In software development, this might be DORA metrics, or in product world the MAU (Monthly Active Users) metric. The goal is to get things started and establish a baseline from which to grow.

What is DORA metrics?

DORA metrics serve as a guide to how well the engineering teams are performing and how successful a company is at delivering software.

They include:

1. Deployment frequency - how often an organisation successfully releases to production

2. Lead Time for Change - the amount of time it takes a commit to get into production

3. Change Failure Rate - the percentage of deployments leading to a degradation in service that must be addressed.

4. Time to Restore Service - how long it takes an organisation to recover from a failure in production.

🚶🏻‍♂️WALK metrics

In the 'WALK' stage, the focus shifts to optimisation and efficiency. Metrics become more sophisticated and are used to refine processes. In engineering it can be Cloud Costs, or performance against SLA. In product, it can be NPS Score, or level of integration of data sources. The idea is to build on the basic capabilities established in the CRAWL stage and start optimising processes.

🏃🏻‍♂️ RUN metrics

At the 'RUN' stage, the organization is focusing on advanced and strategic goals. Metrics at this stage are about driving innovation and leveraging processes for competitive advantage. In engineering, this could mean Team Satisfaction (eNPS), Team Health or Number of Vulnerabilities. In product, it could involve Flow Time, ROI, or Flow Velocity. The goal is to operate at a high level of efficiency and innovation.

HOW should you measure?

There are numbers of way you can measure the above. Currently there are tons of software on the market that helps with that. 

I will be digging deeper in the upcoming releases on what tools I use, and how I assess the metrics, so make sure to subscribe and don’t miss it.

WHY should you measure?

There are numbers of argument that are in favour to do it. Starting with consciousness of your teams' work, through understanding what impacts their flow.

• DORA metrics help in understanding how quickly and efficiently the team can deliver new features and fixes. This understanding helps in streamlining processes, removing bottlenecks, and improving team productivity.

• You will boost the quality of the product you're building.

• Regularly measuring and reflecting on these metrics encourages a culture of continuous improvement. Teams can set goals and track progress over time, fostering a mindset of always seeking to do better.

• By improving deployment frequency and reducing time to restore service, the organization can respond more quickly to customer needs and market changes. This agility can lead to higher customer satisfaction and a better user experience.

• These metrics help align the engineering team’s efforts with broader business goals.

• Faster, more reliable deliveries and improved quality can contribute to the success of the business.

With that you can set the baseline and then optimize. You don't always have to hire another person to your team to get more work done. In fact, hiring has quite the opposite effect when you don't measure the efficiency.

The Goodhart's Law

While metrics are invaluable for understanding and improving the engineering processes, it's crucial to have in mind the Goodhart's Law: 'When a measure becomes a target, it ceases to be a good measure'. It reminds us that over-relying on specific metrics can lead to unintended consequences. When we focus too narrowly on metrics as targets, teams might be incentivised to game the system, optimising for the metric rather than the underlying goal of quality and efficiency. For instance, if we overly emphasise deployment frequency, we might encourage more frequent but less meaningful releases. It's essential, to balance the reliance on metrics with an understanding of their limitations and to continually reassess their relevance and impact on our true objectives. Metrics should guide and inform, but not dictate, our strategies and actions.

🏁 In the next episode...

It's clear that measuring the work of software engineering teams is both an art and a science. The journey from CRAWL to RUN metrics isn't just about numbers - it's about understanding the story they tell about our teams, our processes, and the value we deliver. It's about making informed decisions, driving continuous improvement, and ultimately, creating better software.

In the upcoming issue I'll be diving more into what (and how) I measure my team, and how can you impact these metrics (real life examples).

We'll also explore the human element of Engineering Management. Behind every metric, there's a team of talented individuals striving to make a difference.