☕ The Blend - Issue #10

Hi Friends,

🙌 First milestone achieved - 10th official Issue of ☕ The Blend is here!
Thank you for all the comments, questions and feedback. Means the world to me ♥️. Starting the newsletter for like-minded people was the best decision ever.

In this issue, I wanted to talk about different approaches one can take to tackle the technical debt

As an engineering leader, have you ever been struggling with a dilemma of new feature vs maintenance? You’re probably juggling on a daily basis multiple new business request with an long tail of tech debt.

As the engineering leader you know that tech debt has a very high interest rates and it can blow up in your face anytime. It can either bring down your product, or (what sometimes is much worse, especially in the smaller companies)… it can make people leave your company. Depending on your size, and the way you share knowledge across your organisation, this can be a very painful situation.

I wanted to talk about actionable tips and ideas you can take and put into motion, to start paying the tech debt.

There are 3 steps you can take:

1. Diagnosis

2. Guiding policies

3. Actions

🩺 Diagnosis - understand the situation, define status quo

In the beginning you need to firstly understand how big the tech debt is. There are multiple ways you can try to surface it.

1. Code reviews - in smaller organisation, you’re probably still quite close the code, and you can perform code reviews with your team. This will give you a hands-on information on what is the code’s quality being written .

2. Code analysis tools - there are a lot of different tools like SonarQube or Code Climate that have created the niche for themselves, so called Software Engineering Intelligence. They can provide metrics on code quality, potential bugs, and some sort of quantifiable measure of technical debt.

3. Test coverage - one of your metrics can be the analysis of your test coverage. Low coverage will indicate a higher technical debt, leading to potential bugs and maintenance issues. (You should be aiming between 60-80% of coverage)

4. Documentation review - one of the most underrated part of software product is it’s engineering documentation. Everything seems to be working great until either you have to onboard a new person, or someone just left the company, and you need to take over their part of work. Lack of documentation is almost never listed as part the technical debt, while it can create a huge problem. You should take that into account.

5. Number or bugs in backlog - this one is super important, especially the rate of how fast it’s growing. While you may have a lot of small issues, they can be experienced by a lot of customers, leading to a degrade of their experience. This metric should be tracked alongside of the incoming support requests.

6. Performance metrics - you should be monitoring the performance of your application, period. You should know your platform is struggling, before your clients have the opportunity to tell you that. You should be proactive here. If the performance is worse with each product iteration, you found yourself a new bomb to disarm.

7. Dependency monitoring - there are number of softwares (or services) that can help you monitor the outdated dependencies in the system, making sure you keep the tech stack at least up-to-LTS expiration. Ideally you should be running the newest stack whenever possible, but being realistic - it’s super hard. Your stack should at least have the actively supported by the creators, and getting security patches.

8. Refactoring impact - you should understand how often refactoring is needed. Frequent refactors can be a very good sign - if they are made alongside new features, meaning your team is paying off the tech debt, OR can indicate a massive tech debt. As a tech leader you should know which case is it.

9. Time taken for feature development - you should see how your teams reacts on new features. If they are hiding their faces in hands, and saying it’s gonna take ages… you should start digging. Your team might now much more about the issues that you’re not aware of.

10. 💡BONUS TIP: run the search on your codebase for a TODO comments… it’s really interesting what you’ll gonna find there. It can indicate the debt size taken, with each comment like that.

📄 Guiding policies

You should be fostering a culture of continuous learning, and encouraging team members to stay updated with current best practices and technologies. As you are a team, you should be collectively working on improving your processes, and guiding policies, can be a part of that.

Depending on the size of your organisation, the policies will be different, but here are couple of examples, so that you know what I mean:

1. No cloud policy - and by using the language from one of my previous newsletters, about building and influencing culture, you can set the policy:

2. No new languages - and using a similar approach as in the previous point, you could have a policy

Creating policies that will guide your organisation, are part of the “technology strategy”, and can directly impact the business decisions.

It is super important from many different angles:

1. It defines the business strategy and help Sales with speaking to customers, managing expectations

2. It created the dev culture that everyone agrees on, and don’t make unexpected architectural decisions

3. It is great to have such policies defined in your job description, when you’re looking for new engineers - setting these policies, will draw the attention of like-minded people, that will identify with the approach.

💪 Actions - how to approach tech debt

There is a great quote by Gene Kim, the author of the amazing book “The Phoenix Project” (highly recommend reading it!)

I could not agree more. If you leave the tech debt issue unresolved, you will start eating your own tail, and instead of creating business value for your customers, you will be working just to keep the light on - fixing one bug after another, putting out fire after fire.

For years now, I’ve been working with my teams to find the way to tackle these issues. Here’s what you can do 👇️ 

Apart from the quick insights from the list above like checking the codebase, doing the code reviews, and tracking the number of bugs, there are different, more strategic, ways of handling the tech debt.

As the engineering leader your probably asking about your “technical strategy”, and you might struggle with preparing one, BUT… the tech debt reduction is one of the key part of such plan.

Technical stakeholders

This is an easy scenario… if you are working with the technical C-level business stakeholders, they know how important it is to keep the up-to-date tech stack. Although it doesn’t mean, you’re immediately going to have a green light on paying it off - you should still have a proper tech strategy.

Non-technical stakeholders

This is how the most companies operate, and it requires a lot of effort to showcase the problem, and get the business approval. The first step is to always get your teams’ perspective on the tech debt and try to workshop possible ideas with them.

I want to stress one important thing → the strategy is not to make everyone happy, but to focus limited capabilities and capacity to solve the problem we pursue.
Not everyone in your team will like all the policies and new approach, but they all have to agree to abide by them.

I always loved the meetings we’ve had with my team, called “Teamspective”. It was NOT a meeting about the retrospective on our last cycle of work, but a more strategic discussion. We were discussing metrics and processes we can optimize - striving for excellence.
Being transparent about the metrics I’ve talked about in one of my previous episodes is key (read the issue about the metrics). Really small changes can lead to huge gains.

PR template

As simple change as introducing a PR template with a small checklist can be very beneficial, and sets the standard. You can start with this simple checklist:

# Description
My PR changes/introduces/creates/deletes...

# Checklist:

1. I have manually tested my code before handing it over
2. I have written test to my code
3. I have performed a self-review of my code
4. I have added Linear issue ID to the PR name
5. I have made corresponding changes to the documentation

Super simple, yet so powerful. Seeing such checklist in every single PR, makes people accountable for their work, and they feel more obligated in following the rules, even if it’s a small change.

Code Review process

Check how long usually it takes to Code Review the PR. We’ve seen that sometimes PR’s were sitting, waiting for CR, longer that they’ve actually took to write 🤯 … introducing a policy as simple as “Start your day with Code Review” almost doubled the amount of work delivered (we’ve been working in weekly cycles, ended with a release).

Both Code Review marginal gains, are gatekeepers to the quality, thus tech debt reduction.

Understand the time investment balance

This is a big one, and it requires business understanding. Your business likely has a seasonality in sales and usage. You can prepare a year-round plan for the business stakeholders, based on the data from previous years.

Using historical data, you can plan ahead more time consuming changes and infrastructure updates, knowing when the business is slower, or there is a lesser usage of your platform.

Prepare a plan

Now you know seasonality of your business, and you know which area needs to be fixed/refactored/removed so that you can start paying off the technical debt. It’s time to prepare a plan.

You can’t tackle everything at once, so the best way is to create couple of iterations throughout a year. This is the way I’ve approached it:

1. Twice a year (between Q1/Q2 and Q3/Q4) - 🐛🔫 Bug Hunting. Depending on the size of your organisation, either you’re going to do it yourself or with your Product counter part. You prioritize smaller issues, more of a quick wins, that will make your product better, but there is never a good time to do them, because there is always something more important.
   I would aim to spend 2 weeks (1 cycle of work in my case), as this is the timeframe that you can already ship a lot of meaningful work.

2. Vacation time - In my experience July/August was the time where there was much less people using our platform. This time we were working on improving the broader UX, making our product more usable and eye pleasing. The cognitive friction for users is smaller after some time off.

3. Christmas time - ideally, this is the time when you can work on the biggest changes to the backbone of your stack or infrastructure. This is usually the slowest period, and you can work on the biggest. For that period I’ve planned things like upgrade or Ruby and Rails versions to the newest ones, making sure we’re bulletproof for a year ahead.

Maybe you are a B2B SaaS that want to go through ISO certification to be a better choice for enterprise customers. You have to be aware of that, so that your plan can account for that.

Maybe you will have to run yearly penetration tests with 3rd party companies (that’s also a very important part of the engineering budget).

Maybe there are new laws that will affect your business - like GDPR, or your enterprise customers will require that the data cannot go outside of EU.

As engineering leader you should be on top of all that, making sure it’s planned ahead.