Roadblocks to Engineering Productivity

In recent years, there has been an increased interest in improving engineering productivity. It’s hard to miss the increase in engineering productivity tooling and different articles about what metrics to have and how to measure productivity. This is driven in part by two key factors:

• Software engineer salaries reached an all-time high in 2022, making it really hard to hire and retain engineers.
• Recent tech layoffs put more pressure on the remaining engineers to maintain their complex systems and on managers to deliver more with fewer people.

In this blog post, we are going to explore what are the primary roadblocks to productivity and what companies can do to mitigate them. Let’s dive in!

Finding information

Have you ever had to find the schema of an API and examples of how to use it? Or how to set up a new service in your current infrastructure just to find out the information is outdated or incomplete?

Probably you or your team have had this experience. Documentation in multiple places and with no structure or outdated and incomplete. This leads to valuable time wasted by the teams seeking information and it creates the wrong incentive to interrupt others to ask questions.

Knowledge sharing can improve the overall productivity of the team. You can have weekly share sessions, where each member shares some findings or something useful for the team members. You can have a structured onboarding plan for new members to guarantee they have the knowledge to be up to speed as soon as possible. In this study, the authors correlate the lack of knowledge sharing with the impact on the productivity of the team.

The respondents are aware that the less information is shared, the less productive the team is, which underlines the relevance of social interaction and trust in colleagues.

LLM to the rescue! - bingo word! I actually believe it is the first time I’m mentioning it in this blog 😅. LLM may provide some support in finding the information. Leveraging this technology to help with information retrieval may be an excellent investment. It can summarize internal documentation that is scattered across multiple sources and it can also enrich it with codebase understanding to ensure it's accurate.

Context Switching

So, you finally have a slot without meetings where you can finally code! Yupi! And then, that notification in your Slack... Someone asking for help on how to create a new service. 😅 Meetings, Slack, Emails, shoulder poking… All of these practices kill creativity.

Flow state is essential for productivity. Flow is a state of deep concentration and enjoyment. People experiencing it, are completely absorbed in coding, and their thinking is totally immersed in the experience of coding. The rapid feedback and problem-solving capabilities give them a unique feeling of enjoyment. Those, the lucky ones, like myself, who had felt this kind of enjoyment, understand why coding is amazing and they usually do it as a hobby… It's really hard to understand for those unlucky ones that didn’t experience this.

In a study done with 141 software engineers, 56% said they cannot be fully focused on a typical day. It is important to communicate and ensure alignment, but more often than not, these interruptions are not adding value.

Regardless of the interruption characteristics and type, as stated by our participants, flow change and losing focus are the main reasons which make interruptions costly and harmful to the overall performance of the primary task

A different study shows a correlation between the output quality of some given task and the number of distractions.

The impact on the quality of work was measured in terms of grade points deducted. Fig. 7 shows grade deduction depending on the amount of multitasking overhead.

Some technology starts to appear in this space. An interesting tool that may help reduce the overall time to recover from an interruption is Task Samurai. By suggesting which files to change and the change plan, a developer can quickly return to the flow.

Create a culture that values being in the flow. Protect engineers' time to have deep-quality work done. Set clear communication expectations across the company and walk the walk!

Poor Code Quality

Google's paper on "What improves developer productivity at Google? Code Quality” has the following statement:

We find that increases in perceived code quality tend to be followed by increased perceived developer productivity, but not vice versa, providing the strongest evidence to date that code quality affects individual developer productivity.

The reason is not crystal clear. A possible reason may be found in a different study that shows that the lack of motivation is an internal barrier to achieving the flow state. A theory of mine is that the lack of code quality affects the developer’s motivation, and consequently, their productivity. 

What we know for sure is that increasing code quality has a strong correlation with perceived productivity. Hence, giving teams the necessary time to improve code quality is a good investment for improving overall productivity.

Lack of Psychological Safety

Have you ever worked on a project where a critical bug was found, but everyone was afraid to speak up? Or, have you ever had a project that felt doomed from the start, but everyone was afraid to say anything? That happened to me a few times. Six months of investment in one project that everyone knew would not contribute to the company’s goals. But yet we had to go through it and deliver it… just to shut it down one year later after tremendous losses.

The problem wasn’t just about the company’s money loss. The problem was that no one felt safe saying it to the boss soon enough to make it better.

Psychological safety means that people feel safe to speak up, or that a team knows that it's okay to make mistakes and learn from them. A culture of openness and trust.

Productivity, for me, is not about the output, it's about the outcome. Psychological safety helps both. When people feel safe to contribute and learn from mistakes, they tend to deliver more output. When people feel they can speak up, and that others will listen to them, it opens the door to the right discussions. Having the right conversations and discussions tends to deliver better outcomes.

How do you foster a culture of psychological safety? There are several things you can do:

• Lead by example by demonstrating openness to feedback and showing vulnerability to admit your own mistakes.
• Celebrate learning from mistakes and promote a growth mindset.
• Build trust among the team members.

Poor tooling and standardization

Remember the frustration of pushing a new change to trunk and waiting hours for the CI (Continuous Integration) pipeline to run? Only to find out there are errors? Or maybe the pipeline sometimes fails for no reason? Or remember dealing with different coding styles in the codebase that affect the way teams do innersource?

We have all been here. I remember contributing to some projects in ruby with different RuboCop configurations across the company and I also spent way too much time trying to understand what was the style that a given team preferred. And also remember a project where teams were adopting new pipeline technology and it was always failing. Teams’ productivity and motivation were at an all-time low.

Standardization is key. Now, with the layoffs, it's even more important. Developers are maintaining code they didn’t create because the company went from 5 teams to 2 teams. It's important to understand that scaling down must be part of a technology strategy. It can ease the burden of knowledge transfer after layoff, and also reduce the cost of creating amazing tools for the engineers.

When the tooling just works, people don’t give them enough value. But when it doesn’t work, it's imperative to fix it ASAP.

Environment (Office)

The environment in which the team is working is extremely important. Have you ever worked in a noisy office? And we have to love the open space, right?

The open space is terrible. It is usually crowded if the setup is “onsite” or it's usually too noisy if the setup is “hybrid” - because everyone is on a video call. Even worse, it's an invitation for interruptions.

That’s probably why most developers feel they are more productive and happy when working from home. It's harder for others to interrupt them. This is also one of the main reasons a lot of managers don’t like their team working from home, because they can’t interrupt them as easily 😅.

The impact is exactly the same as we have in the context switching, but I decided to have a section dedicated to this, given the current topic of WFH vs RTO.

At the end of the day, the more you can reduce or plan around potential interruptions, the better.

As a manager, how can you improve engineering productivity, then? Listen to people.

These are some of the primary roadblocks to productivity in the tech industry. They will surface in different formats, but you can use these as buckets. It will probably be hard to fix all of them, but some actions can be taken to mitigate them and ensure your team feels more productive.

There is one I didn’t talk about specifically, but it's worth mentioning in this section. The way you manage your team is also an important factor in productivity! Any leadership position must understand they exist to ensure the right outcome for the business. Management in tech companies can enable the right outcome by ensuring people can perform at their best.

The manager’s function is not to make people work, but to make it possible for people to work. (Peopleware)

Talk to your team to understand what is making it harder for them to deliver better software. These roadblocks will probably arise in the conversation in one format or another, but this is not a recipe.

Your team knows better and they can help you understand what is happening. Work with them and for them.