Full-Cycle Teams

When I first started as a developer, everyone was a “full-cycle[1]” developer. You did the front end; you did the back end; you did the database; you released to production; you owned the production support.

This was known as Dev, and everyone worked that way, even if it was pretty poor compared to what we do now.

Over the years, each of the above tasks has become increasingly complex. The gap between each of these activities has widened and specialisations have emerged. This has led to some of the above tasks no longer being considered part of the standard development cycle. Many of these tasks now come under the purview of DevOps.

The Rise of DevOps

The rise of DevOps has finally brought focus to all activities across the whole SDLC. It is clear that there are benefits to returning to what Netflix label “full-cycle” when you look at things this way. Once again teams cover both the Dev and DevOps side of things and manage everything including production. By embracing DevOps to such an extent, all of these activities just becomes part of Dev again. This is the full-cycle way.

Supporting The Team

However, if that is to be done, the team need to be given the time and support to allow them to do a successful job.

The first problem is always the tooling and the skills required to use that tooling. Building and supporting a production system requires skills beyond that simply required to build the UI and Services. Nowadays you often need a good knowledge of pipelines, containers and infrastructure and getting that knowledge is not easy.

Building The Pipeline

As with all difficult things, the best way to deal with them is to do them every day. For me, building and owning your pipeline is the first step in that journey. By owning the pipeline you have to understand the complete journey that all of your deployment artifacts take before being built and deployed to an environment.

There are always problems in pre-prod environments, and by having to deal with them daily ensures that you fully understand the pipeline. You are also more likely to fully automate if you have to use something every day. That’s where integrations into your other tooling such as slack or custom bots can also off payoff.

Monitoring, Alerting and Logging

Once you’ve built your new system, you are going to have to look after it. The best way to do this is to make sure you build in the Monitoring, Logging and Alerting (MLA) capabilities that you need, right from the start. I cannot stress highly enough how important it is going to be to take a look at a dashboard and see exactly what is going on with your application. You might not get to a single pane view of your entire system from front to back but you must have a simple capability of viewing its overall operation. If you cannot do that, you cannot effectively support your system.

There are many great tools to help with this in a Microservice environment. But there’s a good chance that your Microservices are not generally going to be the problem. It’s often the case that it’s a downstream system that causes the issue. With decent MLA in place, you can pinpoint the exact system causing the issue even when it’s not one you’ve built. Then with some simple synthetic tests in place, the whole incident can be sorted without involving a million unnecessary people. The tests return to green, and all will be well in the world.

Production Support Tickets

So you’ve got your skills, tooling and MLA all sorted – what’s next?

Well, possibly the biggest mindset challenge for teams is the change to treating production support issues as day to day work. In general, people see production issues as either red or green. Production is red if it’s on fire and it requires immediate treatment. Otherwise, production is green and can be ignored until someone else sorts it out. That’s really not going to fly with full-cycle.

Production support issues need to come into the team’s backlog and be prioritised and worked on exactly the same as every other piece of work. This may require a piece of custom tooling to link production support ticketing and day to day tooling. Then it can all be easily managed as the same work. It is also essential that it is dealt with on a daily basis and by the whole team.

Full-Cycle Is Intense!

Full-cycle is pretty intense – especially when combined with out of hours support. It can take a toll on a team, leading to burnout and higher turnover. By putting the right strategies and incentives in place, you can mitigate the chance of this and keep everyone happy.

Is Full-Cycle For You?

After reading all this you might be thinking, “this is not for me!”. That might very well be the case. In the same way that there are pros and cons for full-stack, there are tradeoffs that you have to consider when for full-cycle. Some devs just want to do dev. When hiring, just make sure you get people that are really bought into the idea.

Whilst it may not be for everyone, for curious individuals who like being in charge of their own destiny, full-cycle is the ultimate way to work.

References

[1] Netflix – Operate what you own.
https://netflixtechblog.com/full-cycle-developers-at-netflix-a08c31f83249