The Reports of Devops's death are greatly exaggerated
| The Reports of Devops’s death are greatly exaggerated - Not said by Mark Twain
“Devops is Dead”, is a claim I have heard in hushed conversations and several blog posts over the last couple of years, and it’s turning quickly into more than whispers. However, like most extreme claims this too is grossly exaggerated. There has been a change to the status quo, but it’s rather a transformation than a termination. For good reasons, devops has evolved into a different discipline than before.
The term “devops” itself has a lot of meanings. Devops can refer to either a role or a process for developing and releasing software. Here we are talking about the role which used to specialise in operating systems, monitoring and CI/CD tools with an 80:20 division of work. 80 percent of infra work was done by devops specialists while 20 percent of work(mostly tinkering with config) was handled by developers. For instance, at Myntra way back in 2015, we used to provision VMs in our “cloud” data center by triggering a script manually with the required configuration options. Now in a compute environment like Google Kubernetes Engine, provisioning new VMs happens automatically behind the scenes. A big chunk of that 80 percent work has moved to Cloud Services Providers(AWS, GCP, Azure, Github, Gitlab etc.).
You know what tech roles do ? They evolve.
Every tech role is in a state of constant flux. This is fairly obvious. Although things get muddled when you have organisations latching on to job titles to make hiring easier. The Devops role too has evolved to keep up with a changing infrastructure landscape.
A few observations related to this evolution:
- The tech stack is just a bunch of yamls.
- Deep operating system and networking knowledge is no longer at premium: Docker and k8s ate that lunch.
- Deep knowledge of the inner workings of a particular Cloud Provider(AWS Specialist, GCP Specialist) is still at premium since this can lead to better design and cheaper costs. But the cloud providers know they have a complex ecosystem and are constantly releasing better documentation, better UI controls etc. Think about this: why have a few niche customers(infra specialists), when every software developer is a potential customer.
- Configuring infrastructure is now like writing mini-apps for the Cloud API(aws, gcp, azure) using terraform, deployment manager, cloud formation etc.
- Configuring observability infrastructure is also done over an API. See above. Cloud providers have made available open source SDKs to connect to the monitoring stack. It has become a lot easier for developers to bring in observability than before.
- Permissions management is also an API concern. Although there is still some devops work required to make it work across multiple services( think GCP + Github + Gitlab).
- Designing and building CI/CD pipelines have moved in the hands of developers: Github Actions, Gitlab, CircleCI, https://tekton.dev/ etc. have made writing pipelines less complicated(as compared to Jenkins) and now its just yamls down to the turtles.
- Infrastructure performance is now a cost concern. One can always scale.
- Fully integrated solutions like heroku, netlify, vercel, digitalocean apps are removing the low to medium scale website market from the equation.
- Devops as a process of delivering software has always been an organisational concern mostly led by the devops specialists. But since their specialised skills are no longer required, the process drivers can come from any discipline.
- Devops as teachers of deep infrastructure skills for developers still remains. But if the specialised skills are not being practiced as much, how long can they keep that crown ?
- Devops as budget keepers are being replaced with cost analysis tools which notify app owners to optimise their usage.
- Devops still practice designing infra architecture. At some organisations, this is getting outsourced to cloud services support teams and consultancies.
- The request-response cycle is now a pull request-merge cycle due to the adoption of gitops. The devops team creates the self-service tools and gets out of the way. The devops engineer is now yet another approver.
The devops role has certainly transformed into someone who creates and tends to configurations. Since the complexity has moved to managing increasingly complex configurations, we are seeing the rise of a different kind of discipline and the tech which goes with it. A few examples from last few years:
The common theme in the above is usage of full-fledged programming languages(typescript, python, golang etc.) to define and manage infrastructure. Not just that, we are also seeing the rise of new smaller data templating languages especially created for configurations:
You cannot configure what you do not understand
Although high level constructs are now available to treat infrastructure resources like yet another software object, the abstraction is not complete. For e.g. one still needs to understand when to use k8s deployment vs k8s statefulset, to configure CPU resource limits one still needs to have an understanding of the runtime, to create VPCs, one still needs to know about subnetworks and CIDR blocks. AWS CDK aims to solve this problem by introducing constructs with sane defaults. We have done something similar at Taxfix for Kubernetes and Monitoring(watch out this space for a post on our custom deployment SDK ! ) The knowledge of low level concepts will still be required at some level, but the low level complexity will keep moving towards abstractions built by the behemoths(k8s, container runtimes, GCP, AWS, Azure.). I wonder how long before we start talking about infrastructure solely in terms of software objects, and the languages and tools which can help manage them. IMO we are getting pretty close.
The Devops Role is Dead
Ok maybe devops as a traditional centralised role is dead or at least on its deathbed. For many organisations this is not true, yet. It’s transforming into a discipline where cloud-infra focused engineers work in tandem with application focused engineers to deliver software quickly and efficiently. These “cloud-infra” focused engineers practice classical software principles and full-fledged languages. With respect to their traits, such engineers are closer to application platform engineers than to a traditional devops engineer.
On the other hand, devops as a process is alive and kicking while also evolving. The adoption of Site Reliability Engineering even in medium-sized companies is inline with the aforesaid premise of seeing infra problems as software problems. SRE as a specific implementation of the principles of devops, is pushing infrastructure engineering closer to application engineering.
Following are the activities I have seen while practicing a similar role at Taxfix:
- Use terraform, terraform cdk, pulumi like tools to set up compute, network and datastore infra.
- Generate kubernetes manifests using pulumi, jsonnet, jkcfg, AWS CDK or similar tools.
- Generate monitoring configuration for a Cloud Provider or third-parties like grafana, prometheus etc using SDKs.
- Write or generate CI/CD manifests. Include build steps for code quality, security etc.
- Develop a gitops tool to distribute permissions and secrets.
- Develop a gitops tools to add/remove frequently provisioned resources, e.g. Buckets, Pubsub Queues.
- Develop a gitops tool to automate oncall scheduling and paging.
- Develop platform tools to standardise platform task, e.g: webhook tunnel, notification schedule, API gateways etc.
- Integrate or develop tools to ensure continuous delivery using feature flags.
- Write maintainable, testable code while using established good practices.
A bright new day, A new shiny new title
How about a new title for the devops role then ? Just flinging out some respectable-sounding titles here:
- Infrastructure Tools/Apps Developer/Engineer
- Cloud Native Tools/Apps Developer/Engineer
- Configuration Ops/ConfigOps
- Developer Tools/Apps Engineer
- Cloud Native Specialist
- Cloud Config Ops
- PlumbOps
- Application Platform Engineer(Infrastructure)
I am quite comfortable with the title, “Devops” too, it’s got a nice ring to it. Of course what it means is constantly evolving.
Don’t write an obituary just yet
Far from being dead, devops has grown into an exciting discipline. A few things to highlight before we end:
- Infrastructure is increasingly represented by software objects.
- You cannot configure what you do not understand. Low-level specialists are still needed to design systems well.
- Devops engineers are now software engineers with infra chops. Maybe we need new titles to go with the evolved responsibilities of the devops role.
As devops practitioners, we are no longer just “knob-fiddlers”, now we program the knob to self-fiddle.