The term autonomic computing is nothing new. Ever since IBM used this as a marketing term in 2001, this term has been on the periphery of most people focussed on the future IT trends. Clearly, the term has been suffering from overzealous IBM marketing for a long time and, every time I bring this up in a discussion, people always point to this failed marketing effort. In spite of this pushback, I am a strong believer in autonomic computing and believe that we are heading towards that 2001 vision of IBM with more realistic technologies.
In this blog post, I am going to talk about certain trends we consider as the drivers for the eventual evolution of autonomic computing. Let us be clear here. We are nowhere close to this goal but, as someone who sees patterns and extrapolates it to trends, I do see trends leading towards this future. And, this future will arrive sooner than what many in the industry expect. Keep in mind, none of the vendors are going to accept that their technologies will eventually lead to autonomic computing. Since autonomic systems will lead to dramatic reduction in IT operations, accepting its emergence will add resistance to the vendor sales process. The IT vendors and their employees are incentivized to keep autonomic computing very far away in the future.
Trends shaping this evolution
Public clouds changed the way users thought about infrastructure. The success of public clouds brought mindset changes among the end users leading to a change in behavior when it comes to infrastructure services consumption and automation
- The self-service nature of public cloud pushed users to treat infrastructure as cattle than pets, pushing a code-centric approach to infrastructure operations (programmatic approach to operations). Whether it is early day infrastructure as code trends or GitOps pushed by Weaveworks or Infrastructure as developer friendly code pushed by Pulumi, we are seeing the emergence of a completely new way of dealing with infrastructure
- The responsibility for running compute services moved from data centers to hyperscale cloud providers, resulting in large-scale automation to squeeze cost and resource efficiencies. Now higher levels of automation are fast becoming the path for IT to create operational efficiencies and operate at scale
The large-scale automation is the first step in the (auto) management of any distributed system. The availability of programmable interfaces for managing infrastructure is another important criterion for autonomic systems
The declarative approach to Operations
Another trend that makes automation more efficient and, in the future, will make AI driven operations more seamless, is the declarative approach to infrastructure operations. It gained traction as the infrastructure as a code gained traction (with a big push from Puppet) but the success of Kubernetes has made the declarative approach the default way doing modern IT operations. Since the declarative approach allows for the specification of the preferred end state, we expect this model-driven approach to be the underlying fabric of autonomic systems (at least, in the early versions).
The other trend that is shaping up, and which will be the critical pillar for autonomic computing, is AIOps, AI-driven IT operations. Even though it is way too early for AIOps, we are seeing the application of machine learning and deep learning in areas like observability, security, governance, etc.. We expect AIOps to become mainstream in the next few years. Companies like Splunk, CloudFabrix, CoreStack, Instana, Moogsoft, Insight Engines, etc. are already offering products/services under this category.
The declarative approach to CI/CD pipeline
The modern application infrastructure encompasses the delivery pipeline along with the deployment platform. With the declarative approach to application deployment becoming the norm with Cloud Native Platforms, it is important to extend this approach to the application delivery pipeline. A truly autonomic system should take over the moment a developer checks in the code. We believe that there are plenty of opportunities to inject AI and automation in the CI/CD pipeline. Starting with the analytics layer on the top of the pipeline to handling the automation triggered by this analytics layer, AI will be playing an important role. The declarative approach to the delivery pipeline is a critical first step in this evolution. With tools like Concourse-ci from Pivotal using a declarative approach, expect to see a new generation of tools working closely with the cloud-native platforms.
How far away are we from the autonomic computing future?
As it was pointed out earlier, we are nowhere close to even building prototypes of autonomic systems. We are still in the academic phase with some advances in the areas mentioned above. There are many factors that should come together for this to happen.
- AI itself is in early stages. Even though there is a widespread adoption of machine learning and deep learning across many verticals, we still have to wait for the maturation of these technologies
- Data is still a weak spot for many of the vendors who are trying to implement machine learning/deep learning tools
- I would expect the autonomic system to be better applicable for cloud-native architectures. With enterprises, the proliferation of cloud-native architectures will take time. Till they completely modernize their infrastructure and application architectures, there will be resistance to autonomic computing
Who will lead this evolution?
Even though the current push to AIOps is coming from vendors in the IT operations space with a niche focus, we expect the hyperscale cloud providers to take the lead in delivering autonomic systems. Don’t be surprised if Google comes out and open sources the software and/or the necessary training data that could lead the way towards this trend. Vendors like Microsoft and AWS or Chinese hyperscale providers like Alibaba can lead the way. We cannot dismiss IBM and with their R&D investments, they can take a leadership role in this evolution (hopefully, with less marketing hype than last time). It is too early to say who will lead the innovation towards autonomic computing but we can confidently predict that it will arrive much sooner than what many pundits expect or the timeline vendors will like to accept (to avoid any resistance/backlash due to its potential threat to IT operations jobs). What do you think?
Update: John Willis, DevOps Guru, points out that it was Mark Burgess, founder of CF Engine, who coined the term in 1998 in this paper.