Kubernetes is fast evolving as the standard orchestration platform for cloud native deployments. However, Kubernetes is still not mature enough to support stateful applications and distributed applications like those running on IoT and Edge. Clearly, developers are looking beyond Kubernetes for their needs. While Kubernetes can help to orchestrate an application, a complementary solution is necessary to run distributed application services. Developers need a more modern approach to distributed applications that can complement Kubernetes, like the open source SwimOS platform. In this blog post, we will introduce the SwimOS platform to Kubernetes developers and help them get started with deploying distributed applications using real-time streaming data.
SwimOS is built from ground up to support streaming applications without the overhead of managing app servers, databases and message brokers. SwimOS takes the operational headache out of running distributed applications by offering a vertically-integrated backend stack, so developers can focus solely on writing application code. SwimOS goes beyond the stateless REST API to support a streaming protocol called WARP, which enables bidirectional multiplexing over a single websocket. This enables streaming peer-to-peer communication between application services, so that stateful distributed applications can maintain consistency while taking advantage of multiple data sources.
The SwimOS platform provides its own persistence along with messaging, scheduling, clustering, replication, introspection, and security. SwimOS can be deployed across cloud providers, On-Premises, at the edge and on IoT devices. The versatility of the SwimOS platform allows for building the data fabric necessary to deploy distributed stateful applications. As more and more enterprises embrace Kubernetes and build real time streaming applications using both streaming data and batch data sources, they need a data fabric to support their applications.
As developers use Kubernetes to deploy their cloud-native applications, they can deploy SwimOS applications inside their Kubernetes environments. Deploying SwimOS on Kubernetes is as easy as deploying a Java application. SwimOS can be deployed along with application code as a small war file into a Kubernetes pod. This allows developers to deploy streaming applications on SwimOS platform in hybrid cloud, edge, IoT and other environments that support Kubernetes.
Why SwimOS matters for Kubernetes developers?
Kubernetes makes it easy to run stateless applications and certain stateful applications like web applications. The Kubernetes community is working hard to make StatefulSets more useful to handle persistent storage volumes. While this may be useful to deploy legacy enterprise applications, it falls short in supporting modern distributed applications like real time streaming applications. Developers can put together data fabric needed to support such applications using open source projects like Apache Cassandra for real-time data processing, Apache Kafka for message broker and Apache Flink for stateful data processing. But this DIY approach adds tremendous operational overhead and falls short of handling certain use cases such as enabling insights across heterogeneous data sources.
The requirements for modern distributed applications goes beyond packaging stateful services in stateless containers. It requires a stateful data fabric that weaves a network of stateful streaming links traversing multiple stateful services. While Kubernetes is well suited for running serverless applications and stateful applications that can use a persistent storage volume which can be invoked using a RESTful API, it is ill equipped to handle stateful applications running on dynamic data. This is where the SwimOS platform can complement existing Kubernetes infrastructure. By running SwimOS as a part of Kubernetes environment, there are some distinct advantages:
- Simplified operations with Kubernetes managing SwimOS platform along with the applications
- Stateless applications and traditional cloud applications can now co- exist with streaming applications, breaking down the silos
- Streamlining of DevOps pipeline, thereby, reducing the friction
- Easily take advantage of batch data residing in Kubernetes environments to recompose into a single streaming API, making it easy for developers to access all the data in the environment. This dramatically reduces the complexity in the application architecture, improving developer productivity
Deploying SwimOS on Kubernetes
Deploying SwimOS on Kubernetes is as simple as deploying a Java application in a Docker container. Since Swim applications have no external dependencies, the dockerfile for a Swim application is quite trivial.
Here is a dockerfile that can be used for any Swim application (You may refer to the repo here on how to generate a tar/zip file for a Swim application) :
RUN /bin/bash -c 'apt-get update -y'
RUN /bin/bash -c 'apt-get install -y software-properties-common'
RUN /bin/bash -c 'apt-get install wget unzip -y'
RUN /bin/bash -c 'apt-get install apt-utils -y'
RUN /bin/bash -c 'apt-get install apt-transport-https ca-certificates -y'
# Install java
RUN /bin/bash -c 'add-apt-repository ppa:openjdk-r/ppa'
RUN /bin/bash -c 'apt update -y'
RUN /bin/bash -c 'apt install -y openjdk-11-jdk'
# Open port on which the Swim application is running
# Copy and untar your Swim app from a tar file
COPY build/distributions/your-Swim-app-3.9.4.tar /app/
RUN /bin/bash -c 'tar -xvf your-Swim-app-3.9.4.tar'
#Run the Swim application
Once the Docker image is built, it can be pushed to the container registry of your choice. From there, you can use kubectl to deploy the app in an existing cluster or create a new cluster and deploy the application.
Cloud-native applications are gaining traction fast and Kubernetes is becoming the standard for deploying these applications. But, Kubernetes is limited in its support for real-time streaming applications and applications running on the Edge/IoT. With the open source SwimOS platform, developers can easily build real-time streaming applications and manage it within their Kubernetes environment. This allows organizations to standardize the application infrastructure while giving developers more flexibility to build innovative applications.
Disclosure: Swim.ai is a client of Rishidot Research