Take Our New ‘Simple Sidecar’ Solution for a Spin

The team is very pleased to announce a shiny, new open-source tool, Simple Sidecar. Once implemented, you’ll be able to efficiently extend tooling when injecting sidecar containers into Kubernetes pods.

Below, we’ll cover some sidecar and Kubernetes basics and walk you through the processes of implementing Simple Sidecar. This technique will help to reduce a lot of the friction typically generated by sidecar extensions.

Reducing the Drag of Sidecar Extensions

Sidecar extensions can be a slog. Sometimes, we want to extend the functionality of helm-packaged open-source tools using a sidecar container. And the open nature of these tools means that anything is possible. However, it’s rarely simple. 

With Simple Sidecar, you can easily extend existing helm-packaged tools, requiring almost zero changes to your helm deployment. This significantly reduces the drag that sidecar extensions can have.

Since CentML’s inception in 2022, we’ve remained committed to open sourcing our ML tools and implementation projects. That ethos is a core facet of the company and it’s why the CentML Platform encompasses not only enterprise software like CServe, but also open-sourced (and enterprise-grade) projects like Deepview and Hidet.

In keeping with that mission, we’re rolling out our Simple Sidecar tool for everyone. Let’s roll ⤵

The Basics of Kubernetes & Simple Sidecar Containers

Before we jump into using Simple Sidecar, let’s get a lay of the land with a quick primer on Kubernetes and sidecars. We’ll discuss how they simplify infrastructure ops in large distributed systems, like those used to deploy ML models (our favourite!).

Back to Kubernetes Basics

• What is Kubernetes: Kubernetes is an open-source platform designed to automate the deployment, scaling, and management of containerized applications. In modern infrastructure, it plays a crucial role by providing a robust system for running applications across clusters of machines, ensuring high availability, scalability, and efficient resource utilization.
• Tools: The open-source ecosystem for Kubernetes is huge and forever evolving, with a wide array of tools and projects designed to enhance and extend its capabilities. This ecosystem includes observability tools (like Prometheus and Grafana), CI/CD pipelines (like Argo and Tekton), service meshes (like Istio and Linkerd), security (like Falco and OPA), and storage solutions (like Rook and Ceph).
• Sidecar Containers: These containers add auxiliary functions like logging, monitoring, or traffic management to the main application without code changes. Sharing the same pod, they allow for modular and scalable features like service mesh and secure communication.

Sidecars: Looking Under the Hood

Simply put, sidecars are a powerful way to add new capabilities to applications deployed in Kubernetes. By using them, you can leverage a slew of benefits like improved modularity and maintenance. In turn, these allow for efficient resource usage, improved monitoring, and heightened security.

Here are 10 sidecar basics to keep in mind:

1. Modular Architecture: Sidecars make it possible for us to decouple auxiliary (“side”) functions from the core application’s purpose. We can focus on developing and optimizing the core ML model while simultaneously handling side quests (monitoring, logging, and security) in separate, specialized containers.
2. Language and Runtime Flexibility: Because sidecars can be developed in different programming languages, they can run in separate environments from the core ML application. This means we can use the best tools for each task outside of the constraints of the core framework.
3. Improved Visibility: With sidecars we can track metrics, logs, and traces from the ML model without modifying the core code. This makes it easier to track performance and troubleshoot issues.
4. Simplified Updates and Maintenance: Because sidecars can be updated independently of the main ML application, more frequent updates can be made to auxiliary functions like security patches without disrupting the core ML model’s operation.
5. Resource Optimization: By offloading tasks like feature engineering to sidecars, we can optimize resource allocation for the main ML application. This allows the primary container to focus solely on model inference.
6. Better Security: Encryption, authentication, and authorization can all be implemented in a sidecar without complicating the core code.
7. Easier Integration of MLOps Practices: Sidecars can facilitate the implementation of MLOps practices like A/B testing, canary deployments, and automated model updates. As a result, more agile and robust ML model deployment and management are made possible.
8. Enhanced Scalability: Sidecars can help manage scaling and load balancing for ML models, allowing for more efficient resource utilization as demand fluctuates.
9. Standardization Across Models: Organizations can create standardized sidecar configurations for common tasks like logging, monitoring, and security. This standardization promotes consistency across different ML projects and simplifies management.

Getting Into the Gears: Using Simple Sidecar

We rolled out Simple Sidecar to help our team simplify the all-too-common complexities of ML infrastructure management. Given the nature of our work — developing a platform to run immense ML projects — you can imagine the challenges we’re tackling. We quickly realized that if Simple Sidecar could help our own team, it could be a game-changer for others operating complex infrastructure.

Whether you’re running applications in the cloud or on-premises, Simple Sidecar makes management a whole lot easier. As well, you don’t need to fork or roll your own helm chart just to get a new sidecar added. Your sidecar configuration is defined using official specifications and yaml manifests for precision.

All-in-all, we think Simple Sidecar will help to significantly ease some of your infrastructure management burden, just as it has for us! Here’s how to get started:

Step 1: Install the Simple Sidecar Using Helm

In this first step, we’re going to create a values.yaml for the Simple Sidecar helm installation, containing the sidecar configuration (yaml) you want to ingest. 

🧑‍💻 Note: The configuration for init containers, sidecar containers, and adding to existing container configuration are all supported. Also, it's good to keep in mind that the same Go code that Kubernetes uses to parse YAML workload manifests is used by Simple Sidecar, so the configuration is exactly 1-1.

simpleSidecarConfig:
  ubuntu:
    containers:
    - args:
      - -c
      - sleep infinity
      command:
      - /bin/sh
      image: ubuntu
      name: ubuntu

Next, install the helm chart:

helm repo add simple-sidecar
helm repo update
helm install simple-sidecar simple-sidecar/simple-sidecar -f values.yaml

Step 2: Inject the Ubuntu Container

Next, let’s inject the ubuntu container we configured above into another container. 

First, the sidecar-injection label is set to enabled for the namespaces in which we want simple-sidecar to operate:

kubectl apply -f - << EOF
apiVersion: v1
kind: Namespace
metadata:
  labels:
    simple-sidecar.centml.ai/injection: enabled
  name: injectable
EOF

Step 3: Create a Pod

Now, let’s create a pod with the simple-sidecar.centml.ai/inject annotation pointing to the ubuntu pod configuration by name.

kubectl apply -f - << EOF
apiVersion: v1
kind: Pod
metadata:
 name: my-pod
 namespace: injectable
 annotations:
    simple-sidecar.centml.ai/inject: "ubuntu"
spec:
 containers:
  - name: curl-container
    image: curlimages/curl
    command: ["/bin/sleep"]
    args: ["infinity"]
EOF

Whenever the pod starts/restarts, a ubuntu pod will be injected in your curl pod:

kubectl get pod my-pod -n injectable -o jsonpath='{.spec.containers}' | jq

[
   {
      "args":[
         "infinity"
      ],
      "command":[
         "/bin/sleep"
      ],
      "image":"curlimages/curl",
      "imagePullPolicy":"Always",
      "name":"curl-container",
    …
    {
      "args":[
         "-c",
         "sleep infinity"
      ],
      "command":[
         "/bin/sh"
      ],
      "image":"ubuntu",
      "imagePullPolicy":"Always",
      "name":"ubuntu",
    …
    }
]

Trading in Friction for Speed

By following these steps, you’ll be able to quickly extend your tooling without the typical frictions that sidecars introduce, helping you make certain that your overall deployment remains efficient, modular, and secure. Enhancing your Kubernetes workflows in this way, you’ll be better able to leverage all of your sidecar containers to achieve advanced functionality. 

Take it for a  spin and see how it can simplify your operations. As always, please keep us posted and give us a heads up about the projects you use it for!

Here’s to accessible AI 🦾🚀

---

Ready to Supercharge Your ML and AI Deployments? Learn how CentML can optimize your models, here.