Over a year ago, FP Complete began work on Kube360. Kube360 is a distribution of Kubernetes supporting multiple cloud providers, as well as on-prem deployments. It includes the most requested features we've seen in Kubernetes clusters. It aims to address many pain points companies typically face in Kubernetes deployments, especially around security. And it tries to centralize the burden of keeping up with the Kubernetes treadmill to one centralized code repo that we maintain.

That's all well and good. But what led FP Complete down this path? Who is the target customer? What might you get out of this versus other alternatives? Let's dive in.

Repeatable process

In the past decade, FP Complete has set up dozens of production clusters for hosting container-based applications. We have done this for both internal and customer needs. Our first clusters predate public releases of Docker and relied on tooling like LXC. We have kept our recommendations and preferred tooling up to date as the deployment world has (very rapidly) iterated.

In the past three years, we have seen a consistent move across the industry towards Kubernetes standardization. Kubernetes addresses many of the needs of most companies around deployment. It does so in a vendor-neutral way, and mostly in a developer-friendly way.

But setting up a Kubernetes cluster in a sensible, secure, and standard fashion is far from trivial. Kubernetes is highly configurable, but out of the box provides little functionality. Different cloud vendors offer slightly different features.

As we were helping our clients onboard with Kubernetes, we noticed some recurring themes:

We decided to move ahead with creating a repeatable process for setting up Kubernetes clusters. We dogfooded this on ourselves and have been running all FP Complete services on Kube360 for about nine months now. Our process supports both initial setup as well as upgrades to the latest version of Kubernetes itself, as well as the underlying components.

With this in place, we have been able to get clients up and running far more rapidly with fully functioning, batteries-included clusters. The risk of misconfiguration or mismatch between components has come down significantly. And maintenance costs can now be amortized across multiple projects, instead of falling to each individual DevOps team.

What we included

Our initial collection of tools was based on what we were leveraging internally and what we had seen most commonly with our clients. This is what we consider a "basic batteries included" Kubernetes distribution. The functionality included:

We based our choices of defaults here on best-in-class open-source offerings. These were tools we were already familiar with, with great support across the Kubernetes ecosystem. It also makes Kube360 a less risky endeavor for our clients. We have strived to avoid the common vendor lock-in present in many offerings. With Kube360, you're getting a standard Kubernetes cluster with bells and whistles added. But you're getting it faster, more well tested, and maintained and supported by an external team.

The one curveball in this mix was the inclusion of Istio as a service mesh layer. We had already been investigating Istio for its support of in-transit encryption within the cluster, a feature we had implemented previously. Our belief is that Istio will continue to gain major adoption in the Kubernetes ecosystem, and we wanted to future proof Kube360 to be prepared for this.

Cloud native

Kube360 is designed to run mostly identically across different cloud vendors, as well as on-prem. However, where possible, we've leveraged cloud native service offerings for tighter integration. This includes:

Fully configurable

We've tried to keep the base of Kube360 mostly unopinionated. But in our consulting experience, each organization tends to have at least few modifications needed to a "standard" Kubernetes setup. The most common we experience is integration with SaaS monitoring and logging solutions.

We've designed configurability from the ground up with Kube360. Outside of a few core services, each add-on can be enabled or disabled. We can easily retarget metrics to be sent to a third party instead of intracluster collection. Even more fundamental tooling, such as the ingress controller, can be swapped out for alternatives.

Authentication

The biggest addition we've made to standard tooling comes to authentication. In our experience, the Achilles Heel of many cloud environments, and particularly Kubernetes environments, is mismanagement of authentication. We've seen many setups where credentials leverage:

The reason for this is, in our opinion, quite simple. Doing the right thing is difficult out of the box. We believe this is a major weakness that needs to be addressed. So, with Kube360, we've dedicated significant effort to providing a best-in-class authentication and authorization experience for everyone in your organization.

In short, our goal with Kube360 is to:

We strongly believe by making the right thing easy, we will foster an environment where people will more readily do the right thing. We also believe that making the cluster an open book for the entire organization, including developers, operators, and executives, we can build trust within an organization.

Surprises

Since initial development and release, we've already seen some requests for features that we had not anticipated so early on.

The first was support for deploying Windows Containers. We have previously deployed hybrid Linux/Windows clusters but had always kept the Windows workloads on traditional hosting. That was for a simple reason: our clients historically had not yet embraced Windows Containers. At this point, Kube360 fully supports hybrid Windows/Linux clusters. And we have deployed such production workloads for our clients.

On-prem was next. We've seen far more rapid adoption of "bleeding edge" technology among cloud users. However, at this point, Kubernetes is not bleeding edge. We're seeing the interest in on-prem increase drastically. We're happy to say that on-prem is now a first-class citizen in the Kube360 world, together with AWS and Azure.

The final surprise was multicluster management. Historically, we have supported clients operating on different cloud providers. But typically, deployments within a single operating group focused on a single cluster within a single provider. We're beginning to see a stronger move towards multicluster management across an organization. This fits in nicely with our vision of democratizing access to clusters. We have begun offering high level tooling for viewing status across multiple Kube360 clusters, regardless of where they are hosted.

The future

We are continuing to build up the feature set around Kube360. While our roadmap will be influenced by client requests, some of our short-term goals include:

Learn more

If you're looking to make a move to Kubernetes or are interested in seeing if you can reduce your Kubernetes maintenance costs with a move to a managed product offering, please contact us for more information. We'd love to tell you more about what Kube360 has to offer, demo the product on a live cluster, and discuss possible deployment options.

Learn more about Kube360

Do you like this blog post and need help with DevOps, Rust or functional programming? Contact us.