Part 1: Deploying vSphere with Kubernetes - Prerequisites

vSphere 7 with Kubernetes is finally here and I couldn’t be more excited for it to be available. This blog series will cover the requirements, prequisites and deployment steps in order to deploy a vSphere with Kubernetes (vk8s) environment using vSphere 7 and NSX-T 3.0.

How to Get vSphere with Kubernetes

The initial question, is how can I get licenses for vk8s? VMware has posted a blog that covers this, but the TLDR is you need VCF 4.0. You can learn more at Licensing for vSphere with Kubernetes

An alternative method in which I will use as VCF 4.0 has not been released is you can utilize the evaluation licenses built into vSphere and use those for the Workload Management aspects. All you will need is an NSX-T 3.0 license, and if you don’t have one available the good news its coming to VMUG Advantage!

VMUG Advantage provides access to @VMware's EVALExperience – an avenue to explore cutting-edge #VMware solutions.

You heard it here: #vSphere7, #vSan7 & #NSX-T (COMING SOON!) are now included in #EVALExperience! Follow @VMUGAdv & upgrade your career now: https://t.co/tfGFyvLUyk pic.twitter.com/tnvJ6egVps

— VMware User Group (@MyVMUG) April 9, 2020

If you do not currently have a VMUG Advantage subscription, William Lam has worked on a group buy with VMUG so take a look!

😀 I’m super excited to be partnering up with @MyVMUG to bring to the @VMware Community Special @VMUGAdv #Homelab Group Buy!

This will be great way to get access to latest #vSphere7, vSAN 7, NSX-T 3.0 & upcoming VCF 4.0 which includes vSphere w/Kuberentes!

Please RT & share! https://t.co/isqwGdqWCu

— William Lam (@lamw.bsky.social | @[email protected]) (@lamw) April 10, 2020

Prerequisites

When planning to deploy your vk8s environment, it is important that you understand the prerequisites. These are not only from a software and hardware level, but networking is the most important thing to pay attention to.

VMware covers the minimum requirements and it goes into great detail. Hint: You need lots of CPU and memory!

Compute

In my deployment, I am using nested but the same allocations would apply to a physical deployment. I have my vk8s VC and ESXi hosts deployed to a management VC and my edges will be deployed to my vk8s VC.

I have 8 vCPU’s and 96GB assigned to my hosts. As i am not using VSAN I can get away with 2 hosts, however if using VSAN you must use a minimum of 4 hosts.

Storage

Throughout the beta and with my testing 1-2 TB is a good baseline depending on what you deploy, in this example ive provisioned a 500GB volume for my vk8s environment.

Networking

Throughout the beta and even internally back at VMware the biggest reason an environment didnt function properly was not understanding the networking prerequisites. You WILL need a minimum of 3 separate routable VLANs. This will be for the management, ESXi TEP and Edge TEP interfaces. You will also need PLENTY of IPs.

• From your management network you will need an IP for VC, ESXi Hosts, NSX Manager, NSX Edge and 5 consequitive IPs for the Kubernetes Control Plane VMs.
• You will need a range or CIDR for each of your TEP VLANS. In my example i use a range from .10 - .50 for each.
• You will need from your overlay network (at least a /24) an IP for an uplink on each edge, and a minimum of 2 /27 block carved up from it. This will be used for the ingress and egress CIDR’s. These MUST be in the same network as your edge uplinks

vCenter Server

As of now you cannot use vSphere with Kubernetes alongside some new features in vSphere 7 such as vSphere Lifecycle Manager. If you plan to use vk8s make sure you are still using baselines and not the new image management.

Wrap Up

This post is an introduction of what you need in order to deploy vSphere with Kubernetes. Stay tuned for the next post in the series where we will talk about Configuring your vCenter Server for vk8s.