Getting Started with Kubernetes (at home) — Part 2

In the first part of the series, we looked at installing a bare bones Kubernetes cluster in some CentOS 7 VMs. In this part, we are going to look at setting up some back-end services, like a load balancer and ingress.

Some important things needed to properly run a Kubernetes cluster are a storage class or manual storage configuration via volumes, a load balancer though not strictly needed makes accessing services much easier, and an ingress.

Step-by-step documentation is here.

Setting up Helm

Helm is known as the “package manager for Kubernetes”, it is a tool that is used to template Kubernetes manifest files in a way that makes it easy to install new applications. I use Helm for all my service installation on Kubernetes due to it’s ease, and simplicity when getting started.

If you did not enable the optional Helm installation via Kubespray, we should do that first.

1. Download Helm

If you are on a Mac, you can install Helm via brew install kubernetes-helm. The Helm documentation has instructions for other methods of installation (unpacking a .zip and adding it to the $PATH).

2. Setup a Tiller user with RBAC

You can use this rbac-config.yaml file to setup the user, download it and run the following commands:

kubectl create -f rbac-config.yaml

3. Initialize Helm

Now we have to initialize Helm by deploying Tiller. The simplest method is to run helm init --service-account tiller. We have to specify the service account for tiller to use due to RBAC.

Persistent Storage

Since Docker, and by extension Kubernetes requires storage for any persistent data, it is important to provide Kubernetes some storage. If you were using a cloud provider, this would be handled for you, and you would use the providers default storage solution. In our case, we need to tell Kubernetes what storage to use.

We are going to use NFS storage for our cluster, which requires you to have an existing NFS server.

Once we have an NFS server, we are going to use the nfs-client-provisioner, which is available via Helm, to access that storage via Kubernetes storage class.

Simply run helm install stable/nfs-client-provisioner -name nfs-client -namespace kube-system -set nfs.server=10.0.40.5 -set nfs.path=/mnt/Shared/kubernetes

Let’s break that down a bit, the above command does the following:

1. Tell Helm to install the Helm Chart stable/nfs-client-provisioner
2. Give the release a name nfs-client
3. Install nfs-client to the kube-system namespace
4. Set the NFS Server IP to 10.0.40.5 you should change this as necessary
5. Set the NFS Path to /mnt/Shared/kubernetes you should change this to the share path on your NFS server

We should now check the status of nfs-client to ensure it is running correctly, via helm status nfs-client which should show ready.

Install MetalLB

MetalLB is a load balancer for Kubernetes that allows Kubernetes services to retrieve IP addresses when needed that are on the LAN or WAN. This greatly simplifies network access to services in a bare-metal (or VM) based Kubernetes cluster.

MetalLB has some requirements that we must comply with, namely, we have to have a Kubernetes cluster, a supported cluster network configuration, some IPv4 addresses, and optionally hardware that supports BGP. The first two are automatically handled by Kubespray in section 1, and for the IP addresses, you can provide a block that is outside your DHCP server’s scope on the same network as the Kubernetes nodes. I have not used the BGP option, as I do not have BGP hardware available.

You can install MetalLB via Helm with the following command helm install -name metallb -f values.yaml stable/metallb an example values.yaml file can be found here. Notice the configInline section that defines the IP addresses MetalLB will hand out.

Install Nginx-Ingress

Nginx-Ingress is an Ingress service for Kubernetes. Ingresses serve to manage external access to services in the cluster. It is important to understand how nginx-ingress works, so I recommend you read through the GitHub documentation.

Once you have done that, we can install Nginx-Ingress via Helm. You can review the values.yamlas an example, the main thing to change being the default-ssl-certificate that Nginx should use.

Once you have done that, you can install it via helm install -name nginx-ingress -f values.yaml stable/nginx-ingress -namespace nginx-ingress which does the following.

1. Tell Helm to install the Helm Chart stable/nginx-ingress
2. With the release name nginx-ingress
3. To the nginx-ingress namespace
4. With the values from values.yaml

Since the type is set to loadBalancer, MetalLB will automatically provide an external IP to the service from the provided IP addresses, which you can point your DNS or Port Forwards to in order to access your cluster’s services.

Install Cert-Manager

Cert-Manager is a complex service that allows us to automatically retrieve and renew SSL certificates from certificate providers. It works with Nginx-Ingress to ensure that all of our services are available via a secured connection over HTTPS.

There are a number of steps required to setup Cert-Manager correctly. You should start by reviewing the Helm Chart documentation here.

Prerequisites

1. Install the CustomResourceDefinition resources separately

• kubectl apply -f [https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/00-crds.yaml](https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/00-crds.yaml)

1. Create the namespace for cert-manager

• kubectl create namespace cert-manager

1. Label the cert-manager namespace to disable resource validation

• kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true

1. Add the Jetstack Helm repository

• helm repo add jetstack [https://charts.jetstack.io](https://charts.jetstack.io)

1. Update your local Helm chart repository cache

• helm repo update

Installation via Helm

1. Make any changes to values.yaml notably the podDnsConfig/nameservers
2. Deploy the Helm Chart

• helm install -name cert-manager -namespace cert-manager -f values.yaml jetstack/cert-manager

Verify the Installation

This is covered in the official documentation which you can find here. It has you create a test issuer and get a self-signed certificate to ensure everything is working properly.

Create the Production Issuer

Note : Creation of a test issuer is omitted for brevity, but please ensure you try creating a test issuer first so you do not hit the Let’s Encrypt rate limits.

If you are using Let’s Encrypt and Cloudflare, you can use this letsencrypt.yaml file. Remember to change the values as needed, especially anything marked with changeme.

Once you are ready, run kubectl apply -f letsencrypt.yaml to create the issuer.

Create Cloudflare API Key Secret

Now we need to add our Cloudflare API key into a Kubernetes secret. You can use cloudflare-secret.yaml as an example.

Once you have made your changes, create the secret with kubectl apply -f cloudflare-secret.yaml.

Create the Default Certificate

We can now create the default certificate that Nginx-Ingress will use for serving our sites. See certificate.yaml for an example.

Once you have made your changes to the file, with regard to the names and domains, you can create the certificate with kubectl apply -f certificate.yaml.

Create an Ingress

Now we can create an ingress for the Kubernetes dashboard, so instead of going to https://masternode:6443/proxy/ ect, we can go directly to https://kubernetes.domain.com

The first step is to create the dashboard ingress, which we can do with ingress.yaml once you have made your changes, you can create the ingress with kubectl apply -f ingress.yaml

You can now go to https://kubernetes.domain.com to access your Kubernetes dashboard, after you add the domain to DNS via the nginx-ingress external IP.

You now have a completely setup Kubernetes cluster running at home that you can test your services on, or use as a homelab.