Kubernetes Custom Resource Definition

This scenario takes you through the essentials of defining new Kubernetes Custom Resource Definitions (CRDs). You will extend Kubernetes by creating new resources beyond the standard ones you normally get with any Kubernetes cluster. Once you understand CRDs, you will learn how to further extend Kubernetes using the powerful Operator pattern.

In this scenario you will learn how to:

• Create a CRD and define custom attributes
• Discover how your CRD becomes a new resource
• Add, inspect, and remove instances of a custom resource

Kubernetes Extensibility: Custom Resource Definitions

a. verify your Kubernetes cluster is up and running for this LAB and the Helm package manager is installed

b. Kubernetes Dashboard :You can administer your cluster with the kubectl CLI tool or use the visual Kubernetes Dashboard. Use this script to access the protected Dashboard: token.sh

Create Simple CRD YAML

Let’s begin by inspecting a small example YAML file that declares a simple Thermometer: ccat thermometer-crd-minimal.yaml

In this CRD definition, the Kind is CustomResourceDefinition and the CRD is scoped to namespaces. We also provide the plural and short alias names for the same resource. Later we will define features that we can add to this specification. In its current form, it’s the most simplistic.

In this scenario we are using Kubernetes v1.18, so in the CRD definition we use apiVersion: apiextensions.k8s.io/v1. If you are using Kubernetes v1.16 or newer, you can use v1, which has a slightly improved CRD format.

Before we give this declaration to your cluster, let’s see what is currently on the cluster. There should be no preexisting CRDs: kubectl get crds

Now, submit the Thermometer CRD to Kubernetes:kubectl apply -f thermometer-crd-minimal.yaml

Notice the apply command was used instead of the create because we will be applying additional upgrades to the CRD in the subsequent steps. Kubernetes is now aware of this new resource type: kubectl get crds

Thermometer resources are now listed as a cluster api-resource:

kubectl api-resources --api-group='d2iq.com' -o wide

The resource REST management is also added as /apis/d2iq/ to the Kubernetes API:

kubectl get --raw / | jq . | grep -C3 d2iq

The REST API URL reveals the Thermometer resource:
kubectl get -v=9 --raw /apis/d2iq.com/v1beta1/thermometers/ | jq

Notice that in the last command, we added a verbosity request -v=9. With it set to level 9 (the highest) we get a bit more insight into how the kubectl command is obtaining the resource information.

The command also pipes the output to jq to pretty-print the JSON with syntax highlights.
The CRD definition just defines the resource type and while Kubernetes recognizes the type, there are no instances of the Thermometer resource type:

kubectl get trms

In the next step, let’s add a Thermometer based on this type.

***********************************************************************

Apply Simple Resource

Now that Kubernetes knows about this resource type, let’s add an actual resource. Create a YAML file to declare a Thermometer:

cat <<EOF > thermometer.yaml apiVersion: d2iq.com/v1beta1 kind: Thermometer metadata: name: simple-thermometer namespace: default EOF

The YAML file has been created. There is no data in it, just a name and namespace:

ccat thermometer.yaml

The CRD in the previous step instructed Kubernetes on what the resource kind: Thermometer means. Because of the CRD, this Thermometer declaration can be applied to your Kubernetes cluster:

kubectl apply -f thermometer.yaml

Now you have a new Thermometer resource. This is not a new Pod, Container, or Service; instead it is just a representation of a Thermometer:

kubectl get trm -A

The -A switch means find the specified resource across All namespaces.

With this, you see how to define and create custom resources. However, you can see this Thermometer type is lacking details that make a Thermometer unique, such as a geographic location, a digital metric, and its preferred units. We will expand on this in the next steps but before continuing, let’s remove this current definition.

Remove the resource:

kubectl delete trm -n default --all

Now, the resource is gone:

kubectl get trm -A

• ************************************************************************

Apply Resource with Spec

In your current directory, there is a YAML file for a Thermometer for Stockholm Sweden:

ccat stockholm-thermometer.yaml

Notice the YAML has the name and namespace (as the location) and units defined. Create the namespace:

kubectl create namespace sweden

Apply the new resource declaration:

kubectl apply -f stockholm-thermometer.yaml --validate=false

Get the resource information:

kubectl get trm -A

This is as expected and consistent with the previous example. However, this Thermometer YAML has extra data declared:

unit: Celsius
    example: 'not printed'

We had to pass in the --validate=false option just to get the controller to accept a YAML with the unknown data in it. The CRD controller will aggressively prune any unknown data that is not defined in the schema. Even though the YAML was not validated, the controller still pruned the extra data. This pruning behavior is welcomed because we want to prevent unknown data to lessen accidental or nefarious data getting into our cluster. We'll get into better schema validation in a moment. You can verify the unit and example data was pruned by inspecting the resource:

kubectl get trm stockholm -n sweden -o json | jq

Wouldn’t it be nice if the displayed output from the get command included the additional information defined in the resource details? In the next step, you can define the extra columns then improve the schema.