The CRD API is now available in Kubernetes v1.16 GA, so I’m going to talk about some features and some common issues about CRD v1; I’ll probably mention the basics of CRD briefly, but not too much. The basics can be found in the official Kubernetes documentation. You can use this article more as a quick reference manual when you encounter problems.
CRD is the most common and convenient way to extend Kubernetes. In addition to providing the basic ability to define custom resource (CR), CRD also provides many extensions, including schema, multiple versions, conversion between versions, subresource subresources, etc. This article focuses on the following four parts Contents.
- metadata: defines the basic information of CR, such as API group and name
- versions: defines the version information of the CR
- schema: defines the structure and field types of the CR.
- subresource: defines the subresource information of the CR
The most basic but most important part, metadata contains the following main elements
- The name of the custom resource CR
- The API group the CR belongs to
- scope: whether the resource is at the Namespace or Cluster level
Here is a more complete example (the comments are basically taken from the official K8s documentation, so I won’t translate them), assuming we want to define a new
group must be a domain name, otherwise an exception will be thrown when it is created.
defines the version information of CR, including which versions are available, the structure of each version, the conversion strategy between versions, etc.
There are currently two version conversion policies.
- None: The default policy is to change only
apiVersion, other fields will be ignored, data in fields that are available in the old version but not in the new version will be lost directly, fields with the same name but mismatched structures will be reported as errors.
- Webhook: you can configure a custom conversion, API Server will call to an external webhook to do the conversion
let’s look at the overall structure through an example.
Frequently asked questions about multiple versions.
Each version can be enabled/disabled via the
servedfield; one and only one version can be marked as a stored version (
storage = true)
Versions that get
falsewill throw an exception directly.
Error from server (NotFound): Unable to list "xinzhao.me/v1, Resource=volumesnapshots": the server could not find the requested resource (get volumesnapshots.xinzhao.me)
The version of the created CR will end up being the version with
storage = true, and the previous version will be converted to the new version when it is created.
If there is a previous version of the object, it will be converted to the
storage = trueversion when the object is updated.
The conversion is done through the
When you get a resource (e.g. using
kubectl get), if the specified version is different from the version of the object when it was stored, k8 will handle the conversion between multiple versions (by conversion) and return the version you specified, but this is only a display conversion, in fact the data in etcd is still the old version, and only updates will actually convert. If something goes wrong with the conversion, an exception will be thrown.
Error from server: conversion webhook for xinzhao.me/v1beta1, Kind=VolumeSnapshot failed: Post service "example-conversion-webhook-server" not found
Each version has its own separate schema structure that describes what fields the CR has and what type each field is, etc.; the schema uses the JSON Schema standard, a simple example is as follows.
For this CR we have defined only
spec and only one field
pvcName, so create a YAML file for this CR as follows.
Each field can be set to a default value, and the value of
default in yaml should be whatever type the field is. For example, I have the following three types of fields.
Fields also support validation, for example
int can specify a range of max/min values,
string supports regular expression validation and enum enumeration. See JSON Schema Validation for an example of what each type supports.
Some common problems with schema.
All fields must be defined in
schemaor an exception will be thrown.
error: error validating "snapshot-v1beta1.yaml": error validating data: ValidationError(VolumeSnapshot): unknown field "spec" in ...; if you choose to ignore these errors, turn validation off with --validate=false
If you want to store fields that are not previously defined, you can add the
x-kubernetes-preserve-unknown-fields: trueattribute to the corresponding parent field
Defined fields can be left blank, or you can configure
requiredto require certain fields to be required.
- An exception will be thrown if the
spec.pvc.sizefield is not filled in
- An exception will be thrown if the
If validation is turned off (
kubectl create --validate=false ...), undefined fields will be discarded and not stored
type does not have a map type, you can use
x-kubernetes-preserve-unknown-fieldsto act as a map type (any field is allowed)
Two types of subresource are supported, a Status subresource and a Scale subresource. In the example below, both status and scale subresource are enabled.
With status subresource enabled, the entire CR content will be divided into
status parts, representing the expected status of the resource and the actual status of the resource, respectively. The resource’s API will then expose a new
/status interface, and this CR creation/update interface will verify the entire content (including the
status is ignored in its entirety, and
status can only be updated via the
/status interface. The
/status interface will only update/check the contents of
status, everything else will be ignored.
For client-go, the
UpdateStatus method will be generated whenever
Status is defined by type.
Similar to status subresource, a new
/scale interface will be exposed when enabled. But I’ve never used this, so I won’t go into it too much. You can see from the official documentation that the scale subresource, when enabled, also allows you to control the number of copies of a resource individually using