Build and maintain operators using the Operator Framework toolkit

Operator Framework

The Operator Framework is a CoreOS open source toolkit for rapid development and maintenance of Operator, and contains three main projects.

• Operator SDK: Quickly build an Operator application without having to understand the complex Kubernetes API features.
• Operator Lifecycle Manager (OLM): helps install, update, and manage all operators (and their associated services) running across clusters
• Operator Metering: collects operator metric information and generates reports

With these three sub-projects, Operator Framework completes the whole process of operator generation, maintenance, and monitoring. Let’s take a look at how the Operator Framework toolkit works one by one.

Operator SDK

Operator-sdk helps us generate the operator’s framework in one click, leaving some functions for us to do the business logic.

Install operator-sdk

The operator-sdk can be installed with a single command on MacOS systems.

1

brew install operator-sdk

Linux systems can be installed via binary source files.

1
2
3

$ RELEASE_VERSION=v0.10.0
$ curl -OJL https://github.com/operator-framework/operator-sdk/releases/download/${RELEASE_VERSION}/operator-sdk-${RELEASE_VERSION}-x86_64-linux-gnu
$ chmod +x operator-sdk-${RELEASE_VERSION}-x86_64-linux-gnu && sudo mkdir -p /usr/local/bin/ && sudo cp operator-sdk-${RELEASE_VERSION}-x86_64-linux-gnu /usr/local/bin/operator-sdk && rm operator-sdk-${RELEASE_VERSION}-x86_64-linux-gnu

Initializing the project

Operator is developed in Go language, so you need to install the Go language environment before developing the project. The Operator-sdk initialization project requires only a few lines of code.

1
2
3
4
5
6

$ mkdir -p $HOME/projects/
$ cd $HOME/projects/
$ export GO111MODULE=on
$
$ operator-sdk new app-operator --repo github.com/zwwhdls/app-operator --vendor
$ cd app-operator

Generating CRDs and their controllers

Operator-sdk will mainly generate CRD files and its controller framework for you.

The format of the generate CRD command.

1
2

$ operator-sdk add api --api-version=<自定义资源的 api version> --kind <自定义资源的 kind>
$ operator-sdk add api --api-version=o0w0o.cn/v1 --kind App

Then you will find a few more files in your project, one of which is .../app-operator/pkg/apis/o0w0o/v1/app_types.go.

which creates the App’s resource definitions for you.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

type AppSpec struct {
}

type AppStatus struct {
}

type App struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   AppSpec   `json:"spec,omitempty"`
	Status AppStatus `json:"status,omitempty"`
}

All you need to do is define AppSpec and AppStatus, which describe the properties and status of the resource, respectively. Once defined, just execute the following command to automatically generate the fields you defined in CRD Yaml.

1

$ operator-sdk generate k8s

After defining the CR, we need to tell K8s what we need to do when we create this resource, i.e. we need to build the controller. operator-sdk can generate the framework for the controller with one click.

1
2
3
4
5

$ operator-sdk add controller --api-version=o0w0o.cn/v1 --kind=App
INFO[0000] Generating controller version app.o0w0o.cn/v1 for kind App.
INFO[0000] Created pkg/controller/app/app_controller.go
INFO[0000] Created pkg/controller/add_app.go
INFO[0000] Controller generation complete.

As you can see from the log, Operator creates two files for us.

One of them, add_app.go, is used to register the controller into the manager, which is equivalent to generating a K8s controller for you. And app_controller.go is where the controller is actually defined. In this file, there are two places marked TODO(User), which, as the name implies, are the only two places you need to write yourself.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

func add(mgr manager.Manager, r reconcile.Reconciler) error {
    ... 
	// TODO(user): Modify this to be the types you create that are owned by the primary resource
    ...
	return nil
}

// TODO(user): Modify this Reconcile function to implement your Controller logic.  This example creates
func (r *ReconcileApp) Reconcile(request reconcile.Request) (reconcile.Result, error) {
    ...
}

First of all, we need to know that the operator’s working mode is event-based, listening to specific events and then triggering the corresponding actions. Then it’s easy to understand that the first TODO is to define which resources to listen to; the second TODO is to define which actions to do after listening to the events of those resources.

Deployment

The Operator deployment Yaml is generated by Operator-sdk for you. After you finish the development of Operator, just replace the image in operator.yaml and deploy it by the following steps.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

// 创建 RBAC 相关
$ kubectl create -f deploy/service_account.yaml
$ kubectl create -f deploy/role.yaml
$ kubectl create -f deploy/role_binding.yaml

// 部署 operator CRD
$ kubectl apply -f deploy/crds/app_v1_app_crd.yaml

// 部署 operator
$ kubectl create -f deploy/operator.yaml

Operator Lifecycle Manager

OLM extends Kubernetes to provide a declarative way to install, manage, and upgrade Operator and the resources it depends on in a cluster. It also enforces a number of constraints on the components it manages to ensure a good user experience.

The project enables users to perform the following actions.

• Define an application as a Kubernetes resource that encapsulates requirements and metadata
• Automatically install the application using dependency resolution or manually install the application using kubectl/oc (OpenShift Client)
• Automatically upgrade applications using different approval policies

Defined CR

ClusterServiceVersion can be collected into CatalogSource, which automates the installation by resolving dependencies through InstallPlan and keeps the latest version in use through Subscription.

Defined operators

OLM defines two operators to maintain the lifecycle of custom operators, namely

• OLM operator: creates the application defined in the CSV, listens for changes to the CSV, and maintains its defined state.
• Catalog operator: parses and installs the CSV and its dependent resources; listens for changes to CatalogSources in channels and updates them to the latest available version.

Procedure for use

Generate the CSV and package yaml in your operator project.

1

operator-sdk olm-catalog gen-csv --csv-version xx.yy.zz

If you need to generate a CRD at the same time, add the following to the above command:

1

--update-crds

Test the operator locally to see if it’s legal: https://github.com/operator-framework/community-operators/blob/master/docs/testing-operators.md#manual-testing-on-kubernetes

Finally, upload your own operator’s csv and package yaml files by mentioning pr in the upstream-community-operators subfolder on GitHub.

To deploy an operator on a cluster, you only need to define the subscription file, install the OLM and apply the sub file.

operator Metering

The project generated by operator-sdk comes with a metrics interface. operator Metering essentially provides a metrics interface and then integrates the monitoring data with the data provided by the interface to generate a data report.

The project defines three main CRs.

ReportDataSources: defines what data is available ReportQueries: defines how to query the data defined in ReportDataSources Reports: Generates data reports based on ReportQueries