Kubernetes - operator first experience

Posted by lilwing on Thu, 03 Mar 2022 03:53:16 +0100

This chapter refers to the of operator SDK tutorial Write a memcache operator

1 function introduction

After the memcache operator is created successfully, you can create a memcache service through the following yaml

apiVersion: cache.example.com/v1alpha1
kind: Memcached
metadata:
  name: memcached-sample
spec:
  size: 3

Create memcached based on yaml

kubectl create -f memcached.yaml

After the creation is successful, you can view the status information of memcached through kubectl

# Check whether the corresponding resource information exists
tingshuai.yts@B-5BBCMD6M-2026 ~ % kubectl get memcached
NAME               AGE
memcached-sample   34m

# View memcached details
tingshuai.yts@B-5BBCMD6M-2026 ~ % kubectl get memcached memcached-sample -o yaml
apiVersion: cache.example.com/v1alpha1
kind: Memcached
metadata:
  creationTimestamp: "2022-02-24T07:16:33Z"
  generation: 2
  name: memcached-sample
  namespace: default
  resourceVersion: "3385066472"
  selfLink: /apis/cache.example.com/v1alpha1/namespaces/default/memcacheds/memcached-sample
  uid: 12f26a9f-fc34-492e-a30f-bc17266052aa
spec:
  size: 4
status:
  nodes:
  - memcached-sample-9b765dfc8-w6l26
  - memcached-sample-9b765dfc8-7cwpl
  - memcached-sample-9b765dfc8-kxnmd
  - memcached-sample-9b765dfc8-hsrqd

The main work of memcached operator is to create a deployment according to the spec described in yaml, as shown below:

tingshuai.yts@B-5BBCMD6M-2026 ~ % kubectl get deploy
NAME               READY   UP-TO-DATE   AVAILABLE   AGE
memcached-sample   4/4     4            4           37m

2 overall process

Writing an operator is generally divided into the following steps:

  • Construction of development environment
  • Create an operator project through the operator sdk
  • Create API and controller framework code through operator sdk
  • Write API and controller business code
  • Debugging execution

3 environment construction

The development of controller depends on the following environment. Due to the limited space, this paper only lists the dependent contents, and the installation method is Baidu

  • golang
  • operator-sdk
  • kubectl and k8s corresponding config

4. Create operator project

Create the controller's project through the operator SDK init command

mkdir -p $HOME/projects/memcached-operator
cd $HOME/projects/memcached-operator

# we'll use a domain of example.com
# so all API groups will be <group>.example.com
operator-sdk init --domain example.com --repo github.com/example/memcached-operator

The help document information of operator SDK init is as follows:

% operator-sdk init --help
Initialize a new project including the following files:
  - a "go.mod" with project dependencies
  - a "PROJECT" file that stores project configuration
  - a "Makefile" with several useful make targets for the project
  - several YAML files for project deployment under the "config" directory
  - a "main.go" file that creates the manager that will run the project controllers

Usage:
  operator-sdk init [flags]

Examples:
  # Initialize a new project with your domain and name in copyright
  operator-sdk init --plugins go/v3 --domain example.org --owner "Your name"

  # Initialize a new project defining an specific project version
  operator-sdk init --plugins go/v3 --project-version 3


Flags:
      --component-config         create a versioned ComponentConfig file, may be 'true' or 'false'
      --domain string            domain for groups (default "my.domain")
      --fetch-deps               ensure dependencies are downloaded (default true)
  -h, --help                     help for init
      --license string           license to use to boilerplate, may be one of 'apache2', 'none' (default "apache2")
      --owner string             owner to add to the copyright
      --project-name string      name of this project
      --project-version string   project version (default "3")
      --repo string              name to use for go module (e.g., github.com/user/repo), defaults to the go package of the current working directory.
      --skip-go-version-check    if specified, skip checking the Go version

Global Flags:
      --plugins strings   plugin keys to be used for this subcommand execution
      --verbose           Enable verbose logging

5 create API and controller framework code

Executing the following command will create the framework code of api and controller, and then we can fill in the business logic code according to the requirements on the basis of the framework code:

  • The api code will be in api/v1alpha1/memcached_types.go
  • The controller code will be displayed in controllers/memcached_controller.go
$ operator-sdk create api --group cache --version v1alpha1 --kind Memcached --resource --controller

6. Write API

6.1 api definition

After the api is created, the main codes generated by the operator sdk are as follows:

// MemcachedSpec defines the desired state of Memcached
type MemcachedSpec struct {
        // INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
        // Important: Run "make" to regenerate code after modifying this file

        // Foo is an example field of Memcached. Edit memcached_types.go to remove/update
        Foo string `json:"foo,omitempty"`
}

// MemcachedStatus defines the observed state of Memcached
type MemcachedStatus struct {
        // INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
        // Important: Run "make" to regenerate code after modifying this file
}

//+kubebuilder:object:root=true
//+kubebuilder:subresource:status

// Memcached is the Schema for the memcacheds API
type Memcached struct {
        metav1.TypeMeta   `json:",inline"`
        metav1.ObjectMeta `json:"metadata,omitempty"`

        Spec   MemcachedSpec   `json:"spec,omitempty"`
        Status MemcachedStatus `json:"status,omitempty"`
}

Generally speaking, we need to modify the definition of spec and status to meet our business needs.

The spec is changed as follows:

  • Add the Size field to define the number of memcache copies
  • +kubebuilder:validation:Minimum=0 is marker comments; Operator SDK will generate code according to the description of marker comments. In this example, the function of marker comments is to generate the verification code defined by the api. If the spec is less than 0, the validation check will fail (marker comments will be further described in the next section)
// MemcachedSpec defines the desired state of Memcached
type MemcachedSpec struct {
	//+kubebuilder:validation:Minimum=0
	// Size is the size of the memcached deployment
	Size int32 `json:"size"`
}

status is defined as follows:

  • status has only one slice field of Nodes, which is used to store all pod names corresponding to memcached
  • The controller is responsible for making changes to the status field
// MemcachedStatus defines the observed state of Memcached
type MemcachedStatus struct {
	// Nodes are the names of the memcached pods
	Nodes []string `json:"nodes"`
}

6.2 marker comments

When writing the operator, you can use controller Gen to help us generate golang code and yaml files. The generated rules are described by maker comments.

The format of maker comments is described as follows:

Markers are single-line comments that start with a plus, followed by a marker name, optionally followed by some marker specific configuration

Generally speaking, maker comments can be divided into three categories:

  • empty type: a flag of boolen type, indicating whether to enable a certain communication ability
// +kubebuilder:validation:Optional

  • Anonymous type: accepts parameters of single value type
// +kubebuilder:validation:MaxItems=2

  • Multi option type: multiple parameters are accepted and separated by commas
// +kubebuilder:printcolumn:JSONPath=".status.replicas",name=Replicas,type=string

reference resources:

6.3 controller-gen

Controller Gen is in the bin directory of the project created by the operator sdk. Use as follows:

tingshuai.yts@B-5BBCMD6M-2026 memcached-operator % ./bin/controller-gen -hhh           
Usage:
  controller-gen [flags]

Examples:
        # Generate RBAC manifests and crds for all types under apis/,
        # outputting crds to /tmp/crds and everything else to stdout
        controller-gen rbac:roleName=<role name> crd paths=./apis/... output:crd:dir=/tmp/crds output:stdout

        # Generate deepcopy/runtime.Object implementations for a particular file
        controller-gen object paths=./apis/v1beta1/some_types.go

        # Generate OpenAPI v3 schemas for API packages and merge them into existing CRD manifests
        controller-gen schemapatch:manifests=./manifests output:dir=./manifests paths=./pkg/apis/... 

        # Run all the generators for a given project
        controller-gen paths=./apis/...

        # Explain the markers for generating CRDs, and their arguments
        controller-gen crd -ww


Flags:
  -h, --detailed-help count   print out more detailed help
                              (up to -hhh for the most detailed output, or -hhhh for json output)
      --help                  print out usage and a summary of options
      --version               show version
  -w, --which-markers count   print out all markers available with the requested generators
                              (up to -www for the most detailed output, or -wwww for json output)


Options


generators


+webhook package
        generates (partial) {Mutating,Validating}WebhookConfiguration objects.

+schemapatch package
        patches existing CRDs with new schemata. 
        It will generate output for each "CRD Version" (API version of the CRD type itself) , e.g. apiextensions/v1) available.

        [generateEmbeddedObjectMeta=<bool>]
                specifies if any embedded ObjectMeta in the CRD should be generated
        manifests=<string>
                contains the CustomResourceDefinition YAML files.
        [maxDescLen=<int>]
                specifies the maximum description length for fields in CRD's OpenAPI schema. 
                0 indicates drop the description for all fields completely. n indicates limit the description to at most n characters and truncate the description to closest sentence boundary if it exceeds n characters.


+rbac package
        generates ClusterRole objects.

        roleName=<string>
                sets the name of the generated ClusterRole.

+object package
        generates code containing DeepCopy, DeepCopyInto, and DeepCopyObject method implementations.

        [headerFile=<string>]
                specifies the header text (e.g. license) to prepend to generated files.
        [year=<string>]
                specifies the year to substitute for " YEAR" in the header file.

+crd package
        generates CustomResourceDefinition objects.

        [allowDangerousTypes=<bool>]
                allows types which are usually omitted from CRD generation because they are not recommended. 
                Currently the following additional types are allowed when this is true: float32 float64 
                 Left unspecified, the default is false

        [crdVersions=<[]string>]
                specifies the target API versions of the CRD type itself to generate. Defaults to v1. 
                Currently, the only supported value is v1. 
                 The first version listed will be assumed to be the "default" version and will not get a version suffix in the output filename. 
                 You'll need to use "v1" to get support for features like defaulting, along with an API server that supports it (Kubernetes 1.16+).

        [generateEmbeddedObjectMeta=<bool>]
                specifies if any embedded ObjectMeta in the CRD should be generated
        [maxDescLen=<int>]
                specifies the maximum description length for fields in CRD's OpenAPI schema. 
                0 indicates drop the description for all fields completely. n indicates limit the description to at most n characters and truncate the description to closest sentence boundary if it exceeds n characters.


generic


+paths package
        =<[]string>  represents paths and go-style path patterns to use as package roots.

output rules (optionally as output:<generator>:...)


+output:artifacts package
        outputs artifacts to different locations, depending on whether they're package-associated or not. 
        Non-package associated artifacts are output to the Config directory, while package-associated ones are output to their package's source files' directory, unless an alternate path is specified in Code.

        [code=<string>]
                overrides the directory in which to write new code (defaults to where the existing code lives).
        config=<string>
                points to the directory to which to write configuration.

+output:dir package
        =<string>  outputs each artifact to the given directory, regardless of if it's package-associated or not.

+output:none package
        skips outputting anything.

+output:stdout package
        outputs everything to standard-out, with no separation. 
        Generally useful for single-artifact outputs.

When creating the project, the operator sdk has created a make file for us. Therefore, we can use the make command to create code according to the definition of memcached type.

tingshuai.yts@B-5BBCMD6M-2026 memcached-operator % make generate
/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..." # Real command

Command analysis

  • paths describes the location defined by resource type. "./..." View all subdirectories and subdirectories.
  • object description the generated code includes the implementation of DeepCopy, DeepCopyInto, and DeepCopyObject method

After executing the above command, ZZ will be generated under api/v1alpha1_ generated. deepcopy. Go code, the code content is as follows:

//go:build !ignore_autogenerated
// +build !ignore_autogenerated

/*
Copyright 2022.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// Code generated by controller-gen. DO NOT EDIT.

package v1alpha1

import (
	runtime "k8s.io/apimachinery/pkg/runtime"
)

// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *Memcached) DeepCopyInto(out *Memcached) {
	*out = *in
	out.TypeMeta = in.TypeMeta
	in.ObjectMeta.DeepCopyInto(&out.ObjectMeta)
	out.Spec = in.Spec
	in.Status.DeepCopyInto(&out.Status)
}

// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Memcached.
func (in *Memcached) DeepCopy() *Memcached {
	if in == nil {
		return nil
	}
	out := new(Memcached)
	in.DeepCopyInto(out)
	return out
}

// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *Memcached) DeepCopyObject() runtime.Object {
	if c := in.DeepCopy(); c != nil {
		return c
	}
	return nil
}

// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *MemcachedList) DeepCopyInto(out *MemcachedList) {
	*out = *in
	out.TypeMeta = in.TypeMeta
	in.ListMeta.DeepCopyInto(&out.ListMeta)
	if in.Items != nil {
		in, out := &in.Items, &out.Items
		*out = make([]Memcached, len(*in))
		for i := range *in {
			(*in)[i].DeepCopyInto(&(*out)[i])
		}
	}
}

// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MemcachedList.
func (in *MemcachedList) DeepCopy() *MemcachedList {
	if in == nil {
		return nil
	}
	out := new(MemcachedList)
	in.DeepCopyInto(out)
	return out
}

// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object.
func (in *MemcachedList) DeepCopyObject() runtime.Object {
	if c := in.DeepCopy(); c != nil {
		return c
	}
	return nil
}

// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *MemcachedSpec) DeepCopyInto(out *MemcachedSpec) {
	*out = *in
}

// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MemcachedSpec.
func (in *MemcachedSpec) DeepCopy() *MemcachedSpec {
	if in == nil {
		return nil
	}
	out := new(MemcachedSpec)
	in.DeepCopyInto(out)
	return out
}

// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *MemcachedStatus) DeepCopyInto(out *MemcachedStatus) {
	*out = *in
	if in.Nodes != nil {
		in, out := &in.Nodes, &out.Nodes
		*out = make([]string, len(*in))
		copy(*out, *in)
	}
}

// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MemcachedStatus.
func (in *MemcachedStatus) DeepCopy() *MemcachedStatus {
	if in == nil {
		return nil
	}
	out := new(MemcachedStatus)
	in.DeepCopyInto(out)
	return out
}

Execute make manifest again to generate the yaml file corresponding to the api definition

tingshuai.yts@B-5BBCMD6M-2026 memcached-operator % make manifests
./bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases

Config / crd / bases / cache will be generated example. com_ memcacheds. Yaml file, which describes the definition of crd. The contents of the document are as follows:

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.7.0
  creationTimestamp: null
  name: memcacheds.cache.example.com
spec:
  group: cache.example.com
  names:
    kind: Memcached
    listKind: MemcachedList
    plural: memcacheds
    singular: memcached
  scope: Namespaced
  versions:
  - name: v1alpha1
    schema:
      openAPIV3Schema:
        description: Memcached is the Schema for the memcacheds API
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: MemcachedSpec defines the desired state of Memcached
            properties:
              size:
                description: Size is the size of the memcached deployment
                format: int32
                minimum: 0
                type: integer
            required:
            - size
            type: object
          status:
            description: MemcachedStatus defines the observed state of Memcached
            properties:
              nodes:
                description: Nodes are the names of the memcached pods
                items:
                  type: string
                type: array
            required:
            - nodes
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}
status:
  acceptedNames:
    kind: ""
    plural: ""
  conditions: []
  storedVersions: []

7 write controller business code

7.1 controller framework code analysis

The controller code generated by the framework is located in controllers/memcached_controller.go, the initial content is as follows:

package controllers

import (
        "context"

        "k8s.io/apimachinery/pkg/runtime"
        ctrl "sigs.k8s.io/controller-runtime"
        "sigs.k8s.io/controller-runtime/pkg/client"
        "sigs.k8s.io/controller-runtime/pkg/log"

        cachev1alpha1 "github.com/example/memcached-operator/api/v1alpha1"
)

// MemcachedReconciler reconciles a Memcached object
type MemcachedReconciler struct {
        client.Client
        Scheme *runtime.Scheme
}

//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/finalizers,verbs=update

// Reconcile is part of the main kubernetes reconciliation loop which aims to
// move the current state of the cluster closer to the desired state.
// TODO(user): Modify the Reconcile function to compare the state specified by
// the Memcached object against the actual cluster state, and then
// perform operations to make the cluster state reflect the state specified by
// the user.
//
// For more details, check Reconcile and its Result here:
// - https://pkg.go.dev/sigs.k8s.io/controller-runtime@v0.10.0/pkg/reconcile
func (r *MemcachedReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
        _ = log.FromContext(ctx)

        // your logic here

        return ctrl.Result{}, nil
}

// SetupWithManager sets up the controller with the Manager.
func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error {
        return ctrl.NewControllerManagedBy(mgr).
                For(&cachev1alpha1.Memcached{}).
                Complete(r)
}

The memcachedrecipiler type is first defined in the framework code

  • client.Client is the interface, which defines the CURD related operations of kubernetes object
  • runtime.Scheme defines the method of serializing and deserializing API objects
type MemcachedReconciler struct {
        client.Client
        Scheme *runtime.Scheme
}

The framework code defines the operator's permission information for memcached through mark comment

  • The current operator has get for memcacheds; list; watch; create; update; patch; Delete permission
  • The current operator has get for memcacheds/status; update; Patch permission
  • The current operator has update permission on memcacheds/finalizers
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/finalizers,verbs=update

The framework code generates a Reconcile function for memcached reconciler, which is the main body of the controller

  • Reconcile is used to ensure that the memcached status is consistent with that described in the spec;
  • Whenever the resource monitored by memcached operator changes, the Reconcile function will be called;
  • Parameter meaning:
    • context.Context: A Context carries a deadline, a cancellation signal, and other values across API boundaries;
    • ctrl.Request: contains the name and namespace of the kubernetes object from Reconcile
func (r *MemcachedReconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Result, error) {
        _ = log.FromContext(ctx)

        // your logic here

        return ctrl.Result{}, nil
}

The SetUpWithManager function generated by the framework code is used to describe which resource s the current operator watch es:

  • NewControllerManagedBy() creates a configuration to configure the current manager
  • For (& cachev1alpha1. Memcached {}) set the operator to monitor the changes of memcached, and all Add/Delete/Update of memcached will be changed
// SetupWithManager sets up the controller with the Manager.
func (r *MemcachedReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
    For(&cachev1alpha1.Memcached{}).
    Complete(r)
}

7.2 authority preparation

According to the business logic of memcached operator, you need to add the operation permissions of deployment and pod

//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/status,verbs=get;update;patch
//+kubebuilder:rbac:groups=cache.example.com,resources=memcacheds/finalizers,verbs=update
//+kubebuilder:rbac:groups=apps,resources=deployments,verbs=get;list;watch;create;update;patch;delete
//+kubebuilder:rbac:groups=core,resources=pods,verbs=get;list;watch

7.3 preparation of reconcile loop

The reconcile loop mainly includes the following processes:

  • Get memcached object s related to event s;
  • Judge whether the deployment associated with memcached exists; If it does not exist, execute the create of deployment
  • Judge whether the size of the deployment is the same as that of the spec in memcached. If it is different, execute the update of the deployment
  • Judge whether the pod content in the status field of memcached meets the expectation. If not, update the content of the status field

For the complete code, see: https://github.com/operator-framework/operator-sdk/blob/latest/testdata/go/v3/memcached-operator/controllers/memcached_controller.go

7.3.1 get memcached object

Through req Namespacedname to find the corresponding memcached

	memcached := &cachev1alpha1.Memcached{}
	err := r.Get(ctx, req.NamespacedName, memcached)
	if err != nil {
		if errors.IsNotFound(err) {
			// Request object not found, could have been deleted after reconcile request.
			// Owned objects are automatically garbage collected. For additional cleanup logic use finalizers.
			// Return and don't requeue
			log.Info("Memcached resource not found. Ignoring since object must be deleted")
			return ctrl.Result{}, nil
		}
		// Error reading the object - requeue the request.
		log.Error(err, "Failed to get Memcached")
		return ctrl.Result{}, err
	}

7.3.2 judging the existence of deployment

Find the corresponding deployment through the name and namespace of memcache; If it cannot be found, call the custom deploymentForMemcached to create a deployment

found := &appsv1.Deployment{}
	err = r.Get(ctx, types.NamespacedName{Name: memcached.Name, Namespace: memcached.Namespace}, found)
	if err != nil && errors.IsNotFound(err) {
		// Define a new deployment
		dep := r.deploymentForMemcached(memcached)
		log.Info("Creating a new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
		err = r.Create(ctx, dep)
		if err != nil {
			log.Error(err, "Failed to create new Deployment", "Deployment.Namespace", dep.Namespace, "Deployment.Name", dep.Name)
			return ctrl.Result{}, err
		}
		// Deployment created successfully - return and requeue
		return ctrl.Result{Requeue: true}, nil
	} else if err != nil {
		log.Error(err, "Failed to get Deployment")
		return ctrl.Result{}, err
	}

7.3.3 judging memcached size

If the size is inconsistent with that in the spec, call the user-defined update interface to update

	size := memcached.Spec.Size
	if *found.Spec.Replicas != size {
		found.Spec.Replicas = &size
		err = r.Update(ctx, found)
		if err != nil {
			log.Error(err, "Failed to update Deployment", "Deployment.Namespace", found.Namespace, "Deployment.Name", found.Name)
			return ctrl.Result{}, err
		}
		// Ask to requeue after 1 minute in order to give enough time for the
		// pods be created on the cluster side and the operand be able
		// to do the next update step accurately.
		return ctrl.Result{RequeueAfter: time.Minute}, nil
	}

7.3.4 judge memcached status

The status field of memcached describes the name information of the pod, so each loop should judge whether the status field should be updated:

  • First, get the podlist, and then compare it with that recorded in memcached
  • Update status if different
	// Update the Memcached status with the pod names
	// List the pods for this memcached's deployment
	podList := &corev1.PodList{}
	listOpts := []client.ListOption{
		client.InNamespace(memcached.Namespace),
		client.MatchingLabels(labelsForMemcached(memcached.Name)),
	}
	if err = r.List(ctx, podList, listOpts...); err != nil {
		log.Error(err, "Failed to list pods", "Memcached.Namespace", memcached.Namespace, "Memcached.Name", memcached.Name)
		return ctrl.Result{}, err
	}
	podNames := getPodNames(podList.Items)

	// Update status.Nodes if needed
	if !reflect.DeepEqual(podNames, memcached.Status.Nodes) {
		memcached.Status.Nodes = podNames
		err := r.Status().Update(ctx, memcached)
		if err != nil {
			log.Error(err, "Failed to update Memcached status")
			return ctrl.Result{}, err
		}
	}

8 compilation and debugging

There are generally two ways to debug an operator

  • in cluster: publish the operator to the k8s cluster
  • out cluster: start a golang operator process outside the k8s environment; This paper adopts this method

The command of out cluster in the operator sdk is make install run:

  • First, call controller Gen to create rbac, crd, webhook code and yaml (Note: this step is a redundant operation, which can not be executed in fact)
  • Then, call kustomize to create crd.
  • Finally, execute main go
 memcached-operator % make install run
bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
bin/kustomize build config/crd | kubectl apply -f -
customresourcedefinition.apiextensions.k8s.io/memcacheds.cache.example.com created
bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
go run ./main.go
2022-02-24T15:13:19.997+0800    INFO    controller-runtime.metrics      metrics server is starting to listen    {"addr": ":8080"}
2022-02-24T15:13:19.997+0800    INFO    setup   starting manager
2022-02-24T15:13:19.998+0800    INFO    starting metrics server {"path": "/metrics"}
2022-02-24T15:13:19.998+0800    INFO    controller.memcached    Starting EventSource    {"reconciler group": "cache.example.com", "reconciler kind": "Memcached", "source": "kind source: /, Kind="}
2022-02-24T15:13:19.998+0800    INFO    controller.memcached    Starting Controller     {"reconciler group": "cache.example.com", "reconciler kind": "Memcached"}
2022-02-24T15:13:20.098+0800    INFO    controller.memcached    Starting workers        {"reconciler group": "cache.example.com", "reconciler kind": "Memcached", "worker count": 1}
2022-02-24T15:16:33.317+0800    INFO    controller.memcached    Creating a new Deployment       {"reconciler group": "cache.example.com", "reconciler kind": "Memcached", "name": "memcached-sample", "namespace": "default", "Deployment.Namespace": "default", "Deployment.Name": "memcached-sample"}

Topics: Kubernetes Cloud Native