Preface

Under the ten-year cloud computing wave, DevOps, containers, microservices and other technologies are developing rapidly, and cloud-native has become a trend. Enterprises are moving from “ON Cloud” to “IN Cloud” and becoming “new cloud-native enterprises”, where new capabilities and existing capabilities are established but not broken, with organic synergy to achieve Resource efficiency, application agility, business intelligence, security and trustworthiness. The whole concept of cloud-native is very big, and it is so detailed that we may encounter some small problems in the real scenario, this article will share with you the small demands and solutions we encounter in our daily work.

Background

In my daily use of kubectl to view K8s resources, I want to directly view the corresponding resources of the container name and image name, kubectl does not support the selection, we need to describe then to view, for the cluster itself more, not very convenient, so the development of their own kubectl plug-in to achieve this function.

The first need to call Kubernetes requires the use of client-go project to achieve access to Kubernetes resources, for the plug-in using Golang language development, because it is client-side execution, in order to facilitate the integration to and command-line tools, using the same command-line scaffolding tool Cobra and K8s, and finally released it open source to GitHub.

Golang

In cloud native development, many of Google’s open source projects are developed using Golang, which can be distributed to multiple platforms after cross-platform compilation, and the plug-ins we develop are based on Golang and subsequently support multi-platform use.

Cobra

Cobra is a command line library that is a powerful tool for writing command lines and provides a scaffold for quickly generating Cobra-based application frameworks. We can use Cobra to quickly and easily develop the command line tools we want.

Client-go

In K8s operations, we can use kubectl, client libraries or REST requests to access the K8s API, and in fact, both kubectl and client libraries are tools that encapsulate REST requests. client-go, as a client library, can call the K8s API and implement operations on resource objects in the K8s cluster, including client-go is a client library that can call the K8s API to add, delete, modify, and query resource objects (including deployment, service, Ingress, ReplicaSet, Pod, Namespace, Node, etc.) in a K8s cluster.

krew

Krew is a package management tool for kubectl plugins, similar to apt, dnf or brew, which allows you to easily manage the full cycle of kubectl plugins, including search, download, uninstall, etc.

The kubectl tool is already relatively complete, but for some personalized commands, the goal is to allow developers to release custom kubectl subcommands in an independent and intense form. Plugins can be developed in any language. You need to name the final footer or binary executable with the prefix kubectl- and put it in the PATH. You can use the kubectl plugin list to see which plugins are currently installed.

  • GitHub Action

If you need an Action, you don’t have to write your own complex script, you can just reference someone else’s Action and the whole continuous integration process becomes a combination of Actions. This allows people to define their own Action, which can then be easily reused by others. It is also possible to unify your own or organize some public processes in the build process.

  • gorelease

Developed in Golang, GoReleaser is an automated release tool for Golang projects. Without much configuration, you can easily compile, package and publish cross-platform packages to Github, Gitlab and other repositories with just a few lines of command.

Plugin planning

  • The plugin is named: kubectl-img.
  • Currently only a simple implementation of an image command to view the names of different resource objects (deployments/daemonsets/statefulsets/jobs/cronjobs), and the corresponding container names, image names.
  • JSON format output is supported.
  • Finally use it as a krew plugin.
  • You can view the corresponding resources directly based on the namespace.

Development

Project initialization

  • Install Cobra

Install Cobra in your development environment and go to the command line tool to generate project scaffolding based on this framework, which is also used to generate many of the components in K8s.

1

go get -v github.com/spf13/cobra/cobra

  • Initialize the project

    1
2
3
4
5
6
7
8

    $ cobra init --pkg-name kubectl-img
$ ls
LICENSE cmd     main.go
$ tree
├── LICENSE
├── cmd
│   └── root.go
└── main.go

  • Create go mod, download related packages

    1

    go mod init github.com/redhatxl/kubectl-img

Adding a subcommand

Add a subcommand image, add a subcommand to our plugin here.

1

$ cobra add image

Add parameters

Display different resource image names by subcommand+flag form.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

func Execute() {
 cobra.CheckErr(rootCmd.Execute())
}

func init() {
 KubernetesConfigFlags = genericclioptions.NewConfigFlags(true)
 imageCmd.Flags().BoolP("deployments", "d", false, "show deployments image")
 imageCmd.Flags().BoolP("daemonsets", "e", false, "show daemonsets image")
 imageCmd.Flags().BoolP("statefulsets", "f", false, "show statefulsets image")
 imageCmd.Flags().BoolP("jobs", "o", false, "show jobs image")
 imageCmd.Flags().BoolP("cronjobs", "b", false, "show cronjobs image")
 imageCmd.Flags().BoolP("json", "j", false, "show json format")
 KubernetesConfigFlags.AddFlags(rootCmd.PersistentFlags())
}

Implement the image command

Register subcommands and modify command usage instructions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

var imageCmd = &cobra.Command{
 Use:   "image",
 Short: "show resource image",
 Long:  `show k8s resource image`,
 RunE:  image,
}

func init() {
 rootCmd.AddCommand(imageCmd)
}

Initialize clientset

Since we need to call K8s resources, here we use ClientSet in Client-go to get different resource images based on different flags entered by the user.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

// ClientSet k8s clientset
func ClientSet(configFlags *genericclioptions.ConfigFlags) *kubernetes.Clientset {
 config, err := configFlags.ToRESTConfig()
 if err != nil {
  panic("kube config load error")
 }
 clientSet, err := kubernetes.NewForConfig(config)
 if err != nil {

  panic("gen kube config error")
 }
 return clientSet
}

View resource objects

Use reflection to view specific resource images and image names based on different resource types.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

func image(cmd *cobra.Command, args []string) error {

 clientSet := kube.ClientSet(KubernetesConfigFlags)
 ns, _ := rootCmd.Flags().GetString("namespace")
 // 生命一个全局资源列表
 var rList []interface{}

 if flag, _ := cmd.Flags().GetBool("deployments"); flag {
  deployList, err := clientSet.AppsV1().Deployments(ns).List(context.Background(), v1.ListOptions{})
  if err != nil {
   fmt.Printf("list deployments error: %s", err.Error())
  }
  rList = append(rList, deployList)
 }
  ...
   deployMapList := make([]map[string]string, 0)
 for i := 0; i < len(rList); i++ {
  switch t := rList[i].(type) {
  case *kv1.DeploymentList:
   for k := 0; k < len(t.Items); k++ {
    for j := 0; j < len(t.Items[k].Spec.Template.Spec.Containers); j++ {
     deployMap := make(map[string]string)
     deployMap["NAMESPACE"] = ns
     deployMap["TYPE"] = "deployment"
     deployMap["RESOURCE_NAME"] = t.Items[k].GetName()
     deployMap["CONTAINER_NAME"] = t.Items[k].Spec.Template.Spec.Containers[j].Name
     deployMap["IMAGE"] = t.Items[k].Spec.Template.Spec.Containers[j].Image
     deployMapList = append(deployMapList, deployMap)
    }
   }

Implementing Output

Use Table to output the results, also extending JSON output.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


func GenTable(mapList []map[string]string) *table.Table {
 t, err := gotable.Create(title...)
 if err != nil {
  fmt.Printf("create table error: %s", err.Error())
  return nil
 }
 t.AddRows(mapList)
 return t
}

Final project structure.

project structure

Integrating krew

You need to name the final footer or binary executable with the prefix kubectl- and put it in the PATH. You can use the kubectl plugin list to see which plugins are currently installed.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

$ kubectl plugin list
The following compatible plugins are available:=
/usr/local/bin/kubectl-debug
  - warning: kubectl-debug overwrites existing command: "kubectl debug"
/usr/local/bin/kubectl-v1.10.11
/usr/local/bin/kubectl-v1.20.0
/Users/xuel/.krew/bin/kubectl-df_pv
/Users/xuel/.krew/bin/kubectl-krew

# 将自己开发的插件重新命名为kubectl-img放到可执行路基下
$ cp kubectl-img /Users/xuel/.krew/bin/kubectl-img

$ kubectl plugin list
The following compatible plugins are available:=
/usr/local/bin/kubectl-debug
  - warning: kubectl-debug overwrites existing command: "kubectl debug"
/usr/local/bin/kubectl-v1.10.11
/usr/local/bin/kubectl-v1.20.0
/Users/xuel/.krew/bin/kubectl-df_pv
/Users/xuel/.krew/bin/kubectl-krew
/Users/xuel/.krew/bin/kubectl-img

Use

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

$ kubectl img image -h
show k8s resource image

Usage:
  kubectl-img image [flags]

Flags:
  -b, --cronjobs       show cronjobs image
  -e, --daemonsets     show daemonsets image
  -d, --deployments    show deployments image
  -h, --help           help for image
  -o, --jobs           show jobs image
  -j, --json           show json format
  -f, --statefulsets   show statefulsets image

Global Flags:
      --as string                      Username to impersonate for the operation
      --as-group stringArray           Group to impersonate for the operation, this flag can be repeated to specify multiple groups.
      --cache-dir string               Default cache directory (default "/Users/xuel/.kube/cache")
      --certificate-authority string   Path to a cert file for the certificate authority
      --client-certificate string      Path to a client certificate file for TLS
      --client-key string              Path to a client key file for TLS
      --cluster string                 The name of the kubeconfig cluster to use
      --context string                 The name of the kubeconfig context to use
      --insecure-skip-tls-verify       If true, the server's certificate will not be checked for validity. This will make your HTTPS connections insecure
      --kubeconfig string              Path to the kubeconfig file to use for CLI requests.
  -n, --namespace string               If present, the namespace scope for this CLI request
      --request-timeout string         The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests. (default "0")
  -s, --server string                  The address and port of the Kubernetes API server
      --tls-server-name string         Server name to use for server certificate validation. If it is not provided, the hostname used to contact the server is used
      --token string                   Bearer token for authentication to the API server
      --user string                    The name of the kubeconfig user to use

  • View Resources

    1
2
3
4

    # View the images of all deployments of the entire kubernetes cluster
kubectl img image --deployments
# View the images of all deployments of the entire kubernetes cluster
kubectl img image --deployments -n default

    View Resources

  • View all resources

    1
2
3

    
# view all resource
kubectl img image -bedof

    View all resources

  • JSON formatted output

    1
2

    # Table display is used by default
kubectl img image --deployments -n default -j

    JSON formatted output

Open Source Publishing

Once you’ve finished writing the code, you’ll publish it to GitHub so that more people can learn and share it.

Github Action

Create a .github/workflows/ci.yml file in the project root directory with the following contents.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

name: ci
on:
  push:
  pull_request:
jobs:
  goreleaser:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@master
      - name: Setup Go
        uses: actions/setup-go@v1
        with:
          go-version: 1.16
      - name: GoReleaser
        uses: goreleaser/goreleaser-action@v1
        with:
          version: latest
          args: release --snapshot --rm-dist

Github Action

GO Report Card

Add a Go project report: https://goreportcard.com/

GO Report Card

GoReleaser

For Golang projects, you can use GoReleaser to make a nice Release.

Since macOS is used, here’s how to install it using brew.

1

brew install goreleaser

Generate the .goreleaser.yml configuration in the project root directory.

1

goreleaser init

Once configured, remember to add dist to .gitignore, as goreleaser will output the compiled files to the dist directory by default.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

# This is an example .goreleaser.yml file with some sensible defaults.
# Make sure to check the documentation at https://goreleaser.com
before:
  hooks:
    # You may remove this if you don't use go modules.
    - go mod tidy
    # you may remove this if you don't need go generate
    - go generate ./...
builds:
  - env:
      - CGO_ENABLED=0
    goos:
      - linux
      - windows
      - darwin
archives:
  - replacements:
      darwin: Darwin
      linux: Linux
      windows: Windows
      386: i386
      amd64: x86_64
checksum:
  name_template: 'checksums.txt'
snapshot:
  name_template: "{{ incpatch .Version }}-next"
changelog:
  sort: asc
  filters:
    exclude:
      - '^docs:'
      - '^test:'

project_name: kubectl-img

After GoReleaser is configured, you can first compile and test it by.

Note: To configure GITHUB_TOKEN for the first time using GoReleaser, you can apply for it here, and run the following command to configure GITHUB_TOKEN after the application is done.

1

export GITHUB_TOKEN=<YOUR_TOKEN>

Make sure there are no problems, then you can run Git and GoReleaser to release the release.

1
2
3
4
5

git add .
git commit -m "add goreleaser"
git tag -a v0.0.2 -m "First release"
git push origin main
git push origin v0.0.2

After all is done, execute the following command to publish.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

$ goreleaser release --rm-dist
   • releasing...
   • loading config file       file=.goreleaser.yaml
   • loading environment variables
   • getting and validating git state
      • building...               commit=98703b3b9d9ac7f4661c5669c1e164d2cf3675d2 latest tag=v1.0.0
   • parsing tag
   • running before hooks
      • running                   hook=go mod tidy
      • running                   hook=go generate ./...
   • setting defaults
      • DEPRECATED: skipped windows/arm64 build on Go < 1.17 for compatibility, check https://goreleaser.com/deprecations/#builds-for-windowsarm64 for more info.
   • checking distribution directory
      • --rm-dist is set, cleaning it up
   • loading go mod information
   • build prerequisites
   • writing effective config file
      • writing                   config=dist/config.yaml
   • generating changelog
      • writing                   changelog=dist/CHANGELOG.md
   • building binaries
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_linux_386/kubectl-img
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_linux_amd64/kubectl-img
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_darwin_arm64/kubectl-img
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_linux_arm64/kubectl-img
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_windows_amd64/kubectl-img.exe
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_windows_386/kubectl-img.exe
      • building                  binary=/Users/xuel/workspace/goworkspace/src/github.com/kaliarch/kubectl-img/dist/kubectl-img_darwin_amd64/kubectl-img
   • archives
      • creating                  archive=dist/kubectl-img_1.0.0_Linux_i386.tar.gz
      • creating                  archive=dist/kubectl-img_1.0.0_Darwin_x86_64.tar.gz
      • creating                  archive=dist/kubectl-img_1.0.0_Linux_x86_64.tar.gz
      • creating                  archive=dist/kubectl-img_1.0.0_Windows_x86_64.tar.gz
      • creating                  archive=dist/kubectl-img_1.0.0_Linux_arm64.tar.gz
      • creating                  archive=dist/kubectl-img_1.0.0_Windows_i386.tar.gz
      • creating                  archive=dist/kubectl-img_1.0.0_Darwin_arm64.tar.gz
   • calculating checksums
   • storing release metadata
      • writing                   file=dist/artifacts.json
      • writing                   file=dist/metadata.json
   • publishing
      • scm releases
         • creating or updating release repo=redhatxl/kubectl-img tag=v1.0.0
         • release updated           url=https://github.com/redhatxl/kubectl-img/releases/tag/v1.0.0
         • uploading to release      file=dist/checksums.txt name=checksums.txt
         • uploading to release      file=dist/kubectl-img_1.0.0_Linux_i386.tar.gz name=kubectl-img_1.0.0_Linux_i386.tar.gz
         • uploading to release      file=dist/kubectl-img_1.0.0_Linux_x86_64.tar.gz name=kubectl-img_1.0.0_Linux_x86_64.tar.gz
         • uploading to release      file=dist/kubectl-img_1.0.0_Windows_i386.tar.gz name=kubectl-img_1.0.0_Windows_i386.tar.gz
         • uploading to release      file=dist/kubectl-img_1.0.0_Linux_arm64.tar.gz name=kubectl-img_1.0.0_Linux_arm64.tar.gz
         • uploading to release      file=dist/kubectl-img_1.0.0_Darwin_x86_64.tar.gz name=kubectl-img_1.0.0_Darwin_x86_64.tar.gz
         • uploading to release      file=dist/kubectl-img_1.0.0_Windows_x86_64.tar.gz name=kubectl-img_1.0.0_Windows_x86_64.tar.gz
         • uploading to release      file=dist/kubectl-img_1.0.0_Darwin_arm64.tar.gz name=kubectl-img_1.0.0_Darwin_arm64.tar.gz
   • announcing
   • release succeeded after 183.24s

Check out the Release that was released.

Release

Add the installation method for different platforms in the project README.

Linux

1
2
3
4
5
6

export release=v1.0.0
curl -L -o kubectl-img.tar.gz https://github.com/redhatxl/kubectl-img/releases/download/${release}/kubectl-img_${release}_Linux_arm64.tar.gz
tar -xvf kubectl-img.tar.gz
cp kubectl-img /usr/local/bin/kubectl-img
# use kubectl krew
cp kubectl-img $HOME/.krew/bin

OSX

1
2
3
4
5
6
7


export release=v1.0.0
curl -L -o kubectl-img.tar.gz https://github.com/redhatxl/kubectl-img/releases/download/${release}/kubectl-img_${release}_Darwin_x86_64.tar.gz
tar -xvf kubectl-img.tar.gz
mv kubectl-img /usr/local/bin/kubectl-img
# use kubectl krew
cp kubectl-img $HOME/.krew/bin

Windows

In PowerShell v5+

1
2
3
4
5

$url = "https://github.com/redhatxl/kubectl-img/releases/download/v1.0.0/kubectl-img_1.0.0_Windows_x86_64.tar.gz"
$output = "$PSScriptRoot\kubectl-img.zip"

Invoke-WebRequest -Uri $url -OutFile $output
Expand-Archive "$PSScriptRoot\kubectl-img.zip" -DestinationPath "$PSScriptRoot\kubectl-img"

Badges display tool

Here’s a great tool for displaying Badges: https://shields.io/. This site offers a wide variety of Badges, and you can fill up your GitHub README.md with them if you want.

Badges

Summary

The current implementation is relatively simple, as a way to throw the function, later you can carry out more functions or other plug-in development. From a technical perspective, cloud-native technologies represented by containers, microservices and dynamic orchestration are flourishing and have become an important driving force for enabling business innovation, and have been applied to the core business of enterprises. From the market perspective, cloud-native technologies have been widely verified in many industries such as finance, manufacturing, and Internet, and the business scenarios supported are getting richer and richer, and the industry ecology is becoming more and more prosperous.