If you use Kubernetes a lot, you should be familiar with Helm and Kustomize. Both tools are used to manage Kubernetes resource lists, but they work in different ways.

Helm uses templates, a Helm Chart package that contains a number of templates and value files, and when rendered the variables in the templates are replaced with the corresponding values in the value files. Kustomize uses a template-free approach, which patches and merges YAML files, and Kustomize is also built natively into kubectl. Both tools are widely used in the Kubernetes ecosystem, and they can be used together.

To fork or not to fork

We know that many projects actually provide Helm Chart packages for applications, and that the values of template variables are controlled via value files. A long-standing question is how we should customize the upstream Helm Chart package, for example by adding or a list of Kubernetes resources from the Helm Chart package, and if it’s a generic change, the best option is of course to contribute directly to the upstream repository, but what about a custom change?

Usually we can fork the upstream Helm Chart repository ourselves and then make additional changes to the Chart package in our own repo. But doing so, obviously, creates an extra burden, especially if the Chart package only needs a small change.

In this case, we can use Kustomize to customize the existing Helm Chart without performing a fork operation.

The Helm version used in this article is 3.3.1 and Kustomize 3.8.2.

Customize with Chart Plugin

Kustomize provides a nice ecosystem of plugins that allow to extend the functionality of Kustomize. Among them is a non-built-in plugin called ChartInflator, a non-built-in plugin that allows Kustomize to render Helm Charts and perform any changes needed.

Start by installing the ChartInflator plug-in.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

$ chartinflator_dir="./kustomize/plugin/kustomize.config.k8s.io/v1/chartinflator"

# 创建插件目录
$ mkdir -p ${chartinflator_dir}

# 下载插件
$ curl -L https://raw.githubusercontent.com/kubernetes-sigs/kustomize/kustomize/v3.8.2/plugin/someteam.example.com/v1/chartinflator/ChartInflator > ${chartinflator_dir}/ChartInflator

# 设置插件执行权限
$ chmod u+x ${chartinflator_dir}/ChartInflator

For example, if we want to customize the Vault Helm Chart package, next create the ChartInflator resource list and the Helm values.yaml values file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

# ChartInflator 资源清单
$ cat << EOF >> chartinflator-vault.yaml
apiVersion: kustomize.config.k8s.io/v1
kind: ChartInflator
metadata:
  name: vault-official-helm-chart
chartRepo: https://helm.releases.hashicorp.com  
chartName: vault
chartRelease: hashicorp
chartVersion: 0.7.0
releaseName: vault
values: values.yaml
EOF

# 创建 values 值文件
$ helm repo add hashicorp https://helm.releases.hashicorp.com 
$ helm show values --version 0.7.0 hashicorp/vault > values.yaml

# 创建 Kustomize 文件
$ kustomize init
$ cat << EOF >> kustomization.yaml
generators:
- chartinflator-vault.yaml
EOF

# 为所有资源添加一个 label 标签
$ kustomize edit add label env:dev

# 最后生成的 kustomize 文件如下所示：
$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
generators:
- chartinflator-vault.yaml
commonLabels:
  env: dev

# 整个资源清单目录结构
$ tree .
.
├── chartinflator-vault.yaml
├── kustomization.yaml
├── kustomize
│   └── plugin
│       └── kustomize.config.k8s.io
│           └── v1
│               └── chartinflator
│                   └── ChartInflator
└── values.yaml

5 directories, 4 files

Now it’s time to render the Chart template by executing the command shown below.

1

$ kustomize build --enable_alpha_plugins .

After normal rendering we can see that an env: dev tag has been added to all resources, this is done in real time and does not require any additional files to be maintained.

Customize with a single Chart file

Another way to customize the Chart using Kustomize is to use the helm template command to generate a single list of resources, this way you have more control over the Chart, but it requires more work to come out and handle updating the version control of that generated file.

Often we can use Make to assist in this process, as shown in the following example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

# Makefile
CHART_REPO_NAME   := hashicorp
CHART_REPO_URL    := https://helm.releases.hashicorp.com
CHART_NAME        := vault
CHART_VERSION     := 0.7.0
CHART_VALUES_FILE := values.yaml

add-chart-repo:
    helm repo add ${CHART_REPO_NAME} ${CHART_REPO_URL}
    helm repo update

generate-chart-manifest:
    helm template ${CHART_NAME} ${CHART_REPO_NAME}/${CHART_NAME} \
        --version ${CHART_VERSION} \
        --values ${CHART_VALUES_FILE} > ${CHART_NAME}.yaml

get-chart-values:
    @helm show values --version ${CHART_VERSION} \
    ${CHART_REPO_NAME}/${CHART_NAME}

generate-chart-values:
    @echo "Create values file: ${CHART_VALUES_FILE}"
    @$(MAKE) -s get-chart-values > ${CHART_VALUES_FILE}

diff-chart-values:
    @echo "Diff: Local <==> Remote"
    @$(MAKE) -s get-chart-values | \
    diff --suppress-common-lines --side-by-side ${CHART_VALUES_FILE} - || \
    exit 0

To customize the upstream Vault Helm Chart, we can do the following.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# 初始化 chart 文件
$ make generate-chart-values generate-chart-manifest 

# 创建 Kustomize 文件并添加一个 label 标签
$ kustomize init
$ kustomize edit add resource vault.yaml
$ kustomize edit add label env:dev

# 最后生成的文件结构如下所示
$ tree .
.
├── kustomization.yaml
├── makefile
├── values.yaml
└── vault.yaml

0 directories, 4 files

# kustomize 文件内容如下所示
$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- vault.yaml
commonLabels:
  env: dev

Finally, the same kustomize build command is used to render.

1

$ kustomize build .

You can also see in the rendered result that all the resources have an env: dev tag added to them.

With this approach, you need to somehow run the make command to generate the updated all-in-one resource manifest file, and it can be a bit tricky to integrate the update process with your GitOps workflow.

Customization with Helm post rendering

Post Rendering is one of the new features brought by Helm 3. In the previous 2 methods, Kustomize was the main tool used to handle generating the list of charts, but in this case Kustomize is working as Helm’s helper.

Let’s see how to use this method for Kustomize.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# 创建 Kustomize 文件并添加一个 label 标签
$ kustomize init
$ kustomize edit add label env:dev

# 创建一个包装 Kustomize 的脚本文件，后面在 Helm 中会使用到
$ cat << EOF > kustomize-wrapper.sh
#!/bin/bash
cat <&0 > chart.yaml
kustomize edit add resource chart.yaml
kustomize build . && rm chart.yaml
EOF
$ chmod +x kustomize-wrapper.sh

We can then either render directly using Helm or install Chart.

1
2

$ helm repo add hashicorp https://helm.releases.hashicorp.com 
$ helm template vault hashicorp/vault --post-renderer ./kustomize-wrapper.sh

Normally we can also see that an env:dev tag is added to each resource file that is rendered at the end.

The rest is basically the same as the first approach, except that instead of using the Kustomize plugin, we use Helm’s own functionality to render the upstream Chart package directly.

Summary

We can see that each of the above methods has its own advantages and disadvantages, and which one to use depends mainly on our own working environment and workflow, but at least we have seen the efficiency of using Kustomize in combination with Helm.