No one who uses K8s will be unaware of CNI, but probably most people, most of the time, only care about the installation. Put the binary in /opt/cni/bin, create the configuration file under /etc/cni/net.d/, and leave the rest to K8s or containerd, we don’t care and don’t understand the implementation.

CNI, known as Container Network Interface, is a specification used to define container networks. containerernetworking/cni is a CNCF CNI implementation project, including basic bridge, macvlan and other basic network plugins.

Let’s use this as an example to understand how CNI works.

Installing the CNI Plugin

We will install the CNI plugin by downloading and compiling it ourselves.

1
2
3
4
5

$ cd $GOPATH/src/github.com/containernetworking/plugins
$ ./build_linux.sh
ls bin/
bandwidth  dhcp      flannel      host-local  loopback  portmap  sbr     tuning  vrf
bridge     firewall  host-device  ipvlan      macvlan   ptp      static  vlan

The contents under bin/ are the compiled CNI plugins. We can also put it under the standard /opt/cni/bin.

Configuration file

Our example configuration file is /etc/cni/net.d/10-mynet.conf. The contents are as follows.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

{
  "cniVersion": "0.4.0",
  "name": "mynet",
  "type": "bridge",
  "bridge": "cni0",
  "isGateway": true,
  "ipMasq": true,
  "ipam": {
    "type": "host-local",
    "subnet": "172.19.0.0/24",
    "routes": [
      { "dst": "0.0.0.0/0" }
    ]
  }
}

It is easy to see that our network is named mynet and the network type is bridge, which is both the name of the network type and the name of the network plug-in executable. ipam uses host-local here, which can also be found in the bin/ directory after we compile above.

CNI plug-in invocation rules

CNI plug-ins are invoked directly through exec, not through socket in a C/S way, all parameters are implemented through environment variables, standard input and output, specifically the invocation rules are as follows.

  • Input.
    • Run parameters: environment variables
    • Network configuration: stdin
  • Output.
    • Normal exit: stdout
    • Exception exit: stderr

Run parameters

The parameters passed to the CNI plugin are implemented via environment variables starting with CNI_.

  • CNI_COMMAND: The action to be performed, including ADD, DEL, CHECK, or VERSION.
  • CNI_CONTAINERID: Unique container ID.
  • CNI_NETNS: Network namespace.
  • CNI_IFNAME: The name of the network interface created within the container.
  • CNI_ARGS: Additional parameters passed to the plugin itself, set in the format “FOO=BAR;ABC=123”.
  • CNI_PATH: The path to find the CNI plugin, in the same format as the PATH environment variable, i.e. Linux uses : to split multiple paths, Windows uses ; to split them.

CNI operations

CNI defines 4 operations: ADD, DEL, CHECK, and VERSION. These are passed to the plugin via the CNI_COMMAND environment variable.

Return Value

By general Linux programming convention, success returns 0, failure returns non-zero, and the error message is in the specified format, as shown in the following example.

1
2
3
4
5
6

{
  "cniVersion": "1.0.0",
  "code": 7,
  "msg": "Invalid Configuration",
  "details": "Network 192.168.0.0/31 too small to allocate from."
}

Example

Here we will create and delete some network interfaces manually to see how CNI works.

Adding a network interface

Create a new network namespace, here we use ctr-1 as the name of the network namespace.

1
2
3

$ contid=ctr-1
$ netnspath=/var/run/netns/$contid
$ ip netns add $contid

Set some common environment variables so that you don’t have to set them all over again when adding and removing network devices.

1
2
3
4
5

$ export CNI_PATH=$GOPATH/src/github.com/containernetworking/plugins/bin
$ export PATH=$CNI_PATH:$PATH
$ export CNI_CONTAINERID=$contid
$ export CNI_NETNS=$netnspath
$ export CNI_IFNAME=eth0

The following will allow you to add network devices to the specified namespace.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

$ export CNI_COMMAND=ADD

$ jq -r '.type' /etc/cni/net.d/10-mynet.conf
bridge

$ bridge < /etc/cni/net.d/10-mynet.conf
{
    "cniVersion": "0.4.0",
    "interfaces": [
        {
            "name": "cni0",
            "mac": "e6:4b:0e:c8:52:d0"
        },
        {
            "name": "vethb56e47e8",
            "mac": "de:aa:02:3b:58:a3"
        },
        {
            "name": "eth0",
            "mac": "36:a4:28:8a:da:e0",
            "sandbox": "/var/run/netns/ctr-1"
        }
    ],
    "ips": [
        {
            "version": "4",
            "interface": 2,
            "address": "172.19.0.12/24",
            "gateway": "172.19.0.1"
        }
    ],
    "routes": [
        {
            "dst": "0.0.0.0/0"
        }
    ],
    "dns": {}
}

Here our network type is bridge, so the called binary is also bridge. From the standard output of the bridge command, we can also see the information about the newly created interface, the IP address assigned to it by the bridge/host-local plugin is 172.19.0.12.

Let’s also verify this as follows.

1
2
3
4
5
6
7
8
9

$ ip netns exec $contid ip addr show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0@if6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 36:a4:28:8a:da:e0 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.19.0.12/24 brd 172.19.0.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::34a4:28ff:fe8a:dae0/64 scope link
       valid_lft forever preferred_lft forever

Deleting a network interface

Moving on from the above, let’s continue to see how to delete the network interface you just created.

1
2
3
4
5


$ export CNI_COMMAND=DEL
$ bridge < /etc/cni/net.d/10-mynet.conf
$ echo $?
0

If the deletion is successful, there will be nothing in the standard output, and we can determine if it was successful from the status code returned.

Again, to verify that the interface has been successfully deleted from the specified network namespace.

1
2
3

$ ip netns exec $contid ip addr show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

Don’t forget to delete the network namespace you just created when you’re done testing.

1

$ ip netns delete $contid