[Network Virtualization] MacVLAN

Macvlan allows you to configure multiple virtual network interfaces on a single network interface of the host, these network interfaces have their own independent MAC addresses, or can be configured with IP addresses for communication. virtual machines or container networks under Macvlan are in the same network segment as the host and share the same broadcast domain. macvlan is more similar to Bridge, but because it eliminates the Bridge, it is simpler and more efficient to configure and debug. Macvlan is similar to Bridge, but it is easier and more efficient to configure and debug because it eliminates the presence of Bridge. In addition, Macvlan itself supports VLAN perfectly.

macvlan itself is a linxu kernel module whose function is to allow multiple MAC addresses to be configured on the same physical NIC, i.e. multiple interfaces, each of which can be configured with its own IP. macvlan is essentially a NIC virtualization technology (the biggest advantage is that it has excellent performance)

You can run lsmod | grep macvlan at the linux command line to see if the driver is currently loaded in the kernel; if not, you can load it by modprobe macvlan and check again. Kernel code path /drivers/net/macvlan.c

Bridge VS MACVlan

Bridge Mode

• Bridge is a Layer 2 device and is only used to handle Layer 2 communications.
• Bridge uses the MAC address table to determine how to forward frames (Frame).
• The Bridge learns the MAC address from the communication packets between hosts.
• It can be a hardware device or a pure software implementation (e.g., Linux Bridge).

Tip: Bridge may encounter Layer 2 loops, and STP can be turned on to prevent loops if needed.

MACVlan Mode

• Allows users to set multiple MAC addresses on the same physical NIC.
• The NIC with the MAC address set above is called the sub interface; the physical NIC is called the parent interface.
• The parent interface can be a physical interface (eth0), a sub-interface of 802.1q (eth0.10), or a bonding interface.
• Not only the MAC address can be set on the parent/sub interface, but the IP address can also be set.
• The sub interface cannot communicate directly with the parent interface (a VM or container with a sub interface cannot communicate directly with the host).
• If the VM or container needs to communicate with the host, then an additional sub interface must be created for the host to use.
• The sub interface is usually named as mac0@eth0 for ease of differentiation.

Working Mode

Bridge Mode

The macvlan interfaces belonging to the same parent interface are hooked up to the same bridge and can interoperate on layer 2 (none of the macvlan interfaces can interoperate with the parent interface)

VPEA Mode

Virtual Ethernet Port Aggregator, all interface traffic needs to go to the external switch to reach other interfaces

Private Mode

The interface only accepts messages sent to its own MAC address.

Passthru Mode

The parent interface is bundled with the corresponding MacVLAN interface. This mode allows each parent interface to be bundled with only one Macvlan virtual NIC interface, and the Macvlan virtual NIC interface inherits the MAC address of the parent interface.

Practices

The following experiment creates two macvlan interfaces and puts them into two netns; then verifies that the clients interoperate between these two macvlan ports.

First, use bridge mode to create two macvlan interfaces, whose parent interfaces are both eth0.

1
2

$ sudo ip link add link eth0 name macv1 type macvlan mode bridge
$ sudo ip link add link eth0 name macv2 type macvlan mode bridge

Check the results of the creation and note that each interface has its own mac address.

1
2
3
4
5
6
7

$ sudo ip link
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 52:54:00:f4:3e:1d brd ff:ff:ff:ff:ff:ff
3763: macv1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:76:5f:92:85:8d brd ff:ff:ff:ff:ff:ff
3764: macv2@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 66:90:ba:08:bd:81 brd ff:ff:ff:ff:ff:ff

Create namespace

1
2

$ sudo ip netns add net1
$ sudo ip netns add net2

Insert the macvlan interface into namespace

1
2

$ sudo ip link set macv1 netns net1
$ sudo ip link set macv2 netns net2

Set NIC IP, set NIC UP status

1
2
3
4
5

$ sudo ip netns exec net1 ip addr add 52.1.1.151/24 dev macv1
$ sudo ip netns exec net2 ip addr add 52.1.1.152/24 dev macv2

$ sudo ip netns exec net1 ip link set macv1 up
$ sudo ip netns exec net2 ip link set macv2 up

Check the status of your network card.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

$ sudo ip netns exec net1 ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3763: macv1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:76:5f:92:85:8d brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 52.1.1.151/24 scope global macv1
       valid_lft forever preferred_lft forever
$ sudo ip netns exec net2 ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
3764: macv2@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 66:90:ba:08:bd:81 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 52.1.1.152/24 scope global macv2
       valid_lft forever preferred_lft forever

net1 and net2 can’t ping through the host, ping themselves also can’t. 2 containers ping each other without problems

found that macv1 can not ping the host interface eth0 address.

1
2
3
4
5

$ sudo ip net exec net1 ping 10.0.1.3
PING 10.0.1.3 (10.0.1.3) 56(84) bytes of data.
^C
--- 10.0.1.3 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3008ms

But it is possible to ping the macv2 address.

1
2
3
4
5

$ sudo ip net exec net1 ping 52.1.1.152
PING 52.1.1.152 (52.1.1.152) 56(84) bytes of data.
64 bytes from 52.1.1.152: icmp_seq=1 ttl=64 time=0.044 ms
64 bytes from 52.1.1.152: icmp_seq=2 ttl=64 time=0.041 ms
64 bytes from 52.1.1.152: icmp_seq=3 ttl=64 time=0.047 ms

Reference https://houmin.cc/posts/dccd6e26/