[Enterprise SONiC] VRF(Virtual routing and forwarding) Follow
Tested model & firmware version:
- Switch model name:
DCS203 (AS7362-56X)
DCS204 (AS7726-32X)
DCS201,202 (AS5835-54X(T))
- Edgecore SONiC version:
202006.4
202012.2
202111.3 ~ 202111.9
Restriction:
- The naming rule of VRF begin with "Vrf". The name of the management VRF is "mgmt".
- If you assign IP to the interface without VRF instance binding, it will use the default VRF for this routing entry.
-
When the MGMT VRF is enabled, if you want to use out-of-band management network, you need to assign MGMT VRF to run the command.
- In certain case, the VRF configuration will let the breakout command return failed, detail please refer the "[Enterprise SONiC] Dynamic Port Breakout" section.
- The directly connected routes that leaked from other VRFs will be forwarded to CPU, instead of ASIC, so it causes low performance.(It is enhanced in 202111.10)
- Known issues:
- SONIC-8208: If there is a VRF configuration with a name that includes '-' or '_' and a user assigns the IP address to the VRF's member using FRR, the FRR configurations will be missing after saving and rebooting. Workaround: Please use SONIC command to configure the IP and VRF binding. (It is resolved in 202111.9)
- SONIC-7795: An issue where routes cannot be written to switch ASIC when the management VRF is enabled and a configuration reload is executed.(It is resolved in 202111.8)
- SONIC-6408: The issue of container crashing when adding an interface to a VLAN while the interface is bound to a VRF.(It is resolved in 202111.5)
Basic VRF
Step 1. Create VRF instance
admin@sonic:~$ sudo config vrf add Vrf_01
Caution: When you create VRF instance, it has naming rule. The head of the naming string must be "Vrf".
Step 2. Binding the Ethernet0/VLAN10 to VRF instance.
admin@sonic:~$ sudo config interface vrf bind Ethernet0 Vrf_01
admin@sonic:~$ sudo config interface vrf bind Vlan10 Vrf_01
Caution: If there's an IP address on Ethernet0 after binding the VRF, the IP address will be removed.
Step 3: Adding IP address on Ethernet0 (refer to this article)
Step 4. Checking the VRF
admin@sonic:~$ show vrf
VRF Interfaces
------ ------------
Vrf_01 Ethernet0
Vlan10
admin@sonic:~$ show ip interfaces
Interface Master IPv4 address/mask Admin/Oper BGP Neighbor Neighbor IP
----------- -------- ------------------- ------------ -------------- -------------
Ethernet0 Vrf_01 192.168.1.1/24 up/up N/A N/A
Vlan10 Vrf_01 192.168.10.1/24 up/up N/A N/A
docker0 240.127.1.1/24 up/down N/A N/A
eth0 188.188.97.31/16 up/up N/A N/A
lo 127.0.0.1/8 up/up N/A N/A
Step 5: Checking the routing table.
You may check the routing table for the VRF instance or check all of the routing tables.
admin@sonic:~$ show ip route vrf Vrf_01
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>*192.168.1.0/24 is directly connected, Ethernet0, 00:04:25
K>*192.168.1.1/32 [0/0] is directly connected, Ethernet0, 00:04:25
C>*192.168.10.0/24 is directly connected, Vlan10, 00:04:11
K>*192.168.10.1/32 [0/0] is directly connected, Vlan10, 00:04:11
admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>*192.168.1.0/24 is directly connected, Ethernet0, 00:05:26
K>*192.168.1.1/32 [0/0] is directly connected, Ethernet0, 00:05:26
C>*192.168.10.0/24 is directly connected, Vlan10, 00:05:12
K>*192.168.10.1/32 [0/0] is directly connected, Vlan10, 00:05:12
VRF default:
C>*188.188.0.0/16 is directly connected, eth0, 17:43:26
Management VRF
Step 1. Create Management VRF
admin@sonic:~$ sudo config vrf add mgmt
Step 2. Checking the Management VRF
admin@sonic:~$ show mgmt-vrf
ManagementVRF : Enabled
Management VRF interfaces in Linux:
128: mgmt: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 52:2f:cc:b8:28:b5 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500
vrf table 5000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt state UP mode DEFAULT group default qlen 1000
link/ether 80:a2:35:4f:4f:40 brd ff:ff:ff:ff:ff:ff
129: lo-m: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 0a:25:2e:1f:32:90 brd ff:ff:ff:ff:ff:ff
admin@sonic:~$ show ip interfaces
Interface Master IPv4 address/mask Admin/Oper BGP Neighbor Neighbor IP
----------- -------- ------------------- ------------ -------------- -------------
Ethernet0 Vrf_01 192.168.1.1/24 up/up N/A N/A
Loopback0 10.1.0.1/32 up/up N/A N/A
docker0 240.127.1.1/24 up/down N/A N/A
eth0 mgmt 188.188.97.31/16 up/up N/A N/A
lo 127.0.0.1/8 up/up N/A N/A
lo-m mgmt 127.0.0.1/8 up/up N/A N/A
Step 3: Checking the routing table.
admin@sonic:~$ show ip route vrf mgmt
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF mgmt:
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:12:12
C>* 188.188.0.0/16 is directly connected, eth0, 00:12:12
admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>* 192.168.1.0/24 is directly connected, Ethernet0, 00:01:04
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* 10.1.0.1/32 is directly connected, Loopback0, 00:01:05
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF mgmt:
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:01:21
C>* 188.188.0.0/16 is directly connected, eth0, 00:01:21
Caution: About the Restriction 2, you need to assign MGMT VRF to run the command.
For example:
admin@sonic:~$ ping 8.8.8.8
connect: Network is unreachable ---> If you didn't assign the MGMT VRF to run the command, it will use in-band data plane network (default VRF).
admin@sonic:~$ sudo ip vrf exec mgmt ping 8.8.8.8 -c 5
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=2.86 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=2.64 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=57 time=2.70 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=57 time=2.88 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=57 time=2.83 ms
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 10ms
rtt min/avg/max/mdev = 2.637/2.782/2.877/0.095 ms
Leaking VRF Routes
Restriction:
- The directly connected routes that leaked from other VRFs will be forwarded to CPU, instead of ASIC, so it causes low performance.
Topology:
Setting:
Step 1: Enter Vty shell.
admin@sonic:~$ vtysh
Hello, this is FRRouting (version 8.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
sonic#
Step 2: Set the routing leaking.
sonic# configure terminal
sonic(config)# vrf Vrf200
sonic(config-vrf)# ip route 192.168.10.0/24 Vlan100 nexthop-vrf Vrf100
sonic(config-vrf)# exit
sonic(config)# vrf Vrf100
sonic(config-vrf)# ip route 192.168.20.0/24 Vlan200 nexthop-vrf Vrf200
Step 3: Check the routing table.
sonic# show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF Vrf100:
C>* 192.168.10.0/24 is directly connected, Vlan100, 00:11:44
K>* 192.168.10.254/32 [0/0] is directly connected, Vlan100, 00:11:44
S>* 192.168.20.0/24 [1/0] is directly connected, Vlan200 (vrf Vrf200), weight 1, 00:00:07
VRF Vrf200:
S>* 192.168.10.0/24 [1/0] is directly connected, Vlan100 (vrf Vrf100), weight 1, 00:00:26
C>* 192.168.20.0/24 is directly connected, Vlan200, 00:11:34
K>* 192.168.20.254/32 [0/0] is directly connected, Vlan200, 00:11:34
VRF default:
C>* 188.188.0.0/16 is directly connected, eth0, 00:19:03
Ping from Host1 to Host2
root@ts:~# ping 192.168.20.1 -c 5
PING 192.168.20.1 (192.168.20.1) 56(84) bytes of data.
64 bytes from 192.168.20.1: icmp_seq=1 ttl=63 time=1.19 ms
64 bytes from 192.168.20.1: icmp_seq=2 ttl=63 time=1.47 ms
64 bytes from 192.168.20.1: icmp_seq=3 ttl=63 time=1.46 ms
64 bytes from 192.168.20.1: icmp_seq=4 ttl=63 time=1.34 ms
64 bytes from 192.168.20.1: icmp_seq=5 ttl=63 time=1.33 ms
--- 192.168.20.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4008ms
rtt min/avg/max/mdev = 1.186/1.359/1.472/0.104 ms
Comments
2 comments
In SONiC is it possible to bind a VRF to a VLAN/SVI? Instead of a physical port?
I ask because would it be possible to bind mgmt-vrf to a management VLAN (ie 100) instead of binding it to a port for in-band management
Hello, Johnny
It could bind the VRF to VLAN, and we also support the in-band management.
Here's the way to bind the VRF to VLAN.
And here's the way to enable the in-band management.
If you want to know the details, you may submit the ticket to our system.
Here's the link.
https://support.edge-core.com/hc/en-us/requests/new
Thank you
Please sign in to leave a comment.