[Enterprise SONiC] VRF(Virtual routing and forwarding) Follow
Tested model & firmware version:
- Switch model name:
DCS203 (AS7362-56X)
DCS204 (AS7726-32X)
DCS201,202 (AS5835-54X(T))
- Edgecore SONiC version:
202006.4
202012.2
202111.3 ~ 202111.9
Restriction:
- The naming rule of VRF begin with "Vrf". The name of the management VRF is "mgmt".
- If you assign IP to the interface without VRF instance binding, it will use the default VRF for this routing entry.
-
When the MGMT VRF is enabled, if you want to use out-of-band management network, you need to assign MGMT VRF to run the command.
- In certain case, the VRF configuration will let the breakout command return failed, detail please refer the "[Enterprise SONiC] Dynamic Port Breakout" section.
- The directly connected routes that leaked from other VRFs will be forwarded to CPU, instead of ASIC, so it causes low performance.(This has been resolved in version 202111.10 and later.)
- Known issues:
- SONIC-8208: If there is a VRF configuration with a name that includes '-' or '_' and a user assigns the IP address to the VRF's member using FRR, the FRR configurations will be missing after saving and rebooting. Workaround: Please use SONIC command to configure the IP and VRF binding. (It is resolved in 202111.9)
- SONIC-7795: An issue where routes cannot be written to switch ASIC when the management VRF is enabled and a configuration reload is executed.(It is resolved in 202111.8)
- SONIC-6408: The issue of container crashing when adding an interface to a VLAN while the interface is bound to a VRF.(It is resolved in 202111.5)
Basic VRF
Step 1. Create VRF instance
admin@sonic:~$ sudo config vrf add Vrf_01
Caution: When you create VRF instance, it has naming rule. The head of the naming string must be "Vrf".
Step 2. Binding the Ethernet0/VLAN10 to VRF instance.
admin@sonic:~$ sudo config interface vrf bind Ethernet0 Vrf_01
admin@sonic:~$ sudo config interface vrf bind Vlan10 Vrf_01
Caution: If there's an IP address on Ethernet0 after binding the VRF, the IP address will be removed.
Step 3: Adding IP address on Ethernet0 (refer to this article)
Step 4. Checking the VRF
admin@sonic:~$ show vrf
VRF Interfaces
------ ------------
Vrf_01 Ethernet0
Vlan10
admin@sonic:~$ show ip interfaces
Interface Master IPv4 address/mask Admin/Oper BGP Neighbor Neighbor IP
----------- -------- ------------------- ------------ -------------- -------------
Ethernet0 Vrf_01 192.168.1.1/24 up/up N/A N/A
Vlan10 Vrf_01 192.168.10.1/24 up/up N/A N/A
docker0 240.127.1.1/24 up/down N/A N/A
eth0 188.188.97.31/16 up/up N/A N/A
lo 127.0.0.1/8 up/up N/A N/A
Step 5: Checking the routing table.
You may check the routing table for the VRF instance or check all of the routing tables.
admin@sonic:~$ show ip route vrf Vrf_01
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>*192.168.1.0/24 is directly connected, Ethernet0, 00:04:25
K>*192.168.1.1/32 [0/0] is directly connected, Ethernet0, 00:04:25
C>*192.168.10.0/24 is directly connected, Vlan10, 00:04:11
K>*192.168.10.1/32 [0/0] is directly connected, Vlan10, 00:04:11
admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>*192.168.1.0/24 is directly connected, Ethernet0, 00:05:26
K>*192.168.1.1/32 [0/0] is directly connected, Ethernet0, 00:05:26
C>*192.168.10.0/24 is directly connected, Vlan10, 00:05:12
K>*192.168.10.1/32 [0/0] is directly connected, Vlan10, 00:05:12
VRF default:
C>*188.188.0.0/16 is directly connected, eth0, 17:43:26
Management VRF
Step 1. Create Management VRF
admin@sonic:~$ sudo config vrf add mgmt
Step 2. Checking the Management VRF
admin@sonic:~$ show mgmt-vrf
ManagementVRF : Enabled
Management VRF interfaces in Linux:
128: mgmt: <NOARP,MASTER,UP,LOWER_UP> mtu 65536 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether 52:2f:cc:b8:28:b5 brd ff:ff:ff:ff:ff:ff promiscuity 0 minmtu 68 maxmtu 1500
vrf table 5000 addrgenmode eui64 numtxqueues 1 numrxqueues 1 gso_max_size 65536 gso_max_segs 65535
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt state UP mode DEFAULT group default qlen 1000
link/ether 80:a2:35:4f:4f:40 brd ff:ff:ff:ff:ff:ff
129: lo-m: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt state UNKNOWN mode DEFAULT group default qlen 1000
link/ether 0a:25:2e:1f:32:90 brd ff:ff:ff:ff:ff:ff
admin@sonic:~$ show ip interfaces
Interface Master IPv4 address/mask Admin/Oper BGP Neighbor Neighbor IP
----------- -------- ------------------- ------------ -------------- -------------
Ethernet0 Vrf_01 192.168.1.1/24 up/up N/A N/A
Loopback0 10.1.0.1/32 up/up N/A N/A
docker0 240.127.1.1/24 up/down N/A N/A
eth0 mgmt 188.188.97.31/16 up/up N/A N/A
lo 127.0.0.1/8 up/up N/A N/A
lo-m mgmt 127.0.0.1/8 up/up N/A N/A
Step 3: Checking the routing table.
admin@sonic:~$ show ip route vrf mgmt
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF mgmt:
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:12:12
C>* 188.188.0.0/16 is directly connected, eth0, 00:12:12
admin@sonic:~$ show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF Vrf_01:
C>* 192.168.1.0/24 is directly connected, Ethernet0, 00:01:04
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
C>* 10.1.0.1/32 is directly connected, Loopback0, 00:01:05
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
F - PBR, f - OpenFabric,
> - selected route, * - FIB route, q - queued route, r - rejected route
VRF mgmt:
K>* 0.0.0.0/0 [0/0] via 188.188.1.1, eth0, 00:01:21
C>* 188.188.0.0/16 is directly connected, eth0, 00:01:21
Caution: About the Restriction 2, you need to assign MGMT VRF to run the command.
For example:
admin@sonic:~$ ping 8.8.8.8
connect: Network is unreachable ---> If you didn't assign the MGMT VRF to run the command, it will use in-band data plane network (default VRF).
admin@sonic:~$ sudo ip vrf exec mgmt ping 8.8.8.8 -c 5
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=57 time=2.86 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=57 time=2.64 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=57 time=2.70 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=57 time=2.88 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=57 time=2.83 ms
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 10ms
rtt min/avg/max/mdev = 2.637/2.782/2.877/0.095 ms
Leaking VRF Routes
Restriction:
- The directly connected routes that leaked from other VRFs will be forwarded to CPU, instead of ASIC, so it causes low performance.(This has been resolved in version 202111.10 and later.)
Topology:
Setting:
Step 1: Enter Vty shell.
admin@sonic:~$ vtysh
Hello, this is FRRouting (version 8.1).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
sonic#
Step 2: Set the routing leaking.
sonic# configure terminal
sonic(config)# vrf Vrf200
sonic(config-vrf)# ip route 192.168.10.0/24 Vlan100 nexthop-vrf Vrf100
sonic(config-vrf)# exit
sonic(config)# vrf Vrf100
sonic(config-vrf)# ip route 192.168.20.0/24 Vlan200 nexthop-vrf Vrf200
Step 3: Check the routing table.
sonic# show ip route vrf all
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel, F - PBR,
f - OpenFabric,
> - selected route, * - FIB route, q - queued, r - rejected, b - backup
t - trapped, o - offload failure
VRF Vrf100:
C>* 192.168.10.0/24 is directly connected, Vlan100, 00:11:44
K>* 192.168.10.254/32 [0/0] is directly connected, Vlan100, 00:11:44
S>* 192.168.20.0/24 [1/0] is directly connected, Vlan200 (vrf Vrf200), weight 1, 00:00:07
VRF Vrf200:
S>* 192.168.10.0/24 [1/0] is directly connected, Vlan100 (vrf Vrf100), weight 1, 00:00:26
C>* 192.168.20.0/24 is directly connected, Vlan200, 00:11:34
K>* 192.168.20.254/32 [0/0] is directly connected, Vlan200, 00:11:34
VRF default:
C>* 188.188.0.0/16 is directly connected, eth0, 00:19:03
Ping from Host1 to Host2
root@ts:~# ping 192.168.20.1 -c 5
PING 192.168.20.1 (192.168.20.1) 56(84) bytes of data.
64 bytes from 192.168.20.1: icmp_seq=1 ttl=63 time=1.19 ms
64 bytes from 192.168.20.1: icmp_seq=2 ttl=63 time=1.47 ms
64 bytes from 192.168.20.1: icmp_seq=3 ttl=63 time=1.46 ms
64 bytes from 192.168.20.1: icmp_seq=4 ttl=63 time=1.34 ms
64 bytes from 192.168.20.1: icmp_seq=5 ttl=63 time=1.33 ms
--- 192.168.20.1 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4008ms
rtt min/avg/max/mdev = 1.186/1.359/1.472/0.104 ms
Comments
2 comments
In SONiC is it possible to bind a VRF to a VLAN/SVI? Instead of a physical port?
I ask because would it be possible to bind mgmt-vrf to a management VLAN (ie 100) instead of binding it to a port for in-band management
Hello, Johnny
It could bind the VRF to VLAN, and we also support the in-band management.
Here's the way to bind the VRF to VLAN.
And here's the way to enable the in-band management.
If you want to know the details, you may submit the ticket to our system.
Here's the link.
https://support.edge-core.com/hc/en-us/requests/new
Thank you
Please sign in to leave a comment.