[Enterprise SONiC] MC-LAG Follow
A multi-chassis link aggregation group (MLAG or MC-LAG) is a type of link aggregation group (LAG) with constituent ports that terminate on separate chassis, which provide redundancy in the event one of the chassis fails.
How to configure the MCLAG(Multi-chassis link aggregation group) setting.
Tested model & firmware version:
- Switch model name:
AS7326-56X
- Edgecore SONiC version:
202006.4
202012.2
202111.0
202111.3
202111.8
Restriction
- Support LACP for member only.( If the version is above 202111.2, this restriction is removed.)
- Known issue, MC-LAG function with SAG will have arp learning flapping issue.(This issue is fixed in Edgecore-SONiC_20210505_054937_ec202006_205)
- In current version, we don't have display design on "mclagdctl dump mac" to know mac learning locally or sync from peer. We can check syslog to know mac learned locally or from remote peer.
"cc:37:ab:60:7f:9d" learned locally.
-
The keep alive link can only use TPID 0x8100. There has been no attempt to configure the LINUX Kernel to handle Q-in-Q packets or to change its default 802.1Q TPID 0x8100 to any other value.
NOTICE iccpd#iccpd: [mlacp_prepare_for_mac_info_to_peer.NOTICE] Send MAC messge to peer, port PortChannel0001 mac = cc:37:ab:60:7f:9d, vid = 99, type = add count 0
"cc:37:ab:60:7f:9a" learned from remote peer.
NOTICE iccpd#iccpd: [mlacp_fsm_update_mac_entry_from_peer.NOTICE] Received MAC Info, port[PortChannel0001] vid[99] MAC[cc:37:ab:60:7f:9a] type[add]
We could use following command to filter syslog.
admin@sonic:~$ sudo cat /var/log/syslog | grep 'mlacp.*Received\|mlacp.*Send'
- After the version of 202012.3, we can know MAC address is learning from local or sync from peer with "show mac" command.
- Since the MTU of the VLAN interface cannot be modified, changing the MTU of the keep-alive link might lead to session instability. Therefore, we recommend against changing the MTU of the keep-alive link.
- MCLAG can't work with the Spanning Tree Protocol and VRRP in the current design.
- The virtual Mac feature is enhanced after version 202111.7. Configure it to ensure that the MAC of MCLAG member of standby DUT cannot be changed during rebooting active DUT. Preventing traffic broken temporally. The user MUST set the ACTIVE's MAC address as the virtual MAC on the standby device.
Known issue: SONIC-8346 : [MC-LAG]The MAC address of the VLAN interface may change when the session state changes
MC-LAG L2 scenario configuration
Topology:
Step 1: Create port-channel on all of the switches. Please refer to LAG (Link-Aggregation) tutorial.
MC1 and MC2:
admin@sonic:~$ sudo config portchannel add PortChannel01 admin@sonic:~$ sudo config portchannel add PortChannel02 admin@sonic:~$ sudo config portchannel add PortChannel03 admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0 admin@sonic:~$ sudo config portchannel member add PortChannel02 Ethernet1 admin@sonic:~$ sudo config portchannel member add PortChannel03 Ethernet56 admin@sonic:~$ sudo config portchannel member add PortChannel03 Ethernet60
*Note: On both Active and Standby, the port channel connected to the same device must have the same name.
Step 2: Create two Vlan, one is for MCLAG keepalive another is for data forwarding. As topology, the MCLAG keepalive VLAN is Vlan 10, we should bind it on PortChannel03. Vlan100 is for Host traffic it should bind to PortChannel01 and PortChannel02.(Please refer to VLAN & Inter-VLAN Routing tutorial)
MC1 and MC2:
admin@sonic:~$ sudo config vlan add 10 admin@sonic:~$ sudo config vlan add 100 admin@sonic:~$ sudo config vlan member add 10 PortChannel03 admin@sonic:~$ sudo config vlan member add -u 100 PortChannel01 admin@sonic:~$ sudo config vlan member add 100 PortChannel02 admin@sonic:~$ sudo config vlan member add 100 PortChannel03
Step 3: Create an MC-LAG domain and set the unique-ip on Vlan10.
MC1:
admin@sonic:~$ sudo config mclag add 1 192.168.10.1 192.168.10.2 PortChannel03 admin@sonic:~$ sudo config mclag unique-ip add Vlan10 admin@sonic:~$ sudo config mclag member add 1 PortChannel01 admin@sonic:~$ sudo config mclag member add 1 PortChannel02
MC2:
admin@sonic:~$ sudo config mclag add 1 192.168.10.2 192.168.10.1 PortChannel03 admin@sonic:~$ sudo config mclag unique-ip add Vlan10 admin@sonic:~$ sudo config mclag member add 1 PortChannel01 admin@sonic:~$ sudo config mclag member add 1 PortChannel02
Notice. (Optional) After 202111.7, there is the virtual system MAC supported. We can do the configurations as the below.
MC2:
admin@sonic:~$ sudo config mclag add 1 192.168.10.1 192.168.10.2 PortChannel03
admin@sonic:~$ sudo config mclag system-mac add 1 00:a0:c9:00:00:00
admin@sonic:~$ sudo config mclag unique-ip add Vlan10
admin@sonic:~$ sudo config mclag member add 1 PortChannel01
admin@sonic:~$ sudo config mclag member add 1 PortChannel02
Step 4: Add the IP address on Vlan10.(Please refer to VLAN & Inter-VLAN Routing tutorial)
MC1:
admin@sonic:~$ sudo config interface ip add Vlan10 192.168.10.1/24
MC2:
admin@sonic:~$ sudo config interface ip add Vlan10 192.168.10.2/24
Step 5:Check MC-LAG status. (The below example of the "show mclag brief" is for 202111.0 version.)
MC1:
admin@sonic:~$ show mclag brief
Domain ID : 1
Role : Active
Session Status : Up
Peer Link Status : Up
Source Address : 192.168.10.1
Peer Address : 192.168.10.2
Peer Link : PortChannel03
Keepalive Interval : 1 secs
Session Timeout : 15 secs
System MAC : 00:a0:c9:00:00:00
Number of MCLAG Interfaces : 2
MCLAG Interface Local/Remote Status
PortChannel01 Up/Up
PortChannel02 Up/Up
MC2:
admin@sonic:~$ show mclag brief
Domain ID : 1
Role : Standby
Session Status : Up
Peer Link Status : Up
Source Address : 192.168.10.2
Peer Address : 192.168.10.1
Peer Link : PortChannel03
Keepalive Interval : 1 secs
Session Timeout : 15 secs
System MAC : 00:a0:c9:00:00:00
Number of MCLAG Interfaces : 2
MCLAG Interface Local/Remote Status
PortChannel01 Up/Up
PortChannel02 Up/Up
Note. After MCLAG is up, the system MAC of LAG will be set as role of active.
If the user configure the virtual system MAC settings, the information will be the same as below.
MC1:
admin@sonic:~$show mclag brief
Domain ID : 1
Role : Active
Session Status : Up
Peer Link Status : Up
Source Address : 192.168.10.1
Peer Address : 192.168.10.2
Peer Link : PortChannel03
Keepalive Interval : 1 secs
Session Timeout : 15 secs
System MAC : 00:a0:c9:00:00:00
MCLAG System MAC : 00:a0:c9:00:00:00
Number of MCLAG Interfaces : 2
MCLAG Interface Local/Remote Status
PortChannel01 Up/Up
PortChannel02 Up/Up
MC2:
admin@sonic:~$show mclag brief
Domain ID : 1
Role : Standby
Session Status : Up
Peer Link Status : Up
Source Address : 192.168.10.2
Peer Address : 192.168.10.1
Peer Link : PortChannel03
Keepalive Interval : 1 secs
Session Timeout : 15 secs
System MAC : 00:a0:c9:00:00:00
MCLAG System MAC : 00:a0:c9:00:00:00
Number of MCLAG Interfaces : 2
MCLAG Interface Local/Remote Status
PortChannel01 Up/Up
PortChannel02 Up/Up
Step 6.Check mac learned.
Before the version of 202012.2, since Restriction 3, it cannot know MAC address is learning from local or sync from peer.
MC1:
admin@sonic:~$ show mac No. Vlan MacAddress Port Type ----- ------ ----------------- ------------- ------- 1 10 04:F8:F8:6B:06:91 PortChannel03 Dynamic 2 100 04:F8:F8:6B:06:91 PortChannel03 Dynamic 3 100 8C:EA:1B:30:DA:51 PortChannel02 Dynamic 4 100 8C:EA:1B:30:DA:4F PortChannel01 Dynamic Total number of entries 4 admin@sonic:~$ mclagdctl dump mac -i 1 TYPE: S-STATIC, D-DYNAMIC; AGE: L-Local age, P-Peer age No. TYPE MAC VID DEV ORIGIN-DEV AGE 1 D 04:f8:f8:6b:06:91 10 PortChannel03 PortChannel03 L 2 D 04:f8:f8:6b:06:91 100 PortChannel03 PortChannel03 L 3 D 8c:ea:1b:30:da:4f 100 PortChannel01 PortChannel01 4 D 8c:ea:1b:30:da:51 100 PortChannel02 PortChannel02
MC2:
admin@sonic:~$ show mac No. Vlan MacAddress Port Type ----- ------ ----------------- ------------- ------- 1 100 8C:EA:1B:30:DA:4F PortChannel01 Dynamic 2 100 8C:EA:1B:30:DA:51 PortChannel02 Dynamic Total number of entries 2 admin@sonic:~$ mclagdctl dump mac -i 1 TYPE: S-STATIC, D-DYNAMIC; AGE: L-Local age, P-Peer age No. TYPE MAC VID DEV ORIGIN-DEV AGE 1 D 8c:ea:1b:30:da:51 100 PortChannel02 PortChannel02 2 D 8c:ea:1b:30:da:4f 100 PortChannel01 PortChannel01
After the version of 202012.3.
MC1:
admin@sonic:~$ show mac
No. Vlan MacAddress Port Type
----- ------ ----------------- ------------- -------
1 10 68:21:5F:29:C0:D2 PortChannel03 Static
2 100 B8:6A:97:19:BA:12 PortChannel01 Dynamic
3 100 80:A2:35:5A:22:50 PortChannel02 Dynamic
Total number of entries 3
MC2:
admin@sonic:~$ show mac
No. Vlan MacAddress Port Type
----- ------ ----------------- ------------- ------
1 10 00:A0:C9:00:00:00 PortChannel03 Static
2 100 B8:6A:97:19:BA:12 PortChannel01 Static
3 100 80:A2:35:5A:22:50 PortChannel02 Static
Total number of entries 3
Note:
Dynamic: MAC entry learns locally
Static: MAC entry learns from remote peer
MC-LAG L3 scenario configuration
Topology:
Step 1. Create port-channel on all of the switches. Please refer to LAG (Link-Aggregation) tutorial.
MC1 and MC2:
admin@sonic:~$ sudo config portchannel add PortChannel01 admin@sonic:~$ sudo config portchannel add PortChannel02 admin@sonic:~$ sudo config portchannel add PortChannel03 admin@sonic:~$ sudo config portchannel member add PortChannel01 Ethernet0 admin@sonic:~$ sudo config portchannel member add PortChannel02 Ethernet1 admin@sonic:~$ sudo config portchannel member add PortChannel03 Ethernet56 admin@sonic:~$ sudo config portchannel member add PortChannel03 Ethernet60
Step 2. Binding ip to portchannel interface.
MC1:
admin@sonic:~$ sudo config interface ip add PortChannel01 192.168.11.1/24 admin@sonic:~$ sudo config interface ip add PortChannel02 192.168.12.1/24 admin@sonic:~$ sudo config interface ip add PortChannel03 192.168.10.1/24
MC2:
admin@sonic:~$ sudo config interface ip add PortChannel01 192.168.11.1/24 admin@sonic:~$ sudo config interface ip add PortChannel02 192.168.12.1/24 admin@sonic:~$ sudo config interface ip add PortChannel03 192.168.10.2/24
Step 3. Create MCLAG domain and member binding.
MC1:
admin@sonic:~$ sudo config mclag add 1 192.168.10.1 192.168.10.2 admin@sonic:~$ sudo config mclag member add 1 PortChannel01 admin@sonic:~$ sudo config mclag member add 1 PortChannel02
MC2:
admin@sonic:~$ sudo config mclag add 1 192.168.10.2 192.168.10.1 admin@sonic:~$ sudo config mclag member add 1 PortChannel01 admin@sonic:~$ sudo config mclag member add 1 PortChannel02
Note. In the L3 scenario, peer-link configuration is unnecessary. Member ports are router ports.
(Optional) In this step, you can configure the virtual MAC on standby to match the ACTIVE's MAC.
MC2:
admin@sonic:~$ sudo config mclag add 1 192.168.10.2 192.168.10.1
admin@sonic:~$ sudo config mclag system-mac add 1 00:a0:c9:00:00:00
admin@sonic:~$ sudo config mclag member add 1 PortChannel01
admin@sonic:~$ sudo config mclag member add 1 PortChannel02
Step 4. Check MCLAG status (The below example of the "show mclag brief" is for 202111.0 version.)
MC1:
admin@sonic:~$ show interfaces portchannel
Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available,
S - selected, D - deselected, * - not synced
No. Team Dev Protocol Ports Oper Key Admin Key Fast Rate
----- ------------- ----------- --------------------------- ---------- ----------- -----------
01 PortChannel01 LACP(A)(Up) Ethernet0(S) 101 auto false
02 PortChannel02 LACP(A)(Up) Ethernet1(S) 102 auto false
03 PortChannel03 LACP(A)(Up) Ethernet60(S) Ethernet56(S) 103 auto false
admin@sonic:~$ show mclag brief
Domain ID : 1
Role : Active
Session Status : Up
Peer Link Status :
Source Address : 192.168.10.1
Peer Address : 192.168.10.2
Peer Link :
Keepalive Interval : 1 secs
Session Timeout : 15 secs
System MAC : 00:a0:c9:00:00:00
Number of MCLAG Interfaces : 2
MCLAG Interface Local/Remote Status
PortChannel01 Up/Up
PortChannel02 Up/Up
MC2:
admin@sonic:~$ show interfaces portchannel
Flags: A - active, I - inactive, Up - up, Dw - Down, N/A - not available,
S - selected, D - deselected, * - not synced
No. Team Dev Protocol Ports Oper Key Admin Key Fast Rate
----- ------------- ----------- --------------------------- ---------- ----------- -----------
01 PortChannel01 LACP(A)(Up) Ethernet0(S) 101 auto false
02 PortChannel02 LACP(A)(Up) Ethernet1(S) 102 auto false
03 PortChannel03 LACP(A)(Up) Ethernet60(S) Ethernet56(S) 103 auto false
admin@sonic:~$ show mclag brief
Domain ID : 1
Role : Standby
Session Status : Up
Peer Link Status :
Source Address : 192.168.10.2
Peer Address : 192.168.10.1
Peer Link :
Keepalive Interval : 1 secs
Session Timeout : 15 secs
System MAC : 00:a0:c9:00:00:00
Number of MCLAG Interfaces : 2
MCLAG Interface Local/Remote Status
PortChannel01 Up/Up
PortChannel02 Up/Up
Step 5. Ping from Switch#1 to Switch#2
Step 6. Check the arp sync.
MC1:
admin@sonic:~$ mclagdctl dump arp -i 1
No. IP MAC DEV Flag
1 192.168.12.2 80:a2:35:5a:22:50 PortChannel02 R
2 192.168.11.2 b8:6a:97:19:ba:12 PortChannel01 L
MC2:
admin@sonic:~$ mclagdctl dump arp -i 1
No. IP MAC DEV Flag
1 192.168.12.2 80:a2:35:5a:22:50 PortChannel02 L
2 192.168.11.2 b8:6a:97:19:ba:12 PortChannel01 R
Comments
0 comments
Please sign in to leave a comment.