[Enterprise SONiC] Warm Reboot & Fast Reboot Follow

Example:

1. Warm Reboot
2. Fast Reboot

Tested model & firmware version:

• Switch model name:

       AS7326-56X

• Edgecore SONiC version: 

202012.1

202012.2

Example 1: Warm Reboot

Note: Warm reboot is to be able in-service restart and upgrade SONiC software without impacting the data plane.

https://github.com/Azure/SONiC/blob/master/doc/warm-reboot/SONiC_Warmboot.md#cold-restart-fallback

Note: /etc/sonic/config_db.json won't be reloaded.

Procedure :

Step 1. Upgrade SONiC software. (Refer to this article)

admin@sonic:~$ sudo sonic-installer list
Current: SONiC-OS-Edgecore-SONiC_20210917_063104_ec202012_172
Next: SONiC-OS-Edgecore-SONiC_20210917_063104_ec202012_172
Available: 
SONiC-OS-Edgecore-SONiC_20210917_063104_ec202012_172
admin@sonic:~$ sudo sonic-installer install http://188.188.99.1/ecsonic/Edgecore-SONiC_20211125_074752_ec202012_227.bin -y 
admin@sonic:~$ sudo sonic-installer list
Current: SONiC-OS-Edgecore-SONiC_20210917_063104_ec202012_172
Next: SONiC-OS-Edgecore-SONiC_20211125_074752_ec202012_227
Available: 
SONiC-OS-Edgecore-SONiC_20211125_074752_ec202012_227
SONiC-OS-Edgecore-SONiC_20210917_063104_ec202012_172

Step 2. Warm-reboot to apply the new software version without impacting the data plane. 

admin@sonic:~$ sudo warm-reboot
Job for swss.service canceled.
Job for database.service canceled.
[ 234.127524] kexec_core: Starting new kernel
[ 6.711843] rc.local[544]: + cat+ /etc/sonic/sonic_version.yml
[ 6.718999] rc.local[544]: grep build_version
[ 6.726133] rc.local[544]: + sed -e s/build_version: //g;s/'//g
[ 6.743481] rc.local[544]: + SONIC_VERSION=Edgecore-SONiC_20211125_074752_ec202012_227
[ 6.762887] rc.local[544]: + FIRST_BOOT_FILE=/host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/firsttime
[ 6.782857] rc.local[544]: + SONIC_CONFIG_DIR=/host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config
[ 6.802998] rc.local[544]: + SONIC_ENV_FILE=/host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment
[ 6.826904] rc.local[544]: + [ -d /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config -a -f /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment ]
[ 6.858887] rc.local[544]: + echo moving file /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment to /etc/sonic
[ 6.882908] rc.local[544]: moving file /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment to /etc/sonic
[ 6.906893] rc.local[544]: + mv /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment /etc/sonic
[ 6.933846] rc.local[544]: + logger SONiC version Edgecore-SONiC_20211125_074752_ec202012_227 starting up...
[ 6.954905] rc.local[544]: + grub_installation_needed=
[ 6.970936] rc.local[544]: + [ ! -e /host/machine.conf ]
[ 6.987273] rc.local[544]: + migrate_nos_configuration
[ 7.002909] rc.local[544]: + rm -rf /host/migration
[ 7.018929] rc.local[544]: + mkdir -p /host/migration
[ 7.035407] kdump-tools[553]: /etc/init.d/kdump-tools: 117: /etc/default/kdump-tools: KDUMP_CMDLINE_APPEND+= panic=10 debug hpet=disable pcie_port=compat pci=nommconf sonic_platform=__PLATFORM__: not found
[ 7.067783] rc.local[544]: + cat /proc/cmdline
[ 7.083778] rc.local[544]: + set -- BOOT_IMAGE=/image-Edgecore-SONiC_20211125_074752_ec202012_227/boot/vmlinuz-4.19.0-12-2-amd64 root=UUID=ff252179-4724-4612-8428-a91e76f553af rw console=tty0 console=ttyS0,115200n8 quiet net.ifnames=0 biosdevname=0 loop=image-Edgecore-SONiC_20211125_074752_ec202012_227/fs.squashfs loopfstype=squashfs apparmor=1 security=apparmor varlog_size=4096 usbcore.autosuspend=-1 pcie_aspm=off tg3.short_preamble=1 tg3.bcm5718s_reset=1 SONIC_BOOT_TYPE=warm SONIC_REBOOT_TIME="Tue 15 Feb 2022 10:56:08 AM UTC"
[ 7.150903] rc.local[544]: + [ -n ]
[ 7.162887] rc.local[544]: + . /host/machine.conf
[ 7.178853] rc.local[544]: + onie_arch=x86_64
[ 7.194888] rc.local[544]: + onie_bin=
[ 7.206936] rc.local[544]: + onie_boot_reason=install
[ 7.222910] rc.local[544]: + onie_build_date=2020-03-25T09:57+0800
[ 7.238956] rc.local[544]: + onie_build_machine=accton_as7326_56x
[ 7.254910] rc.local[544]: + onie_build_platform=x86_64-accton_as7326_56x-r0
[ 7.274881] rc.local[544]: + onie_config_version=1
[ 7.294940] rc.local[544]: + onie_dev=/dev/sda2
[ 7.310904] rc.local[544]: + onie_exec_url=file://dev/sdb1/onie-installer.bin
[ 7.330863] rc.local[544]: + onie_firmware=auto
[ 7.347140] rc.local[544]: + onie_grub_image_name=grubx64.efi
[ 7.362880] rc.local[544]: + onie_initrd_tmp=/
[ 7.378879] rc.local[544]: + onie_installer=/var/tmp/installer
[ 7.398901] rc.local[544]: + onie_kernel_version=4.9.95
[ 7.414894] rc.local[544]: + onie_local_parts=
[ 7.430877] rc.local[544]: + onie_machine=accton_as7326_56x
[ 7.446906] rc.local[544]: + onie_machine_rev=0
[ 7.462916] rc.local[544]: + onie_neighs=188.188.1.1,188.188.188.190,[fe80::7151:81d7:3e27:d939-eth0],[fe80::210:74ff:fe48:124c-eth0],[fe80::b848:69ff:fe6c:e385-eth0],[fe80::260:f3ff:fe20:1-eth0],[fe80::200:ff:fe00:0-eth0],[fe80::7acd:8eff:feaf:268e-eth0],[fe80::7272:cfff:fe12:345a-eth0],[fe80::3e2c:99ff:fe04:66cb-eth0],[fe80::3e2c:99ff:feec:596b-eth0],[fe80::302c:99ff:fee3:29e9-eth0],[fe80::1089:9d1d:e518:85a-eth0],[fe80::52e5:49ff:fea2:82e2-eth0],[fe80::103b:5a45:f179:3e76-eth0],[fe80::1eea:bff:fe9a:71f5-eth0],[fe80::7cbe:9dff:fe08:650a-eth0],[fe80::4070:63f3:87c5:8970-eth0],[fe80::21a:c5ff:fe01:3158-eth0],[fe80::5c53:15ff:fe21:790-eth0],[fe80::302c:99ff:fee3:29e8-eth0],[fe80::6f8:f8ff:fe7a:a40f-eth0],[fe80::6867:adff:fe06:d1df-eth0],[fe80::183d:aeff:fe2d:356f-eth0],[fe80::6a21:5fff:fe28:f112-eth0],
[ 7.550916] rc.local[544]: + onie_partition_type=gpt
[ 7.567016] rc.local[544]: + onie_platform=x86_64-accton_as7326_56x-r0
[ 7.582891] rc.local[544]: + onie_root_dir=/mnt/onie-boot/onie
[ 7.598973] rc.local[544]: + onie_skip_ethmgmt_macs=no
[ 7.615001] rc.local[544]: + onie_switch_asic=bcm
[ 7.630905] rc.local[544]: + onie_uefi_arch=x64
[ 7.647024] rc.local[544]: + onie_uefi_boot_loader=grubx64.efi
[ 7.668776] kdump-tools[553]: Starting kdump-tools: no crashkernel= parameter in the kernel cmdline ... failed!
[ 7.696839] rc.local[544]: + onie_vendor_id=259
[ 7.711040] rc.local[544]: + onie_version=2020.02.00.01
[ 7.727302] rc.local[544]: + program_console_speed
[ 7.746604] rc.local[544]: + cat /proc/cmdline
[ 7.766432] rc.local[544]: + grep -Eo console=ttyS[0-9]+,[0-9]+
[ 7.784323] rc.local[544]: + cut -d , -f2
[ 7.801176] rc.local[544]: + speed=115200
[ 7.814951] rc.local[544]: + [ -z 115200 ]
[ 7.826936] rc.local[544]: + CONSOLE_SPEED=115200
[ 7.843012] rc.local[544]: + sed -i s|\-\-keep\-baud .* %I| 115200 %I|g /lib/systemd/system/serial-getty@.service
[ 7.863050] rc.local[544]: + systemctl daemon-reload
[ 7.879150] rc.local[544]: + [ -f /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/firsttime ]
[ 7.899085] rc.local[544]: + echo First boot detected. Performing first boot tasks...
[ 7.919046] rc.local[544]: First boot detected. Performing first boot tasks...
[ 7.938997] rc.local[544]: + [ -n ]
[ 7.951016] rc.local[544]: + [ -n x86_64-accton_as7326_56x-r0 ]
[ 7.967049] rc.local[544]: + platform=x86_64-accton_as7326_56x-r0
[ 7.983108] rc.local[544]: + [ -d /host/old_config ]
[ 7.998833] rc.local[544]: + mv -f /host/old_config /etc/sonic/
[ 8.014879] rc.local[544]: + rm -rf /etc/sonic/old_config/old_config
[ 8.030878] rc.local[544]: + touch /tmp/pending_config_migration
[ 8.047056] rc.local[544]: + touch /tmp/notify_firstboot_to_platform
[ 8.063095] rc.local[544]: + [ ! -d /host/reboot-cause/platform ]
[ 8.079178] rc.local[544]: + [ -d /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/x86_64-accton_as7326_56x-r0 ]
[ 8.102877] rc.local[544]: + dpkg -i /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/x86_64-accton_as7326_56x-r0/sonic-platform-accton-as7326-56x_1.1_amd64.deb
[ 8.133979] rc.local[544]: Selecting previously unselected package sonic-platform-accton-as7326-56x.
[ 8.209384] rc.local[544]: (Reading database ... 30877 files and directories currently installed.)
[ 8.231206] rc.local[544]: Preparing to unpack .../sonic-platform-accton-as7326-56x_1.1_amd64.deb ...
[ 8.254958] rc.local[544]: Unpacking sonic-platform-accton-as7326-56x (1.1) ...
[ 8.424458] rc.local[544]: Setting up sonic-platform-accton-as7326-56x (1.1) ...
[ 8.649036] rc.local[544]: Created symlink /etc/systemd/system/opennsl-modules.service.wants/as7326-platform-handle_mac.service → /lib/systemd/system/as7326-platform-handle_mac.service.
[ 8.974285] rc.local[544]: Created symlink /etc/systemd/system/multi-user.target.wants/as7326-platform-init.service → /lib/systemd/system/as7326-platform-init.service.
[ 9.282448] rc.local[544]: Created symlink /etc/systemd/system/multi-user.target.wants/as7326-platform-monitor-fan.service → /lib/systemd/system/as7326-platform-monitor-fan.service.
[ 9.626047] rc.local[544]: Created symlink /etc/systemd/system/multi-user.target.wants/as7326-platform-monitor-psu.service → /lib/systemd/system/as7326-platform-monitor-psu.service.
[ 9.962805] rc.local[544]: Created symlink /etc/systemd/system/multi-user.target.wants/as7326-platform-monitor.service → /lib/systemd/system/as7326-platform-monitor.service.
[ OK ] Started Accton AS7326-56X …atform initialization service.
[ OK ] Started Accton AS7326-56X …atform Monitoring PSU service.
[ OK ] Started Accton AS7326-56X Platform Monitoring service.
[ OK ] Started Accton AS7326-56X …atform Monitoring FAN service.
[ 26.846311] rc.local[544]: + sync
[ 26.905380] rc.local[544]: + [ -n x86_64-accton_as7326_56x-r0 ]
[ 26.922903] rc.local[544]: + [ -n ]
[ OK ] Started /etc/rc.local Compatibility.
[ 26.939082] rc.local[544]: + mkdir -p /var/platform
[ OK ] Started Getty on tty1.
[ 26.971322] rc.local[544]: + ebtables_config
[ OK ] Started Serial Getty on ttyS0.
[ 27.003062] rc.local[544]: + /usr/sbin/ebtables-restore
[ OK ] Reached target Login Prompts.
[ 27.035100] rc.local[544]: + /usr/sbin/ebtables -t filter --atomic-file /etc/ebtables.filter --atomic-save
[ OK ] Started Reboot cause determination service.
[ 27.071101] rc.local[544]: + sed -i -e s/__PLATFORM__/x86_64-accton_as7326_56x-r0/g /etc/default/kdump-tools
Starting Config chassis_db...
[ 27.107105] rc.local[544]: + rm -rf /sys/fs/pstore/*
[ 27.139101] rc.local[544]: + firsttime_exit
[ 27.151099] rc.local[544]: + rm -rf /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/firsttime
[ 27.171094] rc.local[544]: + exit 0
[ OK ] Started Config chassis_db.
Starting Database container...

Debian GNU/Linux 10 sonic ttyS0

sonic login: admin
Password:
Last login: Tue Feb 15 10:53:30 UTC 2022 on ttyS0
Linux sonic 4.19.0-12-2-amd64 #1 SMP Debian 4.19.152-1 (2020-10-18) x86_64
You are on
____ ___ _ _ _ ____
/ ___| / _ \| \ | (_)/ ___|
\___ \| | | | \| | | |
___) | |_| | |\ | | |___
|____/ \___/|_| \_|_|\____|

-- Software for Open Networking in the Cloud --

Unauthorized access and/or use are prohibited.
All access and/or use are subject to monitoring.

Help: http://azure.github.io/SONiC/

admin@sonic:~$ sudo sonic-installer list
Current: SONiC-OS-Edgecore-SONiC_20211125_074752_ec202012_227
Next: SONiC-OS-Edgecore-SONiC_20211125_074752_ec202012_227
Available:
SONiC-OS-Edgecore-SONiC_20211125_074752_ec202012_227
SONiC-OS-Edgecore-SONiC_20210917_063104_ec202012_172

Example 2: Fast Reboot

Note: Fast-reboot is for updating the control plane, it could let the switch reboot up quickly (<= 25 seconds), so it impacts the data plane with minimum disruption.

https://github.com/Azure/SONiC/wiki/Fast-Reboot

Restriction:

Since the fast-reboot may take 25 seconds to boot the system, there are some restrictions.

• Fast-Reboot should use stale FIB (forwarding information base) information while control plane reboots
• Fast-Reboot must support at least 2000 hosts connected to SONiC VLAN interfaces
• Fast-Reboot must support at least 6000 ipv4 BGP routes and 3000 ipv6 /64 BGP routes
• LACP mode must be in SLOW mode for all LAG interfaces on a SONiC device

Procedure :

admin@sonic:~$ sudo fast-reboot
Job for database.service canceled.
[ 296.948112] kexec_core: Starting new kernel
[ 6.423929] rc.local[547]: + cat /etc/sonic/sonic_version.yml
[ 6.431287] rc.local[547]: + grep build_version
[ 6.437660] rc.local[547]: + sed -e s/build_version: //g;s/'//g
[ 6.453903] rc.local[547]: + SONIC_VERSION=Edgecore-SONiC_20211125_074752_ec202012_227
[ 6.473314] rc.local[547]: + FIRST_BOOT_FILE=/host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/firsttime
[ 6.493438] rc.local[547]: + SONIC_CONFIG_DIR=/host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config
[ 6.513298] rc.local[547]: + SONIC_ENV_FILE=/host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment
[ 6.537356] rc.local[547]: + [ -d /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config -a -f /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/sonic-config/sonic-environment ]
[ 6.562712] rc.local[547]: + logger SONiC version Edgecore-SONiC_20211125_074752_ec202012_227 starting up...
[ 6.577633] kdump-tools[530]: /etc/init.d/kdump-tools: 117: /etc/default/kdump-tools: KDUMP_CMDLINE_APPEND+= panic=10 debug hpet=disable pcie_port=compat pci=nommconf sonic_platform=x86_64-accton_as7326_56x-r0: not found
[ 6.611514] rc.local[547]: + grub_installation_needed=
[ 6.625347] rc.local[547]: + [ ! -e /host/machine.conf ]
[ 6.633306] rc.local[547]: + migrate_nos_configuration
[ 6.649303] rc.local[547]: + rm -rf /host/migration
[ 6.665382] rc.local[547]: + mkdir -p /host/migration
[ 6.672334] rc.local[547]: + cat /proc/cmdline
[ 6.685673] rc.local[547]: + set -- BOOT_IMAGE=/image-Edgecore-SONiC_20211125_074752_ec202012_227/boot/vmlinuz-4.19.0-12-2-amd64 root=UUID=ff252179-4724-4612-8428-a91e76f553af rw console=tty0 console=ttyS0,115200n8 quiet net.ifnames=0 biosdevname=0 loop=image-Edgecore-SONiC_20211125_074752_ec202012_227/fs.squashfs loopfstype=squashfs apparmor=1 security=apparmor varlog_size=4096 usbcore.autosuspend=-1 pcie_aspm=off tg3.short_preamble=1 tg3.bcm5718s_reset=1 SONIC_BOOT_TYPE=fast-reboot SONIC_REBOOT_TIME="Tue 15 Feb 2022 11:01:12 AM UTC"
[ 6.749325] rc.local[547]: + sonic_fast_reboot=true
[ 6.765344] rc.local[547]: + [ -n ]
[ 6.777285] rc.local[547]: + . /host/machine.conf
[ 6.793306] rc.local[547]: + onie_arch=x86_64
[ 6.809298] rc.local[547]: + onie_bin=
[ 6.821316] rc.local[547]: + onie_boot_reason=install
[ 6.837307] rc.local[547]: + onie_build_date=2020-03-25T09:57+0800
[ 6.853360] rc.local[547]: + onie_build_machine=accton_as7326_56x
[ 6.869313] rc.local[547]: + onie_build_platform=x86_64-accton_as7326_56x-r0
[ 6.885341] rc.local[547]: + onie_config_version=1
[ 6.904988] rc.local[547]: + onie_dev=/dev/sda2
[ 6.921328] rc.local[547]: + onie_exec_url=file://dev/sdb1/onie-installer.bin
[ 6.937309] rc.local[547]: + onie_firmware=auto
[ 6.957301] rc.local[547]: + onie_grub_image_name=grubx64.efi
[ 6.973247] rc.local[547]: + onie_initrd_tmp=/
[ 6.989395] rc.local[547]: + onie_installer=/var/tmp/installer
[ 7.005326] rc.local[547]: + onie_kernel_version=4.9.95
[ 7.021315] rc.local[547]: + onie_local_parts=
[ 7.037267] rc.local[547]: + onie_machine=accton_as7326_56x
[ 7.053367] rc.local[547]: + onie_machine_rev=0
[ 7.069375] rc.local[547]: + onie_neighs=188.188.1.1,188.188.188.190,[fe80::7151:81d7:3e27:d939-eth0],[fe80::210:74ff:fe48:124c-eth0],[fe80::b848:69ff:fe6c:e385-eth0],[fe80::260:f3ff:fe20:1-eth0],[fe80::200:ff:fe00:0-eth0],[fe80::7acd:8eff:feaf:268e-eth0],[fe80::7272:cfff:fe12:345a-eth0],[fe80::3e2c:99ff:fe04:66cb-eth0],[fe80::3e2c:99ff:feec:596b-eth0],[fe80::302c:99ff:fee3:29e9-eth0],[fe80::1089:9d1d:e518:85a-eth0],[fe80::52e5:49ff:fea2:82e2-eth0],[fe80::103b:5a45:f179:3e76-eth0],[fe80::1eea:bff:fe9a:71f5-eth0],[fe80::7cbe:9dff:fe08:650a-eth0],[fe80::4070:63f3:87c5:8970-eth0],[fe80::21a:c5ff:fe01:3158-eth0],[fe80::5c53:15ff:fe21:790-eth0],[fe80::302c:99ff:fee3:29e8-eth0],[fe80::6f8:f8ff:fe7a:a40f-eth0],[fe80::6867:adff:fe06:d1df-eth0],[fe80::183d:aeff:fe2d:356f-eth0],[fe80::6a21:5fff:fe28:f112-eth0],
[ 7.157328] rc.local[547]: + onie_partition_type=gpt
[ 7.177321] rc.local[547]: + onie_platform=x86_64-accton_as7326_56x-r0
[ 7.193276] rc.local[547]: + onie_root_dir=/mnt/onie-boot/onie
[ 7.209395] rc.local[547]: + onie_skip_ethmgmt_macs=no
[ 7.225299] rc.local[547]: + onie_switch_asic=bcm
[ 7.245327] rc.local[547]: + onie_uefi_arch=x64
[ 7.264429] rc.local[547]: + onie_uefi_boot_loader=grubx64.efi
[ 7.285343] rc.local[547]: + onie_vendor_id=259
[ 7.301303] rc.local[547]: + onie_version=2020.02.00.01
[ 7.321300] rc.local[547]: + program_console_speed
[ 7.333013] rc.local[547]: + grep -Eo console=ttyS[0-9]+,[0-9]+
[ 7.353177] rc.local[547]: + cat /proc/cmdline
[ 7.372527] rc.local[547]: + cut -d , -f2
[ 7.390739] rc.local[547]: + speed=115200
[ 7.405296] rc.local[547]: + [ -z 115200 ]
[ 7.417337] rc.local[547]: + CONSOLE_SPEED=115200
[ 7.433765] rc.local[547]: + sed -i s|\-\-keep\-baud .* %I| 115200 %I|g /lib/systemd/system/serial-getty@.service
[ 7.453379] rc.local[547]: + systemctl daemon-reload
[ 7.469300] rc.local[547]: + [ -f /host/image-Edgecore-SONiC_20211125_074752_ec202012_227/platform/firsttime ]
[ 7.489351] rc.local[547]: + [ -f /var/log/fsck.log.gz ]
[ 7.506484] rc.local[547]: + gunzip -d -c /var/log/fsck.log.gz
[ 7.521729] rc.local[547]: + logger -t FSCK
[ 7.534221] rc.local[547]: + rm -f /var/log/fsck.log.gz
[ 7.549327] rc.local[547]: + exit 0

Debian GNU/Linux 10 sonic ttyS0

sonic login: