[Wedge100BF-series] FAN Follow

Purpose:

This is the troubleshooting guide for the Fan on Wedge100BF-series.

Model:

Wedge100BF-32X/65X/32QS

Procedure:

Step 0. Identify the FAN number

For Wedge100BF-32X/32QS, it had marked the Fan number on the chassis.

For Wedge100BF-65X, please check the following picture.

Step 1. Identify the problem is on Fan or Fan slot

If the LED of Fan lights Blue means work well, Red means no insert or have some kind of problem.
Use Cross Comparison tests to judge which has the problem, Fan or Fan slot.
There are 5 Fans in one board, swap these Fans to do the Cross Comparison tests.

Note: The Cross Comparison can test when the power on, exchanges 2 of these 5 Fans.
According to the result, decide to run the RMA process with only specific Fan or switch.

If the problem is the Fan speed seems not fit the Thermal Policy, for example, normal temperature but high Fan speed(noisy),or high temperature but didn't raise up the Fan speed, please check the article to judge whether the Fan speed follow the Thermal Policy :

[Wedge100BF-series] Thermal Control Policy with Fan

Step 2. Dump the Fan speed in OpenBMC

root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 12150, 9900, (67%)
Fan 2 RPMs: 12150, 9900, (67%)
Fan 3 RPMs: 12300, 9750, (67%)
Fan 4 RPMs: 12300, 9900, (67%)
Fan 5 RPMs: 12300, 9750, (67%)

Note: If any of the Fan is defective, it's RPMs should not fit to the percentage.

Example of defective Wedge100BF-65X at Fan 6:

root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 9300, 6450, (48%)
Fan 2 RPMs: 9300, 6450, (48%)
Fan 3 RPMs: 9300, 6600, (48%)
Fan 4 RPMs: 9300, 6450, (48%)
Fan 5 RPMs: 9300, 6450, (48%)
Fan 6 RPMs: 0, 0, (100%)
Fan 7 RPMs: 16350, 16200, (100%)
Fan 8 RPMs: 16350, 16650, (100%)
Fan 9 RPMs: 16500, 16500, (100%)
Fan 10 RPMs: 16650, 16500, (100%)
root@bmc:~#

Step 3. Set Fan speed directly for test.

Method 1. Set Fan speed in BMC and check whether the Fan speed percentage fit RPMs

Command: set_fan_speed.sh [Fan speed percentage] [Fan No.]

[Note] Fan speed 30% is lowest available percentage that Fan could operate the RPMs.

For Example: 

Set Fan speed to 15% but the RPMs is actual around 30%.

root@bmc:~# set_fan_speed.sh 15 1
Successfully set fan 1 speed to 15%
root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 6000, 3900, (12%)
Fan 2 RPMs: 9300, 6600, (48%)
Fan 3 RPMs: 9150, 6450, (48%)
Fan 4 RPMs: 9300, 6450, (48%)
Fan 5 RPMs: 9300, 6450, (48%)

[Note] After user set the Fan speed manually about 20 seconds, the Thermal Policy will detect the abnormal between Fan speed and Temperature. The Fan speed will raise to 100% one moment, then return to normal status.

For Example: 

root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 9300, 6450, (48%)
Fan 2 RPMs: 9150, 6450, (48%)
Fan 3 RPMs: 9150, 6600, (48%)
Fan 4 RPMs: 9300, 6450, (48%)
Fan 5 RPMs: 9300, 6600, (48%)
root@bmc:~# set_fan_speed.sh 30 1
Successfully set fan 1 speed to 30%
root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 6150, 3900, (29%)
Fan 2 RPMs: 9300, 6600, (48%)
Fan 3 RPMs: 9300, 6600, (48%)
Fan 4 RPMs: 9300, 6450, (48%)
Fan 5 RPMs: 9300, 6450, (48%)
root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 16500, 16200, (100%)
Fan 2 RPMs: 16500, 16050, (100%)
Fan 3 RPMs: 16500, 16350, (100%)
Fan 4 RPMs: 16500, 16200, (100%)
Fan 5 RPMs: 16650, 16350, (100%)
root@bmc:~# get_fan_speed.sh
Fan 1 RPMs: 9300, 6450, (48%)
Fan 2 RPMs: 9300, 6600, (48%)
Fan 3 RPMs: 9150, 6450, (48%)
Fan 4 RPMs: 9300, 6600, (48%)
Fan 5 RPMs: 9300, 6450, (48%)

Method 2. Login Diag mode to dump the log to us.

To login Diag, please refer to the article: Install and start up Accton Diag

Choose item "72 The Fan speed test" to test.

*********************************************************
(...omitted)
72 The Fan speed test
(...omitted)
*********************************************************
Please enter your choice (0 to quit): 72
set_fan_speed.sh 1 30
set_fan_speed.sh 2 30
set_fan_speed.sh 3 30
set_fan_speed.sh 4 30
set_fan_speed.sh 5 30
(...omitted)
FAN speed test is passed(0, 8, 5, 70).

Item (72) test result: PASSED

[Note] This item tests Fans in every specific RPMs, It can be a good RMA judgement material.