VIAVI TestCenter: PX3-QSFP28-12 (4 x 25G mode) - generator continues to run after 30 sec duration leading to "Port connection will be lost."

Issue: • After running through all the IPv4 frame lengths, and all the IPv6 fixed frame lengths, on the IPv6 IMIX frame length iteration, on "line 16" below in the "test sequence," we see in client-bll.log, that not all the generators have stopped after the 30 second duration. Eventually, waiting for the generators to stop, it timesout and the ports lose connection. • Here are snippets from client-bll.log           Traffic duration is set to 30 seconds                2022-02-18T23:26:59.406Z INFO  4152219328 - user.scripting       - stc::config generatorconfig20 -SchedulingMode RATE_BASED -LoadMode FIXED                                -LoadUnit PERCENT_LINE_RATE -FixedLoad 10 -RandomMinLoad 10 -RandomMaxLoad 100 -DurationMode SECONDS -Duration 30                                -BurstSize 1 -TimeStampLatchMode START_OF_FRAME                2022-02-18T23:26:59.407Z INFO  4152219328 - user.scripting       - stc::apply                     Generators are started                               2022-02-18T23:26:59.418Z INFO  4152219328 - user.scripting       - stc::perform GeneratorStart -GeneratorList {generator1                                      generator2 generator3 generator4 generator5 generator6 generator7 generator8 generator9 generator10 generator11                                      generator12 generator13 generator14 generator15 generator16 generator17 generator18 generator19 generator20}                                                                                 After 60 seconds, a manual stop is performed by the script                               2022-02-18T23:28:05.215Z INFO  4152219328 - user.scripting       - stc::perform GeneratorStop -GeneratorList {generator1                                      generator2 generator3 generator4 generator5 generator6 generator7 generator8 generator9 generator10 generator11                                      generator12 generator13 generator14 generator15 generator16 generator17 generator18 generator19 generator20}                               2022-02-18T23:30:55.417Z INFO  4080008048 - user.port    - Port //3/1: link status changed to Down (remote fault).                                      Then after over 10 minutes, the wait for generator to stop times out and port connection loss is imminent.                               2022-02-18T23:38:06.570Z ERROR 4080008048 - user.rpc  - Timed out waiting for Generator_3.Control response after 600                                    seconds. Port connection will be lost. • this occurs on every time this test is run Other sypmtoms: • Once the system got into this state, we noticed that: • when we tried to Connect or Connect Viewer to the Session Manager, the GUI would hang on the splash screen at "Starting UI..." • As a result, we had to open another GUI: • and saw that the port reservation showed that all the ports were stuck in the "Unavailable" state • and used it to collect the chassis and port IL logs and diagnostic report via this other GUI instance. • The only way to get the ports working again is to reboot the module  

N/A