Interesting MST Troubleshooting
Posted by ltlnetworker on January 23, 2011
I’ve experienced a strange problem on my desk with two switches. I disconnected the uplink to the company network then the two switches lost connectivity with each other. Even if it was December 31th I felt I must find out what was happening.
I was working on two laptops, either connected to a switch*. One of the two switches had an uplink to the company network. I used the following ports on the Cat2960G:
G0/12 laptop
G0/20 uplink to the company network
G0/21 other switch (Catalyst Express 500)
Everything is in VLAN1 but there is a VLAN2 configured so the inter-switch link is a dot1Q trunk.
When I disconnected the cable from G0/20 for the time of quick traffic test between the laptops, they lost each other. No pings any more.
Cat2960G# sh int status
Port Name Status Vlan Duplex Speed Type
Gi0/12 connected 1 a-full a-1000 10/100/1000BaseTX
Gi0/20 notconnect 1 auto auto 10/100/1000BaseTX
Gi0/21 connected trunk a-full a-1000 10/100/1000BaseTX
...
Cat2960G#sh cdp nei
Capability Codes: R - Router, T - Trans Bridge, B - Source Route Bridge
S - Switch, H - Host, I - IGMP, r - Repeater, P - Phone,
D - Remote, C - CVTA, M - Two-port Mac Relay
Device ID Local Intrfce Holdtme Capability Platform Port ID
lab-ce500 Gig 0/21 153 S I WS-CE500- Gig 1
Relevant config lines:
Cat2960G#sh run | i spann
spanning-tree mode mst
spanning-tree etherchannel guard misconfig
spanning-tree extend system-id
spanning-tree pathcost method long
spanning-tree mst 0 priority 45056
spanning-tree vlan 1-4094 priority 40960
interface GigabitEthernet0/12
spanning-tree portfast
spanning-tree bpduguard enable
interface GigabitEthernet0/21
switchport mode trunk
Cat2960G#sh int g0/12 21
GigabitEthernet0/21 is up, line protocol is up (connected)
...
No problems so far.
Cat2960G#sh spanning-tree
MST0
Spanning tree enabled protocol mstp
Root ID Priority 32769
Address 0023.3410.ef80
Cost 20000
Port 21 (GigabitEthernet0/21)
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 45056 (priority 45056 sys-id-ext 0)
Address 0024.5020.7800
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/12 Desg FWD 20000 128.12 P2p Edge
Gi0/21 Root BKN*20000 128.21 P2p Bound(PVST) *PVST_Inc
Link is up, no error disable, but the spanning tree port state is Blocked. The CE500 switch runs PVST+. Let's scan the logs:
00:31:15: %SPANTREE-2-PVSTSIM_FAIL: Blocking root port Gi0/21: Inconsitent inferior PVST BPDU received on VLAN 2, claiming root 32770:0023.3410.ef80
00:31:16: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/20, changed state to down
00:31:16: %LINEPROTO-5-UPDOWN: Line protocol on Interface Vlan2, changed state to down
00:31:17: %LINK-3-UPDOWN: Interface GigabitEthernet0/20, changed state to down
When I connect the uplink, port G0/21 becomes operational, the laptops can ping each other:
01:27:07: %SPANTREE-2-PVSTSIM_OK: PVST Simulation inconsistency cleared on port GigabitEthernet0/21.
01:30:21: %LINK-3-UPDOWN: Interface GigabitEthernet0/20, changed state to up
01:30:22: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet0/20, changed state to up
The Cisco Error Message Decoder has an explanation for this:
When a PVST+ switch is connected to an MST switch, the IST root (MSTOO) becomes the root for all PVST+ spanning trees. Looping occurs if any PVST+ spanning tree has a root with a better preference than IST. To prevent looping, the port, which is on the MST switch that receives the superior message from the PVST+ side, is blocked by root guard. When STP is converging after a new switch or a switch port is added to the topology, this condition occurs transiently. The port unblocks automatically in such cases.
Recommended Action: If the port remains blocked, identify the root bridge as reported in the message, and configure a worse priority for the VLAN spanning tree. There could be better PVST roots than the message indicates, and the port will not recover until all such roots are cleared. If you are unsure whether the roots are cleared, disable and enable the port again.
Solution1:
I went to the CE500 web interface and did not find the spanning tree settings. Luckily, there is a pseudo-CLI URL:
http://10.0.0.1/level/15/exec/-/
so I was able to set the spanning tree priority higher than that of the MST switch:
spanning-tree vlan 1-4094 priority 49152
MST0
Spanning tree enabled protocol mstp
Root ID Priority 45056
Address 0024.5020.7800
This bridge is the root
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Bridge ID Priority 45056 (priority 45056 sys-id-ext 0)
Address 0024.5020.7800
Hello Time 2 sec Max Age 20 sec Forward Delay 15 sec
Interface Role Sts Cost Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Gi0/12 Desg FWD 20000 128.12 P2p Edge
Gi0/21 Desg FWD 20000 128.21 P2p *PVST_Inc
Lessons learned:
If you have MST and PVST+ switches you should treat MST region as kind of a backbone. In other words: make sure MST switches are not placed at the edge of the PVST+ network or at least they have better priority as PVST+ switches. Be extremely cautious with connecting MST switches to a PVST+ network.
Bonus question: why didn’t I have the same problem with G0/20 company uplink? Because it is not a trunk and only VLAN1 BPDUs arrive. In that case, my desk switch’s root port is this port of course.
*A native speaker should help me find the right way to express it.
