Abnormally high amount of Transmit discards reported by Solarwinds for multiple switches

Question

I have several 3750X Cisco switches that, according to our Solarwinds NPM, are producing billions of transmit discards per day. I'm not sure why it's reporting these discards. Many of the ports on the 3750X's have 2960's connected to them and are hardcoded as trunk ports.

Solarwinds NPM version 10.3

Cisco IOS version 12.2(58)SE2

Total output drops: 29139431:

GigabitEthernet1/0/43 is up, line protocol is up (connected)
  Hardware is Gigabit Ethernet, address is XXXX (bia XXXX)
  Description: XXXX
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, media type is 10/100/1000BaseTX
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:47, output 00:00:50, output hang never
  Last clearing of "show interface" counters 1w4d
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 29139431
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 35000 bits/sec, 56 packets/sec
     51376 packets input, 9967594 bytes, 0 no buffer
     Received 51376 broadcasts (51376 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 51376 multicast, 0 pause input
     0 input packets with dribble condition detected
     115672302 packets output, 8673778028 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out

sh controllers gigabitEthernet 1/0/43 utilization:

Receive Bandwidth Percentage Utilization : 0 Transmit Bandwidth Percentage Utilization : 0

Solution

Opened a case with Cisco TAC. The issue is a software bug (CSCso81660) that states Cat3750 stack shows incorrect values for output drops on show interfaces. It's purely cosmetic and the 'show platform port-asic stats drops' command shows the correct values. Which are zero drops.

Comments

[joeqwerty]

What is the device connected to that switch port set to regarding speed and duplex? What happens if you set the switch port to speed 1000 and Full duplex? What happens if you set the switch port to Auto for speed and duplex? Output drops are an indication of a congested interface.

[Jared]

Cisco switches on both ends. Auto speed/duplex at 100Mbps full-duplex which is correct. Average bandwidth monitored on this port is only 50Kbps. This port is a trunk port but I do have other ports on this switch configured for VoIP and auto-QOS. Could the discards be related to QOS since this port doesn't have any QOS configured?

[joeqwerty]

I don't really know anything about QOS. Can you run a "sh port" command for that port and look at the TxPause counter? If there is a large volume of TxPause then it might be a QOS or flow control issue. Ethernet Pause frames (TxPause) is a congestion control method at layer 2 and might be related to your output drops. QOS and flow control can both cause Ethernet Pause frames to be generated.

[joeqwerty]

I just realized that the sh port command may only be available in CatOS. I'm not familiar with the 3750 switch, does it run CatOS or IOS?

[Jared]

It's IOS. I really don't think it's a congestion issue but more of a QOS config issue, but I don't know enough about it to troubleshoot.

[Jason Seemann]

Please share the running configuration from both sides.