Cisco UCS Multihop FCoE QoS Gotcha

See my post on configuration and migration to multihop FCoE for details on my lab setup – https://jeremywaldrop.wordpress.com/2013/04/11/cisco-ucs-fcoe-multihop-configuration-and-migration/

When I first configured UCS multihop FCoE I experienced terrible SAN performance. It was so bad that it took 20 minutes to boot a single virtual machine.

 

I didn’t have much time to troubleshoot as my co-workers needed the lab to be functional to use their test VMs. I posted the question on the Cisco UCS support community – https://supportforums.cisco.com/message/3898514

There were a few responses from folks that seemed to have the same issue so I chalked it up to a bug or something with the generation 1 hardware our lab is on.

After about a month I came to realize that this probably isn’t a bug or either nobody is implementing multihop FCoE. I asked around some more and came across some folks at Cisco that were successfully using multihop FCoE.

I did some more research and thinking about it and came to realize that this could be a QoS issue. I considered this because we typically modify the default UCS QoS system classes and create QoS polices for every traffic type. We mostly do this to place some guard rails around vMotion traffic.

Our typical QoS configuration had us using the Platinum class for VM traffic and this turned out to be the root of the problem.

By default both the Platinum and FC QoS priorities have pause no-drop enabled. This configuration worked fine when using native FC but with FCoE multihop this presented a problem because the pause no-drop was then being used on two different qos groups on the same interfaces.

As soon as I enabled no-drop on the Platinum QoS Priority in UCSM my SAN performance issues went away.

Here is a screen shot of the UCSM QoS System configuration.

image

Advertisement

4 thoughts on “Cisco UCS Multihop FCoE QoS Gotcha

  1. Hello everyone, it’s my first pay a quick visit at this web site, and paragraph is truly fruitful in favor of me, keep up posting these articles. click here

  2. Pingback: QoS and poor SAN performance in Cisco UCS :Cybergavin

  3. Hey Jeremy,
    You mentioned “By default both the Platinum and FC QoS priorities have pause no-drop enabled.” And “As soon as I enabled no-drop on the Platinum QoS Priority in UCSM my SAN performance issues went away.”
    Did you mean by default both have pause no-drop disabled? And that you enabled it afterward?

    Appreciate it!
    -Kris

  4. No, by default the Packet Drop box isn’t checked for Platinum or FC. You can’t and do not want to enable it for FC but you can for Platinum.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s