Presenting 4 vNICs to VMware ESXi 4 with the Cisco UCS VIC (Palo) Adapter

We got our own Cisco UCS lab at the office last week complete with two 6210S, two server blades and the new Cisco VIC (Palo) adapter.

Here is a diagram I copied from the Chassis 1 –> Hybrid Display tab in UCS Manager.

image

What is not in the diagram are two Cisco MDS 9124s and a Cisco 3750E with two Ten Gigabit uplinks.

After we got our first ESXi 4.1 blade up booting from a CX4-120 LUN I was itching to present more than two 10GB adapters to the ESXi host.

When I initially looked at how to add more than two vNICs to a Service Profile I couldn’t figure out how to do it. I was thinking there was some new configuration screen somewhere where you had to go to and enable the additional vNICs. I was also unable to find any good documentation on how to do it so a posted a question to the Unified Computing Cisco Support Community – https://supportforums.cisco.com/community/netpro/data-center/unified-computing?view=discussions&start=0

If you haven’t checked out this community and you are interested in Cisco UCS you should definitely browse through it. There are some good tips in there.

This is a very active community and the two times I posted a question it was answered within 12 hours.

I posted a question on how to configure the new VIC (Palo) adapter and to my surprise it is a lot easier to configure than what I initially thought.

All that I had to was add two more vNICs to my Service Profile template. I don’t know why I didn’t just try that first.

I went into my updating Service Profile Template and added eth2 to fabric A and eth3 to fabric B, here is a screen shot

Now be careful because when you add vNICs or modify the storage of a Service Profile or an updating Service Profile Template it will power cycle the blade that is bound to the template. I don’t know if there is a way to change this behavior but I think this is dangerous.

After my ESX 4i server rebooted I first checked the vmnic list using the esxcfg-nics –l command. Here is the output

~ # esxcfg-nics -l
Name    PCI           Driver      Link Speed     Duplex MAC Address       MTU    Description
vmnic0  0000:08:00.00 enic        Up   10000Mbps Full   00:25:b5:0c:95:af 1500   Cisco Systems Inc VIC Ethernet NIC
vmnic1  0000:09:00.00 enic        Up   10000Mbps Full   00:25:b5:0c:95:bf 1500   Cisco Systems Inc VIC Ethernet NIC
vmnic2  0000:0a:00.00 enic        Up   10000Mbps Full   00:25:b5:0c:95:ae 1500   Cisco Systems Inc VIC Ethernet NIC
vmnic3  0000:0b:00.00 enic        Up   10000Mbps Full   00:25:b5:0c:95:be 1500   Cisco Systems Inc VIC Ethernet NIC

AWESOME!!!

Next I logged into my vSphere Client and checked the Network Adapters and added a new vSwitch for VM traffic.

image

image

Now I can keep my Management Network (Service Console) on a standard vSwitch and put my VM networks in a Distributed vSwitch or in Nexus 1000v without worrying about loosing access because a configuration error.

7 thoughts on “Presenting 4 vNICs to VMware ESXi 4 with the Cisco UCS VIC (Palo) Adapter

  1. Pingback: Virtualization Short Take #37 - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers

  2. Pingback: Virtualization Short Take #37 | Free Techie Blog

  3. Hey
    Why are you separating management and VM traffic with virtual switches, esp with UCS? I’m trying to work out what’s wrong with having a single vDS and two vNICs, one to each fabric, and everything trunked down into it.
    Cheers

  4. For ESX hosts on UCS I have actually been presenting 6 vNICs. There are a few reasons for this; 1). QoS, I have QoS policies tied to the different vNICs, 2 for vMotion, this keeps the vMotion traffic from hogging all of the bandwidth on the FEX links from SAN and VM taffic. 2). To keep SC/MGMT and vMotion on standard vSwitches while the 2 for VM traffic on Nexus 1000v or a VMware vDS.
    3).Logical separation.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s