Cisco and VMWare SDN: Breaking through the FUD


This post is Sponsored by – My Linked Profile

There has been a good deal of FUD spread since VMware has announced their NSX network hypervisor based on software from their $1 billion Nicira acquisition.  VMware is a large corporation but, $1 billion is a lot of money to any company and a big bet on network virtualization.  VMware and Cisco are sounding less and less like partners and more like full on competitors.  There is no question, we are on the edge of the transformation of data center network technology.  We are talking the largest of stakes for both Cisco and VMware as they fight to stay relevant in both their core technologies and are striving to expand beyond their core as their respective industries continue to mature.

For VMware, the server hypervisor is quickly becoming a commodity.  Hyper-V, Xen Serer and KVM are all for the most part good enough for a broad number of use cases.  VMware has looked to both storage and networking as areas of growth.  Specifically, for the network VMware is not looking to add network virtualization as just another feature of their server virtualization platform.  They are looking to transform networking via their NSX platform by creating an overlay on top of the existing physical network there by virtualizing the network.  VMware wants to stop nothing short of disrupting the network industry in the same fashion in which they have the 86x server industry.

Cisco for its part has held on strong to its lead on the enterprise network hardware front.  However, the market for network hardware is extremely mature.  They have looked to diversify from their core routing and switching business partially via a relationship with VMware in the x86 server market.  Cisco’s choice to enter the server market with their UCS platform has been extremely successful and created a strong partnership between them and VMware.  Network hardware is still their core business and in their core business the impact of enterprises moving to cloud based platforms is being felt.  The size and shape of network data is being changed by this movement.  Even according to Cisco themselves the enterprise and provider markets are asking for different capabilities than traditional network hardware has provided.


This places VMware and Cisco on a collision course.  VMware wants to grow its network presence via software based network virtualization resulting in the de-emphasis of the management layer provided by today’s hardware based solutions.  If the physical network is just a less intelligent pipe for the virtualized network overlay this will make the physical hardware that much more a commodity than it is today.  Eventually, the physical network will become just a dumb infrastructure (this will take a long time) and will open the competition for network hardware.  This is all true only if software virtualization can deliver on the promise of meeting the requirements of cloud providers and large enterprises.

So, other than the very future growth of two technology giants, what’s the fuss from a technology side?  Both Cisco and VMware agree that some form of SDN is needed.  The network needs to become an intelligent system that is abstracted from the traditional model of networking to provide agile deployment capability, scale, multinancy and flexibility needed in Cloud based environments.  The growing question has been which approach is the way to take.  At the risk of over simplifying the approaches, VMware takes a software first approach with extended support via virtualization aware hardware (similar to their x86 strategy).  Cisco takes a hardware first approach with software as an added component.  Cisco has been aggressive in framing the argument of why the software first or as they call it software only approach doesn’t work.  We’ll take their argument as a way to highlight the differences in the approach and shed light into what’s real and not.

Cisco’s argument in why software first isn’t the best solution centers around three primary areas.

  • Multi-Hypervisor Support
  • Scale
  • Management Overhead

We’ll examine each area individually starting with multi-hypervisor support.

Multi-Hypervisor Support

Cisco’s claim is that NSX doesn’t support multiple hypervisors.  This may be a splitting hairs type of argument.  In VMware’s NSX approach the network controller or hypervisor integrates on a software layer with the target hypervisors.  Today NSX (Nicira) directly supports vSphere 5.5, KVM and Xen hypervisors.  If you have a Hyper-v installation you must use a gateway in order to support Hyper-V hosts.  This would cause scalability issues for heavy Hyper-V installations.  An interesting note is that a combination of OpenStack and KVM/Xen are normally at the center of conversations around these use cases.  I haven’t heard a ton of demand for virtualized networking over Hyper-V.  Cisco itself has just begun to support Hyper-V with its Nexus 1000v virtual switch.  It seems to me that Microsoft isn’t at the forefront of network virtualization in general.

To Cisco’s point, their hardware first approach would allow organizations that wish to bet big on Microsoft based Cloud infrastructures the flexibility to leverage existing Microsoft software without much dependency on Microsoft making changes to their software platform.  In theory, an ACI based infrastructure could create virtual network ports for VM’s running on any hypervisor supporting existing protocols such as VXLAN.  A hardware first based approach also would not be as vulnerable to version changes in the hypervisor.  We’ll have more on this in the Management Overhead section.

This argument starts to lose some steam when you look at the NSX solution from a partner ecosystem perspective.  VMware has announced several hardware partners with the launch of NSX.  There is nothing preventing these partners from offering the same VXLAN support to extend the virtual network to Hyper-V based hosts in the same manor Cisco would with ACI and managed by NSX.


It’s difficult to know if Cisco has called it correctly in this area as we don’t have enough detail around how exactly ACI will look from a hypervisor perspective.  We have the theory that it would work similar to how existing hardware based networking works today.  On the other side of the aisle, VMware has customers such as Paypal running both ESXi and KVM on a Nicira based virtual network today.


I’m not sure where Cisco’s argument of scale comes into play.  The argument may be around the fact that SDN based solutions establish tunnels for each connection within the network overlay.  I can see this being limited by the capability of the x86 controller and underlying physical network.  Ivan Pepelnjak wrote a great technical piece on why this isn’t a technical issue.

This is a great opportunity to highlight Cisco’s continued advancements or at least desire to continue the advancement of using hardware ASIC.  The software vs. hardware ASIC argument has raged for years.  The basic premise has been that purpose built hardware can always scale and outperform software running on general purpose hardware.  I don’t know if I buy this argument any longer.  Scott Shenker from Stanford gives a great argument how x86 based controllers can perform just as well as ASIC based controllers.  And again since we are talking about two products that haven’t been released yet this is all theoretical.  In theory, VMware’s partners can build ASIC support where needed.  I compare this directly to how Intel and AMD have built virtualization assistance in their chipset over the past few years to close or all out eliminate the performance delta between physical and virtual machines.  F5 for example has announced at VMworld tight integration with NSX.

Again, for a real world perspective, VMware has legacy customers running Nicira at what I assume is scale.  Cisco has proved itself as a network hardware provider that can run the largest workloads on the Internet, so I have little doubt when ACI hits the market that they will be able to deliver on their promise of scale.

Management Overhead

This is maybe Cisco most legit argument against the software first approach but also one of the most difficult ones to support with publicly available information.  I’ve questioned in the past the amount of overhead associated with managing both an overlay network and the underlay physical network.  The networking team still needs to support the full range of IP and physical infrastructure associated with the physical network as well as the overlay network.  The approach does give organizations the option to outsource the physical network in a similar fashion to how MPLS is done today.  However, the difference is that the enterprise network is homogenous.  There is and will still be a need to connect physical nodes to the physical network.  There will be the need for the physical network and virtual network to share subnets and VLANs.  This makes the wholesale outsourcing of the physical network impractical if not impossible.

That leaves the enterprise network team to manage the complexities of two logical networks.  Cisco’s claim is that their ACI approach is one system.  Administrators can use the same tools they use today to manage their SDN network if they base it on the ACI architecture.  This sounds reasonable except we don’t have enough detail to know how a reference ACI network would look like in the wild or what the operations model would look like from a practical sense.  We know conceptually you can leverage your existing Cisco Nexus environment along with the added software components but we don’t know what a network would look like functionally.

We do know what the Nicira network looks like in production.  VMware produced a great piece that looks at the challenges and approach to debugging a virtual network.  It helps to show that real thought needs to be placed into the design and management of the network prior to deploying it to avoid specific pitfalls associated with virtual networks.

Cisco’s additional point in management overhead is change management when going with a software first implementation.  One of the advantages of box to box communication is that it relies on protocols instead of API’s.  When say your hypervisor vendor updates their code there’s a perception of higher risk to the API’s breaking vs. when a network vendor updates a box and the protocol not working.  From my experience of managing servers, I have to say this is an area of concern.  When you talk about multi-hypervisor support there is just something more comfortable with me switching out a box based on protocols vs. installing a new software stack based on API’s and counting on my system still functioning as expected.


I don’t share Cisco’s concern with Multi-hypervisor support or scale.  I believe VMware’s hardware and software partners will eventually work these issues out where they exist.  I do however have reservations when it comes to the management overhead associated with a software first approach.  At the end of the day I keep coming back to the fact that VMware has real customers running Nicira in production while Cisco’s ACI is a great concept that’s still on a whiteboard.

Published by Keith Townsend

Now I'm @CTOAdvisor

18 thoughts on “Cisco and VMWare SDN: Breaking through the FUD

  1. Really great post Keith, looks like you have a good grip on the situation. I do think that VMware is very hardware-aware and much as they did with x86 they will work with partners to drive open hardware supports that allows workload portability between most vendors and cloud providers. Deeper hardware integration is forthcoming but one good point is that VMware for some time has had far deeper integration between partner physical switches and the virtual environments than Cisco has had with their physical, or even between the n1k and n5k without external orchestration. For example all of Dell/Force10 data center switches can API into vcenter and automate the physical network configuration as needed, transparently to the VM administrator – this is not a new feature. As much as Cisco tries to say VMware’s is a software-only approach, VMware has announced several hardware partners before they have even launched NSX.

    I think the biggest difference between VMware and Cisco is that VMware & partners like Dell believe it is possible to drive much greater effeciency in the network. Just like with x86 before virtualization, srvr utilization was <20% on average, and that all changed. We are trying to do the same with the network, hence elasticity + line rate devices + leaf/spine architecture. I assume Cisco doesnt think its possible for most enterprises to have effeciency in their network as they built the UCS around the fex which is hard wired to provide extremely limited average network utilization. If I buy a 5k and attach 4 x 2248's I have 5 switches worth of ports and 1 switches worth of forwarding performance, the math is plain and simple. When your secretaries desktop has less oversubscription than a UCS fabric (this isnt just darts, it is actually accurate) – I see that as a problem.

    1. Thanks Art. I really enjoyed writing the post. It’s a fun topic. I’m looking forward to see if Cisco still has the muscle to move the network industry in the direction they want to take it. The landscape is nothing like it was in the late 90’s and 00’s. Cisco seems to be going at ACI alone and has downplayed the software portion of ACI. We’ll see if they have the chops to make it happen as they envision it.

    2. Efficiency is an interesting topic. Are we trying to use the hardware more efficiently by increasing utilization ? If so, I wouldn’t expect to much from it. I think the point of SDN was to make management of the network more efficient.

  2. Cisco is proven in network technology for 30 years in all industries
    NSX is a risk – if I were wall street, i’d stay away form NSX like a snapped electric wire

    1. Not seeing the rationale. These are tech companies not financial services companies or manufactures. Tech companies have to innovate at the right pace consistently to stay ahead. Do you feel the same way about Unisys? Tech companies can fall asleep and get lapped.

  3. Someone in the tech news industry should go explore what VCE is going to do since that’s the closest intersection point between the two companies where this turf battle will be played out.

    Will VCE allow certified Vblocks to run NSX or will Cisco block them from putting NSX in base configurations now that ACI is included? Or will they break their own rules and become a reference architecture?

    I think this is a big deal if VMware is being blocked from putting NSX into Vblock converged infrastructure. What options will Vblock customers have?

    Can’t wait for The Register to pick this story up.

    1. VCE has already stated that they are going with ACI. I don’t believe there is a way to block NSX. They can just continue to not partner with it.

      VMware for its part wrote an article on integrating NSX and VCE.

      Thanks for the comment.

      1. VMware for its part wrote an article on integrating NSX and VCE… I can’t find any articles on integrating NSX and VCE/Vblock

  4. Great Updates Keith, good neutral coverage. I am curious about the ROI slide that Cisco showed with the $100 “VM Tax” …. looked like straight BS to me but who can know if they dont provide details. At face value though, it looked a lot like they included NSX costs but did not include APIC controller costs, and the opex costs appear to be cisco guesstimates. The one thing that I think many are overlooking is the user experience that VMware has been able to create with their software. I dont think virtualization took off at the rapid pace that it did and become so popular just because there was a business/financial benefit or a technical benefit. IMHO the reason why VMware has had such great success was the user experience they created with vcenter – there simply was nothing that came close to that experience for managing compute. Sadly in my experience selling infrastructure for 15 years to all customer segments, there are more purchasing decisions being made on hype than on reality. And with ACI the hype seems to be the single-pane-of-glass illusion … you dont need to manage multiple disjointed interfaces when you can manage a single cisco system. Well, where does that leave vcenter or vcloud automation center? Today server and application admins continue to be delighted by these interfaces and would enjoy to continue to use these interfaces to define their application profiles. So if I am using ACI with vmware, what is my operational experience, where do I define my policies? Do I use VMware self-service or do I use Cisco self service tools? Do I define my policies in vcac or vcenter, or do I do it in APIC,or do I do it in both? At the end of the day, there will be a set of processes to deploy a workload in a VMware environment, I could do those using VMware’s own software and processes and use a partner like Dell who can ensure the rest of the hardware setup is transparently automated ensuring the application & system administrator interfaces with the VMware-defined processes …. or I use Cisco and the process model and toolsets they define. So assuming VMware can offer tightly integrated solutions with hardware partners, I think the key thing comes down to the operational processes and the user experience provided by vmware vs cisco models, and it is still too early to know what those are.

    1. Good points. But, I think the bigger challenge for Cisco is that this is all academic in their case. Without a shipping controller it’s not much of a debate on which is the best approach or the advantages of their integrated solution.

  5. EMC has published reference architecture for IAAS using NSX and vCAC. There is Supplement document for IAAS on EMC site for NSX and vCAC Integration with vBLOCK.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: