Paul Angus, Cloud Architect at ShapeBlue takes an interesting look at how to separate Cloudstack’s management traffic from its primary storage traffic.
I recently looked at physical networking in a CloudStack environment and alluded to the fact that you cannot separate primary storage traffic from management traffic from CloudStack, but that it is still possible. In this article I will discuss why this is and how to do it.
In the beginning, there was primary storage
The first thing to understand is the process of provisioning primary storage. When you create a primary storage pool for any given cluster, the CloudStack management server tells each hosts’ hypervisor to mount the NFS share or (iSCSI LUN). The storage pool will be presented within the hypervisor as a datastore (VMware), storage repository (XenServer/XCP) or a mount point (KVM), the important point is that it is the hypervisor itself that communicates with the primary storage, the CloudStack management server only communicates with the host hypervisor.
Now, all hypervisors communicate with the outside world via some kind of management interface – think VMKernel port on ESXi or ‘Management Interface’ on XenServer. As the CloudStack management server needs to communicate with the hypervisor in the host, this management interface must be on the CloudStack ‘management’ or ‘private’ network. There may be other interfaces configured on your host carrying guest and public traffic to/from VMs within the hosts but the hypervisor itself doesn’t/can’t communicate over these interfaces.
Figure 1: Hypervisor communications
Separating Primary Storage traffic
For those from a pure virtualisation background, the concept of creating a specific interface for storage traffic will not be new; it has long been best practice for iSCSI traffic to have a dedicated switch fabric to avoid any latency or contention issues.
Sometimes in the cloud(Stack) world we forget that we are simply orchestrating processes that the hypervisors already carry out and that many ‘normal’ hypervisor configurations still apply.
The logical reasoning which explains how this splitting of traffic works is as follows:
1. If you want an additional interface over which the hypervisor can communicate (excluding teamed or bonded interfaces) you need to give it an IP address
2. The mechanism to create an additional interface that the hypervisor can use is to create an additional management interface
3. So that the hypervisor can differentiate between the management interfaces they have to be in different (non-overlapping) subnets
4. In order for the ‘primary storage’ management interface to communicate with the primary storage, the interfaces on the primary storage must be in the same CIDR as the ‘primary storage’ management interface.
5. Therefore the primary storage must be in a different subnet to the management network
Figure 2: Subnetting of Storage Traffic
Figure 3: Hypervisor Communications with Separated Storage Traffic
Other Primary Storage Types
If you are using PreSetup or SharedMountPoints to connect to IP based storage then the same principles apply; if the primary storage and ‘primary storage interface’ are in a different subnet to the ‘management subnet’ then the hypervisor will use the ‘primary storage interface’ to communicate with the primary storage.
Summary
This article has explained the how primary storage traffic can be routed over separate network interfaces from all other traffic on the hosts by adding a management interface on the host for storage and allocating it and the primary storage IP addresses in a different subnet to the CloudStack management subnet.
About the Author
Paul Angus is a Cloud Architect at ShapeBlue, The Cloud Specialists. He has designed numerous CloudStack environments for customers across 4 continents, based on Apache Cloudstack ,Citrix Cloudplatform and Citrix Cloudportal.
When not building Clouds, Paul likes to create scripts that build clouds
..and he very occasionally can be seen trying to hit a golf ball.
Steve is ShapeBlue’s COO and is responsible for all day-to-day administrative and operational functions of the business, including the consulting, programme management, and support functions.
Involved with CloudStack since 2012, Steve has led several large customer engagements including a number of major public and private cloud deployments; co-ordinated and developed worldwide teams and helped implement and deliver an enterprise grade support product.
Prior to ShapeBlue, Steve held senior technical, project and account management roles in enterprise IT outsourcing companies where he gained domain experience in the finance, professional services and defence markets.
Away from work, Steve is a father, keen guitarist, snowboarder and singer (not necessarily in that order).
Away from work, Steve is a music lover and semi-professional musician. Although he doesn’t speak at many technology conferences, he can sometimes be heard providing the evening entertainment.
