In our last blog post, I discussed the storage considerations for the Eucalyptus Walrus component. Toward the end of the post, I talked about pondering some additional implications of using a SAN to back Walrus if you are going to use a Eucalyptus supported SAN with the SAN adapter for the Storage Controller (SC). In this post, we will delve into the various ways you can deploy the Storage Controller (SC) component, building from non-HA to HA with the Eucalyptus SAN adapter. Keep in mind that using the SAN adapter is a requirement for deploying the SC in HA, so if you do not have a Eucalyptus supported SAN (at the time of this publication this includes EMC VNX, NetApp and Dell Equallogic) you can only deploy in non-HA mode.
How the Storage Controller Functions
The SC functions in one of two ways in a Eucalyptus cloud depending on whether you deploy in HA or non-HA mode. They are quite different, so carefully consider each approach when designing the SC component of your cloud.
In non-HA mode, the SC itself becomes a ‘virtual SAN’, utilizing local (or locally mounted) disk space to create virtual block devices that will be exported from itself to the Node Controllers (NCs) and mounted to present block devices to instances (or to the NC itself in the case of Boot from EBS instances.) The key takeaway for non-HA mode is that the SC is ‘in the data path’ meaning that all iSCSI exports are from the SC itself and you are bound by the disk and network performance of the SC.
In HA mode, the SC can be thought of as a proxy that takes EBS requests and orchestrates the provisioning, exporting and mounting of iSCSI LUNs from a supported SAN via the SAN adapter. In this scenario, the SC is NOT in the data path. Node Controllers mount the iSCSI LUNs directly from the SAN, not the SC. Not only is this more performant, in the case of an SC failure, you do not lose data or connectivity. The second SC simply takes over and proxies all of the SC related requests to the SAN and NCs. With all that said, lets take a look at HA and non-HA mode storage considerations.
Deploying the SC in non-HA mode means, at its simplest, that there is only one node (as opposed to two in HA mode). This means that if the SC fails for some reason, you will lose access to all of your EBS volumes. There are five ‘backend providers’ for the Eucalyptus SC, of which two are valid for non-HA: Overlay and DAS. The other three we will discuss in the HA mode section below.
Overlay simply means that Eucalyptus will use the local filesystem and “overlay” storage of your EBS volumes onto that directory. By default, Eucalyptus uses the ‘/var/lib/eucalyptus/volumes’ directory for SC storage in Overlay mode.
Direct Attached Storage (DAS)
When using DAS, you are able to specify a device to use for the SC’s storage space. This can either be a raw device (like /dev/sdb) or a LVM volume group.
The main reason for using DAS instead of Overlay is performance and control, especially if you are using network storage of some type, such as an iSCSI or Fibre Channel LUN. Overlay has the benefit of being much simpler to setup and configure (for small test clouds or POCs) and also allows you to use something like NFS to back the ‘/var/lib/eucalyptus/volumes’ directory instead of a block device. Both have their uses, but DAS is a step up in terms of performance and flexibility which was introduced in 3.2, and is highly recommended if you are able to dedicate a raw device for it’s usage.
HA mode is actually the ‘simplest’ in terms of usage because you only have one option: use the SAN adapter. Eucalyptus currently supports three different SANs for use with the SC: NetApp, EMC VNX and Dell EqualLogic. You can find out more about the SAN specific requirements, like software versions, in our documentation here: http://www.eucalyptus.com/docs/3.2/ig/planning_san.html. In HA mode, the SC is simply an orchestrator that communicates with the SAN directly to perform volume operations such as volume creation, deletion, mounting and snapshots. One thing to note is that the SC is also used as a proxy for volume snapshots in HA mode. Eucalyptus will take a SAN-level snapshot via the SANs own software, then copy that to the SC as a staging point to upload it to the Walrus. The great thing about using a SAN with the Eucalyptus SC is that the robustness, speed and data integrity of your volumes is backed by a piece of hardware designed to be highly-available. Also, since the SC is not in the normal data path for volume usage, the NCs are able to connect directly to the SAN, which typically has more performant connectivity than a simple server. It is common knowledge that data is the most important piece of the IT puzzle and by using the Eucalyptus SAN adapter with a supported SAN, you ensure that your data resides on a rock-solid platform. It is generally recommended to use a SAN with Eucalyptus for your SC configuration.