Running your own EMC Celerra ‘Uber’ Virtual Storage Appliance (VSA) in your vSphere Lab – Part 1

Since starting my role as a vSpecialist at EMC I have had exposure in one form or another to pretty much the full range of EMC products.  Coming from predominantly an HP storage background (ie: MSA, EVA) I’ve enjoyed learning and increasing my knowledge on the EMC range of storage products in particular the mid-range Celerra and CLARiiON appliances.

I’ve been impressed with the flexibility (SAN and/or NAS) and relative ease of use of these storage appliances along with their comprehensive feature set which includes de-duplication, compression, FAST and FAST Cache.  The new Unisphere management interface also offers a nice clean easy to use way of managing these mid-range EMC storage offerings.

Now, getting my hands on a Celerra or CLARiiON for training, testing or just for familiarisation proved difficult sometimes and who wants the risk of ‘learning’ on a production system as it’ll almost always end in tears?  So upon joining EMC and having to bring myself up to speed with the Celerra  I was very pleased to find that they offer a fully featured virtualized instance of the EMC Celerra appliance.  This virtualized Celerra appliance can easily be spun up for non-production use in a VMware Workstation or vSphere environment. Oh, and did I mention it was FREE?  Yes, it is free to download, use and it isn’t time or functionality restricted.  This is quite refreshing as I find it frustrating having to reinstall VMs, applications or utilities after every 30 or 60 days in my vSphere lab.

At this point you can perhaps see how the Celerra VSA is appealing to those, such as myself, wishing to learn and familiarise themselves with the Celerra and for those wanting a fully featured virtual storage appliance (VSA) in their lab environment to run VMs from whilst trying out cool features such a de-duplication, compression and replication.

Now I should mention that of course there are other decent free VSAs such as OpenFiler (which I am a fan of) and FreeNAS though you are less likely to find these in the same number of production IT environments in the real world as you would a Celerra.  From a training perspective, investing the time in learning the Celerra VSA is certainly worthwhile as it provides an excellent learning opportunity to up-skill yourself with experience and knowhow that you can then take to an existing or new employer running mid-level EMC storage.

Now on with the show….

Celerra VSA Blog Post Series – The Game Plan

In this series of blog posts I am going to take you through how to successfully install, configure, run and manage the EMC Celerra VSA in your own VMware vSphere lab using management tools and utilities such as Unisphere, vCenter Server and the EMC vCenter Server plug-ins.  We will also take a look at some of the real value add features such as compression, de-dupe and of course replication between two Celerra VSA instances. Fun times ahead!  For my hardware I am going to take my new HP MicroServer for a spin, I have added 8GB of memory to it along with an extra hard disk. Everything EMC Celerra storage related, even with replication, is going to run from this single lab server.

The series is going to be broken up into the following posts:

• Part 1 – The Basics: A High Level Introduction to the EMC Celerra Physical Storage Appliance
• Part 2 – Installing the EMC Celerra VSA (coming soon)
• Part 3 – Managing your EMC Celerra VSA with Unisphere (integrated & also free) (coming soon)
• Part 4 – Configuring the EMC Celerra VSA for NFS (coming soon)
• Part 5 – Configuring the EMC Celerra VSA for iSCSI (coming soon)
• Part 6 – EMC vCenter Plugs & Other Cool Stuff (coming soon)
• Part 7 – Configuring replication between two EMC Celerra VSAs (coming soon)

There has been a virtualized storage appliance version of the Celerra for a couple of years now though earlier versions could prove a little fiddly (though not impossibly so) to configure especially for those not already familiar with the Celerra product or perhaps who weren’t comfortable with delving into the world of the command line.  However you’ll be pleased to know that the recent versions are very ‘first time user’ friendly with many of the previously manual initial configuration tasks now automated thanks to the scripting brilliance of a vSpecialist colleague of mine, Nick Weaver.

Nick took the Celerra VSA applied a number of performance enhancing ‘tweaks’ to it and as mentioned automated many of the manual configuration steps.  Nick’s version of the Celerra VSA is called the ‘UBER’ edition (ie: EMC Celerra ‘UBER’ VSA) with the current version sitting on v3.2.  Check out Nick’s site, Nickopedia.com here for more details on the Uber VSA and for downloading your very own copy.

Though before we go into detail and start talking about installing and configuring the EMC Celerra ‘Uber’ virtual storage appliance (‘UBER VSA’) it is worth taking a little time to familiarise yourself with the physical EMC Celerra storage appliance and the various high level concepts and terms associated with it.  This will make installing, configuring and running the virtualized version much easier.

The Physical EMC Celerra Storage Appliance

The physical EMC Unified Storage Celerra appliance provides access to both block and file data using the following main protocols:  iSCSI, Fibre Channel (FC), NFS and CIFS.  If you are not familiar with the concepts of block and file storage check out the following Wikipedia articles: Block storage article and File storage (NAS) article.

Many business storage infrastructures generally have a leaning towards using block or file for their storage so the beauty of offering both is that the Celerra provides these businesses with the option to choose one or the other protocol, or maybe a mix depending on preference or the data types being served.  It’s not a case of use one or the other but rather you are given the option to pick, choose and mix as your business’s IT infrastructure requires it. Who doesn’t like choices? If nothing else it lets you hedge your bets. 

The Models (not the cat walk variety)

The next thing to know about Fight Club the EMC Celerra is that there are two separate Celerra offerings.  The first is the full Celerra Unified Storage Appliance and the second is the Celerra Gateway which provides Ethernet IP based iSCSI, CIFS and NFS (and Multi-Path File System (MPFS)) connectivity in front of an existing EMC CLARiiON or Symmetrix SAN appliance. So for example if you currently have Fibre Channel connectivity to your existing EMC CLARiiON and fancy adding NAS functionality, then the Celerra Gateway can provide your SAN with the necessary NAS capabilities.

Below is a diagram showing a summary of the various models of EMC Celerra that make up the range along with some of their basic specifications.  From this diagram you can see the  afore mentioned Celerra offerings: the Unified Storage appliance and the Gateway. 

EMC Celerra Gateway

The hardware components, X-Blades and Control Station, that are used in the full Celerra storage appliance are the same as those used for the Celerra Gateway.  As mentioned above the purpose of a Celerra Gateway appliance is to provide NAS functionality to an existing EMC CLARiiON or Symmetrix SAN you may already have.

The Celerra Gateway has a different model naming convention to that used by the full Celerra storage appliance.  There are two models, the VG2 and the VG8.  You can probably guess that the difference between these two Celerra Gateway models comes down to scalability, with the throughput of the larger model of the two, the VG8, having more potential storage capacity, increased availability and throughput.

For more information on the EMC Celerra Gateways check out EMC’s site here.

EMC Celerra Storage Appliance

There are four core components at the heart of the Celerra storage appliance, these being the X-Blade/Data Movers, the Storage Processor Enclosure (SPE), the Control Station and of course the actual disk shelves or disk array enclosures (DAE) as EMC call them.  The following few sections give a quick overview of these components.  Having this basic knowledge will help you understand how the overall Celerra storage appliance hangs together which you can then apply to the terms found when installing and configuring the Celerra VSA.

The Bits and Pieces

The following front and rear diagrams give an overview of the various components that make up an entry level Celerra.  With the larger models of Celerra you have the same components just more of some of them (eg: more X-Blades, more DAEs).

Here’s a rear view:

In the next few sections we will take a closer look at each of the four core Celerra components.

X-Blade/Data Movers: I have to admit that I quite like the term X-Blade, it sounds rather futuristic and lethal weapon like.  Although not quite as racy as a lethal ray gun or similar it is the name EMC uses to describe the individual servers (X-Blades) which control and bridge the movement of data between the disks via the storage processors within the storage appliance to the IP network.  This provides the Celerra’s NAS capability.  It should be noted however that an iSCSI connection can  be implemented either via the Storage Processor(s) or the X-Blade(s).  It is also important to note that the name for the X-Blade is also used interchangeably with the term ‘Data Mover’ which you will often hear reference to.

A Celerra can have one or more X-Blade/Data Mover installed which provides iSCSI, NFS and/or CIFS shares and there are two models of X-Blade that can be used.  One provides 1Gb Ethernet ports….

… and the other provides a combination of 10-GbE (Optical) and 1Gb Ethernet ports.

Multiple data movers which are grouped together function as a single system which provides high resilience to the environment and are controlled by the Control Station appliance (see section later in this post for more details on the Control Station).

Storage Processor Enclosure (SPE): The storage processor enclosure is a shelf within the Celerra storage system which contains the enclosure, the storage processors (SPs), the Fibre Channel link control cards (LCCs), a couple of power supplies and the fan packs. The SPE uses dual active storage processors (see next section below) for the disk I/O and supports automatic failover should one of the SPs fail.

The image below shows the rear view of a default SPE configuration. As you can see there is plenty of room for iSCSI or FC expansion.

Storage Processors (SPs): The Storage Processors manage the I/O to the disk modules from the Celerra’s host FC adapter.  This host adapter carries FC traffic to and from the DAEs, the X-Blade(s) and any directly attached FC connections such as SAN fabric switches or directly FC attached servers.  As already mentioned you can also add iSCSI connectivity to the SPs if required.  The storage processors run the FLARE microcode.

DAE: Or Disk Array Enclosure is, as the name suggests, an enclosure for disks.  Many of you, including myself, may have always just called it a disk shelf.  The DAEs with the EMC Celerra as with the CLARiiON model have space for 15 hot pluggable 3.5’” Fibre Channel (10K and 15K rpm), SATA or the newer Flash drives.

Control Station: The control station is basically a 1U server (with a nice EMC Celerra bezel) running Red Hat Enterprise Linux which provides a single point of management, via an IP connection, for the entire Celerra Appliance.

The control station provides management via a command-line interface, the Celerra Manager GUI, Unisphere or Celerra Start-up Assistant.  The control station provides a means to configure the data movers and the back-end storage. Here is an example of some of the other tasks that you can perform from the control station:

• • Install, Configure and Manage the X-Blades
  • Monitor the performance of all the components that make up the Celerra
  • Monitor the environmental conditions of the Celerra include X-Blade failovers
  • Manage volumes and the file systems
  • Configure the network interfaces

Control Stations can be installed in pairs to provide resilience, which come as standard in the larger EMC Celerra physical storage appliance offerings.

How It Hangs Together including Front and Back Ends

So how does it all hang together?  It’s definitely a good idea to get a basic understanding on the flow of I/O communication to and from the Celerra and between the components within the overall Celerra storage appliance itself.  Study the diagram below as it gives a great overview on how the I/O traffic flows from an external FC, iSCSI, NFS or CIFS connection right through to it reaching the disks in the DAE.

A concept to become familiar with and is something you’ll hear banded about from time to time is that of a storage appliance’s front end and back end.

Front End:

The ‘front end’ of a storage appliance is considered that part of the SAN/NAS where connectivity from the world outside of the appliance enters and leaves, this can be via, for example, Fibre Channel or Ethernet. Depending on the connectivity protocol (eg: FC, iSCSI, NFS or CIFS) used to the physical Celerra this will determine whether the physical connection comes in via the X-Blade or the Storage Processor.  The following diagram*, similar to the one above, provides a good summary on which component within the Celerra handles which particular protocol.

Back End:

The ‘back end’ is considered the storage processor (SP) part of the appliance which connects the SPs of the appliance with the disks via the DAEs.  The connectivity on the ‘back end’ is via Fibre Channel.

And for those of you who quite like wearing a bright yellow anorak from time to time and knowing the more intricate details on things I have included a cabling diagram for a Celerra NS-120 single blade copper cabling configuration for your viewing pleasure.  Click on the image to get a larger easier to read version.

Here are some other terms that you will also come across when reading or listening to conversations on the Celerra, or the CLARiiON for that matter.

Terminology aka Lingo

DART (Data Access in Real Time): This is the operating system (OS) that runs on the X-Blades/Data Movers of the Celerra which has been optimized for file storage I/O.

EFD: Enterprise Flash Drive/Disk.  The clue is in the name with this one, EFDs are flash disks of an enterprise grade and as such provide increased reliability in an 24×7 IT infrastructure.  Of course Enterprise Flash Drives (EFD) do come with something of an higher price tag over standard Flash drives/disks that are also available in the enterprise though they do offer a better mean time between failure (MTBF) rating.

SPA/SPB: Storage Processor A and/or Storage Processor B – In an EMC Celerra there are two storage processors for performance and/or resilience.  They are both distinguished by being abbreviated to SPA or SPB.

FLARE (Fibre Logic Array Runtime Environment): This is the Operating Environment which is served from the first five drives of the first DAE and is run on the storage processors (SPs).

Other Reading

If you’d like some further semi-technical reading on the EMC Celerra then I recommend you take a look at the ‘Introduction to EMC Celerra Unified Storage’ white paper which you can download from here.  Another great resource around the EMC Celerra, including the Celerra VSA and in fact anything EMC related is Chad Sakac’s ‘Virtual Geek’ blog.

So there we have it, the first installment on the installing, configuring and running the EMC Celerra Virtual Storage Appliance (VSA) in your vSphere lab series. Although not particularly hands-on, this post was intended to equip you with the know-how which will make understanding the Celerra VSA all the more easier.  As mentioned earlier, this series will comprise of the following posts:

• Part 1 – A High Level Introduction to the EMC Celerra Physical Storage Appliance & Virtual Storage Appliance (VSA)
• Part 2 – Installing the EMC Celerra VSA (coming soon)
• Part 3 – Managing your EMC Celerra VSA with Unisphere (also free) (coming soon)
• Part 4 – Configuring the EMC Celerra VSA for NFS (coming soon)
• Part 5 – Configuring the EMC Celerra VSA for iSCSI (coming soon)
• Part 6 – EMC vCenter Plugs & Other Cool Stuff (coming soon)
• Part 7 – Configuring replication between two EMC Celerra VSAs (coming soon)

Hope you’ll join me in building your own Celerra VSA for your vSphere lab environment, it’ll be fun!(and educational) 

*Thanks to EMC TC, Craig Deans for providing one of the diagrams.