#recoverpoint

Something I've enjoyed doing over the years is following the progression of EMC's flagship replication product, RecoverPoint. Personally I have been installing and working with the product going on 3 years now. I have done my fair share of advocation both publicly and in customer shops across this great coast line. With every major release, a wall has tumbled and the customer ends up winning. With the latest release of RP, another wall has tumbled which is probably the most significant.

Its a fact write splitting is easier, less error prone, and cheaper when it is done from the array. While focusing on array splitting for the midrange arrays only, EMC for a while had neglected their roots. Now I say neglect for added effect, but its obvious the Symm world is a bit hesitant to change. I can only envision what went on behind closed doors convincing the "Symm Elders/Mystics" or "Stick in the Muds" as I like to call them, that RecoverPoint is a better long term cross platform solution.Regardless the writing was on the wall with the VMAXe supporting the RecoverPoint splitter at market introduction. And there being no physical constraint between the VMAX and the VMAXe it was only a matter of time before we reached this state.

So as of 3.5 we now have array based write splitting capabilities across the Symmetrix VMAX 40K/20K/10K, VPLEX Local and Metro and VNX/CX arrays. EMC's entire block array line. Kudos to all those who fought those internal battles for the great good of your customers and partners. 

Other changes worth noting, full list here…

FIPS 140-2 Level 1 Compliance

Gen 3 appliance support dropped

Simplified Licensing activation

New copy and replication management capabilities

Kutils replaced with KVSS (support for SQL and Exchange)

VMAX and CX/VNX LUNs in the same Consistency Group

As of yesterday RecoverPoint 3.4 was released to us believers. And even though its code is just under 1G in size its punch is quite far reaching. The biggest? Support for VNX (only, not VNXe). Which means, and is somewhat implied, that if you are a current RP customer you must upgrade to 3.4 to be able to replicate LUNs from a VNX array. Replication between two VNX’s, between a VNX and a CLARiiON or Celerra, all supported, but you must be 3.4 bound.

What else?

Ingestion suites. Yuck. These my friends make my innards zing, whatever the hell that means. Protection suites are what they are called, local and remote. CDP? Local Protection Suite. CRR? Remote Protection Suite. CLR? Remote and Local Protection Suite, licensed per array by the way.

WAN 3D. Shows how much I know, I thought it already did this to some degree. My suspicions are there is a little bit of avamar dust sprinkled in to make this viable. Regardless increased WAN efficiencies are king nowadays and will continue to be. Caveat: You must have Gen4 appliances which assumes you bought within the last 2 years maybe. These are the Dell R610s if you’re playing along at home.

NAS Volume Replication. Sounds cool, but would have to see in this in action. Cabinet level replication is the coined terminology. What it allows you to do is, via CRR, replicate all of your NAS VNX filesystems to another cabinet at a remote site for DR purposes. All or none. I understand the slow release valve way of introducing features but it seems to me if you could provide cabinet FS replication you could support a single filesystem.  Regardless a step in the right direction. After all file level replication support is key if the only two replication technologies going forth long term are SRDF and RecoverPoint.

Storage Awareness and I/O throttling. Two new mechanisms that allow scaling up and scaling back I/O during periods of increased load or outages. Periods such as initialization, or even when a device becomes unresponsive. Surprised this isn’t share based, if you get my drift. Whenever storage efficiencies are introduced you’ll get no complaints from me.

Cisco SANTap-Stale ITL handling. I point this out so the one customer we have running SANTap and RecoverPoint can benefit. This used to be quit cumbersome, as it was process to remove a LUN and its Stale ITLs from SANTap. Over time failure to do so could carry you over your supported ITL count for a single SANTap module. Not to mention its general recommended practice whenever you..

Move a Host from the FE VSAN to the BE VSAN

Expanding, removing or adding LUNs

Replacing HBA, or anything that effects host to LUN pathing.

Moving right along I would say, when will you join us believers…

I tell you, I find these two products just fascinating. The combination of Disaster Avoidance and Disaster Recovery is truly unprecedented. Who would have thought, 10 years ago we would have been standing at a doorway that is open to so many possibilities.

To think beyond a server or rack back then is now thinking beyond a physical data center. The level of abstraction available to us today allows us to dream the impossible. So outside of this post, I challenge you to think beyond whats comfortable in your data center and imagine the unimaginable.   

So anyway, lets look at how these two titans of reverie mesh. Now VPLEX is like a new born infant to me, I don’t yet know how to care for it. But I will do my best. I will do my best. RecoverPoint is of course, old hat :) So let’s start with key concepts of each product…

VPLEX Volumes, and yes it is confusing..

Storage Volumes–these simply are LUNs that you have presented to the VPLEX backend. These LUNs can be from not only EMC arrays but any supported third party array. EMC is really starting to diversify its product line so it’s no longer a “this is my world operate within it” type mentality. RecoverPoint is another example for support of third party arrays.

Extents– these are slivers of a storage volume or a one to one mapping between the said storage volume. This is important for RecoverPoint integration.

Device– these map to extents. A single device can encompass multiple extents or can be one to one. A tell tell sign of a device is its RAID type or protection scheme as its noted. Now an extension of a device is the concept of distributed device. This is two legs of an extent (RAID 1 protection) from two separate VPLEX clusters. This is known as Metro-Plex, sync distances apply. So in summary, a device is typically two extents, one from each array at a single site. A distributed device is two extents, one from each array across two separate sites. And finally..

Virtual Volumes – these are mapped to devices and are “exposed” to the hosts by way of the VPLEX FE ports. Single site = Virtual volumes, 2 sites = Distributed Virtual Volumes.

So here are the associations in directional format..

Storage Volumes ->Extent(s) ->Device ->Virtual Volumes. Say it again in your head. Ok once more..now make yourself a sandwich..

RecoverPoint Volumes, not as confusing..

There are a number of volumes associated with RecoverPoint environments, but in the interest of time I will focus on just the ones pertinent to VPLEX integration...

Production Volume  - these volumes or LUNs are nothing more than your source volume. This is where your data lives, not to sound too much like an EMC marketing ad.

Replica Volume– this is a block for block copy of your source volume. This can exist on your source array or a destination array, depending on what type of replication is in play. For this post, the replica volume will be part of a CRR configuration. CRR stands for Continuous Remote Replication and is unidirectional async replication from Site A to Site B.

Now there are journal volumes, and repository volumes as well but these two volume examples should get us through this example.

So consider the first graphic below… this is a distributed virtual volume between site A and site B. Site C is where our RP replica(referenced as the Target Remote Copy) is in our CRR configuration. What’s important to note, and the significance of this post is VPLEX Metro and RecoverPoint CRR w/ CX4 Splitter Co-exist, they really don’t provide a complete dynamic solution. And here’s why…

For these two products to co-exist, there has to be a one to one mapping between storage volumes and extents, between extents and devices and between devices and virtual volumes. No big deal, right? Right. But this whole design is based on the assumption that the Source Volume (represented as “Source”) never moves. If it does replication is broken. So the act of “Data Mobility”, of moving a source storage volume from one LUN to another or one array to another breaks replication. This assumes the whole reason you bought VPLEX is to have the flexibility of moving your data, non-disruptive, right?  

Queue second graphic…

So the act of performing “Data Mobility”, or moving a source storage volume from one LUN to another or another array, breaks replication, yes I said it again..

I suspect in future RP releases, that there will be the concept of multiple source volumes that operate within a group setting. So as you move a storage volume around on the backend (whether between sites or arrays), RP will track that movement and adjust its replication according. This is manual today within RP and requires a full sweep after reconfiguring the source volume within the consistency group, but doable.

I can tell you this makes a lot of sense in a number of customer environments, having the flexibility to move workloads around yet still maintain DR capabilities is the Data Center of the future that we tout and want. 

..the knowledge sharing cause that is. This is something I have thought about doing for a while but for whatever reason have gone the other direction. There is so much hidden knowledge out there that needs documenting and distributing. Wikipedia has carved out its existence based on this mindset. Make it collaborative, make it quick (wiki) and make it web-based...