Hyper-converged architecture: Aggregation vs. replication

Learn the benefits of aggregation in a hyper-converged environment, how to choose between aggregation and replication, and the compromise to each approach.

At the heart of a hyper-converged architecture system is storage software, which is critical to delivering the...

performance applications demand and reliability users expect. Hyper-converged infrastructures must provide a shared storage resource to enable virtual machine mobility and ensure data remains accessible if there is a component failure. In my last column, we discussed replication, which is simple to implement and provides great flexibility. But replication can consume 3x the storage capacity of its alternative: aggregation.

Aggregation creates a shared, protected storage resource to a hyper-converged architecture. Vendors have their own marketing terms, but this technique essentially aggregates the internal disks in each node in the hyper-converged infrastructure and presents them as a virtual shared pool to the VMs. Each VM writes data to this virtual volume that distributes segments of the data plus a parity bit across all the nodes in the hyper-converged architecture cluster.

Aggregation instantly makes a VM's data accessible from any node in the cluster, but a hyper-converged system exacerbates two issues when using aggregation as the storage architecture:

  1. The calculation of the parity bit, which can occur thousands of times a second, takes a toll on processors. Unlike a dedicated storage system, hyper-converged architecture processors do more than just calculate parity -- they also run applications. Replication merely has to copy changed bits to one or two predesignated nodes in the cluster.
  2. Aggregation, unlike replication, requires all storage I/O to go across a network and touches every node in the cluster. This increased network I/O requires greater attention to the networking hardware design in a hyper-converged infrastructure and calls for extra processing power to be factored into the parity calculation process.

The big benefit to aggregation is that data protection capacity consumption is far more efficient. Aggregation has an overhead of approximately 25% to 30%, and also provides protection against multiple node failures.

Blended model helps with shortcomings

As a compromise to the shortcomings of both types of storage software, some hyper-converged architecture vendors have implemented a blended model.

As a compromise to the shortcomings of both types of storage software, some hyper-converged architecture vendors have implemented a blended model. In this design, a VM's data is stored locally on the host it is currently executing, but a copy of its data is written across the nodes in the cluster via an aggregation technique. Data is read from the local host and written to both the local host and the aggregated storage.

The result is excellent read I/O performance, since most vendors put this local store on flash media. The blended design also reduces network traffic since only write I/O has to touch the network.

Replication or aggregation?

Adding additional CPU power, network I/O and leveraging a blended model alleviates many of the concerns aggregation creates, but each technology component increases the effective cost of the system. The 3x capacity requirement of replication can be a challenge for larger data centers with high-capacity requirements, but many data centers don't have a capacity concern for the applications they will put on the hyper-converged system. Often, the total capacity of the starter configuration is more than they will need. Replication is ideal for these customers, as it is simple and keeps the full cost of the environment under control. For larger data centers where a 3x capacity requirement will be an issue, aggregation is more appropriate.

Next Steps

Could hyper-converged infrastructure replace SAN and NAS?

Data center challenges with hyper-converged architectures

Simplify VM provisioning with converged systems

Dig Deeper on Hyper-Converged Infrastructure Implementation