Hyper-converged architectures bring significant value to organizations that implement them. But many environments...
have a handful of applications that require either very high performance or very specific performance, which can result in hyper-converged network bottlenecks.
In any hyper-converged design, the network is the least converged of the components. A hyper-converged network aggregates storage across nodes, and applications can move between nodes to run on another CPU. Networking is fairly static; bandwidth is not reassignable between nodes, and the physical cable plant of the hyper-converged architecture can be intimidating. It is important to remember that a hyper-converged architecture is a cluster, with potentially many nodes. These nodes all need to intercommunicate to ensure they are in sync.
In the past, legacy clusters had dedicated storage systems that they mutually accessed for shared data. A hyper-converged architecture typically aggregates internal storage from each node into a virtual pool of storage. An environment with heavy storage I/O demands can place a tremendous load on the network. Both the internode communication and the storage I/O communication add latency that may be noticeable in a flash-based, hyper-converged architecture.
Resolving the hyper-converged network problem
Here are a few areas that factor into any attempt to fix a network problem in your hyper-converged environment.
Ensure the network has relatively high performance; 10 gigabit Ethernet (GbE) is the norm. Use fiber, not copper cables, for the interconnections. Most environments will use a top-of-rack switch to connect the nodes for internal node-to-node communication, and then a 40 GbE uplink to the core for connecting to the rest of the environment.
Ideally, these environments have pre-cabling and leverage patch panels to make the final connection between active components. The goal is to not have to open a node to make moves, additions and changes. This design is called true structured cabling and solves the biggest network performance problem of all: suboptimal performance or total loss of connectivity.
Implement a dedicated storage network. Most hyper-converged architectures require Ethernet for all networking (storage and compute), but dedicate the storage I/O to a specific network segment. A dedicated hyper-converged network isolates storage traffic from server-to-server traffic, which results in more consistent performance.
Examine how data is stored in the hyper-converged architecture. All hyper-converged architectures must somehow present shared access to storage even though the storage itself is internal. Vendors thus far have chosen two methods to accomplish this presentation:
- Replication makes a near-real-time image of a virtual machine (VM) on one or more other nodes in the hyper-converged architecture. If a VM needs to migrate, it can instantiate on any other node with that replica. While somewhat wasteful from a storage capacity perspective, it can be less impactful from a storage I/O perspective since only a few nodes are included in the replication.
- Aggregation creates a virtual pool of storage from all the media internal to the nodes in the cluster. While much more efficient from a capacity perspective, the aggregation method typically requires higher I/O.
Vendors have their own tweaks to these two methods, so mileage will vary between offerings.
One major hyper-converged network challenge is scaling. To resolve physical scaling issues, use a structured cabling method as described earlier. The software issues become more complex as nodes are added, especially in the aggregated method. The more nodes, the more storage I/O traffic there is and the more complex the environment becomes. If the organization plans on dozens of nodes and has specific performance expectations, a more traditional architecture with dedicated tiers of compute and storage may be more appropriate.
Author George Crump is president of Storage Switzerland, an IT analyst firm focused on the storage and virtualization segments.
Network infrastructure vital to hyper-converged environments
Hyper-converged offerings bundle storage components
What are the limits to hyper-converged storage?