The Best High Performance Storage

Whether you're a production house, an HE establishment running a High Performance Cluster, or a co-hosting facility in a data centre, chances are that your organisation needs high performance data storage.

Here we take a look at existing and emerging storage protocols and the fact that FCoE (Fibre Channel over Ethernet) looks more and more like a serious bet for performance-minded SAN users such as production houses or data centres. However, in our opinion, FCoE is no iSCSI killer - the competition from FCoE is much more likely to affect traditional Fibre Channel and Infiniband.

 

Is Fibre Channel dead?

A few years ago it seemed as though Fibre Channel (FC) had a tough time ahead, requiring its end-users to purchase relatively expensive and delicate hardware components, but with a technology roadmap that was ostensibly leap-frogged by cheaper and simpler solutions employing Ethernet switches and cables.

The FC roadmap did promise performance enhancements, firstly in a step up to 8Gb/sec, followed by a further step up to 16Gb/sec. However the rapid appearance of the iSCSI protocol deployed with cheap Ethernet hardware offered an alternative path for those interested in the benefits of SAN technology.

FC and iSCSI came to be seen as complimentary technologies, with FC required where performance was the primary consideration.

The seemingly imminent arrival of affordable 10 Gb Ethernet has since been causing a buzz amongst industry pundits for some time. It goes without saying that a throughput of 10 Gb/sec, iSCSI would leapfrog even the prospective 8 Gb/sec of FC in terms of performance. Few were saying it openly but many thought that the writing was on the wall for Fibre Channel as a storage technology.


FC is dead! Long live FC!

Of course, real world technology development is rarely as clear cut as predictions would indicate. In reality 8Gb/sec FC hardware is here with us today. 10 Gb/sec Ethernet is still too expensive to be deployed in edge networks or SMBs. But there is a real threat of lower pricing due to volume sales of iSCSI storage and lower costs of manufacturing due to economies of scale. This threat was enough to prompt the developers and vendors of FC technology into a rethink about their strategies and R&D roadmaps.

The result of this has been the emergence of a technology called Fibre Channel over Ethernet (FCoE). Put simply, FCoE piggy-backs the Fibre Channel protocol on the back of Ethernet. Met initially with some industry scepticism, many felt that FCoE was a solution to a manufacturers' problem with no real technological benefits.

Looking back to eight months ago, vendors appeared to be aligning themselves either for or against FCoE, ready for a technology battle. For example, Dell invested significantly in its acquisition of EqualLogic, a major player in iSCSI. But on the other hand, several notable vendors and storage names lent their weight to FCoE, including Brocade, Cisco, EMC, Emulex, IBM, Intel, Qlogic and Sun.
Something of a battle between titans and gods then.

What features does FCoE offer then?

  • Lower costs - 10Gb Ethernet is still too expensive to have been taken up extensively by many businesses, let alone consumers. If previous enhancements to the Ethernet standard are anything to go by, though, there is no reason to think that 10Gb Ethernet will not fall to commodity price levels. Price has always been one of the most compelling arguments for choosing iSCSI over FC. To some extent FCoE promises the best of both worlds; the efficiency and performance of Fibre Channel plus the affordability of Ethernet networking hardware.
  • Guaranteed performance levels - FCoE takes advantage of a number of enhancements to Ethernet to guarantee performance, avoiding problems that sometimes occur on Ethernet networks which may cause issues with iSCSI. These enhancements include exchange of information on device capabilities, the dynamic allocation of unused “dedicated” bandwidth according to the priority of traffic type, the ability to increase the flow of network traffic by "pausing" some of the less critical traffic, and throttling back data transfer on congested networks.
  • Keep your existing FC hardware - A positive factor in the adoption of FCoE must be the fact that the Fibre Channel protocol is carried transparently across the FCoE Ethernet cabling, meaning that existing FC hardware is fully compatible using converged Ethernet switches that have FC ports attached.
  • Fewer cables - One of the criticisms levelled at iSCSI is that it isn't practical to utilise the same cabling and switching for data and storage. Much better levels of performance are achieved by running a second dedicated Ethernet network to provide connectivity to the SAN. All of this contributes to that nest of cabling to be found around the back of most server racks. Management of large numbers of cables is always a matter for compromise, between tidiness and flexibility: the fewer the cables, the lesser the compromise.
  • Less power or cooling needed - Nuova Systems report that power savings with converged networking and storage are typically between 900 and 1250 Watts per rack. And less power used means less heat and therefore less cooling, which in turn also saves power. There's a lot of talk about saving energy at the moment.
  • Fewer adapters - We're used to NICs (Ethernet networking adaptors) and HBAs (SCSI and FC adaptors). FCoE brings us a new interface card acronym; the CNA (Converged Network Adaptor). The idea is that a single CNA can leverage the bandwidth of 10 GB Ethernet to perform the same combined function as, for example, two 4 GB FC HBAs and two 1Gb Ethernet NICs. This is an attractive proposition when low power consumption and small server footprints are desirable. It might be argued that such convergence imposes an increased risk of critical failure, but in mission critical systems it will no doubt be the norm to install an additional, redundant, CNA. Possible hardware failure is also an argument for investing in high quality hardware.
  • Standardisation - There are a number of moves to standardise FCoE, the best known of which is currently the INCITS (ANSI & ISO advisory) T11 standard. This will ensure the interoperability of FCoE components. Secondly standardisation tends to increase take-up of a technology as R&D is a much less risky proposition for manufacturers: support for standard technologies is expected of them.
  • Blade mezzanine options - It will not have escaped your notice that a number of the benefits of converged storage & data networks will appeal to those organisations that are keen advocates of blade systems and virtualisation. Brocade has been the first to announce FCoE mezzanine cards for the HP C-class Bladesystems. Presumably the mezzanines will be accompanied by the launch of some converged blade switches. Otherwise 10Gb Ethernet "passthru" switches would help.

Is iSCSI dead then?

With these arguments supporting the adoption of FCoE you might reasonably think that iSCSI has no future. But this is not the case -  a number of unique features in iSCSI ensure that it has a complimentary place alongside FCoE:

  • Cost versus performance – Organisations implementing bulk consolidation of storage for certain types of server where disk performance is not a primary consideration (such as in some ancillary application servers or low-demand file servers) will find that iSCSI is perfectly acceptable and that there are still considerable cost savings to be had from its use.
  • Support for Gigabit Ethernet - There are no plans to implement FCoE over existing Gigabit Ethernet, mainly because it doesn't have the necessary performance and features needed for most of the benefits of FCoE. iSCSI SANs have no such demands and work fine over standard Gigabit Ethernet.
  • Routing between locations - Since SCSI packets are carried on a standard IP networking stack it means that storage can exist on a network that is geographically remote from its host computers without any special technology to support it. Obviously performance issues are likely to limit the number of real opportunities to use this feature, but it is there nonetheless.
  • Protected Financial Investment - A lot of money has been invested in iSCSI: Dell's acquisition of EqualLogic is reputed to have cost $1.4 billion; HP recently paid $360 million for LeftHand; and Qlogic, an outspoken champion of FCoE, shipped its QMH4062 ISCSI offload mezzanine card last July. In other words, iSCSI will not disappear over night; supply and demand operate in equilibrium in the technology industry.

Conclusion

Clearly FCoE and iSCSI are complimentary storage technologies. As 10Gb Ethernet becomes affordable and reaches wider adoption, the benefits of FCoE will be ever more appealing. This is not likely to begin until late 2009 though. There has been talk of convergence between Infiniband and FCoE as well, so FCoE has the potential to be the standard protocol for almost all high-end high performance storage networking requirements. Additionally the Ethernet bandwidth roadmap has some direction with proposals for 40Gb/sec and 100Gb/sec.

Fibre Channel over Ethernet promises much & we'll be keeping a close eye on developments.

Page created on 21 October 2008 by Alice Carr.