Networking on steroids
Broadcom has new single chip iSCSI, TOE, RDMA
demonstrating its industry leading NetXtreme II? C-NIC (converged network interface controller) technology at this week's Windows Hardware Engineering Conference in Booth #203. This innovative single-chip C-NIC demonstration simultaneously runs three specific functions -- accelerated TOE (TCP Offload Engine) for data networking, accelerated iSCSI for block storage networking and accelerated RDMA (Remote Direct Memory Access) for high-performance server clustering -- over Ethernet. By converging disparate network traffic over Ethernet, Broadcom's NetXtreme II C-NICs enable a lower total cost of ownership (TCO) versus configuring and running three separate and disparate networks.
Ok what the hell are these technolgies you ask?
RDMA- Remote Direct Memory Access
is a concept whereby two or more computers communicate via Direct Memory Access directly from the main memory of one system to the main memory of another. As there is no CPU, cache, or context switching overhead needed to perform the transfer, and transfers can continue in parallel with other system operations, this is particularly useful in applications where high thoughput, low latency networking is needed such as in massively parallel Linux clusters
Pretty cool huh?
It gets better. RDMA is often used with Infiniband but...
An alternate proposal is RDMA over TCP/IP, in which the TCP/IP protocol is used to move the data over a commodity data networking technology such as Gigabit Ethernet. Unlike conventional TCP/IP implementations, the RDMA implementation would have its TCP/IP stack implemented on the network adapter card, which would thus act as a I/O processor, taking up the load of RDMA processing.
Imagine a cluster of Xserves that can transfer data between them without having to pass the data through the TCP/IP stack and kernel. Speeeeeeed
iSCSI
The iSCSI protocol uses TCP for its data transfer. Unlike other network storage protocols, such as Fibre Channel (which is the foundation of most SANs), it requires only the simple and ubiquitous Ethernet interface (or any other TCP/IP-capable network) to operate. This enables low-cost centralization of storage without all of the usual expense and incompatibility normally associated with Fibre Channel storage area networks.
Critics of iSCSI expect worse performance than Fibre Channel due to the protocol overhead TCP/IP adds to the communication between client and storage. However new techniques like TCP Offload Engine (TOE) help in reducing this overhead. And tests have shown excellent performance of iSCSI SANs, whether TOEs or plain Gigabit Ethernet NICs were used
iSCSI is cool. Currently with Fiber SAN you have to consolidate storage behind your Fiber Switch. How'd you like to utilize your Gigabit network for easy placement of storage resources yet maintain excellent performance. iSCSI allows this if your network is up to snuff. How does 200MBps duplex sound to you? It does this by transferring the SCSI protocol commands(small) over the network thus it works across LANS, MANS, WANS or whatever you have that's IP. No this isn't NAS...NAS only works at the file level so it never appears as though its direct attached storage.
TCP Offload Engine
Networking speed gradually raised over the years. What started as a protocol build for unreliable low speed networks (few Kbytes per second) is now required to run at 1 Gigabit per second. The TCP software implementations on host systems require extensive computing power. Gigabit TCP communication, alone, eats up 100% of a 2.4 GHz Pentium processor.
See? We're choking off computer performance with Gigabit.
Broadcom now has a single chip solution(read more affordable) that can handle 3 hot new technologies. I can see this hitting on future Xserves and then trickling down to Powermacs. Xgrid might be able to hop right on RDMA to speed up clustered performance. Imagine if we get Quad 3GHz chips one day that can blast out gigabit ethernet with barely any CPU intervention. I want...and you should to.
demonstrating its industry leading NetXtreme II? C-NIC (converged network interface controller) technology at this week's Windows Hardware Engineering Conference in Booth #203. This innovative single-chip C-NIC demonstration simultaneously runs three specific functions -- accelerated TOE (TCP Offload Engine) for data networking, accelerated iSCSI for block storage networking and accelerated RDMA (Remote Direct Memory Access) for high-performance server clustering -- over Ethernet. By converging disparate network traffic over Ethernet, Broadcom's NetXtreme II C-NICs enable a lower total cost of ownership (TCO) versus configuring and running three separate and disparate networks.
Ok what the hell are these technolgies you ask?
RDMA- Remote Direct Memory Access
is a concept whereby two or more computers communicate via Direct Memory Access directly from the main memory of one system to the main memory of another. As there is no CPU, cache, or context switching overhead needed to perform the transfer, and transfers can continue in parallel with other system operations, this is particularly useful in applications where high thoughput, low latency networking is needed such as in massively parallel Linux clusters
Pretty cool huh?
It gets better. RDMA is often used with Infiniband but...An alternate proposal is RDMA over TCP/IP, in which the TCP/IP protocol is used to move the data over a commodity data networking technology such as Gigabit Ethernet. Unlike conventional TCP/IP implementations, the RDMA implementation would have its TCP/IP stack implemented on the network adapter card, which would thus act as a I/O processor, taking up the load of RDMA processing.
Imagine a cluster of Xserves that can transfer data between them without having to pass the data through the TCP/IP stack and kernel. Speeeeeeed
iSCSI
The iSCSI protocol uses TCP for its data transfer. Unlike other network storage protocols, such as Fibre Channel (which is the foundation of most SANs), it requires only the simple and ubiquitous Ethernet interface (or any other TCP/IP-capable network) to operate. This enables low-cost centralization of storage without all of the usual expense and incompatibility normally associated with Fibre Channel storage area networks.
Critics of iSCSI expect worse performance than Fibre Channel due to the protocol overhead TCP/IP adds to the communication between client and storage. However new techniques like TCP Offload Engine (TOE) help in reducing this overhead. And tests have shown excellent performance of iSCSI SANs, whether TOEs or plain Gigabit Ethernet NICs were used
iSCSI is cool. Currently with Fiber SAN you have to consolidate storage behind your Fiber Switch. How'd you like to utilize your Gigabit network for easy placement of storage resources yet maintain excellent performance. iSCSI allows this if your network is up to snuff. How does 200MBps duplex sound to you? It does this by transferring the SCSI protocol commands(small) over the network thus it works across LANS, MANS, WANS or whatever you have that's IP. No this isn't NAS...NAS only works at the file level so it never appears as though its direct attached storage.
TCP Offload Engine
Networking speed gradually raised over the years. What started as a protocol build for unreliable low speed networks (few Kbytes per second) is now required to run at 1 Gigabit per second. The TCP software implementations on host systems require extensive computing power. Gigabit TCP communication, alone, eats up 100% of a 2.4 GHz Pentium processor.
See? We're choking off computer performance with Gigabit.
Broadcom now has a single chip solution(read more affordable) that can handle 3 hot new technologies. I can see this hitting on future Xserves and then trickling down to Powermacs. Xgrid might be able to hop right on RDMA to speed up clustered performance. Imagine if we get Quad 3GHz chips one day that can blast out gigabit ethernet with barely any CPU intervention. I want...and you should to.
Comments
Whether the tech challenges Infiniband is not really the issue, as this being available will bring in a whole new class of potential customers for whom fiber optic stuff is just too expensive and cumbersome.
Besides, I want to bump this thread so it won't get lost in all the silly Xbox 360 junk.
We all want faster computers so let's unburden the CPU as much as we can.
And yes the Xserve is slot starved. It's caused us grief. Apple add another slot and put the video on the motherboard. You don't need a freakin 9600 but 8-32MB would be fine.
Originally posted by hmurchison
Apple add another slot and put the video on the motherboard. You don't need a freakin 9600 but 8-32MB would be fine.
Or at least make it onboard standard with the ability to expand if necessary. I've always wondered why they did that in the first place.
Originally posted by wmf
I don't understand the hype about iSCSI. Sure, it's cheaper than FC, but you still need expensive software like Xsan to share data. Just give me wire-speed NFSv4 (it is possible).
Does NFSv4 allow you to access block level data over the network? If so that would be really cool. I agree, I'd like to see SAN software come down to $499 per node but the nice thing is that you don't need a metadata controller with iSCSI as far as I can tell.
iSCSI doesn't require a metadata controller, but Xsan does. Anyway, with the money you save buy not buying SAN software you could get an NFS server.
Originally posted by wmf
What is so special about block level storage?
iSCSI doesn't require a metadata controller, but Xsan does. Anyway, with the money you save buy not buying SAN software you could get an NFS server.
Block-Level access to storage lets you setup your Database or Exchange(or groupware server) on the SAN build redundancy and utilize your Fibre or Gigabit links for speed. Plus adding more storage is as simple as plugging it in. An NFS server isn't going to allow you the storage flexibility of IP-SAN/iSCSI or the speed of 2G/4G Fibre.
There's a reason why SAN and "Disk 2 Disk 2 Tape" are the hottest things going. They cost more upfront but the TCO in administration and functionality is worth it.
Zetera IP based storage
Zetera aims to help the networked home consolidate their storage over IP without the high cost of NAS, SAN and iSCSI. Their technology allows multiple computers to access central storage hooked into the wireless/wired router. Each strip is assigned an IP address and each stripe contains the volume and a mirror of an adjacent drive's volume. As long as you don't lose more than one adjacent drive you'll have the deads drives data mirrored safely. No controller is required and it doesn't use TCP but rather UDP for lower overhead and faster performance.
Zetera is claiming they will achieve performance that exceeds todays NAS at a cheaper price because they are able to use IP as a bridge for their proprietary protocols. Their device can strip not only the volumes but the communication channels.
The cost savings from obviating the need for a specialized controller mean that Zetera IP storage should be cheaper. The only downside seems to be the proprietary nature of their solution. I hear it requires client side software thus Mac users will probably be waiting.