I thought I'd just copy these in so that we all have a frame of reference as to the relative pros and cons: -
[quote]From <a href="http://www.hypertransport.org/doc_faq.htm[qb
At what clock speeds does HyperTransportÂ technology operate?
HyperTransportÂÂ* technology devices are designed to operate at multiple clock speeds from 200MHz up to 800MHz, and utilizes double data rate technology transferring two bits of data per clock cycle, for an effective transfer rate of up to 1,600Mb/sec in each direction. Since transfers can occur in both directions simultaneously, an aggregate transfer rate of 6.4 Gigabytes per second in a 16 bit HyperTransportÂÂ* I/O Link and an aggregate transfer rate of 12.8 Gigabytes per second in a 32-bit HyperTransportÂÂ* I/O Link can be achieved. To allow for system design optimization, the clocks of the receive and transmit links may be set at different rates.
What are the differences and similarities between Infiniband and HyperTransportÂ technology?
HyperTransportÂÂ* technology is a chip-to-chip interconnect primarily intended for use on a system board within distances of up to 24 inches. Infiniband is primarily a box to box link and can cover distances of up to 17 meters. Infiniband can deliver data at rates up to 4 gigabytes per second into a system and HyperTransportÂÂ* technology can easily transport this data within the system, unlike traditional busses that cannot handle data rates this fast. Infiniband is not a technology alternative to HyperTransportÂ .</strong><hr></blockquote>
[quote]From <a href="http://www.rapidio.org/faq[qb
Q: How does the RapidIO technology compare to InfiniBand?
A: InfiniBand is optimized for links between server chassis within an enterprise cluster to form a SAN. In contrast, the RapidIO technology replaces the traditional microprocessor and peripheral bus and is optimized for use between chips on a circuit board. You will likely find RapidIO interconnects co-existing inside InfiniBand subsystems. For example, the RapidIO technology might provide the bridge to PCI-X slots inside a single serer. You may also find the RapidIO technology providing concurrency and bandwidth aggregation inside the storage subsystem. We are quite confident that you will find the RapidIO technology inside networking devices that attach to InfiniBand.
Both InfiniBand and the RapidIO technology reach into the card-to-card communications domain. In this domain where we overlap, InfiniBand provides a more abstracted interface to allow complete decoupling of the subsystems. To accomplish this abstraction, InfiniBand requires modification of legacy software, more transistors to implement, and specialized management software.</strong><hr></blockquote>
I'm trying to find some interesting (?) info on Infiniband, which I had but have mislaid.
However, there are some interesting points here.
1. If you are looking for a signal to travel more than 24 inches (but more likely 18-20 inches), HyperTransport is not your boy.
2. I can't find any solid data, but the quote implies that RapidIO probably has a similar mission in life.
3. In neither RapidIO's or HyperTransport's FAQ, neither of them mention the other which implies a certain sensitivity.
My personal opinion is that. in a high-density blade environment , InfiniBand is probably the interconnect fabric of choice at this time. However the current roadmap ends at a 32X 4 Gbyte/sec interconnect, which I feel may be a constraint as time goes on.
Also I do feel that the 17 meter/55 foot limits will need to be addressed either by repeater technology or by improving the bus technology, 17 meters can get used up quite quickly if you think that a rack can be 42U tall and 22 inches wide, and that cables will have to go into false floors etc.
That said, using a switched fabric, you could still do something like the following.
Bay 9 racks to each other.
Put an InfiniBand fabric switch in the middle rack, alongside two FC-SW based SAN/RAID controllers each controlling some 24 shelves x 10 drives x 240GB, so that you have some 56TB hanging off each controller or 112TB in all.
24 shelves x 3 U = 72U, which split over 8 racks (4 each side of the central controllers) is 3 shelves/rack or 9U/rack. Leaves 33U in each rack. Take up 12U by putting in 4 x 3U of Blade Servers (Thus getting 16 shelves). Each shelf connected to central InfiniBand switch, but shelf has internal HyperTransport bus running at 12.8GB/sec. Leaves 21U
Each shelf has the ability to accomodate up to 8 x 1.75" blade modules, which could be 2-way low-voltage G5 blades (similar to the chip destined for the 2004 iteration of the Powerbook design, running at 1.4 GHz, and 1GB of memory, plus a small, fast internal HD [e.g. 10GB] to handle virtual memory and the booted system image) + an InfiniBand repeater/Shelf Controller (2") + 2 x 1.5" Power Supply (3") = 19"
By the configurable shelf controller, the CPU blades can be dynamically partitioned to act as SMP blocks of between 2-16 processors, whilst the InfiniBand repeaters will be used to join up to 4 Blade shelves with a single 5U I/O expansion chassis, leaving 16U
Internally, the expansion chassis will use a HyperTransport main bus bridging to upto 12 PCI-X (or the flavour of the day) slots, to provide FireWire, Ethernet (100 mbit/1 gbit/10 gbit) plus specialist cards for data acquisition, telephony interfaces, etc.
The repeater mechanism will be important in reducing latency, by removing the need for such I/O chassis systems to be directly connected the core Infiniband switch.
Use up the remaining 16U with 3 5U UPS units + 1 1U Blanking plate.
The result is an aggregate 256-processor farm, with 256GB of RAM, 96 slots of PCI-X I/O + 112TB of SAN main data storage. Throw in a mechanism for controlling a coherent single-memory image cache and you have a really neat solution that can be configured for any type of scientific, MIS, visualisation, streaming media or rendering problem.
And the nice thing is you can do it all in an area of approximately 254 square ft in a room about 12 ft. high, excluding tape backup and network infrastructure of course.
There should also be some location in the set up where you can cook some food.
[ 07-04-2002: Message edited by: Mark- Card Carrying FanaticRealist ]</p>