TOE seems like a technology that is between a rock and a hard place. High end machines can just throw cpus at the problem, low end machines cant afford to begin with, and by the time it is cheap enough cpus will be dual core. 10gigE will change that, I guess I see the TOE vendors waiting for 10gigE to really hit the market.
I agree with much of what you say Mr. Pie, but I'm afraid when it comes to extreme multimedia stuff - HD Video (which is quickly becoming standard on all Macs) and audio, computers need all the CPU horsepower they can get, and loosing a CPU just to handle an IP stack pretty much negates all that could perhaps be gained by distributed processing: it's a pretty good example of the cure being no better than the disease.
I like Programmer's mentioning of the POWER5 fastpath trick - if whatever the next chip Apple comes out with takes advantage of this, this would be a wonderful use to put it to, however - short of this - some form of small, dedicated parallel processor is pretty much a must: I need my 20 Lexicon quality reverbs/real- time HD layered ChromaKey effects etc ... and the only way to guarantee that kind of stuff, is if I have both processors crunching away on my media App.
...loosing a CPU just to handle an IP stack pretty much negates all that could perhaps be gained by distributed processing: it's a pretty good example of the cure being no better than the disease.
I like Programmer's mentioning of the POWER5 fastpath trick ...
This reminds me -- one reason that FastPath (or an offchip TOE) is especially important is because interrupting a deeply pipelined, cache dependent processor is really brutal to its performance. Just dealing with the context changes loses you a great deal of performance -- even if the time spent actually processing the TCP packets is zero. The dedicated TCP processing circuitry, however, is designed to respond quickly and exclusively to these message and thus is very efficient about it... and it leaves the main threads undisturbed.
I have not seen any mention of FastPath from IBM in some time.
iReady (now part of nVidia) and Broadcom supposedly have cheap single-chip 1Gbps TOE chips. Chelsio and NetEffect claim to have 10Gbps TOEs, but those are $$$$.
The downside of TOE that has been discovered (in the few real-world implementations that exist) is that talking to the TOE costs almost as much as just doing the TCP processing.
I agree that eventually TOE will become free, but I am skeptical about the performance benefits.
I have not seen any mention of FastPath from IBM in some time.
...
The downside of TOE that has been discovered (in the few real-world implementations that exist) is that talking to the TOE costs almost as much as just doing the TCP processing.
I agree that eventually TOE will become free, but I am skeptical about the performance benefits.
IBM said hardly anything about FastPath even when they were talking about it. Looking at their server performance numbers compared to their SPEC numbers, however, seems to indicate that they did something significant in their TCP stack handling.
I'm not surprised about the off-chip TOE performance issues -- that's why I emphasize the benefits of on-chip implementations. In the future we may see other kinds of on-chip hardware useful for handling this kind of work (and other things).
I have not seen any mention of FastPath from IBM in some time.
iReady (now part of nVidia) and Broadcom supposedly have cheap single-chip 1Gbps TOE chips. Chelsio and NetEffect claim to have 10Gbps TOEs, but those are $$$$.
The downside of TOE that has been discovered (in the few real-world implementations that exist) is that talking to the TOE costs almost as much as just doing the TCP processing.
Then someone is doing something incorrectly. Generally, a chip's GPIOs (general purpose input output) are tied to a memory buffer (register), and are managed by onboard circuitry. That is, the chip doesn't have to strobe the IOs, it just dumps data to the output register for output, and waits for input interrupts before loading up the input registers. The port circuitry takes care of the command signaling, and ultimately I'd hope that a TOE would be intelligent enough to merely suck up some binary data and spit it out as ethernet.
If this seems fishy to you then build a solution, hunt down a Venture Capital Firm and bring a cheaper solution to market.
Quote:
Originally posted by Splinemodel
Then someone is doing something incorrectly. Generally, a chip's GPIOs (general purpose input output) are tied to a memory buffer (register), and are managed by onboard circuitry. That is, the chip doesn't have to strobe the IOs, it just dumps data to the output register for output, and waits for input interrupts before loading up the input registers. The port circuitry takes care of the command signaling, and ultimately I'd hope that a TOE would be intelligent enough to merely suck up some binary data and spit it out as ethernet.
Programmer and I have different theories about FastPath, but I'll say no more.
Splinemodel: TOE chips aren't attached directly to the processor; they have to go through PCI, which has its own limitations. But I was talking more about the software overhead anyway. To get data to the TOE you have to potentially enter the kernel, set up a descriptor, use a PIO to ring the doorbell, etc. Likewise on the RX side.
Programmer and I have different theories about FastPath, but I'll say no more.
Splinemodel: TOE chips aren't attached directly to the processor; they have to go through PCI, which has its own limitations. But I was talking more about the software overhead anyway. To get data to the TOE you have to potentially enter the kernel, set up a descriptor, use a PIO to ring the doorbell, etc. Likewise on the RX side.
What are workstations currently using to solve the distributed processing problem? Perhaps we'll see something migrating down very shortly?
What are workstations currently using to solve the distributed processing problem? Perhaps we'll see something migrating down very shortly?
I'm not sure what you mean. Workstations still tend to use non-TOE Ethernet. Clusters often use Myrinet or Infiniband, both of which have full transport protocol offload, but I don't see those moving downmarket.
But I was talking more about the software overhead anyway. To get data to the TOE you have to potentially enter the kernel, set up a descriptor, use a PIO to ring the doorbell, etc. Likewise on the RX side.
And this seems to be why TOE is moving towards the specialised area of iSCSI acceleration. The iSCSI protocol is very regimented ( being SCSI ) and so all of the buffers required in the hosts memory can be preallocated. The DMA descriptors can be setup in advance etc etc. The TOE can simply offload incoming data into the buffers while the host consumes them.
What I have read indicates that the real value of TOE comes in receiving data ( not tranmitting ) due to difficulty in making optimisations in the TCP stacks memory handling. Im pretty sure that hosts have an easy time ( excluding PCI saturation ) of pushing the limits of gigabit, particularly in the streaming scenario that started this thread ( TOEs dont seem like a general purpose solution, but might become widely available due to cost ).
Quote:
What are workstations currently using to solve the distributed processing problem? Perhaps we'll see something migrating down very shortly?
It all depends on the parallelism of the problem being solved. Things like distributed compilation work quite nicely over ethernet ( files can be copied while you edit them ). The more communication a task requires the more value you get out of low latency low overhead mechanisms. Hence the use of inifiniband, fibre channel etc. Im pretty sure that both PCI express and Hypertransport are getting out of box connectors specified primarily to attack these sorts of problems.
Of course, at the high end supercomputers are built in big boxes so that high speed interconnects can be used, even shared memory ( although big SMP boxes arent real popular anymore ).
Comments
Originally posted by mmmpie
TOE seems like a technology that is between a rock and a hard place. High end machines can just throw cpus at the problem, low end machines cant afford to begin with, and by the time it is cheap enough cpus will be dual core. 10gigE will change that, I guess I see the TOE vendors waiting for 10gigE to really hit the market.
I agree with much of what you say Mr. Pie, but I'm afraid when it comes to extreme multimedia stuff - HD Video (which is quickly becoming standard on all Macs) and audio, computers need all the CPU horsepower they can get, and loosing a CPU just to handle an IP stack pretty much negates all that could perhaps be gained by distributed processing: it's a pretty good example of the cure being no better than the disease.
I like Programmer's mentioning of the POWER5 fastpath trick - if whatever the next chip Apple comes out with takes advantage of this, this would be a wonderful use to put it to, however - short of this - some form of small, dedicated parallel processor is pretty much a must: I need my 20 Lexicon quality reverbs/real- time HD layered ChromaKey effects etc ... and the only way to guarantee that kind of stuff, is if I have both processors crunching away on my media App.
Sincerely
Still-Kinda-Freaked-At-What-I-Can-Do-With-All-This-Stuff
OT
Originally posted by OverToasty
...loosing a CPU just to handle an IP stack pretty much negates all that could perhaps be gained by distributed processing: it's a pretty good example of the cure being no better than the disease.
I like Programmer's mentioning of the POWER5 fastpath trick ...
This reminds me -- one reason that FastPath (or an offchip TOE) is especially important is because interrupting a deeply pipelined, cache dependent processor is really brutal to its performance. Just dealing with the context changes loses you a great deal of performance -- even if the time spent actually processing the TCP packets is zero. The dedicated TCP processing circuitry, however, is designed to respond quickly and exclusively to these message and thus is very efficient about it... and it leaves the main threads undisturbed.
iReady (now part of nVidia) and Broadcom supposedly have cheap single-chip 1Gbps TOE chips. Chelsio and NetEffect claim to have 10Gbps TOEs, but those are $$$$.
The downside of TOE that has been discovered (in the few real-world implementations that exist) is that talking to the TOE costs almost as much as just doing the TCP processing.
I agree that eventually TOE will become free, but I am skeptical about the performance benefits.
Originally posted by wmf
I have not seen any mention of FastPath from IBM in some time.
...
The downside of TOE that has been discovered (in the few real-world implementations that exist) is that talking to the TOE costs almost as much as just doing the TCP processing.
I agree that eventually TOE will become free, but I am skeptical about the performance benefits.
IBM said hardly anything about FastPath even when they were talking about it. Looking at their server performance numbers compared to their SPEC numbers, however, seems to indicate that they did something significant in their TCP stack handling.
I'm not surprised about the off-chip TOE performance issues -- that's why I emphasize the benefits of on-chip implementations. In the future we may see other kinds of on-chip hardware useful for handling this kind of work (and other things).
Originally posted by wmf
I have not seen any mention of FastPath from IBM in some time.
iReady (now part of nVidia) and Broadcom supposedly have cheap single-chip 1Gbps TOE chips. Chelsio and NetEffect claim to have 10Gbps TOEs, but those are $$$$.
The downside of TOE that has been discovered (in the few real-world implementations that exist) is that talking to the TOE costs almost as much as just doing the TCP processing.
Then someone is doing something incorrectly. Generally, a chip's GPIOs (general purpose input output) are tied to a memory buffer (register), and are managed by onboard circuitry. That is, the chip doesn't have to strobe the IOs, it just dumps data to the output register for output, and waits for input interrupts before loading up the input registers. The port circuitry takes care of the command signaling, and ultimately I'd hope that a TOE would be intelligent enough to merely suck up some binary data and spit it out as ethernet.
If this seems fishy to you then build a solution, hunt down a Venture Capital Firm and bring a cheaper solution to market.
Originally posted by Splinemodel
Then someone is doing something incorrectly. Generally, a chip's GPIOs (general purpose input output) are tied to a memory buffer (register), and are managed by onboard circuitry. That is, the chip doesn't have to strobe the IOs, it just dumps data to the output register for output, and waits for input interrupts before loading up the input registers. The port circuitry takes care of the command signaling, and ultimately I'd hope that a TOE would be intelligent enough to merely suck up some binary data and spit it out as ethernet.
Splinemodel: TOE chips aren't attached directly to the processor; they have to go through PCI, which has its own limitations. But I was talking more about the software overhead anyway. To get data to the TOE you have to potentially enter the kernel, set up a descriptor, use a PIO to ring the doorbell, etc. Likewise on the RX side.
Originally posted by wmf
Programmer and I have different theories about FastPath, but I'll say no more.
Splinemodel: TOE chips aren't attached directly to the processor; they have to go through PCI, which has its own limitations. But I was talking more about the software overhead anyway. To get data to the TOE you have to potentially enter the kernel, set up a descriptor, use a PIO to ring the doorbell, etc. Likewise on the RX side.
What are workstations currently using to solve the distributed processing problem? Perhaps we'll see something migrating down very shortly?
Originally posted by OverToasty
What are workstations currently using to solve the distributed processing problem? Perhaps we'll see something migrating down very shortly?
I'm not sure what you mean. Workstations still tend to use non-TOE Ethernet. Clusters often use Myrinet or Infiniband, both of which have full transport protocol offload, but I don't see those moving downmarket.
Originally posted by wmf
But I was talking more about the software overhead anyway. To get data to the TOE you have to potentially enter the kernel, set up a descriptor, use a PIO to ring the doorbell, etc. Likewise on the RX side.
And this seems to be why TOE is moving towards the specialised area of iSCSI acceleration. The iSCSI protocol is very regimented ( being SCSI ) and so all of the buffers required in the hosts memory can be preallocated. The DMA descriptors can be setup in advance etc etc. The TOE can simply offload incoming data into the buffers while the host consumes them.
What I have read indicates that the real value of TOE comes in receiving data ( not tranmitting ) due to difficulty in making optimisations in the TCP stacks memory handling. Im pretty sure that hosts have an easy time ( excluding PCI saturation ) of pushing the limits of gigabit, particularly in the streaming scenario that started this thread ( TOEs dont seem like a general purpose solution, but might become widely available due to cost ).
What are workstations currently using to solve the distributed processing problem? Perhaps we'll see something migrating down very shortly?
It all depends on the parallelism of the problem being solved. Things like distributed compilation work quite nicely over ethernet ( files can be copied while you edit them ). The more communication a task requires the more value you get out of low latency low overhead mechanisms. Hence the use of inifiniband, fibre channel etc. Im pretty sure that both PCI express and Hypertransport are getting out of box connectors specified primarily to attack these sorts of problems.
Of course, at the high end supercomputers are built in big boxes so that high speed interconnects can be used, even shared memory ( although big SMP boxes arent real popular anymore ).
Horses for courses.
Originally posted by wmf
Programmer and I have different theories about FastPath, but I'll say no more.
Okay, that's just blatant baiting.
Do you think they removed it from the POWER5 spec? Or something else? Any evidence to back that up?
Nvidia have turned on their TOE engine in their nforce pro chipset.
http://www.nvidia.com/page/nforce_Pro.html
They have had firewall functionality in previous chipsets, looks like this is an advance on that.
TOE is coming, TOE is coming.