Severe bottlenecks in today's computers
So we've gone from single-core to dual-core. We've gone from single chips to multiple chips. So we've got two dual-core chips in our computers. Soon we'll be going from dual-core to quad-core. In a few years, we'll have four 8-core chips in our computers (maybe 
).
This is great. Not so for the average consumer though. It's fun to have all these CPUs working for you...but if you don't have the RAM or hard drives to back these CPUs up...you've got yourself a potentially fast computer that is being held back.
I bought my Mac Pro with 1GB of RAM and a 250GB hard drive. Why? Because I knew I'd be making the computer my media center (PVR with eyeTV, iTunes to stream movies and music to other computers in the house, and still be able to use my computer while all of this is happening. So I needed the processing power.
Now, I knew I wouldn't be going far with 1GB of RAM and a 250GB hard drive...but I hit the limits of what a single 7200 hard drive can do and what 1GB of RAM can hold...and I hit that limit FAST. When you've got eyeTV recording a show and converting other shows to H.264 for archival (goes to show that 250GB is really nothing when you've got 3-4 hours of recording per day happening) and you're doing other activities that need fast disk access.
Yes, I am aware of fantastic technologies such as RAID and 10k RPM HDDs. I plan on adding 3 more hard drives when the money finally falls onto my lap and getting a RAID going so that I don't have to sit in front of a Mac Pro and watch a rainbow-colored beachball spinning and laughing at me (oh, it laughs...it mocks me as I sit in incredulity of the whole situation.)
Yes...I'm gonna say it. Hard drives are ridiculously slow. Things have got to change. Hooking drives up in RAID configs or buying 10k RPM HDDs is unrealistic at a consumer's standpoint. I'm doing some very consumerish things here. Ok...I bought a Mac Pro and I should be acting like a big boy (a pro if you will), such it up and buy the expensive gadgets that will allow me to access my hard drive faster and more efficiently.
But consumers already have dual-core setups...and I'm sure they'll be doing something similar to what I'm doing (managing everything from the computer) in a few years. Are hard drives going to keep up or are they going to cripple future computers?
Or will computer manufacturers start shipping consumer computers with RAID setups so that most of the full potential speed is achieved out-of-the-box. Instead of giving the options for 1 500GB HD, stick 4 125GB HDs in there. Instead of 1TB, 4 250GB.
It's either that or, like I said, something has got to improve or replace hard drive tech so that they can keep up with what I consider consumerish activities (recording shows, encoding show, copying files all simultaneously.)
).This is great. Not so for the average consumer though. It's fun to have all these CPUs working for you...but if you don't have the RAM or hard drives to back these CPUs up...you've got yourself a potentially fast computer that is being held back.
I bought my Mac Pro with 1GB of RAM and a 250GB hard drive. Why? Because I knew I'd be making the computer my media center (PVR with eyeTV, iTunes to stream movies and music to other computers in the house, and still be able to use my computer while all of this is happening. So I needed the processing power.
Now, I knew I wouldn't be going far with 1GB of RAM and a 250GB hard drive...but I hit the limits of what a single 7200 hard drive can do and what 1GB of RAM can hold...and I hit that limit FAST. When you've got eyeTV recording a show and converting other shows to H.264 for archival (goes to show that 250GB is really nothing when you've got 3-4 hours of recording per day happening) and you're doing other activities that need fast disk access.
Yes, I am aware of fantastic technologies such as RAID and 10k RPM HDDs. I plan on adding 3 more hard drives when the money finally falls onto my lap and getting a RAID going so that I don't have to sit in front of a Mac Pro and watch a rainbow-colored beachball spinning and laughing at me (oh, it laughs...it mocks me as I sit in incredulity of the whole situation.)
Yes...I'm gonna say it. Hard drives are ridiculously slow. Things have got to change. Hooking drives up in RAID configs or buying 10k RPM HDDs is unrealistic at a consumer's standpoint. I'm doing some very consumerish things here. Ok...I bought a Mac Pro and I should be acting like a big boy (a pro if you will), such it up and buy the expensive gadgets that will allow me to access my hard drive faster and more efficiently.
But consumers already have dual-core setups...and I'm sure they'll be doing something similar to what I'm doing (managing everything from the computer) in a few years. Are hard drives going to keep up or are they going to cripple future computers?
Or will computer manufacturers start shipping consumer computers with RAID setups so that most of the full potential speed is achieved out-of-the-box. Instead of giving the options for 1 500GB HD, stick 4 125GB HDs in there. Instead of 1TB, 4 250GB.
It's either that or, like I said, something has got to improve or replace hard drive tech so that they can keep up with what I consider consumerish activities (recording shows, encoding show, copying files all simultaneously.)
Comments
There are ALU operations (+, -, *, &, |, etc) that are part of C.
There are addressing operations (*, &, =, etc) that are part of C.
There are branching operations ({}, if, else, loops, etc) that are part of C.
There are conditional compares (==, !=, <=, &&, etc) that are part of C.
All of these things translate almost directly into actual hardware features of CPUs. But with C, in order to run processes in parallel, programmers need to use cumbersome methods. Functions and directives were intended to refer to blocks of code, not hardware features. This is why all modern programming languages I'm aware of (other than verilog or vhdl) are unsuitable for programming on machines that have parallelism features built into the hardware.
Until high level programming languages have the VHDL-esque "process() begin ... end" functionality at the language level, parallel program implementation will always be junk.
As an analogue, AutoCAD has been more or less the basic CAD program for a long time. It's shit, but then again so is the x86 architecture. AutoCAD has a fairly pure 2D design flow. Lately engineers have realized they've had to take the plunge and move to pure 3D. It takes a lot of time to relearn, and in the end the job is probably a little bit harder, but the rewards are obvious. Software programmers are going to have to face the facts that they're going to need to rethink the way they do things, give into the fact that their jobs could become more demanding (as will always be the case as new products themselves require ever more complexity) and ditch the 1970's era (and at times 80's era) programming paradigms that they've managed to hold on to for so long.
There's always more ways to get speed the problem is keeping them affordable.
Good RAID controllers support online capacity expansion. Meaning, you want more space? ... open the case and plug a drive in. Don't even bother turning the machine off. Smart software could handle the rest. Failed drive? Simply replace it with another.
Unforunately, user-friendly, bullet-proof RAID isn't here yet. It isn't for lack of trying either. It's just that the technology really is that complicated. Getting RAID ready for the masses isn't rocket science, it's more complex than rocket science.
In other news, my five drive SATA raid is currently bottlenecked by the bandwidth of the PCI slot it is plugged into.
Better filesystems like ZFS or OneFS (www.isilon.com) come without the headaches of RAID. I want my storage to be added to one large contiguous pool. I don't want to worry about volume managers and RAID controllers. One pool of storage ..one global namesource. The future.
Upgrading from 1GB to 2GB of RAM actually appeared to SLOW down InDesign's performance dramatically. Removing the additional DIMMs sped the machine back up again. I don't know if there was a problem with the FB-DIMMs I installed, but they were EXACTLY the same part as the DIMMs that shipped with the machine and they appeared to be fine.
I'm guessing 4GB is the realistic minimum for a Mac Pro, with 8GB being recommended - but I haven't seen a Mac Pro that is 'fast'.
RAID makes a big difference to performance. A Quad with RAID 0 slaughters a stock Mac Pro in the real-world. I wonder how a four-drive RAID 0 stripe-set in a Mac Pro would perform?
I think RAID is a dead end.
Better filesystems like ZFS or OneFS (www.isilon.com) come without the headaches of RAID. I want my storage to be added to one large contiguous pool. I don't want to worry about volume managers and RAID controllers. One pool of storage ..one global namesource. The future.
RAID is the de-facto standard for hard disks in enterprise. Like a couple others said, RAID can be transparent to the user. Everyone thought hard drive were hard to install years ago, now a nine year old can do it. The same thing will happen with RAID, it will turn into a simple idiot-proof GUI. (the way the world likes it).
I generally don't consider RAID 0 to be "RAID" from a consumer perspective. For developers interested in the underlying technology, RAID 0 is similar to other RAID types. But from a consumer perspective it isn't similar because it isn't "Redundant". In fact it is the exact opposite.
When lusting for a consumer-grade zero-config RAID, people are generally thinking of RAID 1, 3 or 5. These offer redundancy as well as improved throughput.
That's exactly what RAID is. What you hate are today's troublesome implementations not the concept itself. If it were done with zero user administration other than plugging in drives. There is no downside.
There are downsides. Having redundancy does involve some performance penalties and removing potential capacity. Simple striping is incredibly easy to set up, is the fastest, but you lose one drive and you lose everything on the volume.
But I do think RAID is fine, it's not a dead end. Maybe the numbered standards will go away eventually, but the general idea behind them won't. If you handle large files, then striping is where you are most likely to see a benefit. Striping does little to nothing for latency in a single user system, so if you handle tons of small files, then the speed boost won't be noticeable.
Hard drive performance will improve with areal density improvements.
It depends. Sequential transfer rate is increasing, but the gap between STR and seek time is also increasing. Many apps are now seek-bound, and it will only get worse. And capacity is increasing faster than STR, meaning traditional backups will take longer.
There's always more ways to get speed the problem is keeping them affordable.
That is a great summary. Four 250GB drives are cheaper and much faster than one 750GB, but that only helps high-end users. An average user might be better off with four 80GB instead of one 250GB, but they won't get it since it costs more.
I don't think any pro would use such a setup because of data loss risk. Unless they had a very frequent backup system.
Any "pro" would backup frequently, so data loss isn't a problem whether you have RAID or not.
Besides, data because of drive failure is not a common problem these days.
I see that data loss because of user error, viruses, controller failure, or another component failure are much more prevelent than the HD failure.
Any "pro" would backup frequently, so data loss isn't a problem whether you have RAID or not.
Besides, data because of drive failure is not a common problem these days.
I see that data loss because of user error, viruses, controller failure, or another component failure are much more prevelent than the HD failure.
It's true that most data is lost by user error rather than hardware failure.
But to say that no pro needs RAID?