It's stuff like this that makes me wonder if true artificial intelligence will ever really be possible.
If we are talking about not a singularitarian divine AI but about multiple smaller AIs each expert on a specific domain and mutually balancing/supervising each other, I think it may be possible...
As long as a human is involved anywhere in the process, there will always be a potential for things to go wrong. Nothing will ever be 100% idiot proof.
"Autonomous" cars are still designed using humans, hence there will be a time when a pedestrian gets killed by one. It WILL happen.
As long as a human is involved anywhere in the process, there will always be a potential for things to go wrong. Nothing will ever be 100% idiot proof.
"Autonomous" cars are still designed using humans, hence there will be a time when a pedestrian gets killed by one. It WILL happen.
The first one may be caught trying to commit insurance fraud, a bigger danger for "autonomous" cars than human error.
What is bad design is consumer-facing services that rely 100% on S3 without any fall-back systems. There were clearly many of those.
Are you aware of the engineering involved in mirroring your storage with 2 separate providers? That was a rhetorical question. Of course you don't.
It's not that hard. There are FUSE modules for several storage providers like S3.
And honestly, if you use externally-hosted storage like S3 for anything but non-critical data, you deserve what you get. If data is critical to your operations, you need to keep it in a place where you can assure its availability.
Why is their system still using command line prompts, this is the reason Unix is not favor for your average IT worker one wrong syntax error and you delete everything. Sounds like Amazon may have gotten off easy on this on.
That's nothing inherent to the command line, just inherent to who you are logged in as (say root), the command you're actually using and your level of familiarity with those commands.
The command line is fast if you know what you're doing.
Some commands like dd can cause untold level of damage in less than 0.1 seconds in the hands of an admin logged in as root (or doing and asroot commdn) (and it can be silent too).
When using the right tool, and logged into an account the proper level of access for your job, the command line is actually much quicker when you want to create something entirely new, or solve something quick. For doing routine tasks though it's a lot less than ideal and would eventually lead to errors due to complacency.
Why is their system still using command line prompts, this is the reason Unix is not favor for your average IT worker one wrong syntax error and you delete everything. Sounds like Amazon may have gotten off easy on this on.
Umm....a lot of things today are still done with command line prompts. You can be a lot more efficient with the command line (running scripts, etc) than with a GUI. If you've ever logged into an enterprise grade Cisco switch you don't do it with a GUI, you do it with a command line interface...sometimes using a DB-9 cable.
You don't get such as stupid mistake like this with GUI. Command lines are so 20th century.
Pfft! Yes you can. I got turned around inside a multiplexer and took down 12 T3s before I realized what I’d done. FCC reportable outage, big fine for AT&T. I almost got three days off without pay. Only the second level manager saved my ass that day. And I was using a point and click GUI to rearrange digital cross-connects. Stupid is as stupid does.
of course they use command line (and many others interface) :
it's efficient,
you can read it a lot before to validate
it's very convenient
you can script
and it's HUGELY USED everywhere.
and of course, in a gui (for example a whatever windows graphical interface), you can click "a little too fast" on a button, you can type just a little too more in some number textfield and click on Apply before thinking "hu... what did I put in the number of processors to shutdown ?"
And of course you can have check and test with a command line. Not everything is a simple than ping.
Errors are errors.
What is bad design is consumer-facing services that rely 100% on S3 without any fall-back systems. There were clearly many of those.
Are you aware of the engineering involved in mirroring your storage with 2 separate providers? That was a rhetorical question. Of course you don't.
It's not that hard. There are FUSE modules for several storage providers like S3.
And honestly, if you use externally-hosted storage like S3 for anything but non-critical data, you deserve what you get. If data is critical to your operations, you need to keep it in a place where you can assure its availability.
FUSE doesn't address syncing two storage providers and having failover. Apple mirrored Amazon with Azure, but not everyone can pull that off.
Hosting images externally is a best practice for many reasons. It improves server performance and scalability, easier to migrate between servers and easier to change server providers. Amazon wants people to put all their resources on their platform, and they just showed why that's a big mistake. Regardless, the issue here is that an employee typing in a wrong command can bring down their entire service world wide. This is a side effect of amazon building such an overly complicated platform like S3.
for some reason this made me think autonomous cars can't come soon enough...imagine all the human error that can go wrong getting behind the wheel at 50 mph
And one human error in programming the software can lead to systemic and failure on a very large scale involving every autonomous car running the same software. No thanks. I'll take my chances with normal human error.
for some reason this made me think autonomous cars can't come soon enough...imagine all the human error that can go wrong getting behind the wheel at 50 mph
Except when some sleep deprived, slightly hungover coder at Ford makes a mistake it causes 100,000 cars to crash instead of one...
You beat me to it. I didn't mean to be redundant. I didn't read far enough.
What is bad design is consumer-facing services that rely 100% on S3 without any fall-back systems. There were clearly many of those.
Are you aware of the engineering involved in mirroring your storage with 2 separate providers? That was a rhetorical question. Of course you don't.
It's not that hard. There are FUSE modules for several storage providers like S3.
And honestly, if you use externally-hosted storage like S3 for anything but non-critical data, you deserve what you get. If data is critical to your operations, you need to keep it in a place where you can assure its availability.
That's overly simplistic. 99+% of companies will have higher overall reliability if they put the hands of their core infrastructure in the hands of a third party. Few companies can afford to hire the engineers and admins (etc.) to build and staff a 100% reliable data center. In fact such a thing is impossible, so it makes perfect sense to pay of the fraction of the cost to have world-class experts take care of that stuff.
for some reason this made me think autonomous cars can't come soon enough...imagine all the human error that can go wrong getting behind the wheel at 50 mph
Yeah, because the automated systems sure won’t screw up ever… OH WAIT, A SIGNIFICANT PORTION OF THE INTERNET JUST WENT DOWN. You thought the opposite of what you should have thought, and that’s terrifying.
Why is their system still using command line prompts, this is the reason Unix is not favor for your average IT worker one wrong syntax error and you delete everything. Sounds like Amazon may have gotten off easy on this on.
Umm....a lot of things today are still done with command line prompts. You can be a lot more efficient with the command line (running scripts, etc) than with a GUI. If you've ever logged into an enterprise grade Cisco switch you don't do it with a GUI, you do it with a command line interface...sometimes using a DB-9 cable.
You don't get such as stupid mistake like this with GUI. Command lines are so 20th century.
Command line prompts let you get into the guts of the system. You cannot build a point and click GUI that enables you to do absolutely everything that is necessary to run even a medium sized system. Sure you can manage BAU aspects because you have a defined set of actions. But once you need to do things that are out of the ordinary you need a command line. What this outage highlights is you really need well trained and responsible operators driving these systems. And you need process and system design that builds resilience. Not saying that this op is a bad op - sometimes shyte happens. But the system needs to be resilient enough that you can have time to recognise a problem and back out if required. Losing your index is a bitch....
Off topic I know but I actually found the way the team, that thought they'd won but didn't, handled it, extremely uplifting and shining example to all and totally made up for a silly error at a very unimportant event (albeit fun to watch for many).
Comments
"Autonomous" cars are still designed using humans, hence there will be a time when a pedestrian gets killed by one. It WILL happen.
And honestly, if you use externally-hosted storage like S3 for anything but non-critical data, you deserve what you get. If data is critical to your operations, you need to keep it in a place where you can assure its availability.
The command line is fast if you know what you're doing.
Some commands like dd can cause untold level of damage in less than 0.1 seconds in the hands of an admin logged in as root (or doing and asroot commdn) (and it can be silent too).
When using the right tool, and logged into an account the proper level of access for your job, the command line is actually much quicker when you want to create something entirely new, or solve something quick. For doing routine tasks though it's a lot less than ideal and would eventually lead to errors due to complacency.
Hosting images externally is a best practice for many reasons. It improves server performance and scalability, easier to migrate between servers and easier to change server providers. Amazon wants people to put all their resources on their platform, and they just showed why that's a big mistake. Regardless, the issue here is that an employee typing in a wrong command can bring down their entire service world wide. This is a side effect of amazon building such an overly complicated platform like S3.
That's overly simplistic. 99+% of companies will have higher overall reliability if they put the hands of their core infrastructure in the hands of a third party. Few companies can afford to hire the engineers and admins (etc.) to build and staff a 100% reliable data center. In fact such a thing is impossible, so it makes perfect sense to pay of the fraction of the cost to have world-class experts take care of that stuff.
Off topic I know but I actually found the way the team, that thought they'd won but didn't, handled it, extremely uplifting and shining example to all and totally made up for a silly error at a very unimportant event (albeit fun to watch for many).