Heck Brad, don't you remember Copland? *THAT* was how far back Apple has been talking about searchable rich metadata tags on files. Copland's Smart Find and Active Folders were precisely implementations of metadata tag search.
And as for that comment about Unix-heads hating metadata... um, couldn't be further from the truth. What most Unix-heads hate is being locked into a particular filesystem. MacOS <X's file and creator types, Finder info, etc, etc, all all explicitly tied into the HFS(+) headers. If you want to move to, say, UFS, you *lose* all of that. (As evidenced by MacOS X installations on UFS. Classic and resource forks have all sorts of little workarounds to get going.) Unix runs on a myriad of filesystems, and if you're going to include metadata, you better make sure that it's accessible from pretty much any filesystem you use, or its chances of surviving and being used are about nil. Unix is all about portability and interoperability. *THAT* is the reality of the current computing world, and what Apple has to aspire to solve if they have any hope of succeeding long term.
Got it? This is so far beyond worrying about migrating a label to the new system that it isn't funny.
Besides, sit there with a straight face and tell me that *no one* will demand, insist, and scream that the old way of labels be retained, even in the face of the new metadata, if Apple were to re-implement labels now, the old way.
As a developer, you're always going to tick someone off. This time, it's you. Apple does it your way, they tick off someone else. I know, I know, small consolation, but that too is the reality.
<strong>If I were you, I'd really stop and think before you try to claim that Amorph doesn't know what he's talking about...</strong><hr></blockquote>
He is clearly wrong when he claims that type/creator doesn't work well with the web. It seems he doesn't understand the OS/Web interaction. You should argue against my argument if you think I'm wrong. Being an administrator doesn't make anyone right per se.
<strong>And as for that comment about Unix-heads hating metadata... um, couldn't be further from the truth. What most Unix-heads hate is being locked into a particular filesystem. MacOS <X's file and creator types, Finder info, etc, etc, all all explicitly tied into the HFS(+) headers. If you want to move to, say, UFS, you *lose* all of that. (As evidenced by MacOS X installations on UFS. Classic and resource forks have all sorts of little workarounds to get going.) Unix runs on a myriad of filesystems, and if you're going to include metadata, you better make sure that it's accessible from pretty much any filesystem you use, or its chances of surviving and being used are about nil.</strong><hr></blockquote>
Well, those are exactly the kind of excuses that make it impossible to ever implement something new. It's very difficult to create a fast filesystem that uses advanced metadata. It's impossible to get the metadata on every filesystem there is in one step. So you are just asking for the unattainable. Unix permissions are undoubtably not supported on every filesystem. So perhaps we should move them to an XML-thingie. And what about username and groupname? Modification date? Hard links are certainly not in every FS, so we should only allow soft links (regardless of the filesystem you are using). Do you think people will like the result, an extremely slow OS with fewer features? Let's use the same argument for texts-files. Would we ever have word processing if we required that every file should be editable with a simple text editor?
Most files never leave a computer, so what is wrong with metadata that is restricted to Macs? How can you ever expect something better until there has been one frontrunner that:
1. Shows that advanced metadata is very useful.
2. proves that one can implement it in a sufficiently fast filesystem.
3. works out all the little details.
Hoping for some big bang or totally transparant transition is just a dream. You need to start somewhere and you need to take it step by step. I think a file system with a database would be extremely powerful, even without full interoperability. Besides, even today you already lose a lot of info. On my PC I run Kazaa which supports descriptions, categories, integrity ratings and keywords, but those disappear as soon as I take the files out of the Kazaa folder. With a new system, that data would be preserved much longer. Only after I move the files to an inferior filesystem would the data be lost. Of course, our superiority might get them to support the system as well. And so the support for metadata increases and increases.
What is your plan? X.400*?
*Overengineered, designed by committee messaging protocol that lost out to the simple, but fairly effective e-mail.
[quote]<strong>Unix is all about portability and interoperability. *THAT* is the reality of the current computing world, and what Apple has to aspire to solve if they have any hope of succeeding long term.</strong><hr></blockquote>
A metadata filesystem doesn't have to prevent the portability and interoperability of systems to a great extent. Besides, the Unix world has seen enough problems with portability and interoperability. What about GCC 2.9->3.1->3.2? What about the libc/glibc problems of the past? Binary kernel modules in Linux that work on more than on specific version of the kernel? Forget it. KDE vs GNOME? So given the wide range of exceptions to the rule, why do you emphasize it so strongly? Arguably metadata has so many advantages that a bit of portability and interoperability may be lost in return, certainly when the problems exist only in new features*.
*It's strange to complain that your labels are lost when you copy files to UFS, when the only alternative is no labels at all.
[quote]<strong>Got it? This is so far beyond worrying about migrating a label to the new system that it isn't funny.</strong><hr></blockquote>
I'm glad that Neumann didn't worry that much ('Can a future version of this computer be used for evil?', 'Will people use it to send spam?', etc). People who change things don't worry till dawn, they just go to work and either succeed or fail, but at least they tried to make their dream come true.
[quote]<strong>Besides, sit there with a straight face and tell me that *no one* will demand, insist, and scream that the old way of labels be retained, even in the face of the new metadata, if Apple were to re-implement labels now, the old way.</strong><hr></blockquote>
People won't scream if the same functionality is retained with the new system*. Why would a labels-menu and coloring be so hard to link to the advanced, new metadata system? Do you understand the difference between functionality and implementation? You argued that a super-advanced system wouldn't support labels, while argueing at the same time that you won't need labels anymore because the new system is so much better. Well, which one is it? Either the new system is so good that it can handle labels without a problem or it's not so good as you say it is.
*How many people complained that their OS X calculator is now a Cocoa app? If they complained about the thing, it was because of lacking functionality, not because of technical details. Of course, sometimes people say one thing and mean another. "The Finder should be rewritten in Cocoa" really means "The Finder is dog slow, halts sometimes and crashesm, fix it".
[quote]<strong>As a developer, you're always going to tick someone off. This time, it's you. Apple does it your way, they tick off someone else. I know, I know, small consolation, but that too is the reality.</strong><hr></blockquote>
I never used labels a lot actually. The point is that Apple has a bad attitude. Their decisions often don't seem to focus on their users (or at least those coming from OS 9). Users have put a lot of effort into learning the system and have organized their files in certain ways. Suddenly Apple says: labels don't work with UFS, so we lose them. But 99% of the users don't care about UFS, so why should they lose the functionality that they rely upon to do their work? Why would it be a problem to have the labels menu be disabled when you highlight a file on UFS or a network share?
The same kind of obstinacy has surfaced time and again during the development of OS X. Why don't they allow another app to take over the role of the dock when so many people dislike it? Why should Apple be the only one that is allowed to use menu extra's? Why not implement WindowShade as an option when so many people like it? Why don't they use gray outlines for resizing instead of opaque resizing that's only good for a gee-whiz demo. I just can't make much sense of decisions where the most important reason seems to be the personal preference of Jobs or Tevanian. That's alright when you work on an OS that nobody uses, but when you've got millions of users, you'd better listen to them.
I think everyone's in agreement that Apple should have had a ready replacement for labels. The pragmatics of the situation are such that it didn't happen. Why should Apple implement something when they KNOW it's going to be replaced by a better system? As Brad and Kick pointed out, Apple's been working on a replacement for some time. It just didn't work out.
So, I'm willing to wager that Apple thought it would be ready in time, but it didn't get done. That let them with 2 choices:
1) They could reimplement labels and have people bitch and moan when they were dropped.
2) or, they could just not include them so that when the system changes you don't have to redo all your organization efforts.
Personally, I'd rather that they do it the second way because I hate organizing my files and I'd hate more than doubly doing it twice. Better things are coming and it's just a matter of time before they get here. Until then, stick to OS 9 or use the haxie. It's not that hard.
wfzelle, I'd love to argue this with you until the cows come home, but frankly, I've got work to do. You're welcome to your opinion, but you haven't done a thing to change mine, sorry.
You're right in that continually thinking and never implementing results in no product, but consider also that immediate implementation and no thinking results in a catastrophically ill formed system. You stated various incompatibilities in the myriad of Unix systems... where do you think they came from? If we can avoid that, then by all means we should. Implementation to shipping is a fine line between over- and under-engineering. I lean towards the over-, and you obviously are firmly in the under- camp. I don't see how our viewpoints are ever going to converge, given that, so I suggest we just agree to disagree.
<strong>I think everyone's in agreement that Apple should have had a ready replacement for labels. The pragmatics of the situation are such that it didn't happen. Why should Apple implement something when they KNOW it's going to be replaced by a better system? As Brad and Kick pointed out, Apple's been working on a replacement for some time. It just didn't work out.</strong><hr></blockquote>
Do you have any proof that this is true? AFAIK Apple's management just doesn't seem to give a damn. Hiring one lone BeFS guy is the closest you can get to proof, but that doesn't mean anything. He could be doing a lot of other things. You guys just keep projecting your hopes on Apple, but no one outside Apple seems to know what's really going on.
[quote]<strong>So, I'm willing to wager that Apple thought it would be ready in time, but it didn't get done.</strong><hr></blockquote>
The wild guesses keep accumulating...
[quote]<strong>That let them with 2 choices:
1) They could reimplement labels and have people bitch and moan when they were dropped.</strong><hr></blockquote>
Read my posts please. They don't need to be dropped (label-bits do, but labels as such don't have to).
[quote]<strong>2) or, they could just not include them so that when the system changes you don't have to redo all your organization efforts.</strong><hr></blockquote>
If they solve the metadata problem well, it will be a piece of cake to change this.
[quote]<strong>Personally, I'd rather that they do it the second way because I hate organizing my files and I'd hate more than doubly doing it twice. Better things are coming and it's just a matter of time before they get here. Until then, stick to OS 9 or use the haxie. It's not that hard.</strong><hr></blockquote>
Doing it twice? If you were using labels under OS 9, you wouldn't have any extra work. You have to migrate anyway. It's exactly the opposite as you say, the people who were using labels and switched to OS X had to reorganize and will have to reorganize again if/when the new system comes (and doesn't contain an easy migration path, which would be easy to create).
<strong>wfzelle, I'd love to argue this with you until the cows come home, but frankly, I've got work to do. You're welcome to your opinion, but you haven't done a thing to change mine, sorry.</strong><hr></blockquote>
I think that I've shown quite a few problems and inconsistenties in your post. It would suit you to answer my questions in that regard. IMHO a debate is not about throwing your opinion over the wall, you should be willing to have it questioned.
[quote]<strong>You're right in that continually thinking and never implementing results in no product, but consider also that immediate implementation and no thinking results in a catastrophically ill formed system.</strong><hr></blockquote>
I never said that one shouldn't think. What I said is that you must not bite more than you can chew. Be content with taking small steps and learning along the way. History has consistently shown that it's rare when solutions succeed that attempt to solve an entire problem at once. They usually die of bloat, architectural problems that weren't caught on paper and basic misunderstanding of the problems that occur when implementing.
[quote]<strong>You stated various incompatibilities in the myriad of Unix systems... where do you think they came from?</strong><hr></blockquote>
Innovation.
[quote]<strong>If we can avoid that, then by all means we should.</strong><hr></blockquote>
I'd rather not stop innovating. If something is innovative enough, it will undoubtably shake up the establishment. Usually there will be several different approaches to solve the same problem because there is no clearly superior solution. Convergence is a powerful force though and in the long run people will strive for decent interoperability and portability. That's already happening with the new Red Hat that integrates KDE and GNOME for instance. Word processing has converged towards .doc and .pdf. RPC is moving towards SOAP. As systems mature it becomes clear what the right solution is and a common standard can be created that supports most possible uses.
[quote]<strong>Implementation to shipping is a fine line between over- and under-engineering. I lean towards the over-, and you obviously are firmly in the under- camp. I don't see how our viewpoints are ever going to converge, given that, so I suggest we just agree to disagree.</strong><hr></blockquote>
I don't think that I lean (strongly) towards underengineering. Your position was quite extreme and I was trying to show the flaws of such a position. I was not argueing in favor of a wild west approach. Again, there is a big difference between limiting the scope of the problem you want to solve and limiting the engineering you do on that scope. The first is much more important than the second when you try to prevent overengineering.
[quote]I think that I've shown quite a few problems and inconsistenties in your post. It would suit you to answer my questions in that regard. IMHO a debate is not about throwing your opinion over the wall, you should be willing to have it questioned.<hr></blockquote>
I'm afraid all you've shown is a set of differing opinions on particular points. I'm perfectly willing to have my opinion questioned, and even proven wrong, as long as the other debater has sufficient evidence that they can provide, or at least a compelling argument that can stand up under scrutiny. I've yet to get either here... there hasn't been any debate, just warring opinions. IMHO, that's pretty low on my priority list of activities to engage in. (And yet here I am again. *sigh*)
[quote]I never said that one shouldn't think. What I said is that you must not bite more than you can chew. Be content with taking small steps and learning along the way. History has consistently shown that it's rare when solutions succeed that attempt to solve an entire problem at once. They usually die of bloat, architectural problems that weren't caught on paper and basic misunderstanding of the problems that occur when implementing.<hr></blockquote>
On that we agree, but where we appear to disagree is on the other approach: ad hoc cobbled together systems that grow, mutate, and otherwise just... *happen*. That's not design, that's an accident. cf: Unix. 30 years, and we have... what? Very little more than we had 10 years, or even 20 years ago. The basics are still the same, and still show the same problems. Why? People keep making the same mistakes. Why? Because the big problems are rarely if ever solved by the bazaar. Sorry. I believe in the open source model, but there are *some* problems that it is ill-suited for.
Metadata across filesystems is such a problem.
[quote]Innovation.<hr></blockquote>
*snort* Okay, you get points for sheer chutzpah.
[quote]I'd rather not stop innovating. If something is innovative enough, it will undoubtably shake up the establishment. <hr></blockquote>
Oh yes, NeXT took over the world, obviously. Windows XP is just a delusion. <img src="graemlins/oyvey.gif" border="0" alt="[No]" />
"Keen and kewl enough and they will come" is what killed NeXT for all intents and purposes, and what almost drove Apple into the grave. "Build it and they will come" DOESN'T work. Linux has been trying this approach for *how* long?
Innovation is wonderful. Flailing around in the dark like an epileptic baboon in a barrel isn't innovation, it's just a waste of resources and time. Infinite monkeys + infinite IDEs = innovation? Well, sure, given infinite *time*... I'd rather not wait that long.
[quote]Usually there will be several different approaches to solve the same problem because there is no clearly superior solution. Convergence is a powerful force though and in the long run people will strive for decent interoperability and portability. That's already happening with the new Red Hat that integrates KDE and GNOME for instance. Word processing has converged towards .doc and .pdf.<hr></blockquote>
Woah, hold it right there... you're claiming that .doc is a) a good standard, b) interoperable, or c) portable? Dear god, give me some of what you're smoking.
.doc is a lousy standard, it's about as far from interoperable as you can get, and it is highly portable... as long as you're running MS product. There are other apps that (mostly) work on the format, but it isn't because of choice, but market pressure from that little thing called a monopoly.
This isn't intelligent design, it's marketing gone haywire.
[quote] RPC is moving towards SOAP. As systems mature it becomes clear what the right solution is and a common standard can be created that supports most possible uses.<hr></blockquote>
As poorly designed systems mature they gain bloat as multiple implementations try and merge and end up producing an incestuous hodge-podge that invariable dies under its own weight.
You can't seriously sit there and tell me that a combination of KDE and Gnome is the 'clearly superior solution' for a Linux GUI, can you? No, it's just what is turning out to be the normative solution. Huge difference.
Even if you have two 95% solutions, trying to merge them to get a 100% solution is going to be a logistical nightmare, and god forbid you have to maintain the system. Better to back off, rethink it, and design *one* system that strives for 100% from the lessons of each. RedHat is headed down a bumpy road.
[quote]I don't think that I lean (strongly) towards underengineering. Your position was quite extreme <hr></blockquote>
Creating an intelligent design from the beginning is extreme? Wow. I had no idea. Here I thought this was software *engineering*, not software horticulture. Guess I've been wasting my time all these years in the wrong discipline.
[quote]and I was trying to show the flaws of such a position. I was not argueing in favor of a wild west approach. Again, there is a big difference between limiting the scope of the problem you want to solve and limiting the engineering you do on that scope. The first is much more important than the second when you try to prevent overengineering.<hr></blockquote>
Yes, it is, but just as important is keeping focus on what you're trying to solve. Limiting the problem down to what is just merely easy to solve results in poor designs, and a crazy quilt mess of a system. But, that seems to be the norm so of *course* it must be the best approach.
On a slightly related note, I'm a fan of XP, but I've seen it horribly abused and used as justification for the *CRAPPIEST* designs. "Well, we were just following XP and did the minimum..." generally is just an excuse for avoiding the actual problem originally under consideration. This is the road that your advocated approach generally ends up in, from my experience. With perfect coders in a perfect world with perfect users, then the bazaar would be *MY* methodology of choice, but unfortunately we don't live in such a world.
Limiting the scope of the problem is fine. Limiting the scope of the problem so far that it is no longer the problem you want to solve, merely because the limited version is easier, is just laziness or a cover for ineptitude and mediocrity.
And since this has long gone jumped the labels on X track, how about we continue this in General Discussion, eh? I won't move or lock this thread, in the hopes that others will continue the discussion about Unsanity's Haxie, but this portion of this thread belongs elsewhere.
[quote]<strong>I already told you that I don't see why it should be so hard to migrate labels. Suppose we have this ultra-slow xml-thingie that some people seem to believe in. What makes it so difficult to move the bit to <label>black</label>? It would just be an extra bit-check during the copying of a file (from a HFS+ partition). If the bit is set you create an xml-label and zap the bit. You lose the ability to migrate back without a special utility to revert the <labels> to the label-bit, but I think that users could live with that.</strong><hr></blockquote>
First, yours is exactly the same argument that people used when Apple went all USB: Why can't they keep the old connectors around for a smooth transition? Because the PC world had tried that, and as a result nobody adopted USB.
Developers are lazy. Generally speaking, they will not support a new technology until they have to. What your system requires is for Apple to support the old way indefinitely: an application can continue to set labels the old way, and the OS will translate to the new way in the background. Indefinitely.
Or they could break it at some point, eliciting the exact same hue and cry they're getting now, only with no available justification and OS X native apps and users who have built their workflows around OS X's legacy support for labels.
The old way means that any file with a label has a resource fork. The Mac's forked filesystem is fundamentally incompatible with the rest of the world, so the resource fork tends to get lost. Remember, Apple is gunning for maximum compatibility here. Having files lose information arbitrarily is a Bad Thing(TM). Being incompatible at a filesystem level with the rest of the world is a Bad Thing(TM). PCs are not islands anymore.
[quote]<strong>Clearly you don't understand what you're talking about. The Web needs more metadata than extensions can provide. That's why both the MacOS and Windows have MIME mappings. Windows maps from/to extensions, MacOS used file types and now a mix of both. Unless you move to a BeFS-like MIME file type, you will be stuck with such a conversion.</strong><hr></blockquote>
What I was talking about was: The MacOS labels are set in the resource fork of the relevant file. This fork - and therefor, MacOS labels - are fundamentally incompatible with the Web, and with other platforms. This has nothing to do with whether filename extensions are an adequate replacement (for the record, I don't think they are anywhere near sufficient, and that should be clear from my prior posts about metadata).
You mention several perfectly good alternatives: Apple could adopt MIME natively and lose the need to map between the Web and its native metadata representation, for example. This would give us a world-standard, robust, extensible, plain-text metadata system. Or Apple could go somewhere else. They could go to a database filesystem, which would be entirely unlike anything else out there, that could be queried for things that looked like files with corresponding MIME types to other systems, and like XML files to others.
They have a lot of options. If they play their cards right, they could come up with something that is at once broadly compatible and astonishingly robust - making label/type/creator look poor by comparison. I don't really mind if they take their time doing it, frankly. If they're really going to do metadata, they'd better do it right.
[quote]<strong>I don't think the first option is that far-fetched. If you've been around Unix-heads (which 90% of the NeXTies are), then you know that most of them despise new metadata (you may not touch their drwxrwxrwx though). Why do you think that Mr Mach moved to file extensions? Would it make sense to move from file type/creator to extensions and then to something more advanced?</strong><hr></blockquote>
This is unfair to the UNIX crowd. For one thing, remember MIME? I think if you looked at the people who are responsible for that, and for HTTP, and for other protocols that recognize metadata, you'll find lots and lots of big, bearded UNIX gurus behind them. They do have a fundamentally different approach to the Mac's old way, however: in UNIX, a file is a stream of data, period, and how its name and contents are interpreted is up to the application looking at the file. Something like Apple's forked filesystem or VMS' indexed, record-based filesystem is alien to the UNIX way of thinking: To them, metadata is an interpretation imposed upon files, rather than an interpretation built into them (this holds for filename extensions as well -- UNIX itself, unlike Windows, could care less about them). Whence all the specifications like MIME that are external to the data whose types they're describing.
[quote]<strong>I would work with the community to try and solve this very difficult problem. Create a document that describes the problems you want to solve ('The new system should do x and y and z') and get feedback from developers (release it at WWDC for instance). Then come up with a possible solution and get feedback on that. The current system is a cludge with some severe problems. It could have used some peer review. </strong><hr></blockquote>
Did Apple work with the community in such an overt way to build its GUI? To design HFS, or extend it to HFS+? No. They are not sourceforge: They'll debut an essentially finished system at a WWDC, and then they'll get feedback from their developers whether they want it or not, and refine it as needed. That's what they do. It's what they've always done. They have some internal pressure as well: As I've pointed out, the iApps themselves cry out for robust metadata.
In the meantime, I can't blame them for getting rid of legacy as soon as they can. I'm a programmer myself, and I've made the mistake of grandfathering things that I fully intended to get around to replacing later, and... oops.
Legacy is a bitch. I'm confident that Apple deprecated labels and type/creator and other resources because they felt they could do better. And they can do much, much better.
<strong>On that we agree, but where we appear to disagree is on the other approach: ad hoc cobbled together systems that grow, mutate, and otherwise just... *happen*. That's not design, that's an accident. cf: Unix. 30 years, and we have... what? Very little more than we had 10 years, or even 20 years ago. The basics are still the same, and still show the same problems. Why? People keep making the same mistakes. Why? Because the big problems are rarely if ever solved by the bazaar. Sorry. I believe in the open source model, but there are *some* problems that it is ill-suited for.
Metadata across filesystems is such a problem.</strong><hr></blockquote>
The problem is that whenever someone proposes an enhancement that might break something, he gets ridiculed. They is no market as conservative as the Unix/open source one. The result is indeed a hodge podge. Why? Because most Unix fans only care about portability and interoperability in the short run. Now and then you need to accept a bit of inconvenience to get a much better system in the long run.
Few Unix guys are willing to accept any new metadata (or anything else) that brings more than an ounce of pain. The result is stagnation, they stick to the current hacks because 'at least those work'. They will continue to work for 20 years of course, but how much will you save by expending a bit of effort today?
[quote]<strong>Oh yes, NeXT took over the world, obviously. Windows XP is just a delusion.
"Keen and kewl enough and they will come" is what killed NeXT for all intents and purposes, and what almost drove Apple into the grave. "Build it and they will come" DOESN'T work. Linux has been trying this approach for *how* long?
Innovation is wonderful. Flailing around in the dark like an epileptic baboon in a barrel isn't innovation, it's just a waste of resources and time. Infinite monkeys + infinite IDEs = innovation? Well, sure, given infinite *time*... I'd rather not wait that long.</strong><hr></blockquote>
OO API's are only one part of an excellent OS, NeXT failed to fill in most of the other 2000 parts you need (and since most users aren't programmers and programmers get paid by users...). I still don't know what's so absolutely wonderful about Linux on the desktop except that it's free. We know from experimental economics that few consumers have price as their main concern, so it's lack of success is certainly not unexpected.
Innovation is not about buzzwords, it's about delivering what people want. Marketing is convincing people that the new features outweigh the faults and switching problems. Given sensible innovators and a correct marketing strategy, you certainly won't have infinite monkeys. Of course, both prerequisites are fairly scarce. When you haven't got money to make it happen, you need the buzz. Unfortunately, when nobody is willing to take metadata seriously, nothing will happen. Thus it is up for a company like Apple to end the status quo.
[quote]<strong>Woah, hold it right there... you're claiming that .doc is a) a good standard, b) interoperable, or c) portable? Dear god, give me some of what you're smoking.
.doc is a lousy standard, it's about as far from interoperable as you can get, and it is highly portable... as long as you're running MS product. There are other apps that (mostly) work on the format, but it isn't because of choice, but market pressure from that little thing called a monopoly.
This isn't intelligent design, it's marketing gone haywire.</strong><hr></blockquote>
90% of the people I know can read Word documents. No other advanced, editable word processing format can beat that (PDF is better for distributing read-only files). It's quite clear that today we need an open, extensible format, but during the rise of word processing, this simply was not a big enough issue. WP and MS were having a hard enough time improving their products and users didn't ask for it.
In short, .doc was a very succesful innovation during the 90's, but today we need a new innovation which resolves a lot of the issues with the .doc format (corruption, extensibility, broad interoperability). I hope that MS's new XDoc will be that innovation, it will make for an easy transition if they do.
[quote]<strong>As poorly designed systems mature they gain bloat as multiple implementations try and merge and end up producing an incestuous hodge-podge that invariable dies under its own weight.
You can't seriously sit there and tell me that a combination of KDE and Gnome is the 'clearly superior solution' for a Linux GUI, can you? No, it's just what is turning out to be the normative solution. Huge difference.
Even if you have two 95% solutions, trying to merge them to get a 100% solution is going to be a logistical nightmare, and god forbid you have to maintain the system. Better to back off, rethink it, and design *one* system that strives for 100% from the lessons of each. RedHat is headed down a bumpy road.</strong><hr></blockquote>
I think it's clear that KDE and Gnome are solving the same GUI problems in similar ways. There has been enough experience with desktop GUI's. They should merge and restrict the differences to area's where real innovation is required. I don't really care whether that happens through a merge or a rewrite. If Apple can put two different API's beneath the same GUI, Linux can have it as well (in theory at least).
[quote]<strong>Creating an intelligent design from the beginning is extreme? Wow. I had no idea. Here I thought this was software *engineering*, not software horticulture. Guess I've been wasting my time all these years in the wrong discipline.</strong><hr></blockquote>
No, you were asking for a solution which solves everything in one go, is fully backwards compatible and other such nonsense. That's as overengineering as you can get IMHO. We have OS X which only works well with HFS+. Why not take advantage of that by solving the limited problem: How do we get advanced metadata on the Mac? Forget converting the entire world for now, create this very useful solution first. It will be hard enough already.
[quote]<strong>Yes, it is, but just as important is keeping focus on what you're trying to solve. Limiting the problem down to what is just merely easy to solve results in poor designs, and a crazy quilt mess of a system. But, that seems to be the norm so of *course* it must be the best approach.</strong><hr></blockquote>
Keeping focus is easier when you restrict the scope. A lot of large designs forget their main purpose. See Mozilla for instance. It contains just about everything from a private programming language to an IRC client.
[quote]<strong>On a slightly related note, I'm a fan of XP, but I've seen it horribly abused and used as justification for the *CRAPPIEST* designs. "Well, we were just following XP and did the minimum..." generally is just an excuse for avoiding the actual problem originally under consideration. This is the road that your advocated approach generally ends up in, from my experience.</strong><hr></blockquote>
Any methodology requires smart architects, developers and managers. There is no panacea that can turn an idiot into a genius. XP can be used to great effect by a small well-led group who have to built a certain class of systems, but it is only one part of the puzzle. Strangely enough some expect unqualified people to create excellent products when we give them tools they cannot use. I wonder if the same people would appreciate being operated in the most advanced operating room by a first year medical student. Somehow I'd expect them to get the point when their lives are at stake.
[quote]<strong>Limiting the scope of the problem is fine. Limiting the scope of the problem so far that it is no longer the problem you want to solve, merely because the limited version is easier, is just laziness or a cover for ineptitude and mediocrity.</strong><hr></blockquote>
What do you think of the scope I selected?
[quote]<strong>And since this has long gone jumped the labels on X track, how about we continue this in General Discussion, eh? I won't move or lock this thread, in the hopes that others will continue the discussion about Unsanity's Haxie, but this portion of this thread belongs elsewhere.</strong><hr></blockquote>
I don't think that anyone is going to continue the discussion about labels in this thread (It seems I'm guilty of hijacking, sorry ). I think that everything that could have been said about the haxie has been said already. Besides, I can't just post this in a new topic, everyone will be confused.
<strong>First, yours is exactly the same argument that people used when Apple went all USB: Why can't they keep the old connectors around for a smooth transition? Because the PC world had tried that, and as a result nobody adopted USB.</strong><hr></blockquote>
That is a different argument altogether. You have to buy new equipment for USB and people wanted to preserve their investment by keeping the old technology. What I propose is that we move to new technology exclusively, but can keep our old equipment (the so called win-win situation). It's like your serial connectors get magically transformed into USB connectors. That is not possible with physical objects of course, but it can be done with label-bits.
[quote]<strong>Developers are lazy. Generally speaking, they will not support a new technology until they have to. What your system requires is for Apple to support the old way indefinitely: an application can continue to set labels the old way, and the OS will translate to the new way in the background. Indefinitely.</strong><hr></blockquote>
Not at all. Apple puts calls into Carbon & Cocoa: getLabel() & setLabel(). Until we get the new metadata system those calls simply manipulate the label-bits. Thereafter they use a label-thingie in the new metadata system. The only problems you will see is when you need to migrate files from the old system to the new, but that is certainly manageable IMHO. Besides, Apple can just force you, so you won't have a choice. MacOS X Metadata will simply ignore the label-bits. So there is no way that people can resist the move to the new system other than not upgrading their OS.
[quote]<strong>The old way means that any file with a label has a resource fork.</strong><hr></blockquote>
No, it's a byte in the filesystem. Try it out, put a label on a file with only a data fork. Resedit will show that a resource fork was not added.
[quote]<strong>MacOS labels - are fundamentally incompatible with the Web, and with other platforms. This has nothing to do with whether filename extensions are an adequate replacement (for the record, I don't think they are anywhere near sufficient, and that should be clear from my prior posts about metadata).</strong><hr></blockquote>
You will lose labels when you copy files to the web or a PC, but I don't think that is a big problem. A lost resource fork that is crucial will break a file. That's why I agree with the move towards packages. A label is never crucial however. It's just a convenient organisation tool on Macs and users can learn not to depend on it whenever they move their files to something inferior.
[quote]<strong>You mention several perfectly good alternatives: Apple could adopt MIME natively and lose the need to map between the Web and its native metadata representation, for example. This would give us a world-standard, robust, extensible, plain-text metadata system.</strong><hr></blockquote>
MIME is only one piece of the puzzle. An advanced metadata system will store much more than just a file type. I think bigger
[quote]<strong>Or Apple could go somewhere else. They could go to a database filesystem, which would be entirely unlike anything else out there, that could be queried for things that looked like files with corresponding MIME types to other systems, and like XML files to others.</strong><hr></blockquote>
MIME would fit in such a system perfectly. The biggest problems are treefold:
1. You will lose data at the perimeter of the metadata system.
2. Some apps might process a file without copying the metadata (UNIX apps will experience this problem severely).
3. Speed.
[quote]<strong>They have a lot of options. If they play their cards right, they could come up with something that is at once broadly compatible and astonishingly robust - making label/type/creator look poor by comparison. I don't really mind if they take their time doing it, frankly. If they're really going to do metadata, they'd better do it right.</strong><hr></blockquote>
I've been waiting 3 years without a peep or hint at a solution. The only thing I've seen is regression: a move to extensions. MS is also aiming at a DB-enhanced filesystem, so Apple shouldn't wait to long.
[quote]<strong>This is unfair to the UNIX crowd. For one thing, remember MIME? I think if you looked at the people who are responsible for that, and for HTTP, and for other protocols that recognize metadata, you'll find lots and lots of big, bearded UNIX gurus behind them.</strong><hr></blockquote>
Ok, perhaps I was a tad unfair. But the point stands that MIME was only accepted at the application level. MIME stops to exist after the browser downloads a webpage or file.
[quote]<strong>They do have a fundamentally different approach to the Mac's old way, however: in UNIX, a file is a stream of data, period, and how its name and contents are interpreted is up to the application looking at the file.</strong><hr></blockquote>
Flawed and easily replaceable by a infinitely better system IMO. Why can't they simply accept that a file has more than one part? Why can't they accept that you might like to know what data you are dealing with? All that is required is that the stream is turned into a structure. If I need the data, I do getProperty('data'). If I need the filename, I do getProperty('filename'). Find out the type with getProperty('MIME'). You caneasily add new metadata in a well-designed system. But no, let's just treat it as one blob. So how do I find out if I'm dealing with a gif? Analyze the data or require the user to tell me. The first method requires magic numbers in the data, so we start mixing metadata with data. How can we easily process that? Do we first need to remove the metadata? What standard does it use? Can we have comments, labels, etc, etc?
And so we hack and kludge at a fundamentally flawed system.
[quote]<strong>Something like Apple's forked filesystem or VMS' indexed, record-based filesystem is alien to the UNIX way of thinking: To them, metadata is an interpretation imposed upon files, rather than an interpretation built into them (this holds for filename extensions as well -- UNIX itself, unlike Windows, could care less about them). Whence all the specifications like MIME that are external to the data whose types they're describing.</strong><hr></blockquote>
And yet a lot of Unix users do like to know what type their files are and so they use extensions like .html and .jpg. The problem is that these idiots don't understand that extra information is never a burden. You can always ignore/override it. And why do they support extensions, but not a MIME type in the file system? If they can handle 'mv a a.html', then why can't they handle 'ct a text/html'? Why is the second method wrong? I cannot see a conceptual difference. The extension is not built into the file anymore than the MIME type.
[quote]<strong>Legacy is a bitch. I'm confident that Apple deprecated labels and type/creator and other resources because they felt they could do better. And they can do much, much better.</strong><hr></blockquote>
Unfortunately, Avie Tevanian and his gang are known for feeling confident that they can do better and then failing. Avie simply doesn't understand the needs of the Mac community and he has to be beaten into submission time and again. Proof:
1) Carbon. Avie wanted everyone to adopt Cocoa, but had to change course after every big developer declined to rewrite their software from scratch. It took Apple three years to turn Rhapsody into MacOS X.
2) Applescript. Avie could do better. Users who had invested millions of dollars in Applescript solutions managed to change his mind. But most new Apple apps don't support applescript.
3) Apple's guidelines that we should move to filename extensions. I can't remember a bigger roar in the developer community.
4) The hierarchical Apple menu was removed and it's replacement is broken and slow.
5) The NeXT installer that replaced the Apple Installer is a broken piece of junk (for over 3 years). They even complained about it at stepwise. Ever wonder why you need a repair privileges utility? It's because the installer changes the privileges of the files that it replaces. This may even stop your system from booting. A more serious bug can destroy files in rare circumstances.
6) Dependency on file paths. I guess that robust file tracking is one more Mac-advantage that we lost.
7) Customization. The ultra-advanced OS X is harder to change than OS 9.
All the evidence points to one conclusion: Avie is more interested in advancing his own personal vision of computing than in serving customer needs. I strongly doubt whether his vision contains advanced metadata and so I'm concerned about the future.
Interesting. This is a nice enhancement. On the other hand, now we know that the BeFS guy that Apple hired is not working on metadata. </strong><hr></blockquote>
Did you even read the article?
[quote]Indeed, Apple apparently enlisted the creator of the Be technology for its Elvis project: Be's BFS journaled file system was written by Dominic Giampaolo, who in March joined Apple as a file system engineer, The Register UK reported.
I'm working on three major projects right now. If the first is released, does that mean I haven't been doing anything else?
Absence of evidence does not prove a negative.</strong><hr></blockquote>
Journaling is not that easy to implement AFAIK. At the very least this new info makes it less likely that he is working on metadata. But my conclusion was probably too strong.
[quote]Journaling is not that easy to implement AFAIK. At the very least this new info makes it less likely that he is working on metadata.<hr></blockquote>
You think metadata is that easy? OSX is now a multi-user system--should metadata be private or should metadata tags be visible to all users? How do permissions and metadata interact? Not so simple in multiuser environment.
never used labels, but the debate has gotten interesting. one point that's important to remember is that it's not terribly smart to put in a feature, then remove it later in a subsequent release. the reason is that people don't all migrate over the newest system en masse. so your os x userbase is already spread out amongst 3 different 10.x versions. there are probably a good amount of computers out there still on 10.0. even in my own house i have two powerbooks on 10.1 and one on 10.2.
if apple implemented an interim solution, such as labels as we knew them in os9, you're saying they'd have to support them all the way until they've implemented their new metadata system. well, what happens when 25% of their userbase is using one metadata system and 75% are using another? it's a pain in the ass. it's better to make a clean break and force people to use a workaround until they can do it right. otherwise they have to explain and support two different metadata systems to both users and developers.<a href="http://http://"; target="_blank">web page</a>
Comments
And as for that comment about Unix-heads hating metadata... um, couldn't be further from the truth. What most Unix-heads hate is being locked into a particular filesystem. MacOS <X's file and creator types, Finder info, etc, etc, all all explicitly tied into the HFS(+) headers. If you want to move to, say, UFS, you *lose* all of that. (As evidenced by MacOS X installations on UFS. Classic and resource forks have all sorts of little workarounds to get going.) Unix runs on a myriad of filesystems, and if you're going to include metadata, you better make sure that it's accessible from pretty much any filesystem you use, or its chances of surviving and being used are about nil. Unix is all about portability and interoperability. *THAT* is the reality of the current computing world, and what Apple has to aspire to solve if they have any hope of succeeding long term.
Got it? This is so far beyond worrying about migrating a label to the new system that it isn't funny.
Besides, sit there with a straight face and tell me that *no one* will demand, insist, and scream that the old way of labels be retained, even in the face of the new metadata, if Apple were to re-implement labels now, the old way.
As a developer, you're always going to tick someone off. This time, it's you. Apple does it your way, they tick off someone else. I know, I know, small consolation, but that too is the reality.
<strong>If I were you, I'd really stop and think before you try to claim that Amorph doesn't know what he's talking about...</strong><hr></blockquote>
He is clearly wrong when he claims that type/creator doesn't work well with the web. It seems he doesn't understand the OS/Web interaction. You should argue against my argument if you think I'm wrong. Being an administrator doesn't make anyone right per se.
[ 10-14-2002: Message edited by: wfzelle ]</p>
<strong>And as for that comment about Unix-heads hating metadata... um, couldn't be further from the truth. What most Unix-heads hate is being locked into a particular filesystem. MacOS <X's file and creator types, Finder info, etc, etc, all all explicitly tied into the HFS(+) headers. If you want to move to, say, UFS, you *lose* all of that. (As evidenced by MacOS X installations on UFS. Classic and resource forks have all sorts of little workarounds to get going.) Unix runs on a myriad of filesystems, and if you're going to include metadata, you better make sure that it's accessible from pretty much any filesystem you use, or its chances of surviving and being used are about nil.</strong><hr></blockquote>
Well, those are exactly the kind of excuses that make it impossible to ever implement something new. It's very difficult to create a fast filesystem that uses advanced metadata. It's impossible to get the metadata on every filesystem there is in one step. So you are just asking for the unattainable. Unix permissions are undoubtably not supported on every filesystem. So perhaps we should move them to an XML-thingie. And what about username and groupname? Modification date? Hard links are certainly not in every FS, so we should only allow soft links (regardless of the filesystem you are using). Do you think people will like the result, an extremely slow OS with fewer features? Let's use the same argument for texts-files. Would we ever have word processing if we required that every file should be editable with a simple text editor?
Most files never leave a computer, so what is wrong with metadata that is restricted to Macs? How can you ever expect something better until there has been one frontrunner that:
1. Shows that advanced metadata is very useful.
2. proves that one can implement it in a sufficiently fast filesystem.
3. works out all the little details.
Hoping for some big bang or totally transparant transition is just a dream. You need to start somewhere and you need to take it step by step. I think a file system with a database would be extremely powerful, even without full interoperability. Besides, even today you already lose a lot of info. On my PC I run Kazaa which supports descriptions, categories, integrity ratings and keywords, but those disappear as soon as I take the files out of the Kazaa folder. With a new system, that data would be preserved much longer. Only after I move the files to an inferior filesystem would the data be lost. Of course, our superiority might get them to support the system as well. And so the support for metadata increases and increases.
What is your plan? X.400*?
*Overengineered, designed by committee messaging protocol that lost out to the simple, but fairly effective e-mail.
[quote]<strong>Unix is all about portability and interoperability. *THAT* is the reality of the current computing world, and what Apple has to aspire to solve if they have any hope of succeeding long term.</strong><hr></blockquote>
A metadata filesystem doesn't have to prevent the portability and interoperability of systems to a great extent. Besides, the Unix world has seen enough problems with portability and interoperability. What about GCC 2.9->3.1->3.2? What about the libc/glibc problems of the past? Binary kernel modules in Linux that work on more than on specific version of the kernel? Forget it. KDE vs GNOME? So given the wide range of exceptions to the rule, why do you emphasize it so strongly? Arguably metadata has so many advantages that a bit of portability and interoperability may be lost in return, certainly when the problems exist only in new features*.
*It's strange to complain that your labels are lost when you copy files to UFS, when the only alternative is no labels at all.
[quote]<strong>Got it? This is so far beyond worrying about migrating a label to the new system that it isn't funny.</strong><hr></blockquote>
I'm glad that Neumann didn't worry that much ('Can a future version of this computer be used for evil?', 'Will people use it to send spam?', etc). People who change things don't worry till dawn, they just go to work and either succeed or fail, but at least they tried to make their dream come true.
[quote]<strong>Besides, sit there with a straight face and tell me that *no one* will demand, insist, and scream that the old way of labels be retained, even in the face of the new metadata, if Apple were to re-implement labels now, the old way.</strong><hr></blockquote>
People won't scream if the same functionality is retained with the new system*. Why would a labels-menu and coloring be so hard to link to the advanced, new metadata system? Do you understand the difference between functionality and implementation? You argued that a super-advanced system wouldn't support labels, while argueing at the same time that you won't need labels anymore because the new system is so much better. Well, which one is it? Either the new system is so good that it can handle labels without a problem or it's not so good as you say it is.
*How many people complained that their OS X calculator is now a Cocoa app? If they complained about the thing, it was because of lacking functionality, not because of technical details. Of course, sometimes people say one thing and mean another. "The Finder should be rewritten in Cocoa" really means "The Finder is dog slow, halts sometimes and crashesm, fix it".
[quote]<strong>As a developer, you're always going to tick someone off. This time, it's you. Apple does it your way, they tick off someone else. I know, I know, small consolation, but that too is the reality.</strong><hr></blockquote>
I never used labels a lot actually. The point is that Apple has a bad attitude. Their decisions often don't seem to focus on their users (or at least those coming from OS 9). Users have put a lot of effort into learning the system and have organized their files in certain ways. Suddenly Apple says: labels don't work with UFS, so we lose them. But 99% of the users don't care about UFS, so why should they lose the functionality that they rely upon to do their work? Why would it be a problem to have the labels menu be disabled when you highlight a file on UFS or a network share?
The same kind of obstinacy has surfaced time and again during the development of OS X. Why don't they allow another app to take over the role of the dock when so many people dislike it? Why should Apple be the only one that is allowed to use menu extra's? Why not implement WindowShade as an option when so many people like it? Why don't they use gray outlines for resizing instead of opaque resizing that's only good for a gee-whiz demo. I just can't make much sense of decisions where the most important reason seems to be the personal preference of Jobs or Tevanian. That's alright when you work on an OS that nobody uses, but when you've got millions of users, you'd better listen to them.
[ 10-14-2002: Message edited by: wfzelle ]</p>
So, I'm willing to wager that Apple thought it would be ready in time, but it didn't get done. That let them with 2 choices:
1) They could reimplement labels and have people bitch and moan when they were dropped.
2) or, they could just not include them so that when the system changes you don't have to redo all your organization efforts.
Personally, I'd rather that they do it the second way because I hate organizing my files and I'd hate more than doubly doing it twice. Better things are coming and it's just a matter of time before they get here. Until then, stick to OS 9 or use the haxie. It's not that hard.
You're right in that continually thinking and never implementing results in no product, but consider also that immediate implementation and no thinking results in a catastrophically ill formed system. You stated various incompatibilities in the myriad of Unix systems... where do you think they came from? If we can avoid that, then by all means we should. Implementation to shipping is a fine line between over- and under-engineering. I lean towards the over-, and you obviously are firmly in the under- camp. I don't see how our viewpoints are ever going to converge, given that, so I suggest we just agree to disagree.
<strong>I think everyone's in agreement that Apple should have had a ready replacement for labels. The pragmatics of the situation are such that it didn't happen. Why should Apple implement something when they KNOW it's going to be replaced by a better system? As Brad and Kick pointed out, Apple's been working on a replacement for some time. It just didn't work out.</strong><hr></blockquote>
Do you have any proof that this is true? AFAIK Apple's management just doesn't seem to give a damn. Hiring one lone BeFS guy is the closest you can get to proof, but that doesn't mean anything. He could be doing a lot of other things. You guys just keep projecting your hopes on Apple, but no one outside Apple seems to know what's really going on.
[quote]<strong>So, I'm willing to wager that Apple thought it would be ready in time, but it didn't get done.</strong><hr></blockquote>
The wild guesses keep accumulating...
[quote]<strong>That let them with 2 choices:
1) They could reimplement labels and have people bitch and moan when they were dropped.</strong><hr></blockquote>
Read my posts please. They don't need to be dropped (label-bits do, but labels as such don't have to).
[quote]<strong>2) or, they could just not include them so that when the system changes you don't have to redo all your organization efforts.</strong><hr></blockquote>
If they solve the metadata problem well, it will be a piece of cake to change this.
[quote]<strong>Personally, I'd rather that they do it the second way because I hate organizing my files and I'd hate more than doubly doing it twice. Better things are coming and it's just a matter of time before they get here. Until then, stick to OS 9 or use the haxie. It's not that hard.</strong><hr></blockquote>
Doing it twice? If you were using labels under OS 9, you wouldn't have any extra work. You have to migrate anyway. It's exactly the opposite as you say, the people who were using labels and switched to OS X had to reorganize and will have to reorganize again if/when the new system comes (and doesn't contain an easy migration path, which would be easy to create).
[ 10-14-2002: Message edited by: wfzelle ]</p>
<strong>wfzelle, I'd love to argue this with you until the cows come home, but frankly, I've got work to do. You're welcome to your opinion, but you haven't done a thing to change mine, sorry.</strong><hr></blockquote>
I think that I've shown quite a few problems and inconsistenties in your post. It would suit you to answer my questions in that regard. IMHO a debate is not about throwing your opinion over the wall, you should be willing to have it questioned.
[quote]<strong>You're right in that continually thinking and never implementing results in no product, but consider also that immediate implementation and no thinking results in a catastrophically ill formed system.</strong><hr></blockquote>
I never said that one shouldn't think. What I said is that you must not bite more than you can chew. Be content with taking small steps and learning along the way. History has consistently shown that it's rare when solutions succeed that attempt to solve an entire problem at once. They usually die of bloat, architectural problems that weren't caught on paper and basic misunderstanding of the problems that occur when implementing.
[quote]<strong>You stated various incompatibilities in the myriad of Unix systems... where do you think they came from?</strong><hr></blockquote>
Innovation.
[quote]<strong>If we can avoid that, then by all means we should.</strong><hr></blockquote>
I'd rather not stop innovating. If something is innovative enough, it will undoubtably shake up the establishment. Usually there will be several different approaches to solve the same problem because there is no clearly superior solution. Convergence is a powerful force though and in the long run people will strive for decent interoperability and portability. That's already happening with the new Red Hat that integrates KDE and GNOME for instance. Word processing has converged towards .doc and .pdf. RPC is moving towards SOAP. As systems mature it becomes clear what the right solution is and a common standard can be created that supports most possible uses.
[quote]<strong>Implementation to shipping is a fine line between over- and under-engineering. I lean towards the over-, and you obviously are firmly in the under- camp. I don't see how our viewpoints are ever going to converge, given that, so I suggest we just agree to disagree.</strong><hr></blockquote>
I don't think that I lean (strongly) towards underengineering. Your position was quite extreme and I was trying to show the flaws of such a position. I was not argueing in favor of a wild west approach. Again, there is a big difference between limiting the scope of the problem you want to solve and limiting the engineering you do on that scope. The first is much more important than the second when you try to prevent overengineering.
I'm afraid all you've shown is a set of differing opinions on particular points. I'm perfectly willing to have my opinion questioned, and even proven wrong, as long as the other debater has sufficient evidence that they can provide, or at least a compelling argument that can stand up under scrutiny. I've yet to get either here... there hasn't been any debate, just warring opinions. IMHO, that's pretty low on my priority list of activities to engage in. (And yet here I am again. *sigh*)
[quote]I never said that one shouldn't think. What I said is that you must not bite more than you can chew. Be content with taking small steps and learning along the way. History has consistently shown that it's rare when solutions succeed that attempt to solve an entire problem at once. They usually die of bloat, architectural problems that weren't caught on paper and basic misunderstanding of the problems that occur when implementing.<hr></blockquote>
On that we agree, but where we appear to disagree is on the other approach: ad hoc cobbled together systems that grow, mutate, and otherwise just... *happen*. That's not design, that's an accident. cf: Unix. 30 years, and we have... what? Very little more than we had 10 years, or even 20 years ago. The basics are still the same, and still show the same problems. Why? People keep making the same mistakes. Why? Because the big problems are rarely if ever solved by the bazaar. Sorry. I believe in the open source model, but there are *some* problems that it is ill-suited for.
Metadata across filesystems is such a problem.
[quote]Innovation.<hr></blockquote>
*snort* Okay, you get points for sheer chutzpah.
[quote]I'd rather not stop innovating. If something is innovative enough, it will undoubtably shake up the establishment. <hr></blockquote>
Oh yes, NeXT took over the world, obviously. Windows XP is just a delusion. <img src="graemlins/oyvey.gif" border="0" alt="[No]" />
"Keen and kewl enough and they will come" is what killed NeXT for all intents and purposes, and what almost drove Apple into the grave. "Build it and they will come" DOESN'T work. Linux has been trying this approach for *how* long?
Innovation is wonderful. Flailing around in the dark like an epileptic baboon in a barrel isn't innovation, it's just a waste of resources and time. Infinite monkeys + infinite IDEs = innovation? Well, sure, given infinite *time*... I'd rather not wait that long.
[quote]Usually there will be several different approaches to solve the same problem because there is no clearly superior solution. Convergence is a powerful force though and in the long run people will strive for decent interoperability and portability. That's already happening with the new Red Hat that integrates KDE and GNOME for instance. Word processing has converged towards .doc and .pdf.<hr></blockquote>
Woah, hold it right there... you're claiming that .doc is a) a good standard, b) interoperable, or c) portable? Dear god, give me some of what you're smoking.
.doc is a lousy standard, it's about as far from interoperable as you can get, and it is highly portable... as long as you're running MS product. There are other apps that (mostly) work on the format, but it isn't because of choice, but market pressure from that little thing called a monopoly.
This isn't intelligent design, it's marketing gone haywire.
[quote] RPC is moving towards SOAP. As systems mature it becomes clear what the right solution is and a common standard can be created that supports most possible uses.<hr></blockquote>
As poorly designed systems mature they gain bloat as multiple implementations try and merge and end up producing an incestuous hodge-podge that invariable dies under its own weight.
You can't seriously sit there and tell me that a combination of KDE and Gnome is the 'clearly superior solution' for a Linux GUI, can you? No, it's just what is turning out to be the normative solution. Huge difference.
Even if you have two 95% solutions, trying to merge them to get a 100% solution is going to be a logistical nightmare, and god forbid you have to maintain the system. Better to back off, rethink it, and design *one* system that strives for 100% from the lessons of each. RedHat is headed down a bumpy road.
[quote]I don't think that I lean (strongly) towards underengineering. Your position was quite extreme <hr></blockquote>
Creating an intelligent design from the beginning is extreme? Wow. I had no idea. Here I thought this was software *engineering*, not software horticulture. Guess I've been wasting my time all these years in the wrong discipline.
[quote]and I was trying to show the flaws of such a position. I was not argueing in favor of a wild west approach. Again, there is a big difference between limiting the scope of the problem you want to solve and limiting the engineering you do on that scope. The first is much more important than the second when you try to prevent overengineering.<hr></blockquote>
Yes, it is, but just as important is keeping focus on what you're trying to solve. Limiting the problem down to what is just merely easy to solve results in poor designs, and a crazy quilt mess of a system. But, that seems to be the norm so of *course* it must be the best approach.
On a slightly related note, I'm a fan of XP, but I've seen it horribly abused and used as justification for the *CRAPPIEST* designs. "Well, we were just following XP and did the minimum..." generally is just an excuse for avoiding the actual problem originally under consideration. This is the road that your advocated approach generally ends up in, from my experience. With perfect coders in a perfect world with perfect users, then the bazaar would be *MY* methodology of choice, but unfortunately we don't live in such a world.
Limiting the scope of the problem is fine. Limiting the scope of the problem so far that it is no longer the problem you want to solve, merely because the limited version is easier, is just laziness or a cover for ineptitude and mediocrity.
And since this has long gone jumped the labels on X track, how about we continue this in General Discussion, eh? I won't move or lock this thread, in the hopes that others will continue the discussion about Unsanity's Haxie, but this portion of this thread belongs elsewhere.
First, yours is exactly the same argument that people used when Apple went all USB: Why can't they keep the old connectors around for a smooth transition? Because the PC world had tried that, and as a result nobody adopted USB.
Developers are lazy. Generally speaking, they will not support a new technology until they have to. What your system requires is for Apple to support the old way indefinitely: an application can continue to set labels the old way, and the OS will translate to the new way in the background. Indefinitely.
Or they could break it at some point, eliciting the exact same hue and cry they're getting now, only with no available justification and OS X native apps and users who have built their workflows around OS X's legacy support for labels.
The old way means that any file with a label has a resource fork. The Mac's forked filesystem is fundamentally incompatible with the rest of the world, so the resource fork tends to get lost. Remember, Apple is gunning for maximum compatibility here. Having files lose information arbitrarily is a Bad Thing(TM). Being incompatible at a filesystem level with the rest of the world is a Bad Thing(TM). PCs are not islands anymore.
[quote]<strong>Clearly you don't understand what you're talking about. The Web needs more metadata than extensions can provide. That's why both the MacOS and Windows have MIME mappings. Windows maps from/to extensions, MacOS used file types and now a mix of both. Unless you move to a BeFS-like MIME file type, you will be stuck with such a conversion.</strong><hr></blockquote>
What I was talking about was: The MacOS labels are set in the resource fork of the relevant file. This fork - and therefor, MacOS labels - are fundamentally incompatible with the Web, and with other platforms. This has nothing to do with whether filename extensions are an adequate replacement (for the record, I don't think they are anywhere near sufficient, and that should be clear from my prior posts about metadata).
You mention several perfectly good alternatives: Apple could adopt MIME natively and lose the need to map between the Web and its native metadata representation, for example. This would give us a world-standard, robust, extensible, plain-text metadata system. Or Apple could go somewhere else. They could go to a database filesystem, which would be entirely unlike anything else out there, that could be queried for things that looked like files with corresponding MIME types to other systems, and like XML files to others.
They have a lot of options. If they play their cards right, they could come up with something that is at once broadly compatible and astonishingly robust - making label/type/creator look poor by comparison. I don't really mind if they take their time doing it, frankly. If they're really going to do metadata, they'd better do it right.
[quote]<strong>I don't think the first option is that far-fetched. If you've been around Unix-heads (which 90% of the NeXTies are), then you know that most of them despise new metadata (you may not touch their drwxrwxrwx though). Why do you think that Mr Mach moved to file extensions? Would it make sense to move from file type/creator to extensions and then to something more advanced?</strong><hr></blockquote>
This is unfair to the UNIX crowd. For one thing, remember MIME? I think if you looked at the people who are responsible for that, and for HTTP, and for other protocols that recognize metadata, you'll find lots and lots of big, bearded UNIX gurus behind them. They do have a fundamentally different approach to the Mac's old way, however: in UNIX, a file is a stream of data, period, and how its name and contents are interpreted is up to the application looking at the file. Something like Apple's forked filesystem or VMS' indexed, record-based filesystem is alien to the UNIX way of thinking: To them, metadata is an interpretation imposed upon files, rather than an interpretation built into them (this holds for filename extensions as well -- UNIX itself, unlike Windows, could care less about them). Whence all the specifications like MIME that are external to the data whose types they're describing.
[quote]<strong>I would work with the community to try and solve this very difficult problem. Create a document that describes the problems you want to solve ('The new system should do x and y and z') and get feedback from developers (release it at WWDC for instance). Then come up with a possible solution and get feedback on that. The current system is a cludge with some severe problems. It could have used some peer review. </strong><hr></blockquote>
Did Apple work with the community in such an overt way to build its GUI? To design HFS, or extend it to HFS+? No. They are not sourceforge: They'll debut an essentially finished system at a WWDC, and then they'll get feedback from their developers whether they want it or not, and refine it as needed. That's what they do. It's what they've always done. They have some internal pressure as well: As I've pointed out, the iApps themselves cry out for robust metadata.
In the meantime, I can't blame them for getting rid of legacy as soon as they can. I'm a programmer myself, and I've made the mistake of grandfathering things that I fully intended to get around to replacing later, and... oops.
Legacy is a bitch. I'm confident that Apple deprecated labels and type/creator and other resources because they felt they could do better. And they can do much, much better.
<strong>On that we agree, but where we appear to disagree is on the other approach: ad hoc cobbled together systems that grow, mutate, and otherwise just... *happen*. That's not design, that's an accident. cf: Unix. 30 years, and we have... what? Very little more than we had 10 years, or even 20 years ago. The basics are still the same, and still show the same problems. Why? People keep making the same mistakes. Why? Because the big problems are rarely if ever solved by the bazaar. Sorry. I believe in the open source model, but there are *some* problems that it is ill-suited for.
Metadata across filesystems is such a problem.</strong><hr></blockquote>
The problem is that whenever someone proposes an enhancement that might break something, he gets ridiculed. They is no market as conservative as the Unix/open source one. The result is indeed a hodge podge. Why? Because most Unix fans only care about portability and interoperability in the short run. Now and then you need to accept a bit of inconvenience to get a much better system in the long run.
Few Unix guys are willing to accept any new metadata (or anything else) that brings more than an ounce of pain. The result is stagnation, they stick to the current hacks because 'at least those work'. They will continue to work for 20 years of course, but how much will you save by expending a bit of effort today?
[quote]<strong>Oh yes, NeXT took over the world, obviously. Windows XP is just a delusion.
"Keen and kewl enough and they will come" is what killed NeXT for all intents and purposes, and what almost drove Apple into the grave. "Build it and they will come" DOESN'T work. Linux has been trying this approach for *how* long?
Innovation is wonderful. Flailing around in the dark like an epileptic baboon in a barrel isn't innovation, it's just a waste of resources and time. Infinite monkeys + infinite IDEs = innovation? Well, sure, given infinite *time*... I'd rather not wait that long.</strong><hr></blockquote>
OO API's are only one part of an excellent OS, NeXT failed to fill in most of the other 2000 parts you need (and since most users aren't programmers and programmers get paid by users...). I still don't know what's so absolutely wonderful about Linux on the desktop except that it's free. We know from experimental economics that few consumers have price as their main concern, so it's lack of success is certainly not unexpected.
Innovation is not about buzzwords, it's about delivering what people want. Marketing is convincing people that the new features outweigh the faults and switching problems. Given sensible innovators and a correct marketing strategy, you certainly won't have infinite monkeys. Of course, both prerequisites are fairly scarce. When you haven't got money to make it happen, you need the buzz. Unfortunately, when nobody is willing to take metadata seriously, nothing will happen. Thus it is up for a company like Apple to end the status quo.
[quote]<strong>Woah, hold it right there... you're claiming that .doc is a) a good standard, b) interoperable, or c) portable? Dear god, give me some of what you're smoking.
.doc is a lousy standard, it's about as far from interoperable as you can get, and it is highly portable... as long as you're running MS product. There are other apps that (mostly) work on the format, but it isn't because of choice, but market pressure from that little thing called a monopoly.
This isn't intelligent design, it's marketing gone haywire.</strong><hr></blockquote>
90% of the people I know can read Word documents. No other advanced, editable word processing format can beat that (PDF is better for distributing read-only files). It's quite clear that today we need an open, extensible format, but during the rise of word processing, this simply was not a big enough issue. WP and MS were having a hard enough time improving their products and users didn't ask for it.
In short, .doc was a very succesful innovation during the 90's, but today we need a new innovation which resolves a lot of the issues with the .doc format (corruption, extensibility, broad interoperability). I hope that MS's new XDoc will be that innovation, it will make for an easy transition if they do.
[quote]<strong>As poorly designed systems mature they gain bloat as multiple implementations try and merge and end up producing an incestuous hodge-podge that invariable dies under its own weight.
You can't seriously sit there and tell me that a combination of KDE and Gnome is the 'clearly superior solution' for a Linux GUI, can you? No, it's just what is turning out to be the normative solution. Huge difference.
Even if you have two 95% solutions, trying to merge them to get a 100% solution is going to be a logistical nightmare, and god forbid you have to maintain the system. Better to back off, rethink it, and design *one* system that strives for 100% from the lessons of each. RedHat is headed down a bumpy road.</strong><hr></blockquote>
I think it's clear that KDE and Gnome are solving the same GUI problems in similar ways. There has been enough experience with desktop GUI's. They should merge and restrict the differences to area's where real innovation is required. I don't really care whether that happens through a merge or a rewrite. If Apple can put two different API's beneath the same GUI, Linux can have it as well (in theory at least).
[quote]<strong>Creating an intelligent design from the beginning is extreme? Wow. I had no idea. Here I thought this was software *engineering*, not software horticulture. Guess I've been wasting my time all these years in the wrong discipline.</strong><hr></blockquote>
No, you were asking for a solution which solves everything in one go, is fully backwards compatible and other such nonsense. That's as overengineering as you can get IMHO. We have OS X which only works well with HFS+. Why not take advantage of that by solving the limited problem: How do we get advanced metadata on the Mac? Forget converting the entire world for now, create this very useful solution first. It will be hard enough already.
[quote]<strong>Yes, it is, but just as important is keeping focus on what you're trying to solve. Limiting the problem down to what is just merely easy to solve results in poor designs, and a crazy quilt mess of a system. But, that seems to be the norm so of *course* it must be the best approach.</strong><hr></blockquote>
Keeping focus is easier when you restrict the scope. A lot of large designs forget their main purpose. See Mozilla for instance. It contains just about everything from a private programming language to an IRC client.
[quote]<strong>On a slightly related note, I'm a fan of XP, but I've seen it horribly abused and used as justification for the *CRAPPIEST* designs. "Well, we were just following XP and did the minimum..." generally is just an excuse for avoiding the actual problem originally under consideration. This is the road that your advocated approach generally ends up in, from my experience.</strong><hr></blockquote>
Any methodology requires smart architects, developers and managers. There is no panacea that can turn an idiot into a genius. XP can be used to great effect by a small well-led group who have to built a certain class of systems, but it is only one part of the puzzle. Strangely enough some expect unqualified people to create excellent products when we give them tools they cannot use. I wonder if the same people would appreciate being operated in the most advanced operating room by a first year medical student. Somehow I'd expect them to get the point when their lives are at stake.
[quote]<strong>Limiting the scope of the problem is fine. Limiting the scope of the problem so far that it is no longer the problem you want to solve, merely because the limited version is easier, is just laziness or a cover for ineptitude and mediocrity.</strong><hr></blockquote>
What do you think of the scope I selected?
[quote]<strong>And since this has long gone jumped the labels on X track, how about we continue this in General Discussion, eh? I won't move or lock this thread, in the hopes that others will continue the discussion about Unsanity's Haxie, but this portion of this thread belongs elsewhere.</strong><hr></blockquote>
I don't think that anyone is going to continue the discussion about labels in this thread (It seems I'm guilty of hijacking, sorry
[ 10-15-2002: Message edited by: wfzelle ]</p>
...
Unfortunately, when nobody is willing to take metadata seriously, nothing will happen. Thus it is up for a company like Apple to end the status quo.
<hr></blockquote>
Thank you. That's been precisely my point.
eWeek is now reporting that a Journaling File System <a href="http://www.eweek.com/article2/0,3959,634711,00.asp"; target="_blank">will soon be available</a> in 10.2.2.
Which as far as I know, is far sooner than anyone has predicted thus far.
[ 10-15-2002: Message edited by: Frank777 ]</p>
<strong>First, yours is exactly the same argument that people used when Apple went all USB: Why can't they keep the old connectors around for a smooth transition? Because the PC world had tried that, and as a result nobody adopted USB.</strong><hr></blockquote>
That is a different argument altogether. You have to buy new equipment for USB and people wanted to preserve their investment by keeping the old technology. What I propose is that we move to new technology exclusively, but can keep our old equipment (the so called win-win situation). It's like your serial connectors get magically transformed into USB connectors. That is not possible with physical objects of course, but it can be done with label-bits.
[quote]<strong>Developers are lazy. Generally speaking, they will not support a new technology until they have to. What your system requires is for Apple to support the old way indefinitely: an application can continue to set labels the old way, and the OS will translate to the new way in the background. Indefinitely.</strong><hr></blockquote>
Not at all. Apple puts calls into Carbon & Cocoa: getLabel() & setLabel(). Until we get the new metadata system those calls simply manipulate the label-bits. Thereafter they use a label-thingie in the new metadata system. The only problems you will see is when you need to migrate files from the old system to the new, but that is certainly manageable IMHO. Besides, Apple can just force you, so you won't have a choice. MacOS X Metadata will simply ignore the label-bits. So there is no way that people can resist the move to the new system other than not upgrading their OS.
[quote]<strong>The old way means that any file with a label has a resource fork.</strong><hr></blockquote>
No, it's a byte in the filesystem. Try it out, put a label on a file with only a data fork. Resedit will show that a resource fork was not added.
[quote]<strong>MacOS labels - are fundamentally incompatible with the Web, and with other platforms. This has nothing to do with whether filename extensions are an adequate replacement (for the record, I don't think they are anywhere near sufficient, and that should be clear from my prior posts about metadata).</strong><hr></blockquote>
You will lose labels when you copy files to the web or a PC, but I don't think that is a big problem. A lost resource fork that is crucial will break a file. That's why I agree with the move towards packages. A label is never crucial however. It's just a convenient organisation tool on Macs and users can learn not to depend on it whenever they move their files to something inferior.
[quote]<strong>You mention several perfectly good alternatives: Apple could adopt MIME natively and lose the need to map between the Web and its native metadata representation, for example. This would give us a world-standard, robust, extensible, plain-text metadata system.</strong><hr></blockquote>
MIME is only one piece of the puzzle. An advanced metadata system will store much more than just a file type. I think bigger
[quote]<strong>Or Apple could go somewhere else. They could go to a database filesystem, which would be entirely unlike anything else out there, that could be queried for things that looked like files with corresponding MIME types to other systems, and like XML files to others.</strong><hr></blockquote>
MIME would fit in such a system perfectly. The biggest problems are treefold:
1. You will lose data at the perimeter of the metadata system.
2. Some apps might process a file without copying the metadata (UNIX apps will experience this problem severely).
3. Speed.
[quote]<strong>They have a lot of options. If they play their cards right, they could come up with something that is at once broadly compatible and astonishingly robust - making label/type/creator look poor by comparison. I don't really mind if they take their time doing it, frankly. If they're really going to do metadata, they'd better do it right.</strong><hr></blockquote>
I've been waiting 3 years without a peep or hint at a solution. The only thing I've seen is regression: a move to extensions. MS is also aiming at a DB-enhanced filesystem, so Apple shouldn't wait to long.
[quote]<strong>This is unfair to the UNIX crowd. For one thing, remember MIME? I think if you looked at the people who are responsible for that, and for HTTP, and for other protocols that recognize metadata, you'll find lots and lots of big, bearded UNIX gurus behind them.</strong><hr></blockquote>
Ok, perhaps I was a tad unfair. But the point stands that MIME was only accepted at the application level. MIME stops to exist after the browser downloads a webpage or file.
[quote]<strong>They do have a fundamentally different approach to the Mac's old way, however: in UNIX, a file is a stream of data, period, and how its name and contents are interpreted is up to the application looking at the file.</strong><hr></blockquote>
Flawed and easily replaceable by a infinitely better system IMO. Why can't they simply accept that a file has more than one part? Why can't they accept that you might like to know what data you are dealing with? All that is required is that the stream is turned into a structure. If I need the data, I do getProperty('data'). If I need the filename, I do getProperty('filename'). Find out the type with getProperty('MIME'). You caneasily add new metadata in a well-designed system. But no, let's just treat it as one blob. So how do I find out if I'm dealing with a gif? Analyze the data or require the user to tell me. The first method requires magic numbers in the data, so we start mixing metadata with data. How can we easily process that? Do we first need to remove the metadata? What standard does it use? Can we have comments, labels, etc, etc?
And so we hack and kludge at a fundamentally flawed system.
[quote]<strong>Something like Apple's forked filesystem or VMS' indexed, record-based filesystem is alien to the UNIX way of thinking: To them, metadata is an interpretation imposed upon files, rather than an interpretation built into them (this holds for filename extensions as well -- UNIX itself, unlike Windows, could care less about them). Whence all the specifications like MIME that are external to the data whose types they're describing.</strong><hr></blockquote>
And yet a lot of Unix users do like to know what type their files are and so they use extensions like .html and .jpg. The problem is that these idiots don't understand that extra information is never a burden. You can always ignore/override it. And why do they support extensions, but not a MIME type in the file system? If they can handle 'mv a a.html', then why can't they handle 'ct a text/html'? Why is the second method wrong? I cannot see a conceptual difference. The extension is not built into the file anymore than the MIME type.
[quote]<strong>Legacy is a bitch. I'm confident that Apple deprecated labels and type/creator and other resources because they felt they could do better. And they can do much, much better.</strong><hr></blockquote>
Unfortunately, Avie Tevanian and his gang are known for feeling confident that they can do better and then failing. Avie simply doesn't understand the needs of the Mac community and he has to be beaten into submission time and again. Proof:
1) Carbon. Avie wanted everyone to adopt Cocoa, but had to change course after every big developer declined to rewrite their software from scratch. It took Apple three years to turn Rhapsody into MacOS X.
2) Applescript. Avie could do better. Users who had invested millions of dollars in Applescript solutions managed to change his mind. But most new Apple apps don't support applescript.
3) Apple's guidelines that we should move to filename extensions. I can't remember a bigger roar in the developer community.
4) The hierarchical Apple menu was removed and it's replacement is broken and slow.
5) The NeXT installer that replaced the Apple Installer is a broken piece of junk (for over 3 years). They even complained about it at stepwise. Ever wonder why you need a repair privileges utility? It's because the installer changes the privileges of the files that it replaces. This may even stop your system from booting. A more serious bug can destroy files in rare circumstances.
6) Dependency on file paths. I guess that robust file tracking is one more Mac-advantage that we lost.
7) Customization. The ultra-advanced OS X is harder to change than OS 9.
All the evidence points to one conclusion: Avie is more interested in advancing his own personal vision of computing than in serving customer needs. I strongly doubt whether his vision contains advanced metadata and so I'm concerned about the future.
[ 10-15-2002: Message edited by: wfzelle ]</p>
<strong>Just to interject:
eWeek is now reporting that a Journaling File System <a href="http://www.eweek.com/article2/0,3959,634711,00.asp"; target="_blank">will soon be available</a> in 10.2.2.
Which as far as I know, is far sooner than anyone has predicted thus far.</strong><hr></blockquote>
Interesting. This is a nice enhancement. On the other hand, now we know that the BeFS guy that Apple hired is not working on metadata.
<strong>
Interesting. This is a nice enhancement. On the other hand, now we know that the BeFS guy that Apple hired is not working on metadata.
Did you even read the article?
[quote]Indeed, Apple apparently enlisted the creator of the Be technology for its Elvis project: Be's BFS journaled file system was written by Dominic Giampaolo, who in March joined Apple as a file system engineer, The Register UK reported.
<hr></blockquote>
It boggles the mind...
<strong>Did you even read the article?</strong><hr></blockquote>
Yes. We already knew that Apple hired the guy, but we didn't know what he was working on. Some hoped he was working on metadata, but he clearly isn't.
So what did I supposedly get wrong?
<strong>
Yes. We already knew that Apple hired the guy, but we didn't know what he was working on. Some hoped he was working on metadata, but he clearly isn't.
So what did I supposedly get wrong?</strong><hr></blockquote>
See italicized section above.
I'm working on three major projects right now. If the first is released, does that mean I haven't been doing anything else?
Absence of evidence does not prove a negative.
[ 10-16-2002: Message edited by: Kickaha ]</p>
<strong>
See italicized section above.
I'm working on three major projects right now. If the first is released, does that mean I haven't been doing anything else?
Absence of evidence does not prove a negative.</strong><hr></blockquote>
Journaling is not that easy to implement AFAIK. At the very least this new info makes it less likely that he is working on metadata. But my conclusion was probably too strong.
You think metadata is that easy? OSX is now a multi-user system--should metadata be private or should metadata tags be visible to all users? How do permissions and metadata interact? Not so simple in multiuser environment.
if apple implemented an interim solution, such as labels as we knew them in os9, you're saying they'd have to support them all the way until they've implemented their new metadata system. well, what happens when 25% of their userbase is using one metadata system and 75% are using another? it's a pain in the ass. it's better to make a clean break and force people to use a workaround until they can do it right. otherwise they have to explain and support two different metadata systems to both users and developers.<a href="http://http://"; target="_blank">web page</a>
[ 10-21-2002: Message edited by: admactanium ]</p>