What hasn't been discussed much yet is the underlying storage model that would work well with a "metadata-driven" interface abstraction. Is it necessary to replace the hierarchical filesystem or can something successfully be built on top of it?
Picking a couple familiar apps, iPhoto and iTunes already have the ability to store imported data in hierarchies of their design. As we know, that's a gnarly issue for users who'd rather pre-organize that content however they choose. I usually prefer letting those apps store content as they choose since the interfaces to organize/manage it are sufficient. But I also recognize issues with certain attributes associated with that content being unavailable outside those apps. Part of that problem is the "metadata" existing in separate databases. A few cooperating apps can "access" (view/read) iTunes playlists and iPhoto albums, but it's not pervasive enough so any app (including Finder!) can recognize them and even modify them.
I think BeOS partially solved some of that but I've never used it.
I don't care if my data is stored in hierarchical filesystems but I don't want to be tediously managing/organizing them within the rigid, location-dependent limitations of that file/folder structure. I want every "folder" to be a group/view of data independent of where it's actually stored, similar to iTunes/iPhoto playlists/albums, Yojimbo collections, etc. Sharable, too.
Jumping more to the main thread theme ...
If Apple doesn't adequately "fix Finder" I'm curious whether Bruce Horn's iFile might become a decent Finder replacement.
Well, that's kind of the idea. The existing folder mechanism would survive, and be completely usable to those who feel comfortable in it. No changes to their workflow needed. The visible user-accessible portion would remain exactly the same, but the guts would be ripped out and replaced wholesale.
But... smart folders would take on a whole new level of usefulness because the user's folder organization would automatically be used for metadata tagging.
Traditional folders would be view/controller objects (document objects? now there's a funky idea) into the metadata soup, while smart folders would be straight view objects.
Backwards compatibility, but leveraging all the metadata folks have built up over the years in their existing folder structures without replacing it, ala iTunes, et al. Various apps could then hook into this through a unified API and metadata access system, from Finder through the iApps, and on to third party apps.
I don't know about them, but every CMS I build stores files in a database. It's far easier to manage and flexible, but there is a speed hit involved (though that's because of the environment, not the concept of a DBFS).
Is anyone here familiar with the Reiser4 filesystem? If Apple worked with Namesys to develop a version of that filesystem with the necessary plugins for OS X it would be the holy freakin' grail.
Yes many of us Linux users are familiar with Reiser4 and if you think Hans would get anywhere with Apple on what he wants "changed" in XNU forget it.
In fact, I'm fascinated by the idea of no folders. It makes perfect sense (not being sarcastic, although I realize it may sound so).
livatlantis': A Dream.
I can almost picture Steve Jobs on stage, explaining, "Throw anything into OS X, and it will organize it for you. You don't need to remember where you put your photos or movies or documents, it's all in the OS database. Just throw in some information, and boom! - It's there."
"Of course, we know you need folders for some occasions. You need structure to create websites, to build applications... We are not doing away with folders. You can always have folders just like in Tiger.
But the beauty is that they won't be neccessary to use your compter. You won't need to depend on in. Let your computer organize it."
Then there could be a demo of the concept - a 3D world, a massive universe of information (camera floating around millions of icons in black space). Then there's sound of keyboard keys clicking.
A few of those millions of icons suddenly light up, and others fade away. The user is then presented with just the icons he/she is looking for.
And that's what Spotlight will do.
This sequence of events shows me that Apple is investing R&D into Federal areas. If they can perfect this technology and provide a public version and then a government version more options will open up for them.
folders are not dying as a methafor (it just groups files belong to eachother for some reason), they are just changing in a technical sense (saved Spotlight search-results) which make them more versitle.
You cant't just throw folders away; how are you going to upload your website to a Linux server then, eh?
I think that Apple will only keep the smart folders. Imagine if you move photos into a folder called "Holiday in Spain". Spotlight will then add the keyword/value: folder="Holiday in Spain". This keyword is then is used by the OS to build the hierarchy in Finder, but it would probably be just as easy to do it by "photocamera aperture" or something.
How about 'keep the smart folders visible' for unless I'm mistaken Spotlight is a layer above the BSD layer and therefore the inode system is in tact as it should be.
What's really amusing is that we're actually converging back on something like the original Mac had in 1984, and was universally panned for.
The current filesystem is HFS+. The H stands for "Hierarchical". As in folders. Now since this seems like an obvious part of a filesystem, why name it that way? Because the predecessor that shipping in Macs until, oh, 1987, was MFS - Macintosh File System, and it had no folders.
Yup, no folders. Everything was on floppy at that time, so it was kind of hard to lose your file in only 1.44MB of space. Instead, files were expected to be tagged with multiple types of metadata to help you recognize them, organize them, etc, but on a flat disk structure.
Unfortunately, they never really got the metadata thing nailed down right, and then hard drives gave people *PLENTY* of space (a whole 10MB!) to lose their files in, and folders were introduced in HFS. Metadata took a back seat, and now twenty years later, everyone's rushing to return to essentially MFS+.
I love cycles.
Unless Apache2/3, Java, etc., are suddenly metadata aware only tools you aren't going to see hierarchies poof away. A layer above the filesystem to quickly map to unique references [embedded as tags inside documents] due in part because the filesystem and all its contents keeps a generated blob of information about it can be very useful. Of course it also means the more people use it the more dependent upon it they become and when you don't have that "tool" at your disposal you forget a lot of basic steps.
Relying on a spreadsheet, calculator or other tools to crunch all your problems are examples that are great when used effectively and horrible when relied upon.
Take any engineering discipline: Say Heat Transfer. You model your blackbody solution, you determine your unknowns and you use your best engineering "judgement" of laws/rules to determine your unknown constraints.
You can develop tools to crunch it all out but if you really know how to solve the problems these hairy solutions reduce themselves quite rapidly and leave very straightforward and reduced equations to solve. Then applying these equations to programming routines in which millions of calculations over various material property constants are plugged in gives various possible scenarios. This is where computers can be useful tools.
If you dump ten million documents into a single box, go about your day while the system maps the meta data and then comeback to do a search and the results aren't as flattering as billed you realize that you can either wait for years of refinement in the meta tagging techniques or provide a solution that works with the strengths of hierarchical filesystems and the strengths of Spotlight.
Yes many of us Linux users are familiar with Reiser4 and if you think Hans would get anywhere with Apple on what he wants "changed" in XNU forget it.
I actually haven't heard his opinion on XNU, but I'm sure it's not terribly different from most people think. (she's a beast!)
Regardless, it would take some pretty significant changes to kernel to pull off something like this. I've heard suggestions that Apple should hire Namesys to help implement Reiser4 in OS X, and I'd have to agree. Let him bitch-slap them and make some cash while doing it. It's a win-win-win situation. Just because OS X is leading the pack doesn't mean they shouldn't fix what's broken.
Two-pronged approach... isn't that pretty much what I described in the long posts discussing folders, traditional and otherwise?
And really, what apps, other than disk utilities, do you know of that access the filesystem *DIRECTLY*? They don't. They go through very basic APIs, right? So... to the Unix layer, the APIs remain the same, and it acts just like a regular file/directory system, ala the regular folder look and feel in the Finder. But the files are being handled behind the scenes differently. No tweaks to Apache needed.
In fact, I'm fascinated by the idea of no folders. It makes perfect sense (not being sarcastic, although I realize it may sound so).
livatlantis': A Dream.
I can almost picture Steve Jobs on stage, explaining, "Throw anything into OS X, and it will organize it for you. You don't need to remember where you put your photos or movies or documents, it's all in the OS database. Just throw in some information, and boom! - It's there."
"Of course, we know you need folders for some occasions. You need structure to create websites, to build applications... We are not doing away with folders. You can always have folders just like in Tiger.
But the beauty is that they won't be neccessary to use your compter. You won't need to depend on in. Let your computer organize it."
Then there could be a demo of the concept - a 3D world, a massive universe of information (camera floating around millions of icons in black space). Then there's sound of keyboard keys clicking.
A few of those millions of icons suddenly light up, and others fade away. The user is then presented with just the icons he/she is looking for.
And that's what Spotlight will do.
This is a great vision, and I hope you are correct.
I wouldn't want this functionality for everything. I still need to separate work files from home, etc, but for basic filing, I hope you are correct.
Well, one could say that the metaphor of a file is just as unnecessary as that of a folder; everything could be one sea of hookable information. But frankly, I don't think many people are going to like that.
Any program can use the metadata, if the hooks are there for it. Right now, Apple hasn't evolved the concept as to what metadata they want exposed. They have exposed some of it, but not all.
THe interesting thing is that a program that is properly written will allow the user to define what data is filed. If we have a list, we can choose to use whichever defined data types are on that list. We should be able to add and subtract those types at any time.
By simply defining what types we want, the filing will be done automatically.
Right now, we have to manually file data. Some programs, as mentioned above will do some of that for us. But, it's still primitive.
The power will come when all data types will automatically come with the proper tags. Then we won't have to spend the time tagging them ourselves first. That is one of the most difficult, and tiresome tasks.
An example is iTunes. Put a cd in, and if you are connected to the net, the info for the cd pops up automatically, after a few seconds. Previously, we had to add all of that data by hand. Tedious indeed.
The next step would be if the program categorized all of the info in a database. It does some of that now, but only some.
The computer really doesn't have to organize anything until you open a program that makes use of it, unless you have some process running in the background that makes use of it.
Otherwise, the data can remain seemingly unorganized in the machine. There really isn't any reason why it would have to be organized. The point of use would determine the organization. If fact, organizing data outside of the need at the time, would be wasteful of resources. The very same data might be needed by numerous processes in different ways, in combination with other data. To have data permanently organized in various ways actually mediates against the entire concept of metadata.
The "hook" I mentioned earlier would be charged to only "catch" the metadata it needed. If the metadata was otherwise organized, it would still require the vast number of pointers that systems use now. A waste of resourses. Smart Folders are an in between repository of ever-changing data. They would contain the only pointers to data the system ever needs. It's not clear that even Smart Folders will survive. All they really are is a defined search, so the pointers aren't really needed. They look like folders because, well, that's what we feel comfortable with. As machines get quicker, well done metafile searches will be almost instantaneous.
This is a very deep subject. There is a lot to say.
Well, one could say that the metaphor of a file is just as unnecessary as that of a folder; everything could be one sea of hookable information. But frankly, I don't think many people are going to like that.
That's what Newton did. That type of mechanism can be extremely intuitive in situations where there is either limited data or a very powerful AI layer between the user and the data.
Unfortunately, neither of those are the situation. Not to mention that it would break nearly every piece of software in existence.
One thing that's missing from livatlantis' concept (and the discussion in ngeral) is adding your own data to manage your file, how much is really required. I think this is the weak link in the chain of all this -- how much will be demanded of users to add metadata (info about the files) and/or create these interrelationships?
This touches on a very good point which has bothered me since long before Tiger. Right now, OS X has far too many filesystems. As well integrated as Apple wants you to believe Spotlight is, it sadly is not. The Spotlight filesystem resides on a layer above HFS+ in the OS, and only ties in via the mdimport process. This becomes a serious issue when a disk is modified outside Tiger (or possibly even after a crash). The layers no longer match up correctly and the index becomes inaccurate. For example, my iBook's Spotlight index suddenly disappear recently for no apparent reason. If OS X were currently making better use of the data it would have been a very serious problem.
There are other disjointed filesystem layers as well. Programs like iTunes, iPhoto and Aperture all have internal databases to store metadata, while the files themselves are stored down on the HFS+ filesystem in folders. There are two major problems with a system like this.
First, the files and associated metadata are now located in three different places. HFS+ holds the data, the application's database holds custom metadata (such as iPhoto's image ranking etc) and Spotlight holds standard metadata. As a result, there is no way to view the metadata of one system from the other two. Even worse, some of the metadata is replicated unchanged through all three systems (such as EXIF data). The result is an unmanageable mishmash without even a hint of a "humane interface".
One thing that's missing from livatlantis' concept (and the discussion in ngeral) is adding your own data to manage your file, how much is really required. I think this is the weak link in the chain of all this -- how much will be demanded of users to add metadata (info about the files) and/or create these interrelationships?
I absolutely love how some apps implement their metadata editing systems. If Apple does implement a system like this it's very important that there are guidelines for which metadata tags are used and how. Standard types such as image resolution should be shared between programs.
One interesting thing I've learned about HFS+ is that it has had support for arbitrary metadata via the resource fork since its release with Mac OS 8.1. This ability wasn't accessible until Tiger, however, and even now it isn't being used by the OS (though I could be wrong). Actually, can anyone confirm that?
Yeah, it's all very complicated. I hope the lack of quality in the OS X Finder is due to Apple concentrating on its resourced on the Finder's replacement. Hey, I can dream, can't I?
That's what Newton did. That type of mechanism can be extremely intuitive in situations where there is either limited data or a very powerful AI layer between the user and the data.
Unfortunately, neither of those are the situation. Not to mention that it would break nearly every piece of software in existence.
This doesn't have to be done all at once. Companies are already putting metadata tags onto their file types for Spotlight to find. They are also incorporating the search into their programs. Both systems can co-exist for as long as they have to.
At some point, the metadata will become so "required" that companies will just have to change.
One thing that's missing from livatlantis' concept (and the discussion in ngeral) is adding your own data to manage your file, how much is really required. I think this is the weak link in the chain of all this -- how much will be demanded of users to add metadata (info about the files) and/or create these interrelationships?
This touches on a very good point which has bothered me since long before Tiger. Right now, OS X has far too many filesystems. As well integrated as Apple wants you to believe Spotlight is, it sadly is not. The Spotlight filesystem resides on a layer above HFS+ in the OS, and only ties in via the mdimport process. This becomes a serious issue when a disk is modified outside Tiger (or possibly even after a crash). The layers no longer match up correctly and the index becomes inaccurate. For example, my iBook's Spotlight index suddenly disappear recently for no apparent reason. If OS X were currently making better use of the data it would have been a very serious problem.
There are other disjointed filesystem layers as well. Programs like iTunes, iPhoto and Aperture all have internal databases to store metadata, while the files themselves are stored down on the HFS+ filesystem in folders. There are two major problems with a system like this.
First, the files and associated metadata are now located in three different places. HFS+ holds the data, the application's database holds custom metadata (such as iPhoto's image ranking etc) and Spotlight holds standard metadata. As a result, there is no way to view the metadata of one system from the other two. Even worse, some of the metadata is replicated unchanged through all three systems (such as EXIF data). The result is an unmanageable mishmash without even a hint of a "humane interface".
In conclusion, OS X needs fixing.
That's exactly why I was saying that there shouldn't BE any filing of data, other than when it is needed. It wouldn't actually be filing anyway, just the results of a search. I'm not certain that indexing will be required in the long run. At this time, because of the slow data searching a HD is only capable of, it IS required. but, when that issue is gone, then it shouldn't be required any longer. When you open a program, the search will automatically be done, almost instantly, and will be forgotten when you close down. Nothing more is required.
Right now, it's as much a technological issue as a software one.
At this time, because of the slow data searching a HD is only capable of, it IS required. but, when that issue is gone, then it shouldn't be required any longer. When you open a program, the search will automatically be done, almost instantly, and will be forgotten when you close down. Nothing more is required.
Right now, it's as much a technological issue as a software one.
If near-real-time searching of arbitrary amounts of (meta)data were already possible we'd be having a different discussion.
Comments
Picking a couple familiar apps, iPhoto and iTunes already have the ability to store imported data in hierarchies of their design. As we know, that's a gnarly issue for users who'd rather pre-organize that content however they choose. I usually prefer letting those apps store content as they choose since the interfaces to organize/manage it are sufficient. But I also recognize issues with certain attributes associated with that content being unavailable outside those apps. Part of that problem is the "metadata" existing in separate databases. A few cooperating apps can "access" (view/read) iTunes playlists and iPhoto albums, but it's not pervasive enough so any app (including Finder!) can recognize them and even modify them.
I think BeOS partially solved some of that but I've never used it.
I don't care if my data is stored in hierarchical filesystems but I don't want to be tediously managing/organizing them within the rigid, location-dependent limitations of that file/folder structure. I want every "folder" to be a group/view of data independent of where it's actually stored, similar to iTunes/iPhoto playlists/albums, Yojimbo collections, etc. Sharable, too.
Jumping more to the main thread theme ...
If Apple doesn't adequately "fix Finder" I'm curious whether Bruce Horn's iFile might become a decent Finder replacement.
But... smart folders would take on a whole new level of usefulness because the user's folder organization would automatically be used for metadata tagging.
Traditional folders would be view/controller objects (document objects? now there's a funky idea) into the metadata soup, while smart folders would be straight view objects.
Backwards compatibility, but leveraging all the metadata folks have built up over the years in their existing folder structures without replacing it, ala iTunes, et al. Various apps could then hook into this through a unified API and metadata access system, from Finder through the iApps, and on to third party apps.
Ommmmmm.....
Originally posted by Ringo
I don't know about them, but every CMS I build stores files in a database. It's far easier to manage and flexible, but there is a speed hit involved (though that's because of the environment, not the concept of a DBFS).
Is anyone here familiar with the Reiser4 filesystem? If Apple worked with Namesys to develop a version of that filesystem with the necessary plugins for OS X it would be the holy freakin' grail.
Yes many of us Linux users are familiar with Reiser4 and if you think Hans would get anywhere with Apple on what he wants "changed" in XNU forget it.
Originally posted by livatlantis
This image shocks:
In fact, I'm fascinated by the idea of no folders. It makes perfect sense (not being sarcastic, although I realize it may sound so).
livatlantis': A Dream.
I can almost picture Steve Jobs on stage, explaining, "Throw anything into OS X, and it will organize it for you. You don't need to remember where you put your photos or movies or documents, it's all in the OS database. Just throw in some information, and boom! - It's there."
"Of course, we know you need folders for some occasions. You need structure to create websites, to build applications... We are not doing away with folders. You can always have folders just like in Tiger.
But the beauty is that they won't be neccessary to use your compter. You won't need to depend on in. Let your computer organize it."
Then there could be a demo of the concept - a 3D world, a massive universe of information (camera floating around millions of icons in black space). Then there's sound of keyboard keys clicking.
A few of those millions of icons suddenly light up, and others fade away. The user is then presented with just the icons he/she is looking for.
And that's what Spotlight will do.
This sequence of events shows me that Apple is investing R&D into Federal areas. If they can perfect this technology and provide a public version and then a government version more options will open up for them.
Originally posted by dacloo
folders are not dying as a methafor (it just groups files belong to eachother for some reason), they are just changing in a technical sense (saved Spotlight search-results) which make them more versitle.
You cant't just throw folders away; how are you going to upload your website to a Linux server then, eh?
I think that Apple will only keep the smart folders. Imagine if you move photos into a folder called "Holiday in Spain". Spotlight will then add the keyword/value: folder="Holiday in Spain". This keyword is then is used by the OS to build the hierarchy in Finder, but it would probably be just as easy to do it by "photocamera aperture" or something.
How about 'keep the smart folders visible' for unless I'm mistaken Spotlight is a layer above the BSD layer and therefore the inode system is in tact as it should be.
Originally posted by Kickaha
What's really amusing is that we're actually converging back on something like the original Mac had in 1984, and was universally panned for.
The current filesystem is HFS+. The H stands for "Hierarchical". As in folders. Now since this seems like an obvious part of a filesystem, why name it that way? Because the predecessor that shipping in Macs until, oh, 1987, was MFS - Macintosh File System, and it had no folders.
Yup, no folders. Everything was on floppy at that time, so it was kind of hard to lose your file in only 1.44MB of space. Instead, files were expected to be tagged with multiple types of metadata to help you recognize them, organize them, etc, but on a flat disk structure.
Unfortunately, they never really got the metadata thing nailed down right, and then hard drives gave people *PLENTY* of space (a whole 10MB!) to lose their files in, and folders were introduced in HFS. Metadata took a back seat, and now twenty years later, everyone's rushing to return to essentially MFS+.
I love cycles.
Unless Apache2/3, Java, etc., are suddenly metadata aware only tools you aren't going to see hierarchies poof away. A layer above the filesystem to quickly map to unique references [embedded as tags inside documents] due in part because the filesystem and all its contents keeps a generated blob of information about it can be very useful. Of course it also means the more people use it the more dependent upon it they become and when you don't have that "tool" at your disposal you forget a lot of basic steps.
Relying on a spreadsheet, calculator or other tools to crunch all your problems are examples that are great when used effectively and horrible when relied upon.
Take any engineering discipline: Say Heat Transfer. You model your blackbody solution, you determine your unknowns and you use your best engineering "judgement" of laws/rules to determine your unknown constraints.
You can develop tools to crunch it all out but if you really know how to solve the problems these hairy solutions reduce themselves quite rapidly and leave very straightforward and reduced equations to solve. Then applying these equations to programming routines in which millions of calculations over various material property constants are plugged in gives various possible scenarios. This is where computers can be useful tools.
If you dump ten million documents into a single box, go about your day while the system maps the meta data and then comeback to do a search and the results aren't as flattering as billed you realize that you can either wait for years of refinement in the meta tagging techniques or provide a solution that works with the strengths of hierarchical filesystems and the strengths of Spotlight.
Apple will use a two pronged approach.
Originally posted by mdriftmeyer
Yes many of us Linux users are familiar with Reiser4 and if you think Hans would get anywhere with Apple on what he wants "changed" in XNU forget it.
I actually haven't heard his opinion on XNU, but I'm sure it's not terribly different from most people think. (she's a beast!)
Regardless, it would take some pretty significant changes to kernel to pull off something like this. I've heard suggestions that Apple should hire Namesys to help implement Reiser4 in OS X, and I'd have to agree. Let him bitch-slap them and make some cash while doing it. It's a win-win-win situation. Just because OS X is leading the pack doesn't mean they shouldn't fix what's broken.
And really, what apps, other than disk utilities, do you know of that access the filesystem *DIRECTLY*? They don't. They go through very basic APIs, right? So... to the Unix layer, the APIs remain the same, and it acts just like a regular file/directory system, ala the regular folder look and feel in the Finder. But the files are being handled behind the scenes differently. No tweaks to Apache needed.
Originally posted by livatlantis
This image shocks:
In fact, I'm fascinated by the idea of no folders. It makes perfect sense (not being sarcastic, although I realize it may sound so).
livatlantis': A Dream.
I can almost picture Steve Jobs on stage, explaining, "Throw anything into OS X, and it will organize it for you. You don't need to remember where you put your photos or movies or documents, it's all in the OS database. Just throw in some information, and boom! - It's there."
"Of course, we know you need folders for some occasions. You need structure to create websites, to build applications... We are not doing away with folders. You can always have folders just like in Tiger.
But the beauty is that they won't be neccessary to use your compter. You won't need to depend on in. Let your computer organize it."
Then there could be a demo of the concept - a 3D world, a massive universe of information (camera floating around millions of icons in black space). Then there's sound of keyboard keys clicking.
A few of those millions of icons suddenly light up, and others fade away. The user is then presented with just the icons he/she is looking for.
And that's what Spotlight will do.
This is a great vision, and I hope you are correct.
I wouldn't want this functionality for everything. I still need to separate work files from home, etc, but for basic filing, I hope you are correct.
THe interesting thing is that a program that is properly written will allow the user to define what data is filed. If we have a list, we can choose to use whichever defined data types are on that list. We should be able to add and subtract those types at any time.
By simply defining what types we want, the filing will be done automatically.
Right now, we have to manually file data. Some programs, as mentioned above will do some of that for us. But, it's still primitive.
The power will come when all data types will automatically come with the proper tags. Then we won't have to spend the time tagging them ourselves first. That is one of the most difficult, and tiresome tasks.
An example is iTunes. Put a cd in, and if you are connected to the net, the info for the cd pops up automatically, after a few seconds. Previously, we had to add all of that data by hand. Tedious indeed.
The next step would be if the program categorized all of the info in a database. It does some of that now, but only some.
The computer really doesn't have to organize anything until you open a program that makes use of it, unless you have some process running in the background that makes use of it.
Otherwise, the data can remain seemingly unorganized in the machine. There really isn't any reason why it would have to be organized. The point of use would determine the organization. If fact, organizing data outside of the need at the time, would be wasteful of resources. The very same data might be needed by numerous processes in different ways, in combination with other data. To have data permanently organized in various ways actually mediates against the entire concept of metadata.
The "hook" I mentioned earlier would be charged to only "catch" the metadata it needed. If the metadata was otherwise organized, it would still require the vast number of pointers that systems use now. A waste of resourses. Smart Folders are an in between repository of ever-changing data. They would contain the only pointers to data the system ever needs. It's not clear that even Smart Folders will survive. All they really are is a defined search, so the pointers aren't really needed. They look like folders because, well, that's what we feel comfortable with. As machines get quicker, well done metafile searches will be almost instantaneous.
This is a very deep subject. There is a lot to say.
Originally posted by Placebo
Well, one could say that the metaphor of a file is just as unnecessary as that of a folder; everything could be one sea of hookable information. But frankly, I don't think many people are going to like that.
That's what Newton did. That type of mechanism can be extremely intuitive in situations where there is either limited data or a very powerful AI layer between the user and the data.
Unfortunately, neither of those are the situation. Not to mention that it would break nearly every piece of software in existence.
Originally posted by melgross
Also, Spotlight IS a file system.
This touches on a very good point which has bothered me since long before Tiger. Right now, OS X has far too many filesystems. As well integrated as Apple wants you to believe Spotlight is, it sadly is not. The Spotlight filesystem resides on a layer above HFS+ in the OS, and only ties in via the mdimport process. This becomes a serious issue when a disk is modified outside Tiger (or possibly even after a crash). The layers no longer match up correctly and the index becomes inaccurate. For example, my iBook's Spotlight index suddenly disappear recently for no apparent reason. If OS X were currently making better use of the data it would have been a very serious problem.
There are other disjointed filesystem layers as well. Programs like iTunes, iPhoto and Aperture all have internal databases to store metadata, while the files themselves are stored down on the HFS+ filesystem in folders. There are two major problems with a system like this.
First, the files and associated metadata are now located in three different places. HFS+ holds the data, the application's database holds custom metadata (such as iPhoto's image ranking etc) and Spotlight holds standard metadata. As a result, there is no way to view the metadata of one system from the other two. Even worse, some of the metadata is replicated unchanged through all three systems (such as EXIF data). The result is an unmanageable mishmash without even a hint of a "humane interface".
In conclusion, OS X needs fixing.
Originally posted by BuonRotto
One thing that's missing from livatlantis' concept (and the discussion in ngeral) is adding your own data to manage your file, how much is really required. I think this is the weak link in the chain of all this -- how much will be demanded of users to add metadata (info about the files) and/or create these interrelationships?
I absolutely love how some apps implement their metadata editing systems. If Apple does implement a system like this it's very important that there are guidelines for which metadata tags are used and how. Standard types such as image resolution should be shared between programs.
One interesting thing I've learned about HFS+ is that it has had support for arbitrary metadata via the resource fork since its release with Mac OS 8.1. This ability wasn't accessible until Tiger, however, and even now it isn't being used by the OS (though I could be wrong). Actually, can anyone confirm that?
Yeah, it's all very complicated. I hope the lack of quality in the OS X Finder is due to Apple concentrating on its resourced on the Finder's replacement. Hey, I can dream, can't I?
Originally posted by Ringo
That's what Newton did. That type of mechanism can be extremely intuitive in situations where there is either limited data or a very powerful AI layer between the user and the data.
Unfortunately, neither of those are the situation. Not to mention that it would break nearly every piece of software in existence.
This doesn't have to be done all at once. Companies are already putting metadata tags onto their file types for Spotlight to find. They are also incorporating the search into their programs. Both systems can co-exist for as long as they have to.
At some point, the metadata will become so "required" that companies will just have to change.
Originally posted by BuonRotto
One thing that's missing from livatlantis' concept (and the discussion in ngeral) is adding your own data to manage your file, how much is really required. I think this is the weak link in the chain of all this -- how much will be demanded of users to add metadata (info about the files) and/or create these interrelationships?
I addressed that in my post above.
Originally posted by Ringo
This touches on a very good point which has bothered me since long before Tiger. Right now, OS X has far too many filesystems. As well integrated as Apple wants you to believe Spotlight is, it sadly is not. The Spotlight filesystem resides on a layer above HFS+ in the OS, and only ties in via the mdimport process. This becomes a serious issue when a disk is modified outside Tiger (or possibly even after a crash). The layers no longer match up correctly and the index becomes inaccurate. For example, my iBook's Spotlight index suddenly disappear recently for no apparent reason. If OS X were currently making better use of the data it would have been a very serious problem.
There are other disjointed filesystem layers as well. Programs like iTunes, iPhoto and Aperture all have internal databases to store metadata, while the files themselves are stored down on the HFS+ filesystem in folders. There are two major problems with a system like this.
First, the files and associated metadata are now located in three different places. HFS+ holds the data, the application's database holds custom metadata (such as iPhoto's image ranking etc) and Spotlight holds standard metadata. As a result, there is no way to view the metadata of one system from the other two. Even worse, some of the metadata is replicated unchanged through all three systems (such as EXIF data). The result is an unmanageable mishmash without even a hint of a "humane interface".
In conclusion, OS X needs fixing.
That's exactly why I was saying that there shouldn't BE any filing of data, other than when it is needed. It wouldn't actually be filing anyway, just the results of a search. I'm not certain that indexing will be required in the long run. At this time, because of the slow data searching a HD is only capable of, it IS required. but, when that issue is gone, then it shouldn't be required any longer. When you open a program, the search will automatically be done, almost instantly, and will be forgotten when you close down. Nothing more is required.
Right now, it's as much a technological issue as a software one.
Originally posted by melgross
At this time, because of the slow data searching a HD is only capable of, it IS required. but, when that issue is gone, then it shouldn't be required any longer. When you open a program, the search will automatically be done, almost instantly, and will be forgotten when you close down. Nothing more is required.
Right now, it's as much a technological issue as a software one.
If near-real-time searching of arbitrary amounts of (meta)data were already possible we'd be having a different discussion.