OSX has a problem moving large number of individual files?

eotku · October 13, 2003 6:30PM

We have been finally upgrading our machines at work over to OSX. Last week I got a 1.8 G5 and since then have come across a problem with OSX that basically has screwed up my workflow so much that if Panther doesn't fix it I may have to go back to OS 9. I work with large amounts of small files and have to move, organize, combine and batch process for use in a multimedia CD's. I have found when opening a folder and selecting say 1200 individual files and try to drag them to a new location it can take around 6 minutes for the files to move. It doesn't matter if the files are photos or text files. Now in OS 9 it only takes 12 seconds to move the exact same files on my old G4 400. I routinely have to move as many as 5000 images at a time and when I try to do that I have waited as long as 30 minutes before having to forced quit of the finder. When it comes back up the files have moved. We have 2 G5's and they both do the same thing.

I have been working with apple and they say they had never seen this problem before or at least the tech guys I talked to said so. They we able to reproduce it in their lab but refuse to tell me if Panther will fix this issue. It is so frustrating to move to a G5 and have my work flow slow down by 30% as a result.

Anyone know if this is fixed in Panther? Anyone else experience this?

johnq · October 13, 2003 7:17PM

First, what Finder view are you using?

List View will be the worst, Icon View and Column View should be roughly the same as eachother and much better.

Second, are you moving entire folder contents, or just contiguous/discontiguous portions of a larger set of files in a folder?

If you are moving entire folders, you don't need to drag and drop, only select all then copy. Then paste into the target location. You even have one Undo for this, if you make a mistake.

I rarely suggest using Terminal, but it does allow you to move huge numbers of files very quickly, with less error as far as dropping into the wrong target (although it is rife with risk if you don't know the basics of Terminal/unix commandline).

You also might want to come up with some custom Applescripts.

Or you could try things like Pathfinder, or other Finder-like utilities. I can't vouch for any being faster but it's worth a try.

It is not normal to have problems moving files, I have no problem in the 1000-2000 file range, even on a G3 iBook. Only over Networks do I see a slowdown/stall.

fulmer · October 14, 2003 2:51AM

I've found the same thing on my dual 2Ghz system. When I select a number of files (between 400~1500) files, the finder hanges. I'm usually using list view to see the file sizes and dates.

ghost_user_name · October 14, 2003 6:41AM

The problem lies within the Finder itself, unfortunately or fortunately, depending on your perspective. If you can find an alternate solution to moving files such as an AppleScript or a terminal shell script, you will find it to be magnitudes faster. The amount by which the Finder stalls is directly proportional to the size of the open directory you are using. One theory behind the high CPU usage is that the Finder is trying to redraw the window contents with each step and is therefore simply hanging because it is changing too quickly for the CPU to keep up.

In Panther, Finder operations are much improved but still not as fast as they *should* be.

Hope this helps.

eotku · October 14, 2003 10:18AM

I normally need to use the list view because I always need to select contiguous / discontiguous portions of files and to select them I usually have to sort them by either name, date modified or kind of file. I have been trying to use column view more often to select but it is much harder to select the exact files I need although it does work a little bit faster and I do not usually have to restart the finder. 5000 files will move in 10 minutes.

Almost all of the time I need to move files and not copy them as I need to remove them from the source folder as well as add them into the destination folder. I guess I could copy and paste them into the destination folder and then go back to the source folder reselect the files and delete them but these steps take extra time which again makes OS 9 the much faster solution.

I do not want to move back to OS 9 as that would mean having to give up my G5.

ast3r3x · October 14, 2003 10:21AM

Not at my computer right now but can't you CUT instead of COPY?

pensieve · October 14, 2003 10:33AM

Quote:

Originally posted by ast3r3x

Not at my computer right now but can't you CUT instead of COPY?

There is no cut option. Only copy and paste.

To the OP: If you're on a G5, there's no "going back" to OS 9 anyway.

eotku · October 14, 2003 10:45AM

Quote:

To the OP: If you're on a G5, there's no "going back" to OS 9 anyway. :

Well we also just bought 3 of the dual boot 1.25 G4's and I'm sure one of those people would be happy to change with me. That is as long as I don't tell them about this issue.

code master · October 14, 2003 11:14AM

One option may be to select the files you want in list view, then change to column or icon view before actually moving them.

johnq · October 14, 2003 11:39AM

I don't understand:

Just select the files, then Copy, then open the destination folder in a new window and Paste. Return to the window and directory you copied from, which will still have your selections retained, and hit command-Delete. Not as elegant as Cut, I know, but hardly a major hassle.

However, Pathfinder, I believe, has a Move command, so you can do it in one step.

Anyway, drang and drop is not meant for huge amounts of files, I don't care what OS or platform. I know Mac OS 9 did it zippier, but that's just a bad habit.

Agreed that Mac OS X should not be so slow doing that, nonetheless.

gargoyle · October 14, 2003 12:36PM

Just out of interest, where are you moving the files from and to? Across internal HD's, from one partition to another on the same HD, the network, or just to another folder on the same partition ?

I would like to test this myself, but I do not have my mac... (come on TNT..)

I figure this is how the os performing the move.

from one disk to another:

copy then delete.

from one partition to another on the same disk:

copy then delete.

from one folder to another on the same partition:

reshuffle of the file index table, the actual files dont physically move on the disk.

from your disk to the network:

copy and delete.

from one network drive to another:

God knows!!! Probably network -> your puter -> network.

ghost_user_name · October 14, 2003 1:16PM

Quote:

Originally posted by johnq

I don't understand:

Just select the files, then Copy, then open the destination folder in a new window and Paste. Return to the window and directory you copied from, which will still have your selections retained, and hit command-Delete. Not as elegant as Cut, I know, but hardly a major hassle.

That's right; you don't understand.

Doing this will duplicate all of those thousands of files. This will take time and disk space. Then you have to wait for the system to delete the original files as well. This is hardly any better than just waiting for the system to move them.

eotku · October 14, 2003 1:37PM

I move files from one folder to another in the same partition. They are usually in the same sub folder so yes the file index is the only thing that is really changing.

I just completed some tests.

4755 files.

Test 1) I opened the source folder in list view so I could sort by date. Selected the files I wanted, changed to column view, copied the files. I then opened a new window for the destination folder in column view and pasted the files. Once the new files were there I then went back to the source folder where the files were still selected and then deleted them. This operation took 11 minutes and 5 seconds. The majority of time was spent moving the original files to the trash.

Test 2) I opened the source folder in list view so I could sort by date. Selected the files I wanted, changed to column view and then dragged the files to a different open window with the destination folder in column view. This operation took 7 minutes and 45 seconds.

So moving the files is faster than coping and then deleting.

ghost_user_name · October 14, 2003 2:06PM

I ran a test for you on my 3-year-old dual 500 G4. I created 2048 empty files. As you said, the size doesn't matter -- just the number of files.

In Panther, I selected all and dragged of them from one open window into another open window. Both were in icon view, set to 48x48 icon size and keep arranged by name. The drag operation itself stalled the finder for 40 seconds. Then the move started and everything was responsive again. The status was displayed in a little copy/move window with a nonterminating progress bar. The move of everything took 30 seconds.

So, we have a total of one minute and ten seconds, only the first half of which the Finder was not responding. Is this an improvement over your time, considering how much slower my Mac is?

Yes, it's still not on par with OS9 or even moves via the command line, but it seems to be getting better.

ghost_user_name · October 14, 2003 2:10PM

I did another test.

Same 2048 files. This time, the source folder was set to list view and the destination folder was not open. I simply dragged to its icon.

The Finder stalled for 50 seconds before responding, but the move then only took 10 seconds. One minute total here.

eotku · October 14, 2003 4:00PM

Brad thanks for the testing. Wow it sounds like Panther may indeed make this issue tollerable.

I did a test with the same 4755 files and with the first window in icon view. Drug over the files to another closed folder and it took 8 minutes to complete. I am running Jag 10.2.8 so my hopes have been raised a bit by your test.

One of the technicians I spoke to at Apple did give me a hint that this may be the case. After being put on hold for the seventh time he came back and told me that his recomemencation was to go try Panther as soon as it came out. He said he couldn't confirm that it would fix the issue as they are not allowed to comment on products that haven't been released but I think I sensed a grin on the other end of the phone.

Well looks like that's my choice now.

Thanks everyone!!

eotku · October 14, 2003 4:25PM

Hey, I did find a faster way to transfer. I transfered exactly like my test above and just did a force restart of the finder after a minute an a half and the files did transfer.

I hate to have to do this everytime but it might get me through until I get a copy of Panther.

ryaxnb · October 14, 2003 6:00PM

According to Dan, author of XvsXp.com, it's fixed.

ryaxnb · October 14, 2003 6:01PM

the Terminal is really fast at file moving/deletion,

ghost_user_name · October 14, 2003 7:32PM

Quote:

Originally posted by ryaxnb

According to Dan, author of XvsXp.com, it's fixed.

No, he mentions file *deletion* which is a whole different monster. That's a simple command-delete and they are gone to the Trash. When you have to drag a thousand+ files, THAT'S when the Finder slows down.

I intend to write a note to Dan so he is aware of this MAJOR discrepancy.

applenut · October 14, 2003 7:37PM

Quote:

Originally posted by johnq

I don't understand:

Just select the files, then Copy, then open the destination folder in a new window and Paste. Return to the window and directory you copied from, which will still have your selections retained, and hit command-Delete. Not as elegant as Cut, I know, but hardly a major hassle.

However, Pathfinder, I believe, has a Move command, so you can do it in one step.

Anyway, drang and drop is not meant for huge amounts of files, I don't care what OS or platform. I know Mac OS 9 did it zippier, but that's just a bad habit.

Agreed that Mac OS X should not be so slow doing that, nonetheless.

you don't understand that that is more work than it should be and the finder is crap?

i have this same problem all the time and its frustrating as hell.

OSX has a problem moving large number of individual files?

Comments