moving thousands of files

Posted:
in Genius Bar edited January 2014
I have a that have several layers of sub-folders upon sub-folders in it. Inside all of these folders are JPEGs that I want to get out and put into a single folder.



I've tried doing a find for .jpg files in the main folder (both in the finder and with easyfind) but the resulting list is just too long (anywhere from 6000 to 17,000) and I have trouble moving the files. I was thinking about using the terminal to move the files but I can't figure out the syntax. Whatever I try copies the folder structure as well; that's not what I want.



I tried FileBuddy but I can't figure out a way to make it select only the jpegs inside all these sub folders and move them; I have to go one folder at a time or click the disclosure triangle on every single folder to get the JPEGs to display and then I can select them... that takes much too long. It also shows the insivible files which I don't need to see in this case but that's a minor issue.



I don't mind using the terminal to do this but I would prefer some utility as the other person doing this doesn't know enough to use the terminal. Moving around a folder with 17,000+ items in the finder ain't easy but that's something FileBuddy or the terminal can do easily once everything is in one folder I think.



Can anyone suggest a fast and easy way to do this?



Thanks
«1

Comments

  • Reply 1 of 26
    well don't I feel stupid..... I can use the search feature in filebuddy!



    Other than FileBuddy, is there any other way to do this?



    Thanks
  • Reply 2 of 26
    Can Automator do that?
  • Reply 3 of 26
    That's a good idea except I can't figure out how to get Automator to do it. I can't figure out how to have it search a folder full of sub folders for JPEGs and then copy all the found items to a specified folder. My find finder items seems to turn up zero results all the time so the next action, copy finder items, doesn't copy anything.



    If you know how to use Automator, please share your secrets
  • Reply 4 of 26
    jvbjvb Posts: 210member
    Quote:
    Originally Posted by FreddyFredFred View Post


    That's a good idea except I can't figure out how to get Automator to do it. I can't figure out how to have it search a folder full of sub folders for JPEGs and then copy all the found items to a specified folder. My find finder items seems to turn up zero results all the time so the next action, copy finder items, doesn't copy anything.



    If you know how to use Automator, please share your secrets



    IMO Automator is one of the most poorly constructed applications of all time.
  • Reply 5 of 26
    SpamSandwichSpamSandwich Posts: 33,407member
    Quote:
    Originally Posted by jvb View Post


    IMO Automator is one of the most poorly constructed applications of all time.



    I'm glad I'm not the only one who thinks so. It's a dumbed-down programming tool for non-programmers that can only be used by programmers. Terrible.
  • Reply 6 of 26
    jvbjvb Posts: 210member
    Quote:
    Originally Posted by SpamSandwich View Post


    I'm glad I'm not the only one who thinks so. It's a dumbed-down programming tool for non-programmers that can only be used by programmers. Terrible.



    I also love when Automator is mentioned in interviews as "the best programming tool for Mac". It makes absolutely no sense. I must be missing something.
  • Reply 7 of 26
    f1turbof1turbo Posts: 257member
    I haven't used Automator much, but it looks to me like this might work:



    - launch Automator

    - Select Finder

    - Drag "Find Finder Items" to the workflow

    - Drag "Get Selected Finder Items" to workflow

    - Drag "Move Finder Items" to workflow (in this order)



    Then go into your options for each of the three steps to select the files you want and move them to the location you want. Not sure if you need the middle step (Get) or not.



    Try some test runs and see if it's doing what you want it to do.
  • Reply 8 of 26
    Create a SMART folder that searches for .JPG files from the top level folder. Open the SMART folder and you will see all of the files in there. Move them as you would normally move them via Finder.
  • Reply 9 of 26
    Half the time it's easier to write a shell script than make an automator script. You could write a recursive script that pulls all jpegs (or whatever) into a target folder, and deletes the source folder if empty.
  • Reply 10 of 26
    Well..... nothing was working, even the smart folders until I remembered I put that folder of images in the spotlight privacy tab. Took it out and things started to work a little better.



    Quote:
    Originally Posted by F1Turbo


    I haven't used Automator much, but it looks to me like this might work:



    - launch Automator

    - Select Finder

    - Drag "Find Finder Items" to the workflow

    - Drag "Get Selected Finder Items" to workflow

    - Drag "Move Finder Items" to workflow (in this order)



    Then go into your options for each of the three steps to select the files you want and move them to the location you want. Not sure if you need the middle step (Get) or not.



    Try some test runs and see if it's doing what you want it to do.



    This does work and I don't need the "get selected finder items" action.





    Quote:
    Originally Posted by talksense101


    Create a SMART folder that searches for .JPG files from the top level folder. Open the SMART folder and you will see all of the files in there. Move them as you would normally move them via Finder.



    This one works too and I think I like it better than the automator solution because I can see the entire list of files which helps me find problems with naming. It doesn't seem to work as nicely when there are duplicate names though. I tried to select 3 files (2 duplicates, 1 not) and the finder refused to copy any of them.



    Is there any way to use smart folders to find duplicate file names in that big list of file names I end up with?



    Quote:
    Originally Posted by Splinemodel


    Half the time it's easier to write a shell script than make an automator script. You could write a recursive script that pulls all jpegs (or whatever) into a target folder, and deletes the source folder if empty.



    I like that solution except I can't write the scripts. About all I know is to do ls -R and that's about it. I don't know how to write the script to do it and it's not a user friendly solution because only I would be able to modify it. The other person doing this wouldn't know how to modify or use a shell script and the last thing I need is for them to have a typo in there that messes up everything.





    It looks like a Smart Folder or FileBuddy is the way to go. A smart folder is free and gets the job done but file buddy seems to be better at finding and fixing the little problems before I put everything into a single folder.



    Speaking of which, I do a search for .jpg in filebuddy and come up with 6208 items. I do a search for creator type of JPEG and come up with 6201. A Smart folder determines there are 6205 files with a name containing (or ending with) .jpg. Who knows where the problem is?
  • Reply 11 of 26
    Unless the finder can do this, I think I'm going to have to go with FileBuddy. The difference between the 6201 nd 6208 in filebuddy were 7 tiff files. I added the type column to the list in filebuddy and sorted by type. Filebuddy showed me 7 files with a file type of tiff that had an ending of .jpg.



    Still don't know how the finder is getting 6205 though It's 3 less than filebuddy shows when searching for a .jpg.



    The fun continues....
  • Reply 12 of 26
    splinemodelsplinemodel Posts: 7,311member
    Quote:
    Originally Posted by FreddyFredFred View Post


    I like that solution except I can't write the scripts. About all I know is to do ls -R and that's about it. I don't know how to write the script to do it and it's not a user friendly solution because only I would be able to modify it. The other person doing this wouldn't know how to modify or use a shell script and the last thing I need is for them to have a typo in there that messes up everything.



    You have to start somewhere. . . Look up "Bash scripts" or simply "shell scripts" (you are probably running the bash shell). If you are willing to learn, it might be worth the time. If you have a little bit of programming experience, this will be very quick. If not, shell scripting is not a bad intro to programming.



    And yes, you should practice on duplicated subsets of the data, which in this case are the folders of jpegs.



    Lastly, you would most likely run the shell script as a file i.e. "./elscripto" so it behaves just like a program. Just make sure to give it odd permission bits (e.g. chmod 777 elscripto) so that it's executable. All of this should be covered in any basic shell scripting primer.
  • Reply 13 of 26
    lundylundy Posts: 4,466member
    This little script will copy the items over. It will skip files that would overwrite files with the same name in the destination folder. The first choose folder is for the destination folder which must exist (make sure it is not inside the source folder) and the second choose folder is the folder containing the items to be copied.



    Test it out. Click the link below to load it into Script Editor on your machine.



    CLICK HERE to automatically place this script into Script Editor.
    Quote:

    set theDestination to (choose folder)

    tell application "Finder"

    \tset theList to items of (choose folder)

    \ttell me to copyTheItems(theList, theDestination)

    end tell





    to copyTheItems(theList, theDestination)

    \ttell application "Finder"

    \t\trepeat with theItem in theList

    \t\t\tif kind of theItem is "folder" then

    \t\t\t\tset theNewList to items of theItem

    \t\t\t\ttell me to copyTheItems(theNewList, theDestination)

    \t\t\telse

    \t\t\t\ttry

    \t\t\t\t\tduplicate theItem to theDestination

    \t\t\t\tend try

    \t\t\tend if

    \t\tend repeat

    \tend tell

    end copyTheItems



  • Reply 14 of 26
    Starting with lundy's most wonderful and helpful script, I managed to hack the rest of what I wanted into it.



    I found a bit of code on apple's web site to help me log errors to a file so now I will know which files have duplicates so I can track down the problems (or track down any other file that may have failed to duplicated I guess). It's not perfect (if I forget to close the file before running it again, I don't think it updates the file) but it's not the end of the world.



    Here's the code I'm using. It might not be pretty but it looks like it will do the job. If anyone sees something horribly wrong with it, please let me know.



    Now to find a way to use finder and command+F to perform all the same tricks as filebuddy



    Code:




    on write_to_file(this_data, target_file, append_data)

    try

    set the target_file to the target_file as text

    set the open_target_file to ¬

    open for access file target_file with write permission

    if append_data is false then ¬

    set eof of the open_target_file to 0

    write this_data to the open_target_file starting at eof

    close access the open_target_file

    return true

    on error

    try

    close access file target_file

    end try

    return false

    end try

    end write_to_file





    display dialog ("Select destination folder")

    set theDestination to (choose folder)



    display dialog ("Select error file location")

    global errorFileLocation

    set errorFileLocation to (choose folder)



    tell application "Finder"

    display dialog ("Choose source folder")

    set theList to items of (choose folder)

    tell me to copyTheItems(theList, theDestination)

    end tell





    to copyTheItems(theList, theDestination)

    tell application "Finder"

    repeat with theItem in theList

    if kind of theItem is "folder" then

    set theNewList to items of theItem

    tell me to copyTheItems(theNewList, theDestination)

    else

    try

    duplicate theItem to theDestination

    on error error_message

    set this_error to "Error: " & error_message & ": " & name of theItem & return

    --on error error_message number error_number

    --set this_error to "Error: " & error_number & ". " & ¬

    --error_message & theItem & return

    set the log_file to ((errorFileLocation) as text) & "Script Error Log"

    my write_to_file(this_error, log_file, true)

    end try

    end if

    end repeat

    end tell

    end copyTheItems



  • Reply 15 of 26
    lundylundy Posts: 4,466member
    I considered adding code to move all duplicates to a separate folder, but you could again have dupes, so I think your method of making a list is better in that regard. You will still have the files that "would have been dupes" remaining in the source folder, and you could add the line of code in your "on error" block



    set label index of theItem to 2



    Which would label it Red. Then you could easily find the files that were not moved.
  • Reply 16 of 26
    SpamSandwichSpamSandwich Posts: 33,407member
    All right you script kiddies (as long as you're on this)...



    What would I have to do to clean up my iPhoto files?



    After one of the previous Apple sytem updates, and my trying to locate a file, I mistakenly managed to erase my entire library of photos. After manually recopying them to my computer (from a backup DVD) I found iPhoto keeps Originals and smaller (file size) duplicates with the same names.



    Here's the question: Is there a way to eliminate the secondary unneeded smaller dupes and keep the originals (and also the slightly edited for "rotation") versions? I'm wondering if I could recover Gigs of HD space by carefully doing this with a properly conceived script.
  • Reply 17 of 26
    splinemodelsplinemodel Posts: 7,311member
    Nice to see AppleScript put to use, although it confuses the hell out of me. Applescript has always been a little bit mystical to me, kind of like makefiles.
  • Reply 18 of 26
    lundylundy Posts: 4,466member
    Yep. Those are most likely Thumbnails. The trick is defining a way to identify them. The best way is to go by the file size and dimensions. The file size of a thumbnail is usually 19 KB and its dimensions are either 180 by 240 or 240 by 180. The file size isn't available directly from iPhoto, so you have to get the Finder to do it, after getting the pointer to the file from iPhoto. Here is a script that will loop through all of the photos and place those that are less than 20 kilobytes or have the dimensions 240 by 180 into a separate album. Then you can peruse those and see if in fact they are all thumbnails and not just small photos that you actually want.



    CLICK HERE to automatically place this script into Script Editor.



    Quote:

    property minSize : 20 * 1024

    property thumbAlbum : "Thumbnail-Sized Photos"

    tell application "iPhoto"

    \tif (not (exists album thumbAlbum)) then new album name thumbAlbum

    \tset numThumbs to 0

    \trepeat with theCurrentPhoto in photos

    \t\tset theCurrentPhotoPath to image path of theCurrentPhoto

    \t\ttell application "Finder" to set sizeOfTheCurrentPhoto to (size of (info for (theCurrentPhotoPath as POSIX file)))

    \t\tset theDims to dimensions of theCurrentPhoto

    \t\tif (sizeOfTheCurrentPhoto is less than or equal to minSize) and (theDims is {240.0, 180.0} or theDims is {180.0, 240.0}) then

    \t\t\tadd theCurrentPhoto to album thumbAlbum

    \t\t\tset numThumbs to numThumbs + 1

    \t\tend if

    \tend repeat

    \t

    \tif numThumbs is 0 then

    \t\tremove album thumbAlbum

    \t\tselect photo library album

    \t\tdisplay dialog "No thumbnail-sized photos were detected." buttons {"OK"} default button 1 with title "Thumbnail Search Results"

    \telse

    \t\tif numThumbs is 1 then

    \t\t\tset {pluralS, wasWere, letterA} to {"", "was", "a "}

    \t\telse

    \t\t\tset {pluralS, wasWere, letterA} to {"s", "were", ""}

    \t\tend if

    \t\t

    \t\tselect album thumbAlbum

    \t\tactivate

    \t\tdisplay dialog (numThumbs as text) & " photo" & pluralS & " appeared to be " & letterA & "thumbnail" & pluralS & " and " & wasWere & " moved to the Album " & quote & thumbAlbum & quote & "." buttons {"OK"} default button 1 with title "Thumbnail Search Results"

    \tend if

    end tell





    Quote:
    Originally Posted by SpamSandwich View Post


    All right you script kiddies (as long as you're on this)...



    What would I have to do to clean up my iPhoto files?



    After one of the previous Apple sytem updates, and my trying to locate a file, I mistakenly managed to erase my entire library of photos. After manually recopying them to my computer (from a backup DVD) I found iPhoto keeps Originals and smaller (file size) duplicates with the same names.



    Here's the question: Is there a way to eliminate the secondary unneeded smaller dupes and keep the originals (and also the slightly edited for "rotation") versions? I'm wondering if I could recover Gigs of HD space by carefully doing this with a properly conceived script.



  • Reply 19 of 26
    SpamSandwichSpamSandwich Posts: 33,407member
    Fantastic. Thanks for the very helpful bit of programmery. One question about this, though... if they are thumbnails, maybe I should not remove them. Does this detect duplicate images based on the name and file size?



    Again... thanks! This could be an extremely useful thread for us incompetent wannabe scripters.
  • Reply 20 of 26
    lundylundy Posts: 4,466member
    Quote:
    Originally Posted by SpamSandwich View Post


    Fantastic. Thanks for the very helpful bit of programmery. One question about this, though... if they are thumbnails, maybe I should not remove them. Does this detect duplicate images based on the name and file size?



    Nope. But nothing is getting deleted - it just moves the "suspected" thumbnails to their own album so that you can peruse them.



    Finding dupes would be another script, much more complex and would take a long time on big libraries. Better to use the Sort menu item and Sort by Title.

    Quote:

    Again... thanks! This could be an extremely useful thread for us incompetent wannabe scripters.



    My pleasure. Keep 'em coming. I've been coding Applescript almost every day since it came out in 1993 with System 7 Pro. It still sometimes takes me an hour to figure out a particular syntax. Thank goodness for Script Debugger from Late Night Software.
Sign In or Register to comment.