Leica, Apple working to fix Monochrom DNG bug that could erase Photos app library

2»

Comments

  • Reply 21 of 40
    wigginwiggin Posts: 2,265member
    Quote:
    Originally Posted by netrox View Post



    That should NEVER happen - ever. That's why I don't like the idea of Photo Library. An error can cause the entire DB to get corrupted. It's insane.

    Agreed it should never happen, but I question the article's statement that your Photos library is destroyed. The library or the database within the library?

     

    Quote:
    Originally Posted by jfc1138 View Post

     



    Indeed, Leica has really gone downhill. Maybe they should have stuck to film?


    No matter how malformed the DNG file is (and it's apparently not so mangled that other apps can't work with it without issue), the Photos app should never let that affect any data outside of the data related to that specific file.

     


    Quote:
    Originally Posted by cnocbui View Post

     

    I have never liked or used iPhoto precisely because I found the whole database idea to be both stupid and a pain.  Thank goodness iTunes was never saddled with this nonsense.  I was rather hoping Photos wouldn't follow the same paradigm.


     

     

    Quote:
    Originally Posted by Kevinneal View Post



    It may corrupt the database but surely you can "show package contents" of the library and still get to your photo's in the "masters" folder, or are they saying it corrupts every single image file (unlikely)

     

    As Kevinneal says, your photos aren't in a database. They are in a normal, Finder navigable folder structure just like iTunes. You just need to be able to open the Photos library package (right-click) which is really just a folder in disguise. Slightly less hidden than your Library folder which contains all of your email, for example. The metadata and edits recipe are in a database which is also inside of the package folder. Even if that were corrupted your photos are still in their folders. I think the only reason the iTunes media folder hasn't been hidden inside of a package is because iTunes is cross platform so they leave it as a regular folder for consistency with Windows.

     

    Of course, for the average user it's a moot whether it's the library database or the library itself (including your photos) which is destroyed. If Photos allows the database to get corrupted all they know is Photos won't display their photos, and they won't have any clue how to get into the package folder to find and recover at least the original photos.

  • Reply 22 of 40
    jfc1138jfc1138 Posts: 3,090member
    Quote:

    Originally Posted by popnfresh View Post





    Yeah, like you have a clue what you're talking about.



    Testy aren't we? 

     

    Try investing in a monitor that better resolves the sarcasm font.

  • Reply 23 of 40
    jfc1138jfc1138 Posts: 3,090member
    Quote:
    Originally Posted by Evilution View Post

     

    $7500 for a black and white camera! Holy sh......

    A good SLR and Photoshop, job done.

     

    People must be buying these for the name or as a show off.


     

    If people want the Leica jewelry (and they ARE works of art), this is a bargain compared to 15 to 25 grand for a dSLR body alone.... and the results of those lenses, on film anyway, produced images that looked like you could touch the rocks and feel the roughness..... the trick for these would be the quality of the sensor... though given they've eliminated the low-pass filter they expect to deliver.

     

    And for the Monochrom they have an explanation, at any rate: with just "light" sensor elements there's less image processing than with a color multi-sensor element per data point sensor to get the image detail..

    http://www.adorama.com/lcm246.html

     

    Less image processing could lead to higher detail resolution, plus physics does mean sensors take up space so having three instead of one per data "point" would have an impact.

     

    A niche product but then B&W has long been popular alongside color...

  • Reply 24 of 40
    bobschlobbobschlob Posts: 1,074member
    Jumpin jimminies! How the fk does something like that happen??
  • Reply 25 of 40
    bobschlobbobschlob Posts: 1,074member
    Quote:

    Originally Posted by netrox View Post



    That should NEVER happen - ever. That's why I don't like the idea of Photo Library. An error can cause the entire DB to get corrupted. It's insane.



    That's the nature of a DB. Something like that certainly should not happen, but there is no such thing as zero risk. Don't use databases. (?)

  • Reply 26 of 40
    bobschlobbobschlob Posts: 1,074member
    Quote:

    Originally Posted by bdkennedy1 View Post

     
    Quote:
    Originally Posted by NolaMacGuy View Post





    sorry, but bugs happen. not having a backup is insane.



    files get screwed up. thats why every enterprise has nightly backups. you can too, with Time Machine or similar.



    No third party should have he ability to corrupt a database. This is an Apple problem.




    Sorry, but I am a third party, and I can corrupt any database you'd care to give me.

  • Reply 27 of 40
    jabohnjabohn Posts: 582member
    Quote:

    Originally Posted by CMF View Post

     

    More importantly, why don't they fix the bug with large JPEG images in Preview and QuickLook? I'm still on 10.9 for this reason as the conditions under which the system hangs up or crashes are not that well understood (or publicized, for that matter). Hopefully this will come with 10.10.4. At this rate, I'd expect point updates for both OS X and iOS at WWDC.


    This was fixed a few weeks ago. It was a separate updater for 10.10.3 and will be rolled into 10.10.4 when it's released.

     

    https://support.apple.com/kb/DL1808?locale=en_US

  • Reply 28 of 40
    popnfreshpopnfresh Posts: 139member
    jfc1138 wrote: »

    Testy aren't we? 

    Try investing in a monitor that better resolves the sarcasm font.

    Try not mouthing off about things you know absolutely nothing about, for a change.
  • Reply 29 of 40
    s.metcalfs.metcalf Posts: 968member
    netrox wrote: »
    That should NEVER happen - ever. That's why I don't like the idea of Photo Library. An error can cause the entire DB to get corrupted. It's insane.

    The "registry" of photos! I hate it in Windows so why the hell would I want it for photos?

    Seriously Apple, stop overcomplicating things. The fact this could even happen shows that the programming model and implementation is pathetic and breaks any form of beat practise in programming.

    Programming is like engineering, build robust system, reduce risk and if possible dependencies, increase resilience, have backups/redundancy in place and enforce technical rigor. Simpler is often better and these problems wouldn't happen!
  • Reply 30 of 40
    boriscletoboriscleto Posts: 159member

    I'm sure this is a concern to the 3 people on the planet that this would affect.

  • Reply 31 of 40
    cmfcmf Posts: 64member
    Quote:
    Originally Posted by Marvin View Post





    Apple has an aim to hide the filesystem from users, which is understandable but it leads to problems like this because the application is then left to do the file management behind the scenes. iTunes is an example of doing this where if you change a song's details inside iTunes, it will rename the file and move it to an appropriate folder. When it comes to a single point of failure against a huge collection of important files, it opens up the possibility for a lot of damage to be done from minor bugs.



    I don't know why there has to be a library that manages the files at all. When you look at Spotlight, it is a database that indexes files as soon as they arrive on a drive but Spotlight itself never does anything to the files. SSDs are fast enough that you should be able to put files anywhere, even on externals and any app can know about the file but any metadata changes and edits go into the app's own space. When a bug like this arises, the app's own data could be renamed and reinitialized but the original photos left untouched and the app data can potentially be repaired to get back edits or other metadata.



    As strange as it sounds, this approach in iTunes has actually worked well for me, provided you follow Apple's guidance about how things should be organized. I do recall one incident about 10-11 years ago where I had serious metadata corruption and had to reinstall. In that case, I ended up talking to an Apple Support rep who walked me through the entire process and explained how it all worked. As a result, I am keenly aware of how the different pieces of the app function. Even with almost 130 GB of media on an external drive, I've done iTunes updates, OS upgrades, and migrated to a new machine without any big problems.

     

    The reasons why this works may be due to the fact that iTunes also has to run on Windows, but that's just speculation on my part...

     

    For an example of how this can go wrong, just look at Outlook for Mac. While I haven't used it personally, I've heard many stories of DB corruption (They do include a repair tool for this purpose - so the team is aware of the potential pitfalls).

  • Reply 32 of 40
    cmfcmf Posts: 64member
    Quote:



    Originally Posted by jabohn View Post

     

    This was fixed a few weeks ago. It was a separate updater for 10.10.3 and will be rolled into 10.10.4 when it's released.

     

    https://support.apple.com/kb/DL1808?locale=en_US




    I know they released a separate fix, but (at least according to Apple) this is iMac only.

     

    Suffice it to say, ANY graphics related bugs make me nervous. My last MacBook Pro had an 8600M GT in it. From what I've seen, performance has improved significantly under 10.10.3, but if I can't still turn on "Reduce Transparency" without everything going to shit, I'm concerned.

  • Reply 33 of 40
    MarvinMarvin Posts: 15,280moderator
    s.metcalf wrote: »
    The "registry" of photos! I hate it in Windows so why the hell would I want it for photos?
    CF wrote:
    For an example of how this can go wrong, just look at Outlook for Mac. While I haven't used it personally, I've heard many stories of DB corruption.

    Microsoft's databases tend to be more monolithic whereas Apple at least tries to maintain data in separate files so a single bad write to a file can't corrupt everything but Mail is another example of the way Apple does it. They keep all the emails as separate files but link them all to individual accounts so that if you just do something basic like hit the minus button on an email account because you don't want to get any new mail from that account, it removes every email with that single click along with the account, which leads to people making guides on how to get around it:

    https://support.tigertech.net/mail-safely-delete-account

    They are far too quick to wipe out people's data like connecting an iPhone or iPad to new iTunes accounts where hitting sync can wipe the whole device. Bulk deletion of libraries and devices full of data should be the absolute last resort, not the first. The OS should really go out of its way to stop you doing this, even when the user tries manually.

    This Photos bug might be a standard recovery procedure where if an app can't launch repeatedly then it resets it back to factory settings wiping the library out in the process but it should have a procedure for recovering from this like take a snapshot of when the app worked fine (it can do a recursive file listing in no time). Then after a faulty import process, it can take another snapshot of the filesystem to see what changed and the recovery would involve just reverting back to the previous snapshot by moving imported files and reverting back to the last metadata.

    The annoying thing is they do this where it isn't needed. They switched Textedit to do this automatically and it now saves every single change to a text document but they hide it away so hardly anyone will know its even doing that if they did benefit from it. When you hover near the title of a text document, an arrow appears and you can click to browse versions and it will show a whole history of edits to the file. These revisions can go right back years. It shouldn't be too hard to maintain snapshots for a few days in important apps so that if something really messes up, it can go back to how the computer was the day before without resorting to a backup.

    Having a system-wide snapshot would be very useful in tracking down issues too. It just needs to be a map of the entire filesystem of every file, even inside bundles with names, sizes, creation and modification dates. You can make these files very easily with the following command in a terminal:

    ls -alR / > ~/Desktop/snapshot.txt

    That took about 2 minutes to do my whole filesystem and the snapshot was ~150MB. On subsequent snapshots, the filesystem just has to track which nodes changed for reindexing. If you had a piece of malware installed or you noticed something going wrong, there can be an app or system pref that lets you see which files were changed on a day to day basis. For files that are small enough, they can retain the old one for a certain period of time to allow recovery and beyond that, remove it. In the interests of privacy, they can allow turning off and removing snapshots but have them on by default to allow recovering from errors.

    It looks like Aperture didn't support the Leica Monochrome DNG format and just failed to import:

    http://www.postseek.com/meta/423dc860a001273c15db36892f43b775
  • Reply 34 of 40
    jfc1138jfc1138 Posts: 3,090member
    Quote:

    Originally Posted by popnfresh View Post





    Try not mouthing off about things you know absolutely nothing about, for a change.



    Try not so hard at "Winning the Internet", as a suggestion?

  • Reply 35 of 40
    jfc1138jfc1138 Posts: 3,090member
    Quote:

    Originally Posted by boriscleto View Post

     

    I'm sure this is a concern to the 3 people on the planet that this would affect.




    Beyond that there IS the puzzle of why, if it's even true, the Photos library file data would be susceptible to corruption from a mere incompatible file type? 

     

    Other than the standard Apple hating the underlying concern would be if this library vulnerability is symptomatic of a more vulnerable file structure that is unstable for normal usage?

     

    Any application program I've used in many decades, when presented with an incompatible file type, gives me the rasberry or displays gibberish across the screen: leaking through to corrupt other data? That's an uh oh if true as it might not be isolated to that one oddball black and white digital file format.

     

    How about a slightly damaged SD card (hey, it happens) that you try and load files off of and one of the bad images does this same thing rather than a "cannot be read" rejection? Though that's for further examination of the issue.

  • Reply 36 of 40
    MarvinMarvin Posts: 15,280moderator
    jfc1138 wrote: »
    there IS the puzzle of why, if it's even true, the Photos library file data would be susceptible to corruption from a mere incompatible file type?

    Leica says it will cause a crash continuously on loading, maybe this arises from trying to make a thumbnail or something and this crashing is what can corrupt the database. Crashes won't always corrupt files but if the crash happened during a file write then it could damage the file. Photos might then not be able to find a working library and initialize a new one over it. It sounds like they are being overly cautious about it to make people aware of the possibilities. It's like when you see medicine labels where they list all the possible side effects - explosive diarrhoea, blindness, paralysis, erectile dysfunction, death. It doesn't mean they'll happen but the warning helps make sure you take whatever else they're saying seriously.

    If Photos does have a step where it initializes a new library over one with just a broken index file and removes files in the process then that is something they should consider changing.
  • Reply 37 of 40
    bigmikebigmike Posts: 266member

    Backup. Backup. Backup... Drives are cheap. Bugs can be expensive.

  • Reply 38 of 40
    jfc1138jfc1138 Posts: 3,090member
    Quote:

    Originally Posted by Marvin View Post





    Leica says it will cause a crash continuously on loading, maybe this arises from trying to make a thumbnail or something and this crashing is what can corrupt the database. Crashes won't always corrupt files but if the crash happened during a file write then it could damage the file. Photos might then not be able to find a working library and initialize a new one over it. It sounds like they are being overly cautious about it to make people aware of the possibilities. It's like when you see medicine labels where they list all the possible side effects - explosive diarrhoea, blindness, paralysis, erectile dysfunction, death. It doesn't mean they'll happen but the warning helps make sure you take whatever else they're saying seriously.



    If Photos does have a step where it initializes a new library over one with just a broken index file and removes files in the process then that is something they should consider changing.



    Ah that makes sense: like back in the day when some browsers were notorious hard drive killers (cough, Netscape, cough) for that very reason: constantly messing with caches on the drive and then when they crashed the write dribbled gibberish all over the disk, and as you say the usual suspect in a write abort is scrambling the index.... So may be a "flaw" in Photos if it's maniacal about keeping a temp backup or something on the drive.... writing opening that window of vulnerability.

     

    @bigmike: yep a terabyte isn't as expensive as it used to be... but I'm emotionally attached to my photos and so I can sympathize with the shock of finding the main library smashed even when I know intellectually it's recoverable off my backup.

  • Reply 39 of 40
    mr. memr. me Posts: 3,221member
    OK.

    After reading the OP by AppleInsider's robot and the nonsensical comments that followed, I inspected my [I]Photos Library[/I] for myself. I had inspected and even edited my [I]iPhoto Library[/I] going back to the time it was just an ordinary folder within the [I]Pictures[/I] folder. Unfortunately, too many children scattered their [I]iPhoto Library[/I] contents hither and yon. Eventually, Apple set the bundle bit and converted the [I]iPhoto Library[/I] into an impenetrable construction of Satan. As children, they did not know about [I]Show Package Contents[/I]. Within the new [I]iPhoto Library[/I] package was the same old folder that had always been there doing the same old thing that it had always done.

    With the installation of Apple's new demon works, [B]Photos[/B], was an all new and even less accessible construction of Lucifer the Lightbearer, the [I]Photos Library[/I]. The contents of the [I]Photos Library[/I] may have differences with the [I]iPhoto Library[/I], but they are minor. In the case of my Apple photo application libraries, they contain files dating back more than eight years and from several different cameras. I have pictures taken with cameras spanning the gamut from a $5 key chain camera to a $1500 Olympus DSLR. I have stored pictures taken with a relative's Fuji onto the smart card from my Olympus. I downloaded them all [I]en masse[/I] to my Macs.

    Make no mistake. I do not excuse damage or loss of photographs. I also want to hold the responsible party responsible. Having used [B]Image Capture[/B], [B]iPhoto[/B], [B]Photos[/B], and manufacturers' proprietary software with my Macs over the last 15 years, I cannot reflexively blame Apple. OTOH, I have no experience with Leica.

    That said, it is not at all clear that anyone is at fault. This may be a one-off problem. Also, no evidence was presented that any real damage has been done except to the specific files being downloaded from the Leica. It would be very interesting to go through the [I]Photos Library[/I] to see what is really there. If the Leica mounts on the OS X Desktop, then it would be very interesting to navigate the file structure of its smart card. Even if it does not mount on the Desktop, then its smart card can be removed and mounted in a card reader so that its file system can be navigated.

    It is better to light a candle than to curse the darkness.
  • Reply 40 of 40
    Quote:

    Originally Posted by NolaMacGuy View Post





    every bug has its origin.



    But was this one created or did it evolve from another?

Sign In or Register to comment.