A closer look at Apple's new ProRes 422 video format

[AppleInsider]

Going into extreme detail, Apple has said its new HD video standard was designed not just to beat opposing formats but to clear bottlenecks -- some of which are stifling for video editors.

Although it touched on ProRes 422 as part of its Final Cut Studio 2 announcement, Apple later offered a more detailed explanation of why the format was created and its real advantages to film crews.

The software developer observes that one of its key trump cards, compressing full-quality HD to manageable file sizes, isn't new. In fact, the company notes that in some cases the variable bit rate encoding of ProRes -- which tosses away data in simpler frames -- isn't as purely efficient in crunching data as with other standards. The new RED One camera, for example, compresses a 4000-pixel wide or "4K" video image into a 200 megabit per second stream. By contrast, a high-quality 1920x1080 ProRes clip (roughly a quarter the size of the 4K video) is larger at 220Mbps.

But while this fits the storage needs of the camera, the processing overhead required to decode the video during editing can be crushing to all but the best systems -- a problem for video editors who may need to preview one or more clips at once. It leaves little room for adding effects and seldom scales down smoothly.

This is where ProRes 422 excels, says Apple.



Despite compressing HD footage to a size smaller than raw standard definition video, the codec's demands on the CPU and disk speed are light enough that a MacBook Pro can play editing-grade video in full HD resolution. This lets pros test out their editing work in the field without a desktop computer or a large amount of desktop-class storage, either of which can cripple the budget for smaller or already expensive projects.

Those fortunate enough to work at a Mac Pro are poised to see even more of a benefit, Apple boasts. An improved half-resolution decoding method means that Final Cut Pro 6's unlimited real-time preview sees an exponential increase in the number of active previews: where only a single 1080i, 60Hz video would be visible on an eight-core Mac Pro in uncompressed form, the use of ProRes 422's high quality mode allows four. Dropping the quality to 720p at a film-level 24Hz sees the number of streams jump from three to fourteen on the same system and increases further with the normal 145Mbps ProRes bitrate.



And while already impressive to users of Final Cut Pro 5, this performance boost is explicitly adapted to multi-core systems. ProRes is said to scale almost linearly with the number of cores available to process video, cutting the latency in drawing a finished frame in half every time the available processing power doubles.



Recording is also made easier. A Mac Pro with an HD SDI video card, used for capturing live broadcasts, can automatically transform raw HD into a ProRes 422 clip without losing visual data, optionally sending it back to other hardware at the same quality. Since video is recorded at either normal 8-bit or richer 10-bit color depths regardless of the resolution, video producers can get subtler color gradients without moving to a rarer format or consuming more space. Capturing 10-bit isn't even an option for videographers using Panasonic's DVCPRO-50 or Sony's IMX-50 HD video formats, Apple points out.

Finally, the space and workload savings from the format can be applied whether or not the source footage matches: even when editing pure HDV or XDCAM HD, ProRes can be applied solely to effects layered on top of the source for a smaller overall bitrate, a faster render, and to preserve the quality of the underlying video.

The net effect of all these factors is to make HD editing more ubiquitous, according to Apple. With the sharper resolutions filtering down from feature-length movies to live footage, the company believes that the ability to trim videos with less than ideal equipment is increasingly essential -- and, of course, that its own Mac hardware and Final Cut Studio software should be the first to make this a practical reality.

[ View this article at AppleInsider.com ]

[ Digg this story ]

[Bageljoey]

Will this help with my Sony HandyCam?

[dp123]

Avid has a similar format at the same bitrates, also 4.2.2. It's an open format. The real question is: is ProRes 4.2.2 just the same format in a different container. These details without a comparison to Avid's format, it's closest competitor, is almost irrelevant.

Yes, it's good to know Apple has their own that they are providing cheaper. But the comparison to DNxHD is what people really want to know.

[hmurchison]

It's funny Apple doesn't mention ProRes 422's handling of 4K in their whitepaper.

[solipsism]

How will help those cam versions of newly released movies floating around on torrent sites?

[mugwump]

Question: How does this codec compare to uncompressed footage? Is it truly "lossless"?

[MySchizoBuddy]

Originally Posted by mugwump

Question: How does this codec compare to uncompressed footage? Is it truly "lossless"?

Does it say anywhere thats its a lossless codec?
Why are people assuming thats its lossless

plus lossless is lossless. there isn't anything like "truly" lossless.
Is a Supersonic Jet truly Supersonic.

[JeffDM]

Originally Posted by hmurchison

It's funny Apple doesn't mention ProRes 422's handling of 4K in their whitepaper.

I couldn't open that whitepaper, the site just gives me a very fancy version of a 404.

I can't find anything anywhere on Apple's site that says that they support 4k at all. There is one booth at the Apple exhibit that says something to the effect of "2k Red ProRes 422". If FCS2 can do 4k, I don't understand why they wouldn't mention that somewhere.

[melgross]

Originally Posted by mugwump

Question: How does this codec compare to uncompressed footage? Is it truly "lossless"?

No, it's not lossless.

the company notes that in some cases the variable bit rate encoding of ProRes -- which tosses away data in simpler frames --

[paprochy]

Originally Posted by MySchizoBuddy

Does it say anywhere thats its a lossless codec?
Why are people assuming thats its lossless

plus lossless is lossless. there isn't anything like "truly" lossless.
Is a Supersonic Jet truly Supersonic.

Yes and Yes.

It means what it says. A supersonic jet, really is supersonic, in other words, it flies faster than the speed of sound.

Lossless compression is just that, something that compresses an image without destroying any image data. For example LZW compression in TIFF, it shrinks the filesize but no essential data about the image is lost leaving the image looking exactly as it did when it was RAW or Vector.

I'm not saying that ProRes422 is lossless. I just don't agree with your analogy. Makes no sense.

[ecking]

This has me drooling but I personally don't need it! lol

[Chucker]

Originally Posted by ecking

This has me drooling but I personally don't need it! lol

LAWL!

[Chucker]

Originally Posted by MySchizoBuddy

plus lossless is lossless. there isn't anything like "truly" lossless.

The distinction between lossless compression and lossy compression is actually very clear, unambiguous and simple.

If you take uncompressed data (e.g., TIFF, WAV, AIFF), compress it losslessly (e.g., PNG, TIFF-LZW, AIFC, ALAC, FLAC), then decompress it again, the image/audio data will be 100% perfectly identical to the original data.

If you, however, use a lossy compression (e.g., JPEG, HD Photo, MP3, AAC, WMA), then decompress, the data will be significantly different. Most parts taken out won't be as noticeable to us humans, but the differences are measurable, and worse, each successive conversion loses you even more data.

[palegolas]

Originally Posted by Chucker

... If you, however, use a lossy compression (e.g., JPEG, HD Photo, MP3, AAC, WMA), then decompress, the data will be significantly different. Most parts taken out won't be as noticeable to us humans, but the differences are measurable, and worse, each successive conversion loses you even more data.

The problem is that if you want to do color adjustments (levels, curves, contrast, color balance etc) on compressed material most of the time you will spot the compression straight away. Brightening up a dark area that is compressed surely brings up visible compression artefacts. But doing the same on uncompressed material could provide a good result, depending on the source of the footage. Of course the best would be to have RAW footage or 16 bit footage. I don't even know if that's available. But refining a RAW still photo is like magic compared to refining a high quality TIFF image.

I usually work with 95% compressed Photo JPEG codec though. I think it's good enough. The fixed bit rate codecs usually doesn't cut it for me since I'm most of the time working with drawn animation.. I'm excited that ProRes is VBR.

But people and insiders and sources.. WE NEED PICTURES OF PRORES, not specs.. Anyone could present specs, but if it doesn't look and perform good (like Pixlet.. horrible horrible Pixlet) then it's just bogus.

[digitalclips]

I wonder if Apple have any new streaming capabilities up their sleeves or can 422 encoded material be used for that also? Is this in any way a replacement for H.264? We all know iTunes has to start offering at least 720p one day soon and ideally be watchable as it downloads on a modern broadband connection.

[digitalclips]

Originally Posted by palegolas

The problem is that if you want to do color adjustments (levels, curves, contrast, color balance etc) on compressed material most of the time you will spot the compression straight away. Brightening up a dark area that is compressed surely brings up visible compression artefacts. But doing the same on uncompressed material could provide a good result, depending on the source of the footage. Of course the best would be to have RAW footage or 16 bit footage. I don't even know if that's available. But refining a RAW still photo is like magic compared to refining a high quality TIFF image.

I usually work with 95% compressed Photo JPEG codec though. I think it's good enough. The fixed bit rate codecs usually doesn't cut it for me since I'm most of the time working with drawn animation.. I'm excited that ProRes is VBR.

But people and insiders and sources.. WE NEED PICTURES OF PRORES, not specs.. Anyone could present specs, but if it doesn't look and perform good (like Pixlet.. horrible horrible Pixlet) then it's just bogus.

Given RED are on board with 422 at 4K and with the new Color application in FCPro Suite too, I have to believe Apple know what they are doing regarding your concerns and it is going to blow our socks off. Think about this, it is only a very short while ago experts discussed the merits of high end BetaCam SP or DigiBeta and all that at 640 x 480 We have come a long way in a very short time ... what a ride !

[aplnub]

Sounds to me that Apple is nipping at the heels of AVID for more and more market share. Editing on a MBP sounds very tempting for those in-the-field shots 30 miles from no-wheresville Canada.

Probably not any real innovation here, as mentioned above, more just about making progress toward bettering the video editing process at a way cheaper cost. That has to get a few people's attention.

[vinney57]

ProRes422 is 'visually lossless'. Which basically means you can't see any difference when using the HQ (10 bit) codec. It remains to be seen what this means in practice but there wouldn't be a lot of point if it didn't work. Its clear that it is to be used through the entire VFX and colour timing chain and then converted, if required, back to 10bit HD via software or hardware (AJA).

Apple mentioned 4k but didn't push it for a number of reasons. I suspect support at this time is minimal and that frankly nobody is actually asking for it. Its a necessary buzzword but its not a real requirement just yet. I know 'Color' doesn't support 4K and Motion 3 probably doesn't either.

There is a misconception about the RED compression. RED have achieved great compression ratios (Graham Natrass is a genius) because they are using wavelet compression on the RAW info from their Bayer style chip. This data is not usable in this form, it has to be converted to another format (any QT codec) using the software that they supply. Apple are going to be offering a native REDCODE codec (real time RED RAW Bayer de-mosaicing) for Final Cut Pro but it is not ready yet, and I suspect that this was the only justification the Apple had for mentioning 4k at all! There are no 4k monitors and only one projector (Sony).

When all those RED cameras hit the streets over the next year, 4k RAW compressed will be an ideal acquisition and archive format with 2k being an excellent and practical desktop workflow.

[Pascal007]

Originally Posted by palegolas

But people and insiders and sources.. WE NEED PICTURES OF PRORES, not specs.. Anyone could present specs, but if it doesn't look and perform good (like Pixlet.. horrible horrible Pixlet) then it's just bogus.

That's interesting. I thought about Pixlet when Apple presented ProRes 422 and I kept wondering why didn't Pixlet take off. When it was presented, Pixlet was supposed to be so promising... For some reason, it probably wasn't, but I don't have the expertise to understand why.

Would you care to expand on what made Pixlet so horrible ?

[palegolas]

Originally Posted by Pascal007

That's interesting. I thought about Pixlet when Apple presented ProRes 422 and I kept wondering why didn't Pixlet take off. When it was presented, Pixlet was supposed to be so promising... For some reason, it probably wasn't, but I don't have the expertise to understand why.

Would you care to expand on what made Pixlet so horrible ?

Horrible Pixlet was my initial reaction.. perhaps I should have just put it disappointing Pixlet, or Pixlet the sad story.

Pixlet was presented somewhat like ProRes, like "The codec that provides studio grade HD quality at SD bitrates" or something like that. But it didn't. The advantage over JPEG compression is that it preserves dark and bright colors much better, I give them that. But they should never have released it. Should have kept working on it until they reached ProRes (supposing ProRes is good.) They really never followed up on Pixlet. I know updating a codec isn't a good idea, but I actually was waiting for them to update it, since it was only available for QuickTime users anyway.

Pixlet is too CPU heavy. Just impossible to work in realtime with Pixlet. And then it uses a strange compression method that has especially animation look pretty bad around lines and areas with high contrasts. It almost looks like bulky plastic around the edges. Strange. It is possible to select compression level, but as soon as you go below 100% the "bulky plastic" effect takes off. And if you're using 100% it looks good on photographic material, regular filmed material, but you could as well reduce the CPU usage with 75% and cut the bitrate nearly in half with kind of the same result with the Photo JPEG codec.

I tested a lot of codecs when Pixlet was launched. Tested it for HD editing with bitrate, quality and usability in mind. I thought that if JPEG 2000 wasn't so heavy on the processor it would have been the best choise (supporting alpha and low bitrate with very good color and picture quality and all), but it always comes down to the regular Photo JPEG codec. High hopes for ProRes 422 to be what Pixlet should have been.

[Wil Maneker]

I'm wondering how long it will take to see this technology integrated into a consumer device such as the Apple TV. Live HD Recording and compression, seems good to me!

[reallynotnick]

Originally Posted by Wil Maneker

I'm wondering how long it will take to see this technology integrated into a consumer device such as the Apple TV. Live HD Recording and compression, seems good to me!

The point of this codec is for editing, it is not meant for Apple TV.

I think with some dedicated H.264 chips we could probably record that in real time, not quite sure on the cost of those right now though.

[Louzer]

Originally Posted by MySchizoBuddy

Does it say anywhere thats its a lossless codec?
Why are people assuming thats its lossless

plus lossless is lossless. there isn't anything like "truly" lossless.
Is a Supersonic Jet truly Supersonic.

Originally Posted by paprochy

Yes and Yes.

It means what it says. A supersonic jet, really is supersonic, in other words, it flies faster than the speed of sound.

Lossless compression is just that, something that compresses an image without destroying any image data. For example LZW compression in TIFF, it shrinks the filesize but no essential data about the image is lost leaving the image looking exactly as it did when it was RAW or Vector.

I'm not saying that ProRes422 is lossless. I just don't agree with your analogy. Makes no sense.

Originally Posted by Chucker

The distinction between lossless compression and lossy compression is actually very clear, unambiguous and simple.

Apparently no one caught his original point, "lossless" is "lossless". The use of the word "truly" is completely unnecessary and unrequired. You can't say you're 'lossless' and then actually lose data, just like you can't be 'sort of' pregnant. So, to say "truly" lossless implies that it might be 'kind of' lossless, maybe just losing 'some' bits.

That's all he was saying.

[palegolas]

Originally Posted by Louzer

Apparently no one caught his original point, "lossless" is "lossless". The use of the word "truly" is completely unnecessary and unrequired. You can't say you're 'lossless' and then actually lose data, just like you can't be 'sort of' pregnant. So, to say "truly" lossless implies that it might be 'kind of' lossless, maybe just losing 'some' bits.

That's all he was saying.

Reminds me of the Apple Lossless audio codec.. that's a strange one. Is it lossless or is it not?

[melgross]

Originally Posted by vinney57

ProRes422 is 'visually lossless'. Which basically means you can't see any difference when using the HQ (10 bit) codec. It remains to be seen what this means in practice but there wouldn't be a lot of point if it didn't work. Its clear that it is to be used through the entire VFX and colour timing chain and then converted, if required, back to 10bit HD via software or hardware (AJA).

Apple mentioned 4k but didn't push it for a number of reasons. I suspect support at this time is minimal and that frankly nobody is actually asking for it. Its a necessary buzzword but its not a real requirement just yet. I know 'Color' doesn't support 4K and Motion 3 probably doesn't either.

There is a misconception about the RED compression. RED have achieved great compression ratios (Graham Natrass is a genius) because they are using wavelet compression on the RAW info from their Bayer style chip. This data is not usable in this form, it has to be converted to another format (any QT codec) using the software that they supply. Apple are going to be offering a native REDCODE codec (real time RED RAW Bayer de-mosaicing) for Final Cut Pro but it is not ready yet, and I suspect that this was the only justification the Apple had for mentioning 4k at all! There are no 4k monitors and only one projector (Sony).

When all those RED cameras hit the streets over the next year, 4k RAW compressed will be an ideal acquisition and archive format with 2k being an excellent and practical desktop workflow.

Right.

[melgross]

Originally Posted by palegolas

Reminds me of the Apple Lossless audio codec.. that's a strange one. Is it lossless or is it not?

Yes, it is.

Lossles codec's can give about 50% compression, some a bit better, some a bit worse.

It's similar to what is done with Zip and Stuffit. Both are lossless, and both deliver about 50%.

[vinney57]

Originally Posted by Pascal007

That's interesting. I thought about Pixlet when Apple presented ProRes 422 and I kept wondering why didn't Pixlet take off. When it was presented, Pixlet was supposed to be so promising... For some reason, it probably wasn't, but I don't have the expertise to understand why.

Would you care to expand on what made Pixlet so horrible ?

Personally I thought it was fine when used at 100%. It was never intended as an editing format and was never promoted as such. It was a wavelet based compression developed in house by Pixar (hence the name) and I think it was thrown into QT simply because they could. I don't know anybody that took any notice of it and I don't think it is related to ProRes422 in any way.

[11thIndian]

So, I take it this opens up some options for those working on external FW800 drives? Right now, I can edit up to XDCAMHD 35, and even uncompressed 10bit SD (though I'm sure I'd loose frames on a long tape output; but I don't master here at my office.). Looking at that chart, it looks like you could edit standard Prores 422 at up to 1280x720 29.97, but they're talking about not needing RAIDS anymore. Am I reading this right?

[vinney57]

Originally Posted by 11thIndian

So, I take it this opens up some options for those working on external FW800 drives? Right now, I can edit up to XDCAMHD 35, and even uncompressed 10bit SD (though I'm sure I'd loose frames on a long tape output; but I don't master here at my office.). Looking at that chart, it looks like you could edit standard Prores 422 at up to 1280x720 29.97, but they're talking about not needing RAIDS anymore. Am I reading this right?

Over FW800 a single stream of PreRes422 at up to 1080 29.97 is no problem. You could probably get two streams. Certainly a viable capture and playback scenario. For editing several streams you would need to move to a simple raid of internal or external SATA drives. There are ExpressCard SATA interfaces that work extremely well with ProBooks.

[hmurchison]

Originally Posted by palegolas

Reminds me of the Apple Lossless audio codec.. that's a strange one. Is it lossless or is it not?

To truly be called "lossless" a codec needs to be able to compresses and then decompress back to the orginal state without losing bits. Apple Lossless Audio can compress a file and then decompress back to a bit-for-bit duplicate of the orginal file.

Pixlet was meant to view "Daylies" without showing compression artifacts in the frame. Think about it you need a codec that doesn't give you a false impression of what you've recorded and Pixlet is that codec. It's not meant for final distribution. Apple kind of misrepresented what Pixlet really is and does. ProRes 422 is an editing codec which means it's full of I frames that make frame accurate editing a joy. It doesn't mean that it's going to look any better than AVC/h.264 is once distributed as packaged media.

[11thIndian]

Originally Posted by vinney57

Over FW800 a single stream of PreRes422 at up to 1080 29.97 is no problem. You could probably get two streams. Certainly a viable capture and playback scenario. For editing several streams you would need to move to a simple raid of internal or external SATA drives. There are ExpressCard SATA interfaces that work extremely well with ProBooks.

Hold on. You mean the HQ setting? that's 220Mbps. I thought FW800 maxed out at around 90Mbps...

[melgross]

Originally Posted by 11thIndian

Hold on. You mean the HQ setting? that's 220Mbps. I thought FW800 maxed out at around 90Mbps...

220 Mbps vs 90 MBps. That's an eightfold difference. 90MBps equals 720 Mbps. To put it another way, 220 Mbps equals 27.5 MBps.

You're mixing up Bytes with bits.

[Johnny Mozzarella]

Originally Posted by MySchizoBuddy

Is a Supersonic Jet truly Supersonic.

Yes, unless it is really Superdupersonic.

[11thIndian]

Originally Posted by melgross

220 Mbps vs 90 MBps. That's an eightfold difference. 90MBps equals 720 Mbps. To put it another way, 220 Mbps equals 27.5 MBps.

You're mixing up Bytes with bits.

Son of a bitch. I'd mixed those up before. I'm very happy to be mistaken on this. So, basically the entire gammit of the Prores422 is open to Firewire800 users. That's pretty exciting, and I'd imagine at this kind of compression, one might even be able to work with 2K media, 4K would probably be pusing it.

[noirdesir]

Originally Posted by MySchizoBuddy

Does it say anywhere thats its a lossless codec?
Why are people assuming thats its lossless

plus lossless is lossless. there isn't anything like "truly" lossless.
Is a Supersonic Jet truly Supersonic.

Zip is lossless.

[vinney57]

Originally Posted by 11thIndian

Son of a bitch. I'd mixed those up before. I'm very happy to be mistaken on this. So, basically the entire gammit of the Prores422 is open to Firewire800 users. That's pretty exciting, and I'd imagine at this kind of compression, one might even be able to work with 2K media, 4K would probably be pusing it.

2K would be fine in theory, though Apple aren't saying that yet. 4K at 24fps might be doable but I think NAB 2008 will '4K workflow NAB'

[hmurchison]

Originally Posted by vinney57

2K would be fine in theory, though Apple aren't saying that yet. 4K at 24fps might be doable but I think NAB 2008 will '4K workflow NAB'

Yeah that sounds about right. I think next year we'll have 4k support and quite honestly I don't expect "good" support (meaning ..non whimpering computers when working on 4k) until some time in 2009 with the successor to Leopard. My hunch is that working on 4k is going to take something along the lines of a 16-core Nehalem system with a software engine to match.

Apple has really busted ass on QTKit which is now the way to deliver Quicktime functionality in 64-bit but you can only get so much done. The QTKit in 10.6 will have yet another couple of years of developmental maturation.

I'd love to see a roadmap of what Apple plans for Quicktime. Pretty soon blowing 4k video through the computer will be relatively easy but here are some big hurdles to jump before we get there. I guess PCI Express 2.0 an 600MBps SATA 3 should help along with a fast filesystem (ZFS perhaps).

[Hiro]

Originally Posted by noirdesir

Zip is lossless.

But is it "truely" lossless?

<ducks & runs away >

[nicky g]

ProRes was my favorite news of NAB, well, that, Color, Motion 3, and Final Cut Server. OK, so it was a good NAB.

But really, ProRes HQ in theory turns all of the SANs I've sold to my clients, that were engineered for a single stream of uncompressed 10-bit 4:2:2 HD, into 5-6 stream SANs. Instantly, with no re-engineering. And of course now they'll be able to hold 5-6 times as much footage. This is HUGE, and will make my clients very happy.

It also brings the price of doing very high-quality HD post waaaay down, to the point where I'd say this development will actually help more facilities offer HD post services, thus accelerating the adoption of HD in the broadcast industry.

Gosh, this kicks the crap out of posting "online" in DVCPROHD so much, it hurts. And don't even think about posting in HDV ever again. Or frankly any other format, for 95% of all users (assuming ProRes lives up tot he hype, which I assume it will.) FULL RASTER PEOPLE! 10-bit at ~220mbps?! 4:2:2 chroma sampling?! My gosh it's a good day for Final Cut users (and VARs, hah!)

[hmurchison]

Originally Posted by nicky g

Gosh, this kicks the crap out of posting "online" in DVCPROHD so much, it hurts. And don't even think about posting in HDV ever again. Or frankly any other format, for 95% of all users (assuming ProRes lives up tot he hype, which I assume it will.) FULL RASTER PEOPLE! 10-bit at ~220mbps?! 4:2:2 chroma sampling?! My gosh it's a good day for Final Cut users (and VARs, hah!)

Nicky..nice read.

Could you explain to me the benefits of full raster video. I can't get my head around the features/benefits of this. I'm glad to see ProRes supports square pixels and 422 color sampling but how does full raster improve my quality? Thanks in advance.