Inside High Sierra: HEVC promises to keep quality high, file sizes low for videos, photos

AppleInsider · June 12, 2017 7:30PM

Support for HEVC, also known as H.265, was announced at the WWDC Keynote by Apple Senior Vice President Craig Federighi. AppleInsider examines the technology from a user's standpoint, said to be needed to keep 4K videos down to a manageable size.

Editor's note: this examination was published shortly after the reveal of HEVC at the WWDC. It has been updated to reflect the shipment of High Sierra, iOS 11, and tvOS 11.

MPEG-4 begat H.264, which led to HEVC

Apple adopted H.264, also known as AVC in with the launch of QuickTime 7 in April 2004, with Mac OS X 10.4. Apple moved the implementation to Windows in 2005. It boosted the then-standard MPEG-4 from two years prior in much the same way that HEVC improves H.264.

The newer HEVC was born in 2012, and is an evolution of H.264. The newer codec includes better motion compensation for fast-moving scenes, larger computational errors for difference-coding, and other improvements designed to keep file sizes small, at the expense of processor power needed to encode and decode.
The inclusion of the HEVC codec is transformational, in the same way that H.264 was a decade ago.
Cutting back on the processor demand of coding and decoding HEVC video, Apple is implementing hardware acceleration of HEVC in the late 2015 27-inch iMac, the early 2016 MacBook, and the 2016 MacBook Pro and newer.

Utilities to take advantage of the hardware acceleration have started to arrive.

Motion, Final Cut, and Compressor will see hardware encoding support for HEVC at some point in the future -- but a timetable for release is still not yet known. Apple's iTunes supports HEVC in High Sierra now, to some extent.

Intel's Kaby Lake processor used in the most recent range of Macs, and the AMD Radeon Pro chipset included in the 2016 MacBook Pro have HEVC decoding support. Some Nvidia GPUs such as the GTX 9 and 10 series cards do as well. Other machines will have to rely on software decoding.

Processing demands can be tested now

Inspired by a YouTube test, AppleInsider used the 42-second Universal studios pre-movie "bumper" in 1080p transferred from Blu-Ray from the opening of music and the black screen through the animation, and back to a black screen as a test bed. We then transcoded the video using H.264 and H.265 encoders in Handbrake 1.0.7, both set for 1000 kbit/sec on an 8-core, keeping file sizes and all other parameters the same.

The H.264 video clearly had more compression artifacts than the H.265 video. Plus, the movement was smoother in the HEVC video than the older codec.

Compression time was more than double on the 2012 8-core Mac Pro used for conversion. The transcoding progressed at 41 frames per second with HEVC on the average, and 94 frames per second with H.264.

Competing codecs

HEVC is not open source. As a result, there are a few codecs designed for 4K content.

Google's VP9 is used in YouTube and currently not supported in Safari or the Apple TV 4K. The Alliance for Open Media's AV1 is expected to see the light of day before the end of 2017.

It is not clear who Apple has chosen to license HEVC from -- but it is likely around $25 million annually, given MPEG LA's terms for using the technology.

Apple has not declared any intention to natively support VP9, nor AV1. The lack of support is cited as a reason why 4K content will not play through a video embedded in Safari.

More than just High Sierra

To keep Apple's costs down, it needs wide support of whatever codec it can manage that will keep bandwidth demands to a minimum.

HEVC is that codec. Bringing it to iOS and macOS is only the start.

The inclusion of the HEVC codec is transformational, in the same way that H.264 was a decade ago. It will allow for the quality that Apple users demand, while meeting the needs of Apple itself, at the same time.

Soli · June 12, 2017 8:11PM

I’d like to see AI conduct that same encode test on one of their 2017 MBPs since those machines have built-in HW acceleration.

Mike Wuerthele · June 12, 2017 8:16PM

Soli said:

I’d like to see AI conduct that same encode test on one of their 2017 MBPs since those machines have built-in HW acceleration.

Hardware decode only, it appears.

I personally don't have a 2017 on hand, but we'll see if any of the encoding utilities have been updated for High Sierra yet.

MacPro · June 13, 2017 1:56AM

I was able to encode several HD and 4K videos on a 6 core Mac Pro using the new Quick Time app that was installed with High Sierra developer beta, the interface is slick. Once done I was able to play it back smoothly with VLC to my surprise.

Sadly, the encoding time was beyond belief (as in go out for lunch for a few seconds of video). I realize I have no dedicated hardware for encoding but I had hoped given the GPU horse power in a new Mac Pro it might have been usable, as is it's not. In FCPro X only one GPU is utilized for compression/decompressionI know, but I'd love to think Apple could allow Compressor at least to work using both in a Catalyst type scenario at least on such an ancient and underpowered 6 core Mac Pro, Xeon CPU with Dual GPUs, RAID 0 and all! OK that was sarcasm brought on my disappointment, and this is only beta 1. Fingers crossed.

Mike Wuerthele · June 13, 2017 2:07AM

MacPro said:

I was able to encode several HD and 4K videos on a 6 core Mac Pro using the new Quick Time app that was installed with High Sierra developer beta, the interface is slick. Once done I was able to play it back smoothly with VLC to my surprise.

Sadly, the encoding time was beyond belief (as in go out for lunch for a few seconds of video). I realize I have no dedicated hardware for encoding but I had hoped given the GPU horse power in a new Mac Pro it might have been usable, as is it's not. In FCPro X only one GPU is utilized for compression/decompressionI know, but I'd love to think Apple could allow Compressor at least to work using both in a Catalyst type scenario at least on such an ancient and underpowered 6 core Mac Pro, Xeon CPU with Dual GPUs, RAID 0 and all! OK that was sarcasm brought on my disappointment, and this is only beta 1. Fingers crossed.

I suspect that there will be speed enhancements in the coming days.

Soli · June 13, 2017 2:18AM

Mike Wuerthele said:

Soli said:

I’d like to see AI conduct that same encode test on one of their 2017 MBPs since those machines have built-in HW acceleration.

Hardware decode only, it appears.

I personally don't have a 2017 on hand, but we'll see if any of the encoding utilities have been updated for High Sierra yet.

I thought that the Intel chips included a HW encoder and decoder. Are you saying they don't or that High Sierra is only supporting HW decoding? If the latter, is this due to cost?

This is at least the right step. I've said it's the first domino that had to fall before we get a 4K Apple TV and 4K iTS content because I don't see Apple releasing those without offering HEVC codec support. Where it gets more iffy is if they'll offer H.264 for 4K, too, as well as HEVC for 1080p, or even 720p content. My guess is that they'll likely not change anything else, but start offering HEVC specifically for 4K content from the iTunes Store video, all around the same time as High Sierra, iTunes 13, iOS 11, and a 4K Apple TV later this year.

MacPro · June 13, 2017 2:40AM

Mike Wuerthele said:

MacPro said:

I was able to encode several HD and 4K videos on a 6 core Mac Pro using the new Quick Time app that was installed with High Sierra developer beta, the interface is slick. Once done I was able to play it back smoothly with VLC to my surprise.

Sadly, the encoding time was beyond belief (as in go out for lunch for a few seconds of video). I realize I have no dedicated hardware for encoding but I had hoped given the GPU horse power in a new Mac Pro it might have been usable, as is it's not. In FCPro X only one GPU is utilized for compression/decompressionI know, but I'd love to think Apple could allow Compressor at least to work using both in a Catalyst type scenario at least on such an ancient and underpowered 6 core Mac Pro, Xeon CPU with Dual GPUs, RAID 0 and all! OK that was sarcasm brought on my disappointment, and this is only beta 1. Fingers crossed.

I suspect that there will be speed enhancements in the coming days.

Me too, I am fighting the inner cynic that says selling new hardware is the prime objective ...

OK back to flying my helicopter GTA V in Windows 10 on my Mac Pro, at least there I have Catalyst!

entropys · June 13, 2017 4:03AM

This reminded me to go over to slashdot and see how the open source Stasi at slashdot reacted to Apple's HEVC announcement. Predictable, but entertaining none the less. You will be pleased to know that AV1 will be the best when its bitstream is finally sorted, and Apple has made a huge mistake. Just you wait!
i remember the same slashdot arguments about codecs when the iPod first came out, then when Apple adopted H.264.

suddenly newton · June 13, 2017 2:13PM

Soli said:

Mike Wuerthele said:

Soli said:

I’d like to see AI conduct that same encode test on one of their 2017 MBPs since those machines have built-in HW acceleration.

Hardware decode only, it appears.

I personally don't have a 2017 on hand, but we'll see if any of the encoding utilities have been updated for High Sierra yet.

I thought that the Intel chips included a HW encoder and decoder. Are you saying they don't or that High Sierra is only supporting HW decoding? If the latter, is this due to cost?

First, with regards to Intel's CPUs, if you mean hardware support for encoding HEVC _in realtime_, no. Decoding, yes. I understand too that the decoding support in hardware is a matter of efficiency. A software-only decoder is posible, but, as it was a generation ago with H.264, extracted a penalty in battery life.

Soli · June 13, 2017 2:22PM

suddenly newton said:

Soli said:

Mike Wuerthele said:

Soli said:

I’d like to see AI conduct that same encode test on one of their 2017 MBPs since those machines have built-in HW acceleration.

Hardware decode only, it appears.

I personally don't have a 2017 on hand, but we'll see if any of the encoding utilities have been updated for High Sierra yet.

I thought that the Intel chips included a HW encoder and decoder. Are you saying they don't or that High Sierra is only supporting HW decoding? If the latter, is this due to cost?

First, with regards to Intel's CPUs, if you mean hardware support for encoding HEVC _in realtime_, no. Decoding, yes. I understand too that the decoding support in hardware is a matter of efficiency. A software-only decoder is posible, but, as it was a generation ago with H.264, extracted a penalty in battery life.

Intel states that they've included a HW accelerated HEVC encoder since 6th gen Core. The realtime SW decoder was a 4th gen Core feature.

https://software.intel.com/en-us/blogs/2015/12/11/codecs-are-they-slowing-you-down

brucemc · June 13, 2017 7:22PM

Soli said:

Mike Wuerthele said:

Soli said:

I’d like to see AI conduct that same encode test on one of their 2017 MBPs since those machines have built-in HW acceleration.

Hardware decode only, it appears.

I personally don't have a 2017 on hand, but we'll see if any of the encoding utilities have been updated for High Sierra yet.

I thought that the Intel chips included a HW encoder and decoder. Are you saying they don't or that High Sierra is only supporting HW decoding? If the latter, is this due to cost?

This is at least the right step. I've said it's the first domino that had to fall before we get a 4K Apple TV and 4K iTS content because I don't see Apple releasing those without offering HEVC codec support. Where it gets more iffy is if they'll offer H.264 for 4K, too, as well as HEVC for 1080p, or even 720p content. My guess is that they'll likely not change anything else, but start offering HEVC specifically for 4K content from the iTunes Store video, all around the same time as High Sierra, iTunes 13, iOS 11, and a 4K Apple TV later this year.

IMO Apple will only offer 4K content using HEVC. I expect they will offer 1080p content encoded in HEVC as well (though perhaps this comes later due to priority being on 4K content). There is significant file size and b/w savings to be had in moving to HEVC for 1080p content, which would improve the existing iTS content services to hundreds of millions of users (impactful to far more users than 4K, since many still would not have a 4K TV or the broadband speeds to utilize it).

gregalexander · June 15, 2017 6:34AM

I thought they said that new phones would use the new codec, and it'd work with current Macs Photos libraries. I assume they'll handle the new HEVC and HEIC formats. I'm hoping that iMovie will have a more efficient codec for storage but perhaps only for the latest Macs.

I'm also wondering if my "optimised Photos Library" on my laptop will start using HEVC and HEIC optimised versions.... but I assume that question is way to early to have an answer for?

Soli · June 15, 2017 7:09AM

Will iTunes 13 offer a conversation tool for your H.264 content on High Sierra?

Mike Wuerthele · June 15, 2017 10:46AM

Soli said:

Will iTunes 13 offer a conversation tool for your H.264 content on High Sierra?

There's no tool yet, but we're still early. I doubt it, though.

ksec · June 15, 2017 11:32AM

entropys said:

This reminded me to go over to slashdot and see how the open source Stasi at slashdot reacted to Apple's HEVC announcement. Predictable, but entertaining none the less. You will be pleased to know that AV1 will be the best when its bitstream is finally sorted, and Apple has made a huge mistake. Just you wait!
i remember the same slashdot arguments about codecs when the iPod first came out, then when Apple adopted H.264.

AV1 isn't better then HEVC for crying out loud. They are on the same level, and one may be better then another depending on scenario, as with all encoding codec. The AV1 Encoding and Format is like a living thing and changes constantly, HEVC format and standard was done quite a while the encoder still need more time to tune and test before it reaches x264 level of maturity.

What's different though is that there are hundreds of millions of HEVC Hardware decoding SoC shipped already.

>>>>It is not clear who Apple has chosen to license HEVC from -- but it is likely around $25 million annually, given MPEG LA's terms for using the technology.

You dont get to choose who to license from, You have to license them all. MPEG LA, HEVC Advance Group, and the newly announced Group from Qualcomm, Sony, Panasonic.... etc The first two combined to $75M / year. The third group has not announced their pricing, and there is also technicolor, they pulled out of MPEG-LA last year.

Compared to the $5M / year for H.264. HEVC is insanely expensive.

xTradaTrade · June 21, 2017 8:43AM

Oh heck yes, this is transformational and needed for 4k content. "Apple may be going it alone, again" Not sure what that comment was referring to. Apple has been way late to the h265 party, dragging it's feet. They should've supported this long ago, at least with the content creation apps like FCPx. Windows 10 has been supporting h265 as well as Adobe Premiere which allows you to directly import h265 and work on them. Even budget video editors like Power Director allow this. Samsung even released a semi-pro grade camera, NX1, that directly recorded (encoded) 4k to h265 in 2014! Regardless, it's a welcome edition and let's hope Apple pushes this through all it's software, hardware, and online platforms. Hopefully the iPhone 8 will have an option to record 4k to h265, saving us all some much needed storage space.

mtbnut · September 26, 2017 6:40PM

AppleInsider said:

It is not clear who Apple has chosen to license HEVC from -- but it is likely around $25 million annually, given MPEG LA's terms for using the technology.

How does this actually work? Why can't they buy the rights and open source it, to promote availability and compatibility?

Avieshek · September 26, 2017 7:08PM

Tell us more about AV1.

gatorguy · September 26, 2017 7:21PM

ksec said:

entropys said:

This reminded me to go over to slashdot and see how the open source Stasi at slashdot reacted to Apple's HEVC announcement. Predictable, but entertaining none the less. You will be pleased to know that AV1 will be the best when its bitstream is finally sorted, and Apple has made a huge mistake. Just you wait!
i remember the same slashdot arguments about codecs when the iPod first came out, then when Apple adopted H.264.

AV1 isn't better then HEVC for crying out loud. They are on the same level, and one may be better then another depending on scenario, as with all encoding codec. The AV1 Encoding and Format is like a living thing and changes constantly, HEVC format and standard was done quite a while the encoder still need more time to tune and test before it reaches x264 level of maturity.

What's different though is that there are hundreds of millions of HEVC Hardware decoding SoC shipped already.

>>>>It is not clear who Apple has chosen to license HEVC from -- but it is likely around $25 million annually, given MPEG LA's terms for using the technology.

You dont get to choose who to license from, You have to license them all. MPEG LA, HEVC Advance Group, and the newly announced Group from Qualcomm, Sony, Panasonic.... etc The first two combined to $75M / year. The third group has not announced their pricing, and there is also technicolor, they pulled out of MPEG-LA last year.

Compared to the $5M / year for H.264. HEVC is insanely expensive.

Apple is part of the MPEG-LA/ HVEC Advance standards group with IP contributed, which typically means they are receiving licensing revenue from others who take a license to h.265. It may not be costing Apple anything, though Velos Media might be receiving a bit since they are another one of the 4 standards groups all claiming a piece of the action.

polish · September 26, 2017 8:07PM

I'm hoping for a FCPX update imminently... Obviously, High Sierra is a pre-requisite, but equally, HEVC compatibility in FCPX is quite an important prerequisite for its usefulness in Lumix GH5 2.0 firmware... which is due "To be release in the end of September 2017".

Just a hunch.

yoyo2222 · September 26, 2017 9:06PM

gregalexander said:

I thought they said that new phones would use the new codec, and it'd work with current Macs Photos libraries. I assume they'll handle the new HEVC and HEIC formats. I'm hoping that iMovie will have a more efficient codec for storage but perhaps only for the latest Macs.

I'm also wondering if my "optimised Photos Library" on my laptop will start using HEVC and HEIC optimised versions.... but I assume that question is way to early to have an answer for?

After upgrading my iMac to High Sierra my new iPhone 8 told me it was going to start using HEIC (and I assume HEVC too). I don't know if that means it was going to deliver the previously shot full resolution photos from iCloud Photo Library to my phone or just that the phone is going to start shooting/saving in that format.

Inside High Sierra: HEVC promises to keep quality high, file sizes low for videos, photos

MPEG-4 begat H.264, which led to HEVC

Processing demands can be tested now

Competing codecs

More than just High Sierra

Comments