QuickTime 6.6 to support high-resolution audio, B-Frame video

AppleInsider

The next version of Apple Computer's QuickTime media software will include major new features for users, developers, and content creators, while introducing improvements to the QuickTime architecture, file format, and API, AppleInsider has learned.



More specifically, the new version of QuickTime will boast significant improvements to Audio, Video, and Metadata, as well as add new Cocoa APIs, and numerous other enhancements.



Apple first seeded the software to developers last month as "QuickTime 6.6 Developer Preview." The release was bundled as part of the first external build of Mac OS X 10.4 "Tiger," which was available to select Apple partners and developers who pay annual fees to receive developmental software builds from the company.



Audio Enhancements



QuickTime 6.6 will break free of the limitations of the Sound Manager, adding many new features that support high-quality audio, including a high-resolution audio, a new sound description, a new audio media handler, and new audio sequence grabber components and compressor components.\t



For developers, the software will offer new APIs for movie audio, sound conversion, audio extraction, movie export, and level and frequency metering. Support has also been added for use of Core Audio input/output units in QuickTime.



The changes will result in a much richer approach to sound in QuickTime, with support for higher sampling rates, such as 96 kHz and 192 kHz, multiple channels and multiple channel layouts, including 5.1 surround sound and 8 or more discrete channels. Additionally, the release will introduce support for a variety of more accurate audio representations, such as 24-bit uncompressed audio, during capture, playback, and export.



QuickTime 6.6 audio will also feature improve synchronization and access to uncompressed audio on a per-sample basis, including access to raw PCM audio samples from VBR-compressed audio sources.



Along with support for high-resolution audio, the new version of QuickTime will deliver a more flexible architecture for connecting audio components, letting developers capture multiple sequence grabber channels from a single device at the same time, even if the device doesn?t permit multiple clients directly.



Additionally, sources said the new QuickTime software will convert audio from one format to another on the fly, performing channel mix-down or remapping, upsampling or downsampling, and sample conversion as needed.



Video Enhancements



QuickTime 6.6 will also feature several video enhancements, such as support for B-Frame and H.264 video compression. The release will also introduce a new visual context that provides an abstraction layer decoupling QuickTime from GWorlds.



B-Frame video compression is a major advancement that involves new sample tables for video, allowing video frames to have independent decode and display times. This will result in improved display, editing, and capture in H.264 and other advanced video codec formats. The implementation will reportedly include a new API for working with media sample times, adding and finding samples, and a new Image Compression Manager API.



The H.264 codec is the latest standards-based video codec. Also known as the Advanced Video Codec (AVC) and as MPEG-4 Part 10, the H.264 codec promises to deliver better live streaming characteristics and better image quality at lower bit rates than the current MPEG-4 video codec.



Meanwhile, a new QuickTime visual context will provide an abstraction layer, decoupling QuickTime movies from GWorlds. The context allows programmers to work in QuickTime without needing to understand QuickDraw concepts and structures, and making it easier to render QuickTime output using engines such as OpenGL.



Likewise, QuickTime 6.6 with also mark the debut of an OpenGL texture context, a specific implementation of the aforementioned visual context that will provide a movie's visual data to a client applications as OpenGL textures. These textures can then be rendered to the screen using OpenGL, composited with other graphics, or run through CoreImage filters.



QuickTime Metadata Enhancements and API's



Another feature of QuickTime 6.6 is a new extensible metadata storage format that allows more flexible and efficient storage of metadata, including encapsulated storage of metadata without translating to and from a defined QuickTime format, sources said.



QTKit Framework for QuickTime



Also bundled with the new version of QuickTime is a Cocoa framework that provides developers with support for displaying and editing QuickTime movies. It uses abstractions and data types that are already familiar to Cocoa programmers, such as delegation and notification, and it will introduce new data types for QuickTime-related operations only when necessary.



Apple has designed this framework to encapsulates a tremendous amount of QuickTime functionality in a small, easily-mastered API with a handful of new objects, classes, and methods. The company hopes that the new API will introduce QuickTime programming to new groups of developers without requiring them to learn the large, complex C/C++ QuickTime API.



New API's for QuickTime Developers



Rounding out the list of enhancements contained within QuickTime 6.6 is the addition of two new developer APIs: New Movie from Properties and QT Sample Table.



The New Movie from Properties API will let developers set up properties before creating a movie. This will also allow developers to create movies that are not necessarily associated with a GWorld and movies that can render their output to a visual context.



Meanwhile, the new QT Sample Table API can be used to obtain or set sample attributes such as their size, location, and sample descriptions.



Requirements and Availability



The QuickTime 6.6 Developer Preview requires Mac OS X 10.4 "Tiger" Developer Preview, a PowerPC G3 or better processor running at 400 MHz or higher, and at least 128 MB of RAM.



Although the software is only available for pre-release builds of Mac OS X 10.4 at the moment, sources expect several of the new QuickTime features to be made available for Mac OS X 10.3 "Panther."

[ View this article at AppleInsider.com ]

kim kap sol

A much needed overhaul. This is the best thing since the release of Mac OS X and is of much greater importance than the release of 10.2, 10.3 and 10.4 combined, IMO.



Metadata will allow Spotlight to search for movies...and would allow iTunes-like apps but for movies/clips to be created and used on 10.3 (or lower?). The change to CoreAudio from SoundManager will be a godsend. Being able to use CoreAudio, CoreImage/Video with QuickTime should produce some amazing stuff...and applying them as a texture via OpenGL should bring some very interesting possibilities to apps and most probably games.



This is the rebirth of QuickTime...most likely going to be released as QuickTime 7.0. Or QuickTime X? Or QuickTime Extreme?



scratch that last one.



The question is...has the codebase going to been split up? Is this the end of QuickTime updates for Windows? How is Apple going to bring all these features to the Windows QuickTime?



Part of me says "good, people should switch over to the Mac side if they want to use a real multimedia platform"...but the other part of me says "no more significant QuickTime updates on the PC side means less QuickTime content on the internet."



Apple may be working its ass off to bring all of this on Windows...but again, part of me says "well, if the Windows folks get this update of QuickTime as well, they'll never be compelled to switch over to Macs"...but the other part of me says "KICK SOME MS BUTT, APPLE!!!! YEAH!!!"

rok

Quote:

Originally posted by kim kap sol

Is this the end of QuickTime updates for Windows? How is Apple going to bring all these features to the Windows QuickTime?

i can't comment on any of the other stuff you brought up, but they will not stop quicktime for windows. they can't, without losing massive ground to real and microsoft, and steve just will not let that happen. plus, movie trailers are so often released in quicktime format because they can make the claim that it's accessible cross-platform witha heavy installed base. can you imagine how many studios would stop issuing quicktime trailers if they knew they were going to lose the windows crowd?



then again, who the hell knows what goes on in steve's head...

kim kap sol

Quote:

Originally posted by rok

i can't comment on any of the other stuff you brought up, but they will not stop quicktime for windows. they can't, without losing massive ground to real and microsoft, and steve just will not let that happen. plus, movie trailers are so often released in quicktime format because they can make the claim that it's accessible cross-platform witha heavy installed base. can you imagine how many studios would stop issuing quicktime trailers if they knew they were going to lose the windows crowd?



then again, who the hell knows what goes on in steve's head...

They probably won't stop the updates but I wonder how they'll get the CoreAudio stuff working on Windows and are they going to offer .net frameworks?

kickaha

My guess is that since Apple pretty much 0\\/\\/nz the DCC market in many ways, that QT as a *development* API will be limited on Windows... I mean really, who on Windows uses the more advanced features of the API *anyway*? Anyone? Or do they just use the basic features for exporting a QT .mov file?



The QT API will continue to be pushed as far as possible on the Mac, and the QuickTime Player will continue to be distributed to Windows clients... but note that only means Apple has to use the QT/Win APIs. I suspect that it will simply be that playback will be assured to be cross-platform, but advanced content *creation* APIs will remain Mac only. The basic APIs? Sure. Let them stand as cross-plat. But I can't see them porting CoreAudio in its entirety to Windows just for QT. Seems like a nightmare. Ditto for CoreVideo. (Which won't even be possible until Longhorn, due to graphics limitations of Windows XP...)



Bottom line: end users won't know the difference. Developers for QT apps on Windows may find themselves eyeing the Mac just that much more. Mac developers will chortle.

hmurchison

This is Quicktime 7 for all intents and purposes. I couldn't see them shipping this as QT 6.6 as there are far too many new features. I'm sure most of this stuff is under NDA because not much info on QT 6.6 has been released. I don't see Apple not delivering a version for PC. I agree though that the PC version will not have parity. It'll become more and more like a QT runtime environment.



Breaking the audio limitation is going to be huge for Final Cut Pro. The OpenGL texture thing is interesting.

hmurchison

Thanks to Drunkenbatman we have a description of what the various frames do.



* I frames

These are basically full reference frames; consider these to be snapshots of the movie that the encoder/decoder uses to tell it what's going on. Movies generally need to start with these.



* P frames

These allow the decoder to use frames that have been decoded in the past to help it reconstruct the next frame. Here's the idea; very often not everything in the scene will change from frame to frame, so its a hell of a lot more efficient to just tell the decoder "Change these blocks to these colors, but leave the others just where they are". As an example, lets say you're encoding a movie of yourself talking into the camera.



Assuming you aren't getting your groove on while you're talking, remarkably little about the scene actually changes over a period of a few seconds. So the decoder simply takes the last frame that was constructed and changes what needs to for a nice data savings. Hopefully this is pretty simple; the decoder looks at the reference frame and just keeps making changes to it until it hits another frame, at which point it starts all over.



The farther apart your keyframes, the more the image has to be 'constructed' by the decoder and why if you've ever tried to scrub back in forth in a movie that has keyframes set to something wacky, like 1 keyframe for every 1,400 frames, things grind to a halt. Things are fine when you're just playing the movie, but when you try to, say, jump to the halfway mark you're sitting there waiting while the CPU identifies the frame you want to see, where the last reference frame was, and reconstructs the scene up to that point.

* B frames

These are almost exactly like P frames, with the exception that while P frames are able to look at the last frame and see what needs to change, B frames are able to look at future frames too. This is a great thing in terms of quality and efficiency, and helps keep down those gawd-awful image problems where you're in between keyframes and suddenly the encoder is told everything has to change. But if you reference the P-frame example, and the idea of tradeoffs, you can get an idea of the kind of hurt progressions like these put on the CPU.



Thought this might clear up any confusion over what the various frames do in a nutshell.

rwintheiser

The question remains for me, what effect will this have on iTunes? And further (showing my lack of knowledge perhaps) how will the h.264 spec change/enhance iTunes.

hmurchison

Quote:

Originally posted by rwintheiser

The question remains for me, what effect will this have on iTunes? And further (showing my lack of knowledge perhaps) how will the h.264 spec change/enhance iTunes.

Multichannel Audio could be sold and played back in iTunes.



The Videos could be encoded in the more efficient h.264 for larger vids without the large datarate.

curiousuburb

Looked to me like the lines about "new codecs" and "supports higher bitrates" and "sampling VBR" suggest that the ripping and playback performance will improve for MPEG-4 (including, I thought I saw, better extraction of audio from movies).



Higher quality would by my bet, although more efficient CPU load would be nice too.

(Though I expect most of the offload/performance gain will be due to Core A/V as we approach Tiger)