Apple looks to improve visual quality of streaming video

Posted:
in iPod + iTunes + AppleTV edited January 2014
Apple has investigated a method to make imperfections in compressed video files less visible to the human eye, a new patent application revealed this week shows.



The described technology would not do away with compression, which is necessary given bandwidth restrictions. Instead, it details a new technology that would mask artifacting that often appears when video files are compressed to reduce their file size.



"Disadvantageously, most video compression standards use lossy data compression techniques in which data determined by a particular compression algorithm to be of lesser importance to the overall content, but which is nonetheless discernible and objectionable to the user, is discarded," the application reads. As a result, certain video compression algorithms may introduce visual artifacts into the decoded video stream, which may be distracting to a user when viewing the decoded video data."



It continues: "Such visual artifacts are generally attributable to the latent error in lossy data compression and may appear more frequently as higher video compression rates are used. Moreover, such artifacts are exacerbated when the decoded video images are scaled to larger high definition displays. "



Earlier this year, Apple was rumored to be working on a new service called iTunes Replay. It was said to allow users to stream iTunes video purchases directly from the company's servers for playback anywhere. Such a system would prevent the need for local storage to save large video files.



Apple has also taken interest in streaming video on the iPhone, with the latest 3.0 software allowing the HTTP Live Streaming standard. Apple has provided the QuickTime Streaming Server for the last decade, but that format has run into issues as it is often blocked by firewalls. But the new HTTP streaming technology allows simple real-time streaming of content and can offer multiple versions of clips in different formats which could automatically switch based on bandwidth availability.







The patent application describes a system that would introduce "random noise" into a video stream to reduce or mask the visibility of the effects of compression on video. The technology would analyze the video and determine appropriate places based on the picture's characteristics to insert the noise and mask the artifacting, otherwise known as "fuzziness" in an overly compressed image. Such a system could be used on a computer, an Apple TV, or an iPhone.



"(A device), upon receiving streaming compressed video data from a network connection via network (device), may decode the compressed video data and temporarily buffer or cache the decoded video data in the (memory)," the patent application states. "In one embodiment, the noise generation (system) may sequentially process each frame of the video data buffered in the (memory) on a pixel-by-pixel basis. Each buffered pixel is read from the (memory) via an input data (bus) coupled to the noise generation."
«13

Comments

  • Reply 1 of 49
    So Apple wants to mask artifacts by introducing random noise that makes it less obvious that an artifact occurred? So I won't notice it because thoe whole screen will be snowy?

    Uhhh.... ok.
  • Reply 2 of 49
    Or maybe they could start using higher levels of h.264 for more efficient compression and/or better encoders.
  • Reply 3 of 49
    teckstudteckstud Posts: 6,476member
    Simple- make Macs handle FLASH properly. Streaming Flash looks great.
  • Reply 4 of 49
    The thread title is a bit misleading - this technology could be applied to all video, not just streaming video.
  • Reply 5 of 49
    cubertcubert Posts: 728member
    I'm assuming this is somewhat analogous to one sound wave cancelling out another?
  • Reply 6 of 49
    cubertcubert Posts: 728member
    Quote:
    Originally Posted by teckstud View Post


    Simple- make Macs handle FLASH properly. Streaming Flash looks great.



    That should be "Make Adobe create a better Flash plug-in for Macs."



    The problem is entirely on their end with shoddy coding and indifference to the platform.
  • Reply 7 of 49
    MarvinMarvin Posts: 15,301moderator
    Quote:
    Originally Posted by huntercr View Post


    So Apple wants to mask artifacts by introducing random noise that makes it less obvious that an artifact occurred? So I won't notice it because thoe whole screen will be snowy?

    Uhhh.... ok.



    I'd expect it to be more like when you get blocky artifacts, it breaks those blocks down into noisier and less uniform shapes so they appear more natural and less digital. Like how an old VHS tape with image degradation still looks natural whereas a realplayer stream doesn't.
  • Reply 8 of 49
    if you have the bandwidth to send randomized noise that attempts to "motion-blur the eff out of compression artifacts," why not just encode the original file with less compression, thus more effectively utilizing the bandwidth?
  • Reply 9 of 49
    Quote:
    Originally Posted by oakie View Post


    if you have the bandwidth to send randomized noise that attempts to "motion-blur the eff out of compression artifacts," why not just encode the original file with less compression, thus more effectively utilizing the bandwidth?



    I think this is more a client technology in the player, rather than sending the noise in the stream.
  • Reply 10 of 49
    zoolookzoolook Posts: 657member
    Quote:
    Originally Posted by teckstud View Post


    Simple- make Macs handle FLASH properly. Streaming Flash looks great.



    Flash is shit. I'd rather take Silverlight or HTML 5.



    I don't know how Apple encodes its TV shows, but some look great and some look absolutely horrible. BSG Season 2 in SD is appalling (I managed to encode Season 1 from DVD at the same bit-rate with far better quality using free software). But Seasons 3 and 4 were better (although not perfect, even in HD there was a lot of pixelating and aliasing).



    However Fringe looks great (even in SD), and Mad Men in HD varies between looking amazing and then looking like an AVI from the 1990s. I had Dexter Season 3, great until the penultimate episode which was encoded in the wrong aspect ratio and had lots of artifacts. To their credit, iTunes store has always given a credit for anything I have complained about.



    iTunes videos definitely suffer from artifacts when there are lots of dark areas or when scenes are fairly still. The reason is the fixed bit-rate (1500kbps isn't a lot, even for SD) - a constant quality encoder (available on any x264 encoder) allows for bit-rates to rise when scenes require more quality, but keep the overall file sizes lower. These work fine on ATV and the iPhone, but not on older iPods that are limited to 1500kbps or 2500kbps decoding.
  • Reply 11 of 49
    I just hope that one day, in the far future, maybe there will be a 1,024-core mac that can play a youtube video over a broadband link without stuttering.
  • Reply 12 of 49
    Quote:
    Originally Posted by Marvin View Post


    I'd expect it to be more like when you get blocky artifacts, it breaks those blocks down into noisier and less uniform shapes so they appear more natural and less digital. Like how an old VHS tape with image degradation still looks natural whereas a realplayer stream doesn't.



    What the hell do you mean by a VHS tape image degradation looking "natural"?!? Whenever I look at reality, things do not look "degradated", mkay? Any kind of "degradation" due to it being "recorded" or "streamed" or whatever have you is not natural, it's by definition, artificial. mkay?
  • Reply 13 of 49
    Sounds like a rubbish patent

    - certainly from the information presented here, there's no actual algorithm, so it's impossible to say what it does, or how it does it

    - saying 'add some noise to make the artefact go away' isn't enough for a patent...

  • Reply 14 of 49
    boogabooga Posts: 1,082member
    Quote:
    Originally Posted by samurai1999 View Post


    Sounds like a rubbish patent

    - certainly from the information presented here, there's no actual algorithm, so it's impossible to say what it does, or how it does it

    - saying 'add some noise to make the artefact go away' isn't enough for a patent...





    AppleInsider, if they want to discuss patents responsibly, really needs to include the patent # or application # and preferrably a link.



    The one in question appears to be patent application 20090257507. The patent application is actually fairly specific. As patents generally go, some of the later claims get more and more general to try to cover as broad an area as possible, but the major claims seem to be fairly interesting if the effect they're describing is real.



    Would you rather watch a super-blocky degraded digital signal or a somewhat snowy degraded analog signal? This makes some attempt to find a happy medium.
  • Reply 15 of 49
    Quote:
    Originally Posted by samurai1999 View Post


    Sounds like a rubbish patent

    - certainly from the information presented here, there's no actual algorithm, so it's impossible to say what it does, or how it does it

    - saying 'add some noise to make the artefact go away' isn't enough for a patent...





    Yeah, because hey, what's better than patenting an algorithm?



    Don't know?



    Why hell, to announce to the world the friggin' algorithm!
  • Reply 16 of 49
    Quote:
    Originally Posted by oakie View Post


    if you have the bandwidth to send randomized noise that attempts to "motion-blur the eff out of compression artifacts," why not just encode the original file with less compression, thus more effectively utilizing the bandwidth?



    Because some people have good bandwidth and some don't, and current systems will on the fly reduce the quality when the bandwidth gets constrained.
  • Reply 17 of 49
    newbeenewbee Posts: 2,055member
    Quote:
    Originally Posted by huntercr View Post


    So Apple wants to mask artifacts by introducing random noise that makes it less obvious that an artifact occurred? So I won't notice it because thoe whole screen will be snowy?

    Uhhh.... ok.



    Introducing "white noise" into headphones for sound cancelling ability has been done for ages. This just sounds like a similar arrangement. Should be interesting to see the results.
  • Reply 18 of 49
    newbeenewbee Posts: 2,055member
    Quote:
    Originally Posted by teckstud View Post


    Simple- make Macs handle FLASH properly. Streaming Flash looks great.



    Don't you ever get tired of shoveling that same load of crap?
  • Reply 19 of 49
    newbeenewbee Posts: 2,055member
    Quote:
    Originally Posted by LuisDias View Post


    What the hell do you mean by a VHS tape image degradation looking "natural"?!? Whenever I look at reality, things do not look "degradated", mkay? Any kind of "degradation" due to it being "recorded" or "streamed" or whatever have you is not natural, it's by definition, artificial. mkay?



    It doesn't matter what it is ... only how it looks .... chill out.
  • Reply 20 of 49
    teckstudteckstud Posts: 6,476member
    Quote:
    Originally Posted by newbee View Post


    Don't you ever get tired of shoveling that same load of crap?



    Open your mind NEWBEE and embrace different forms of streaming- ones that already exists- ones that work excellently on 90% of computers out there and excellently on the other 10%, provided you don't use any other application simultaneously.
Sign In or Register to comment.