Android vs iPhone web page loading speed contest flawed

Posted:
in iPhone edited January 2014
Test results promoted by Blaze Software that purport to prove that Android is much faster at loading web pages than Apple's iOS 4.3 did so using a poorly performing custom iPhone app, rather than using Safari itself.



The results of the test, according to Bloomberg, said that an Android-based Nexus S phone performed 52 percent faster on average after loading more than 45,000 pages from 1,000 websites compared to iPhone 4.



The average speed difference was about a second longer page load on iPhone 4: 2.14 seconds compared to 3.25 seconds. The more complex the page, the greater the performance difference, Blaze reported. Guy Podjarny, the firm's chief technology officer, said "it?s not that Apple doesn?t care about speed, but Google is fanatical about it."



However, while Blaze maintained that its benchmarks used the newly released iOS 4.3, suggesting that it took into account the fast new Safari browser with Apple's new Nitro JavaScript engine, the way it performed the tests completely bypassed those improvements.



Rather than using Apple's Safari browser directly, Blaze tested page loading on iPhone 4 using the company's own proprietary app that did not take advantage of the new improvements in iOS 4.3.



As noted in a previous report by AppleInsider, apps that implement Apple's UIWebView to provide web browsing functions within an app (as Blaze did), in addition to full screen web apps, do not take advantage of the new web acceleration features Apple introduced in iOS 4.3, including Nitro and a variety of other improvements to the mobile Safari browser.



While Apple hasn't officially commented on the disparity between the newly revamped Safari and the features of the UIWebView framework, it appears that the difference relates to both to the fact that Apple wanted to rapidly roll out new WebKit features quickly to mainstream iOS users in Safari (and simply didn't have time to retrofit every other element of the system with the new code), and also to security considerations.



Apple's new Nitro JavaScript engine (originally called Squirrel Fish Extreme) competes against Google's Chrome V8 and Mozilla's FireFox TraceMonkey to speed JavaScript (the programming language behind the web) using various different approaches, each of which has different strengths and advantages.



Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.



Running an automated test on page loading using the actual Safari browser on iPhone 4 would be far more difficult to perform, but would also fail to account for other, likely more important differences between iOS and devices running Android.



These include overall stability and usability of the platform, power management and battery life, hardware quality, and easy access to iTunes music and movie rentals, iBooks, and App Store, three features Apple has started promoting in series of new ads that end with the line, "if you don?t have an iPhone, well, you don?t have an iPhone.?



«134

Comments

  • Reply 1 of 78
    Any time some study comes out pro or against an Apple product, it inevitably fans flames from all sides. Go have fun Apple Devotes and Android Fans. I want to see a big old fight all over a 1 second difference in web page loading on a mobile device. I'll eat my popcorn.



    What I found interesting was this:

    Quote:
    Originally Posted by AppleInsider View Post


    Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as a both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.



    Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...
  • Reply 2 of 78
    jpcgjpcg Posts: 114member
    Quote:
    Originally Posted by camroidv27 View Post


    Any time some study comes out pro or against an Apple product, it inevitably fans flames from all sides. Go have fun Apple Devotes and Android Fans. I'll eat my popcorn.



    What I found interesting was this:





    Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...



    Relax, it will be fixed in an update. All apps go through approval so I don't see a problem here.



    These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.

    People relax! Its just an OS.
  • Reply 3 of 78
    tenobelltenobell Posts: 7,014member
    I think this is more about preventing malware from breaching security holes in webkit. I would be willing to give Apple more time to figure it out so that its secure and functional.



    Quote:
    Originally Posted by camroidv27 View Post


    Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...



  • Reply 4 of 78
    I'm hoping the Nitro engine will eventually be able to be used by my favourite browser, Atomic Web.
  • Reply 5 of 78
    Quote:
    Originally Posted by jpcg View Post


    These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.

    People relax! Its just an OS.



    Indeed! I would be the first to point out that all OSs, regardless of the company attached, have positives and negatives against the other OSs. I love aspects of the webOS, and I love aspects of the iOS, and love aspects of the Android OS (I have not had experience with the newest WinMobile OS, so I can't vouch there). There are also aspects I hate on all. Its a to each their own, and respect the decisions and argument points. Studies like this I feel just increase a "Mine is Better/Bigger than Yours" when in the real world, its all about personal preference. Just cause a study says that the thing you have is now "devalued", doesn't mean it really is. If it works great for you, and you like how it works, then the study doesn't mean anything for you. If you happen to have issues that a study shows, then that's something different.
  • Reply 6 of 78
    gwydiongwydion Posts: 1,067member
    I think that those "my d*** is longer than yours" contest are stupid so I don't mind if Android Webview loads a page 1 millisecond faster or slower than Apple's UIwebview but: how Nitro JS can make pages load faster. The only thing I whink it can make loading pages faster is asynchonous loading so it's not so flawed the contest.
  • Reply 7 of 78
    jb510jb510 Posts: 124member
    Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.



    Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...



    ..and no link to the source, bad AI... bad....



    For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/
  • Reply 8 of 78
    gwydiongwydion Posts: 1,067member
    Quote:
    Originally Posted by jb510 View Post


    Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.



    Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...



    ..and no link to the source, bad AI... bad....



    For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/



    The have issued a clarification, but is hard for Dan to check anything

    http://www.blaze.io/business/embeded...ment-167303524
  • Reply 9 of 78
    tenobelltenobell Posts: 7,014member
    The complaint is more against the methodology of the test. That the methodology does not prove what the test claims to prove. Its more about that than the actual results of the test.



    Quote:
    Originally Posted by camroidv27 View Post


    If you happen to have issues that a study shows, then that's something different.



  • Reply 10 of 78
    tenobelltenobell Posts: 7,014member
    The title of the test is about whose browser is faster. They assumed that the embedded browser and native browser were exactly the same. Security concerns are the reason Apple has not opened Nitro to its embedded browser. They would need to sandbox the process so that it becomes a trusted chain of executable code.



    Quote:
    Originally Posted by jb510 View Post


    Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps...



  • Reply 11 of 78
    anonymouseanonymouse Posts: 6,576member
    Quote:
    Originally Posted by Gwydion View Post


    The have issued a clarification, but is hard for Dan to check anything

    http://www.blaze.io/business/embeded...ment-167303524



    Not much of a "clarification" when they say, "We were wrong but we're still right."



    Or this bit:



    Quote:

    ?Despite this fundamental testing flaw they still only found an average of a second difference in loading Web pages?.



    We see this is a bad interpretation of our results. First and foremost, our tests were run over networks and conditions more favorable than the average user browsing on his mobile device. Second, on many sites the gap was greater in absolute terms (for example, on wsj.com we saw a 5-10 second gap). The median gap was only one second, thanks in part to the great network conditions.



    So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.
  • Reply 12 of 78
    anonymouseanonymouse Posts: 6,576member
    Quote:
    Originally Posted by camroidv27 View Post


    ... Am I wrong in seeing a double standard here? ...



    Yes, you are wrong.
  • Reply 13 of 78
    Quote:
    Originally Posted by Gwydion View Post


    The have issued a clarification, but is hard for Dan to check anything

    http://www.blaze.io/business/embeded...ment-167303524



    Yeah, but to believe the clarification (where they still claim they were right BTW), you have to believe that these guys are either total dumbasses, that they purposely skewed the test, or both.



    So it's either a non-story about a bunch of idiots that are posing as technical wizards when they don't know their ass from a screwdriver, or it's a tale of pure deception. Take your pick.
  • Reply 14 of 78
    anonymouseanonymouse Posts: 6,576member
    Quote:
    Originally Posted by jb510 View Post


    ... Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. ...



    It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.
  • Reply 15 of 78
    gwydiongwydion Posts: 1,067member
    Quote:
    Originally Posted by anonymouse View Post


    It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.



    No, they didn't claim that, in their report there is the metodology and how they made the tests.



    And despite the metodology, the tests are stupid, it's irelevant that a page load a second faster or slower
  • Reply 16 of 78
    gwydiongwydion Posts: 1,067member
    Quote:
    Originally Posted by anonymouse View Post


    So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.



    No, thy are saying that with worse network the gap will be greater.
  • Reply 17 of 78
    Quote:
    Originally Posted by anonymouse View Post


    Yes, you are wrong.



    I enjoy the short and sweet.



    However, how so? Please explain. If you explain it well enough, I'll change my opinion.
  • Reply 18 of 78
    Quote:
    Originally Posted by TenoBell View Post


    The complaint is more against the methodology of the test. That the methodology does not prove what the test claims to prove. Its more about that than the actual results of the test.



    I'm not arguing that. According to this article, yes the methodology of this test is faulty. I was talking in general.
  • Reply 19 of 78
    Two seconds vs. three seconds. OMG. I'm Glad I live in a remote area of Alaska and have access to the web using a satellite connection (Starband). They promise that my maximum download speed will not exceed 0.5 Mb per second and in practice I get 0.45 Mb / sec. This is achieved with large downloads such as software updates from Apple. The iOS 4.3 update downloaded in just over 3 hours (actually a total time of 4.5 hours since the first attempt failed halfway through and iTunes started over when I attempted a second download).



    The weak link in my setup definitely is not my MacBook. I don't think it will be my refurbished iPad 1 either. Apple Technical Services is sending me a mailer to return the refurbished unit which takes 73 seconds to boot after the white apple appears on the screen. I see that white apple occasionally (actually occasionally x 10, which is frequently, but it's more fun to try to spell occasionally), Usually when I try to type, or when I click on an icon. They sent the mailer a week ago and it should arrive any day. If it's a FED EX mailer I will have to drive 130 miles to Fairbanks to the Fed Ex office which is located about 2000 feet from MacHaus of Fairbanks, the nearest Apple Service Provider.



    Two seconds vs three seconds. You gotta be kiddin'
  • Reply 20 of 78
    Fandroid fail



    Again.
Sign In or Register to comment.