Android vs iPhone web page loading speed contest flawed
Test results promoted by Blaze Software that purport to prove that Android is much faster at loading web pages than Apple's iOS 4.3 did so using a poorly performing custom iPhone app, rather than using Safari itself.
The results of the test, according to Bloomberg, said that an Android-based Nexus S phone performed 52 percent faster on average after loading more than 45,000 pages from 1,000 websites compared to iPhone 4.
The average speed difference was about a second longer page load on iPhone 4: 2.14 seconds compared to 3.25 seconds. The more complex the page, the greater the performance difference, Blaze reported. Guy Podjarny, the firm's chief technology officer, said "it?s not that Apple doesn?t care about speed, but Google is fanatical about it."
However, while Blaze maintained that its benchmarks used the newly released iOS 4.3, suggesting that it took into account the fast new Safari browser with Apple's new Nitro JavaScript engine, the way it performed the tests completely bypassed those improvements.
Rather than using Apple's Safari browser directly, Blaze tested page loading on iPhone 4 using the company's own proprietary app that did not take advantage of the new improvements in iOS 4.3.
As noted in a previous report by AppleInsider, apps that implement Apple's UIWebView to provide web browsing functions within an app (as Blaze did), in addition to full screen web apps, do not take advantage of the new web acceleration features Apple introduced in iOS 4.3, including Nitro and a variety of other improvements to the mobile Safari browser.
While Apple hasn't officially commented on the disparity between the newly revamped Safari and the features of the UIWebView framework, it appears that the difference relates to both to the fact that Apple wanted to rapidly roll out new WebKit features quickly to mainstream iOS users in Safari (and simply didn't have time to retrofit every other element of the system with the new code), and also to security considerations.
Apple's new Nitro JavaScript engine (originally called Squirrel Fish Extreme) competes against Google's Chrome V8 and Mozilla's FireFox TraceMonkey to speed JavaScript (the programming language behind the web) using various different approaches, each of which has different strengths and advantages.
Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.
Running an automated test on page loading using the actual Safari browser on iPhone 4 would be far more difficult to perform, but would also fail to account for other, likely more important differences between iOS and devices running Android.
These include overall stability and usability of the platform, power management and battery life, hardware quality, and easy access to iTunes music and movie rentals, iBooks, and App Store, three features Apple has started promoting in series of new ads that end with the line, "if you don?t have an iPhone, well, you don?t have an iPhone.?
The results of the test, according to Bloomberg, said that an Android-based Nexus S phone performed 52 percent faster on average after loading more than 45,000 pages from 1,000 websites compared to iPhone 4.
The average speed difference was about a second longer page load on iPhone 4: 2.14 seconds compared to 3.25 seconds. The more complex the page, the greater the performance difference, Blaze reported. Guy Podjarny, the firm's chief technology officer, said "it?s not that Apple doesn?t care about speed, but Google is fanatical about it."
However, while Blaze maintained that its benchmarks used the newly released iOS 4.3, suggesting that it took into account the fast new Safari browser with Apple's new Nitro JavaScript engine, the way it performed the tests completely bypassed those improvements.
Rather than using Apple's Safari browser directly, Blaze tested page loading on iPhone 4 using the company's own proprietary app that did not take advantage of the new improvements in iOS 4.3.
As noted in a previous report by AppleInsider, apps that implement Apple's UIWebView to provide web browsing functions within an app (as Blaze did), in addition to full screen web apps, do not take advantage of the new web acceleration features Apple introduced in iOS 4.3, including Nitro and a variety of other improvements to the mobile Safari browser.
While Apple hasn't officially commented on the disparity between the newly revamped Safari and the features of the UIWebView framework, it appears that the difference relates to both to the fact that Apple wanted to rapidly roll out new WebKit features quickly to mainstream iOS users in Safari (and simply didn't have time to retrofit every other element of the system with the new code), and also to security considerations.
Apple's new Nitro JavaScript engine (originally called Squirrel Fish Extreme) competes against Google's Chrome V8 and Mozilla's FireFox TraceMonkey to speed JavaScript (the programming language behind the web) using various different approaches, each of which has different strengths and advantages.
Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.
Running an automated test on page loading using the actual Safari browser on iPhone 4 would be far more difficult to perform, but would also fail to account for other, likely more important differences between iOS and devices running Android.
These include overall stability and usability of the platform, power management and battery life, hardware quality, and easy access to iTunes music and movie rentals, iBooks, and App Store, three features Apple has started promoting in series of new ads that end with the line, "if you don?t have an iPhone, well, you don?t have an iPhone.?
Comments
What I found interesting was this:
Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as a both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.
Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...
Any time some study comes out pro or against an Apple product, it inevitably fans flames from all sides. Go have fun Apple Devotes and Android Fans. I'll eat my popcorn.
What I found interesting was this:
Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...
Relax, it will be fixed in an update. All apps go through approval so I don't see a problem here.
These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.
People relax! Its just an OS.
Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...
These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.
People relax! Its just an OS.
Indeed! I would be the first to point out that all OSs, regardless of the company attached, have positives and negatives against the other OSs. I love aspects of the webOS, and I love aspects of the iOS, and love aspects of the Android OS (I have not had experience with the newest WinMobile OS, so I can't vouch there). There are also aspects I hate on all. Its a to each their own, and respect the decisions and argument points. Studies like this I feel just increase a "Mine is Better/Bigger than Yours" when in the real world, its all about personal preference. Just cause a study says that the thing you have is now "devalued", doesn't mean it really is. If it works great for you, and you like how it works, then the study doesn't mean anything for you. If you happen to have issues that a study shows, then that's something different.
Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...
..and no link to the source, bad AI... bad....
For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/
Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.
Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...
..and no link to the source, bad AI... bad....
For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/
The have issued a clarification, but is hard for Dan to check anything
http://www.blaze.io/business/embeded...ment-167303524
If you happen to have issues that a study shows, then that's something different.
Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps...
The have issued a clarification, but is hard for Dan to check anything
http://www.blaze.io/business/embeded...ment-167303524
Not much of a "clarification" when they say, "We were wrong but we're still right."
Or this bit:
?Despite this fundamental testing flaw they still only found an average of a second difference in loading Web pages?.
We see this is a bad interpretation of our results. First and foremost, our tests were run over networks and conditions more favorable than the average user browsing on his mobile device. Second, on many sites the gap was greater in absolute terms (for example, on wsj.com we saw a 5-10 second gap). The median gap was only one second, thanks in part to the great network conditions.
So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.
... Am I wrong in seeing a double standard here? ...
Yes, you are wrong.
The have issued a clarification, but is hard for Dan to check anything
http://www.blaze.io/business/embeded...ment-167303524
Yeah, but to believe the clarification (where they still claim they were right BTW), you have to believe that these guys are either total dumbasses, that they purposely skewed the test, or both.
So it's either a non-story about a bunch of idiots that are posing as technical wizards when they don't know their ass from a screwdriver, or it's a tale of pure deception. Take your pick.
... Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. ...
It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.
It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.
No, they didn't claim that, in their report there is the metodology and how they made the tests.
And despite the metodology, the tests are stupid, it's irelevant that a page load a second faster or slower
So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.
No, thy are saying that with worse network the gap will be greater.
Yes, you are wrong.
I enjoy the short and sweet.
However, how so? Please explain. If you explain it well enough, I'll change my opinion.
The complaint is more against the methodology of the test. That the methodology does not prove what the test claims to prove. Its more about that than the actual results of the test.
I'm not arguing that. According to this article, yes the methodology of this test is faulty. I was talking in general.
The weak link in my setup definitely is not my MacBook. I don't think it will be my refurbished iPad 1 either. Apple Technical Services is sending me a mailer to return the refurbished unit which takes 73 seconds to boot after the white apple appears on the screen. I see that white apple occasionally (actually occasionally x 10, which is frequently, but it's more fun to try to spell occasionally), Usually when I try to type, or when I click on an icon. They sent the mailer a week ago and it should arrive any day. If it's a FED EX mailer I will have to drive 130 miles to Fairbanks to the Fed Ex office which is located about 2000 feet from MacHaus of Fairbanks, the nearest Apple Service Provider.
Two seconds vs three seconds. You gotta be kiddin'
Again.