or Connect
AppleInsider › Forums › Mobile › iPhone › Android vs iPhone web page loading speed contest flawed
New Posts  All Forums:Forum Nav:

Android vs iPhone web page loading speed contest flawed

post #1 of 78
Thread Starter 
Test results promoted by Blaze Software that purport to prove that Android is much faster at loading web pages than Apple's iOS 4.3 did so using a poorly performing custom iPhone app, rather than using Safari itself.

The results of the test, according to Bloomberg, said that an Android-based Nexus S phone performed 52 percent faster on average after loading more than 45,000 pages from 1,000 websites compared to iPhone 4.

The average speed difference was about a second longer page load on iPhone 4: 2.14 seconds compared to 3.25 seconds. The more complex the page, the greater the performance difference, Blaze reported. Guy Podjarny, the firm's chief technology officer, said "its not that Apple doesnt care about speed, but Google is fanatical about it."

However, while Blaze maintained that its benchmarks used the newly released iOS 4.3, suggesting that it took into account the fast new Safari browser with Apple's new Nitro JavaScript engine, the way it performed the tests completely bypassed those improvements.

Rather than using Apple's Safari browser directly, Blaze tested page loading on iPhone 4 using the company's own proprietary app that did not take advantage of the new improvements in iOS 4.3.

As noted in a previous report by AppleInsider, apps that implement Apple's UIWebView to provide web browsing functions within an app (as Blaze did), in addition to full screen web apps, do not take advantage of the new web acceleration features Apple introduced in iOS 4.3, including Nitro and a variety of other improvements to the mobile Safari browser.

While Apple hasn't officially commented on the disparity between the newly revamped Safari and the features of the UIWebView framework, it appears that the difference relates to both to the fact that Apple wanted to rapidly roll out new WebKit features quickly to mainstream iOS users in Safari (and simply didn't have time to retrofit every other element of the system with the new code), and also to security considerations.

Apple's new Nitro JavaScript engine (originally called Squirrel Fish Extreme) competes against Google's Chrome V8 and Mozilla's FireFox TraceMonkey to speed JavaScript (the programming language behind the web) using various different approaches, each of which has different strengths and advantages.

Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.

Running an automated test on page loading using the actual Safari browser on iPhone 4 would be far more difficult to perform, but would also fail to account for other, likely more important differences between iOS and devices running Android.

These include overall stability and usability of the platform, power management and battery life, hardware quality, and easy access to iTunes music and movie rentals, iBooks, and App Store, three features Apple has started promoting in series of new ads that end with the line, "if you dont have an iPhone, well, you dont have an iPhone.

post #2 of 78
Any time some study comes out pro or against an Apple product, it inevitably fans flames from all sides. Go have fun Apple Devotes and Android Fans. I want to see a big old fight all over a 1 second difference in web page loading on a mobile device. I'll eat my popcorn.

What I found interesting was this:
Quote:
Originally Posted by AppleInsider View Post

Apple's Nitro uses a JIT (just-in-time) compiler as opposed to a traditional interpreter. This requires that Nitro obtain additional security privileges required to compile data into executable code, something Apple reserves for the iOS itself and its bundled apps. Third party iOS apps can't compile code as a both a security feature and, apparently, a limitation that prevents middleware platforms (such as Adobe Flash) from competing for iOS developers' attention.

Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
post #3 of 78
Quote:
Originally Posted by camroidv27 View Post

Any time some study comes out pro or against an Apple product, it inevitably fans flames from all sides. Go have fun Apple Devotes and Android Fans. I'll eat my popcorn.

What I found interesting was this:


Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...

Relax, it will be fixed in an update. All apps go through approval so I don't see a problem here.

These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.
People relax! Its just an OS.
post #4 of 78
I think this is more about preventing malware from breaching security holes in webkit. I would be willing to give Apple more time to figure it out so that its secure and functional.

Quote:
Originally Posted by camroidv27 View Post

Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...
post #5 of 78
I'm hoping the Nitro engine will eventually be able to be used by my favourite browser, Atomic Web.
post #6 of 78
Quote:
Originally Posted by jpcg View Post

These Flame Wars go on my nerves. You cannot read the comment section of engadget anymore because there is so much hate.
People relax! Its just an OS.

Indeed! I would be the first to point out that all OSs, regardless of the company attached, have positives and negatives against the other OSs. I love aspects of the webOS, and I love aspects of the iOS, and love aspects of the Android OS (I have not had experience with the newest WinMobile OS, so I can't vouch there). There are also aspects I hate on all. Its a to each their own, and respect the decisions and argument points. Studies like this I feel just increase a "Mine is Better/Bigger than Yours" when in the real world, its all about personal preference. Just cause a study says that the thing you have is now "devalued", doesn't mean it really is. If it works great for you, and you like how it works, then the study doesn't mean anything for you. If you happen to have issues that a study shows, then that's something different.
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
post #7 of 78
I think that those "my d*** is longer than yours" contest are stupid so I don't mind if Android Webview loads a page 1 millisecond faster or slower than Apple's UIwebview but: how Nitro JS can make pages load faster. The only thing I whink it can make loading pages faster is asynchonous loading so it's not so flawed the contest.
post #8 of 78
Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...

..and no link to the source, bad AI... bad....

For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/
post #9 of 78
Quote:
Originally Posted by jb510 View Post

Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...

..and no link to the source, bad AI... bad....

For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/

The have issued a clarification, but is hard for Dan to check anything
http://www.blaze.io/business/embeded...ment-167303524
post #10 of 78
The complaint is more against the methodology of the test. That the methodology does not prove what the test claims to prove. Its more about that than the actual results of the test.

Quote:
Originally Posted by camroidv27 View Post

If you happen to have issues that a study shows, then that's something different.
post #11 of 78
The title of the test is about whose browser is faster. They assumed that the embedded browser and native browser were exactly the same. Security concerns are the reason Apple has not opened Nitro to its embedded browser. They would need to sandbox the process so that it becomes a trusted chain of executable code.

Quote:
Originally Posted by jb510 View Post

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps...
post #12 of 78
Quote:
Originally Posted by Gwydion View Post

The have issued a clarification, but is hard for Dan to check anything
http://www.blaze.io/business/embeded...ment-167303524

Not much of a "clarification" when they say, "We were wrong but we're still right."

Or this bit:

Quote:
Despite this fundamental testing flaw they still only found an average of a second difference in loading Web pages.

We see this is a bad interpretation of our results. First and foremost, our tests were run over networks and conditions more favorable than the average user browsing on his mobile device. Second, on many sites the gap was greater in absolute terms (for example, on wsj.com we saw a 5-10 second gap). The median gap was only one second, thanks in part to the great network conditions.

So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.
post #13 of 78
Quote:
Originally Posted by camroidv27 View Post

... Am I wrong in seeing a double standard here? ...

Yes, you are wrong.
post #14 of 78
Quote:
Originally Posted by Gwydion View Post

The have issued a clarification, but is hard for Dan to check anything
http://www.blaze.io/business/embeded...ment-167303524

Yeah, but to believe the clarification (where they still claim they were right BTW), you have to believe that these guys are either total dumbasses, that they purposely skewed the test, or both.

So it's either a non-story about a bunch of idiots that are posing as technical wizards when they don't know their ass from a screwdriver, or it's a tale of pure deception. Take your pick.
post #15 of 78
Quote:
Originally Posted by jb510 View Post

... Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. ...

It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.
post #16 of 78
Quote:
Originally Posted by anonymouse View Post

It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.

No, they didn't claim that, in their report there is the metodology and how they made the tests.

And despite the metodology, the tests are stupid, it's irelevant that a page load a second faster or slower
post #17 of 78
Quote:
Originally Posted by anonymouse View Post

So, they are saying that, yes, it's only a second, but if there are network issues between you and the server the times could be meaninglessly different. I don't think they know how to interpret what results they do have. They actually sound like a bunch of bozos.

No, thy are saying that with worse network the gap will be greater.
post #18 of 78
Quote:
Originally Posted by anonymouse View Post

Yes, you are wrong.

I enjoy the short and sweet.

However, how so? Please explain. If you explain it well enough, I'll change my opinion.
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
post #19 of 78
Quote:
Originally Posted by TenoBell View Post

The complaint is more against the methodology of the test. That the methodology does not prove what the test claims to prove. Its more about that than the actual results of the test.

I'm not arguing that. According to this article, yes the methodology of this test is faulty. I was talking in general.
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
post #20 of 78
Two seconds vs. three seconds. OMG. I'm Glad I live in a remote area of Alaska and have access to the web using a satellite connection (Starband). They promise that my maximum download speed will not exceed 0.5 Mb per second and in practice I get 0.45 Mb / sec. This is achieved with large downloads such as software updates from Apple. The iOS 4.3 update downloaded in just over 3 hours (actually a total time of 4.5 hours since the first attempt failed halfway through and iTunes started over when I attempted a second download).

The weak link in my setup definitely is not my MacBook. I don't think it will be my refurbished iPad 1 either. Apple Technical Services is sending me a mailer to return the refurbished unit which takes 73 seconds to boot after the white apple appears on the screen. I see that white apple occasionally (actually occasionally x 10, which is frequently, but it's more fun to try to spell occasionally), Usually when I try to type, or when I click on an icon. They sent the mailer a week ago and it should arrive any day. If it's a FED EX mailer I will have to drive 130 miles to Fairbanks to the Fed Ex office which is located about 2000 feet from MacHaus of Fairbanks, the nearest Apple Service Provider.

Two seconds vs three seconds. You gotta be kiddin'
post #21 of 78
Fandroid fail

Again.
post #22 of 78
Quote:
Originally Posted by Gwydion View Post

No, thy are saying that with worse network the gap will be greater.

Not due to differences in "browsers", simply due to network issues. That whole point they tried to make was completely stupid. And, the gap will only be wider under controlled conditions. In the wild, which is what they are talking about, where you can't control conditions, the gap could become smaller or reverse. They're idiots.
post #23 of 78
Quote:
Originally Posted by camroidv27 View Post

Am I wrong in seeing a double standard here? Granted, it is Apple's devices, and they can do what they want, but still...

Of course there is a double-standard. That's the advantage of being the builder of the device and the OS. You get to install your apps on the shipping device. You get access to private APIs for your applications that nobody else is allowed to use. It's not right or wrong, it just "is". It's Apple's embeded browser API they are using. They could always build their own browser right into the application. They used the one Apple provided, and regardless of the reasons, Apple decided to not provide that browser the same enhancments Safari got.

Quote:
Originally Posted by TenoBell View Post

The title of the test is about whose browser is faster. They assumed that the embedded browser and native browser were exactly the same. Security concerns are the reason Apple has not opened Nitro to its embedded browser. They would need to sandbox the process so that it becomes a trusted chain of executable code.

Quote:
Originally Posted by anonymouse View Post

It's totally fair. They claimed they were testing browser against browser but didn't test any browsers. They deserve every bit of criticism they get, but mostly since they were deceptive about it.

But until the story was reported after 4.3 was released, how many people had the reasonable expectation that the embedded browser used the same engine as Safari? That's the way it's been since the beginning of Safari on both the Mac and iOS, right? Other applications used the same WebKit code that Safari used. Apple didn't tell anyone that in iOS 4.3 it was any different. So how was Blaze supposed to know? They've even put a disclaimer at the very top of their original story. So please, explain to me where the "deception" is.

Now that they know, should they try to repleat the test using Safari? Sure. But as even AI stated, that could be very difficult to execute.
post #24 of 78
Quote:
Originally Posted by Gwydion View Post

No, they didn't claim that, in their report there is the metodology and how they made the tests.

And despite the metodology, the tests are stupid, it's irelevant that a page load a second faster or slower

This is their headline:

Quote:
iPhone vs. Android 45,000 Tests Prove Whose Browser is Faster

What part of that didn't they claim?
post #25 of 78
lol this is an easy argument to solve and its why we continue to see these trumpeted stories of superiority from android supporters.
Solution:

An android user and, iphone user walk into a room of 20 people round a large table. Sit, take out your device and place it on the table... now flip it over so the back is showing. As i've tested this, 13 out of 20 ask if thats the iPhone and how do you like it? Out of the remaining 7; 5 Already own an iPhone, . 1 owns a blackberry (haha) and the lone Android user comes to the realization that he doesn't have the cool device.

Easy solution.
post #26 of 78
Quote:
Originally Posted by Wiggin View Post

... But until the story was reported after 4.3 was released, how many people had the reasonable expectation that the embedded browser used the same engine as Safari? That's the way it's been since the beginning of Safari on both the Mac and iOS, right? Other applications used the same WebKit code that Safari used. Apple didn't tell anyone that in iOS 4.3 it was any different. So how was Blaze supposed to know? They've even put a disclaimer at the very top of their original story. So please, explain to me where the "deception" is.

For one thing, their headline is completely dishonest. Secondly, they made a bunch of assumptions that were stupid to make, so that's no defense for their dishonest lead.
post #27 of 78
Quote:
Originally Posted by anonymouse View Post

This is their headline:



What part of that didn't they claim?

They hadn't claimed that they measured Safari and Chrome, they claimed that they measured iOS and Android browsers

"The measurement itself was done using the custom apps, which use the platforms embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone. Manual verification showed that page load performance of the embedded browsers, when properly configured, is effectively identical to the stand-alone browsers. The load times are calculated using the Document Complete callback from the browser, which is a standard way of measuring a web pages load time. As mentioned above, the agents are now a part of a free service available at http://blaze.io/mobile/, and we encourage you to try it out."

And yes, they used the browsers and until iOS 4.3 the performance of embedded browser and Safari browser was the same as it is with android embedded and Chrome browsers.
post #28 of 78
Quote:
Originally Posted by camroidv27 View Post

I enjoy the short and sweet.

However, how so? Please explain. If you explain it well enough, I'll change my opinion.

There's no double standard at all. Apple only allows code it has direct control over to run interpreted/downloaded code. It's well known that this is done for security reasons; allowing all apps to run interpreted or other downloaded code makes it impossible to screen them for malicious intent. So, it's not a double standard, it's the only way that they can provide any level of security, and it's a single well understood standard based on well understood reasons.
post #29 of 78
Quote:
Originally Posted by jb510 View Post

Normally I find AI's writting exemplary, but the biased tone in the first half of this posting is really bothersome.

Suggesting that this is solely Blaze's fault that Apple's new JS engine wasn't used is unfair. Apple should have made the engine available to Apps and WebApps... I'd even argue Blaze was right to assume they had, although I'd expect Blaze to issue a clarification upon learning that WebApp and Apps using the web framework don't get to utilize the new JS engine that their tests are only valid for comparing apps to other apps, not browser to browser...

..and no link to the source, bad AI... bad....

For those that actually care about facts: http://www.blaze.io/uncategorized/mo...ser-is-faster/

I disagree. Blaze did not make it clear enough that they weren't using the browser directly which means people using their iPhone are not experiencing the same thing.

It is up to the people doing the test to make sure their equipment is correctly calibrated to give an objective result which obviously was not the case here.

Based on their defective test there is nothing anyone can deduce except that their app is not able to max out on iOS.
post #30 of 78
Quote:
Originally Posted by Gwydion View Post

They hadn't claimed that they measured Safari and Chrome, they claimed that they measured iOS and Android browsers ...

Take your sophistry elsewhere. Safari and Chrome are the iOS and Android browsers.

(I mean, really, that was just the most pathetic defense of the indefensible I've ever seen on these forums.)
post #31 of 78
Quote:
Originally Posted by anonymouse View Post

There's no double standard at all. Apple only allows code it has direct control over to run interpreted/downloaded code. It's well known that this is done for security reasons; allowing all apps to run interpreted or other downloaded code makes it impossible to screen them for malicious intent. So, it's not a double standard, it's the only way that they can provide any level of security, and it's a single well understood standard based on well understood reasons.

Gotcha, thanks!
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
Go Linux, Choose a Flavor!
"I aim to misbehave"
Reply
post #32 of 78
Quote:
Originally Posted by anonymouse View Post

There's no double standard at all. Apple only allows code it has direct control over to run interpreted/downloaded code. It's well known that this is done for security reasons; allowing all apps to run interpreted or other downloaded code makes it impossible to screen them for malicious intent. So, it's not a double standard, it's the only way that they can provide any level of security, and it's a single well understood standard based on well understood reasons.

And what is the difference of running Facebook JS code in Safari and in UIwebview? Or any other web app, for that matter?

If it's insecure on uiwebview it's also insecure on Safari
post #33 of 78
Quote:
Originally Posted by Gwydion View Post

No, they didn't claim that

They sure did, right in the headline.

iPhone vs. Android 45,000 Tests Prove Whose Browser is Faster

iPhone's browser is Safari, and that's what people are going to assume the title is talking about, not some "embedded browser". At the very least, the headline is imprecise and misleading.
post #34 of 78
They used the embedded browsers in both platforms.

Blaze's conclusion was that the embedded browser in Android was faster than the embedded browser in iOS. That conclusion holds up to scrutiny.

Perhaps it escapes AppleInsider, but in the real world of computer performance testing, a fair set of benchmarks compares platforms as equally across the table as possible. It might make AI happy to see a set of unequal comparisons, but that would hardly be considered fair outside the confines of fanboidom.
post #35 of 78
That is what happens when you make an assumption without doing your due diligence to check. The onus is on the tester to make sure they understand all of the variables of their test. Before they release the results as fact.

Quote:
Originally Posted by Wiggin View Post

But until the story was reported after 4.3 was released, how many people had the reasonable expectation that the embedded browser used the same engine as Safari? That's the way it's been since the beginning of Safari on both the Mac and iOS, right? Other applications used the same WebKit code that Safari used. Apple didn't tell anyone that in iOS 4.3 it was any different. So how was Blaze supposed to know? They've even put a disclaimer at the very top of their original story. So please, explain to me where the "deception" is.
post #36 of 78
Quote:
Originally Posted by anonymouse View Post

Take your sophistry elsewhere. Safari and Chrome are the iOS and Android browsers.

(I mean, really, that was just the most pathetic defense of the indefensible I've ever seen on these forums.)

Sophistry and pathetic? Which was the difference 2 weeks ago between Safari and uiwebview?
post #37 of 78
Wow...
Talk about an organization trying to establish itself as a reliable source of testing, Blaze has sure shot itself in the foot.
They're particularly getting hammered in their own comments section under the article for a clearly flawed and misleading 'study'.

For more...
http://daringfireball.net/2011/03/nitro_ios_43
post #38 of 78
Quote:
Originally Posted by anonymouse View Post

For one thing, their headline is completely dishonest. Secondly, they made a bunch of assumptions that were stupid to make, so that's no defense for their dishonest lead.

"Dishonest" and "deceptive" means that they knew and were intentionally misleading in their report. Everytime anyone says anything non-positive, and even sometimes when they say something positive but use a word like "overkill", you instantly assume that they are out to get Apple, that they are Apple haters and have a chip on their shoulder. It's a very paranoid world you live in.

Did they make a mistake? Yes. Did they admit that their test didn't test Safari? Yes. And they were open about it by putting the disclaimer on their orignal story. Let's see AI be as forthcoming about the deceptive writing by DED (deceptive or delusional, take your pick). As of the time I'm writing this, DED still has not acknowledged that Blaze discovered the error in their testing and has added a disclaimer on their story. And he probably never will. He doesn't even included a link to Blaze's orignial article. Why? Oh right, because then people would also see the disclaimer.

So just who is being deceptive and dishonest here?
post #39 of 78
Quote:
Originally Posted by ks2problema View Post

They used the embedded browsers in both platforms.

Blaze's conclusion was that the embedded browser in Android was faster than the embedded browser in iOS. That conclusion holds up to scrutiny.

Perhaps it escapes AppleInsider, but in the real world of computer performance testing, a fair set of benchmarks compares platforms as equally across the table as possible. It might make AI happy to see a set of unequal comparisons, but that would hardly be considered fair outside the confines of fanboidom.

No, if you want to compare embedded apps against each other, that's fine. Perhaps even of interest.
But they are clearly (let's all say the word together) LYING when they say that their study analyzes BROWSER speed. That would be Safari against whatever browser is built into Android.
Words have meaning.
post #40 of 78
Quote:
Originally Posted by ks2problema View Post

They used the embedded browsers in both platforms.

Blaze's conclusion was that the embedded browser in Android was faster than the embedded browser in iOS. That conclusion holds up to scrutiny.

Perhaps it escapes AppleInsider, but in the real world of computer performance testing, a fair set of benchmarks compares platforms as equally across the table as possible. It might make AI happy to see a set of unequal comparisons, but that would hardly be considered fair outside the confines of fanboidom.

You do realize this "real world of computer performance testing" is different than the "real world," right?

Statistics and the benchmarks generated by performance testing are meaningless without correct analysis and interpretation.

As stated, the article said "iPhone v. Android: 45000 tests prove which browser is faster."
Not which "embedded" browser is faster. And they certainly didn't put it in real world terms, eg, this affects third party browsers, not the safari browser (which is the browser used by the vast majority of users).

Their poor interpretation and presentation did a disservice to the real and useful statistics that they produced.
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: iPhone
AppleInsider › Forums › Mobile › iPhone › Android vs iPhone web page loading speed contest flawed