W3C publishes "extremely silly" HTML5 test results suggesting win for Internet Explorer 9
A preview version of Microsoft's Internet Explorer 9 browser engine passed more elements of a web standards test than a random selection of other browsers in the Worldwide Web Consortium's "vastly incomplete" HTML5 test suite, creating a news story that W3C members decried as "incorrect information" based on a "rather buggy test results page."
A variety of sources have reported the published results of the W3C test suite without noting that the browser Microsoft submitted for testing on October 28 was its "hot off the press" Internet Explorer Platform Preview 6, introduced to developers at the company's Professional Developers Conference just days earlier. Microsoft's "platform preview" was compared against Apple's currently shipping Safari 5.0.2 (publicly released two months ago), as well a beta version of Google Chrome 7, Mozilla's Firefox 4 beta 6, and an alpha build of Opera 11.
The results, published and amplified by Microsoft in a blog posting, suggest exceptional HTML5 web standards compliance in the company's upcoming Internet Explorer 9 and depict a significant lead over other browsers in web standards performance.
"Bogus," "buggy test results" used in "creating PR events"
The W3C's internal HTML5 test results were copied to a public web site labeled "Official HTML5 Test Suite Conformance Results," and were subsequently picked up by the tech media to falsely suggest Internet Explorer 9 "outperforms other browsers in HTML5 compliance."
The result was then discussed in emails between W3C members, including Anne van Kesteren of Opera, Maciej Stachowiak of Apple, Jonathan Griffin of Mozilla, Mikkel Toudal Kristiansen of Google, and Kris Krueger of Microsoft.
Opera's van Kesteren wrote, "This test suite is vastly incomplete. Publishing unverified results of a vastly incomplete test suite without a big fat warning is extremely silly. Why was this done?"
Stachowiak responded, "It's also strange that the results include alpha/beta/preview versions of most browsers, but the stable version of Safari [rather than the latest nightly build]. Wouldn't be a big deal other than the fact that this rather buggy test results page was labeled as 'Official' and then picked up in the press as authoritative. We should probably be cautious about the chance of creating PR events based on incorrect information."
Google employee Ian Hickson, the author and maintainer of the Acid2 and Acid3 tests and the HTML 5 specification itself, added, "I agree with Anne that it's rather pointless to be publishing results for this test suite. Realistically speaking the test suite isn't even 0.1% complete yet."
Another member of the email thread, identified only as "[email protected]," responded, "Apologies to all. That was just a transmission of a test result to the public list. I did not expect that this result is published immediately." The individual's email address literally means "Miss Understanding" in German, with the placeholder domain name "only for spam."
Microsoft's big push to catch up in HTML5, reach iOS users
Microsoft's currently shipping version of Internet Explorer 8 fails nearly every HTML5, SVG, CSS3, DOM, and JavaScript test across the board. The dramatic reversal of Microsoft's browser performance in the W3C's HTML5 test suite appears to align with the company's recently stated intent to switch gears in pushing HTML5 web standards over its own Silverlight proprietary web-replacement plugin, in large part an effort to reach Apple's installed base of iOS users who can't install Silverlight.
Microsoft has also channeled resources in its money-hemorrhaging Online Services Division to focus Bing web search and maps features on using HTML5 embellishments rather than using Silverlight, an effort the company profiled on stage at Apple's WWDC event this summer. At the time, Apple's Steve Jobs announced that both iOS devices and Safari 5 would gain access to Microsoft's Bing service as a web search option.
The company has also changed its mind about supporting Canvas, an element of HTML5 used to create dynamic, scriptable rendering of 2D shapes and bitmap images within web pages. Canvas, originally created by Apple and contributed royalty free to the HTML5 specification, enables HTML5 developers to create animations and games without needing a plugin helper like Flash or Silverlight.
This spring, Microsoft was promising HTML5 support in IE9 but was still reserving the right to exclude support for Canvas, likely in an effort to protect a market for Silverlight. The company is now including support, albeit in a limited fashion.
Microsoft aces its own tests
However, there's another reason why Microsoft has vaulted from dead last to first place in HTML5 benchmarks: the company has essentially written the W3C's test suite used to generate the scores.
Microsoft has openly contributed thousands of test cases to the W3C pertaining to elements of the HTML5 technology portfolio (which collectively includes CSS, JavaScript and a variety of other components in addition to HTML itself).
Additionally, the test also completely ignores HTML5 features critical to real world web applications, such support for drag and drop, Web Workers, the File API, local storage, and CSS3 transforms and other animations. Further, Microsoft only reports IE's performance against its own tests.
This allows Microsoft's engineers to focus on scoring well on the tests they wrote, while not necessarily competing with other browsers to actually support web standards properly. Internet Explorer certainly does not lead in the performance rankings of independent web standards tests that were not invented by Microsoft, such as Acid3, which focuses on DOM and JavaScript compliance.
Internet Explorer 9 still isn't standards compliant
For example, despite publishing scores that suggest Internet Explorer 9 will pass all of the W3C tests it submitted related to SVG (scalable vector graphics) on its website, Microsoft's data "doesn't actually show SVG compliance as such," noted "Haavard," a blogger who works at Opera.
Haavard cited CodeDread, a site detailing real world SVG support in different browsers. "As you can see, IE9 still does poorly compared to other browsers. So while Microsoft's own page would give you the impression that IE9 has excellent SVG support, that is not the reality," Haavard wrote.
CodeDread's tests, based on the official SVG Test Suite, assigned shipping versions of Safari, Chrome and Opera "A" scores, while Firefox got a "C" and Internet Explorer 8 earned an "F," failing every test. Microsoft's IE9 Previews have only inched up from "F" to "D" over the past year, despite passing with flying colors the limited set of SVG tests that Microsoft invented for the W3C.
A variety of sources have reported the published results of the W3C test suite without noting that the browser Microsoft submitted for testing on October 28 was its "hot off the press" Internet Explorer Platform Preview 6, introduced to developers at the company's Professional Developers Conference just days earlier. Microsoft's "platform preview" was compared against Apple's currently shipping Safari 5.0.2 (publicly released two months ago), as well a beta version of Google Chrome 7, Mozilla's Firefox 4 beta 6, and an alpha build of Opera 11.
The results, published and amplified by Microsoft in a blog posting, suggest exceptional HTML5 web standards compliance in the company's upcoming Internet Explorer 9 and depict a significant lead over other browsers in web standards performance.
"Bogus," "buggy test results" used in "creating PR events"
The W3C's internal HTML5 test results were copied to a public web site labeled "Official HTML5 Test Suite Conformance Results," and were subsequently picked up by the tech media to falsely suggest Internet Explorer 9 "outperforms other browsers in HTML5 compliance."
The result was then discussed in emails between W3C members, including Anne van Kesteren of Opera, Maciej Stachowiak of Apple, Jonathan Griffin of Mozilla, Mikkel Toudal Kristiansen of Google, and Kris Krueger of Microsoft.
Opera's van Kesteren wrote, "This test suite is vastly incomplete. Publishing unverified results of a vastly incomplete test suite without a big fat warning is extremely silly. Why was this done?"
Stachowiak responded, "It's also strange that the results include alpha/beta/preview versions of most browsers, but the stable version of Safari [rather than the latest nightly build]. Wouldn't be a big deal other than the fact that this rather buggy test results page was labeled as 'Official' and then picked up in the press as authoritative. We should probably be cautious about the chance of creating PR events based on incorrect information."
Google employee Ian Hickson, the author and maintainer of the Acid2 and Acid3 tests and the HTML 5 specification itself, added, "I agree with Anne that it's rather pointless to be publishing results for this test suite. Realistically speaking the test suite isn't even 0.1% complete yet."
Another member of the email thread, identified only as "[email protected]," responded, "Apologies to all. That was just a transmission of a test result to the public list. I did not expect that this result is published immediately." The individual's email address literally means "Miss Understanding" in German, with the placeholder domain name "only for spam."
Microsoft's big push to catch up in HTML5, reach iOS users
Microsoft's currently shipping version of Internet Explorer 8 fails nearly every HTML5, SVG, CSS3, DOM, and JavaScript test across the board. The dramatic reversal of Microsoft's browser performance in the W3C's HTML5 test suite appears to align with the company's recently stated intent to switch gears in pushing HTML5 web standards over its own Silverlight proprietary web-replacement plugin, in large part an effort to reach Apple's installed base of iOS users who can't install Silverlight.
Microsoft has also channeled resources in its money-hemorrhaging Online Services Division to focus Bing web search and maps features on using HTML5 embellishments rather than using Silverlight, an effort the company profiled on stage at Apple's WWDC event this summer. At the time, Apple's Steve Jobs announced that both iOS devices and Safari 5 would gain access to Microsoft's Bing service as a web search option.
The company has also changed its mind about supporting Canvas, an element of HTML5 used to create dynamic, scriptable rendering of 2D shapes and bitmap images within web pages. Canvas, originally created by Apple and contributed royalty free to the HTML5 specification, enables HTML5 developers to create animations and games without needing a plugin helper like Flash or Silverlight.
This spring, Microsoft was promising HTML5 support in IE9 but was still reserving the right to exclude support for Canvas, likely in an effort to protect a market for Silverlight. The company is now including support, albeit in a limited fashion.
Microsoft aces its own tests
However, there's another reason why Microsoft has vaulted from dead last to first place in HTML5 benchmarks: the company has essentially written the W3C's test suite used to generate the scores.
Microsoft has openly contributed thousands of test cases to the W3C pertaining to elements of the HTML5 technology portfolio (which collectively includes CSS, JavaScript and a variety of other components in addition to HTML itself).
Additionally, the test also completely ignores HTML5 features critical to real world web applications, such support for drag and drop, Web Workers, the File API, local storage, and CSS3 transforms and other animations. Further, Microsoft only reports IE's performance against its own tests.
This allows Microsoft's engineers to focus on scoring well on the tests they wrote, while not necessarily competing with other browsers to actually support web standards properly. Internet Explorer certainly does not lead in the performance rankings of independent web standards tests that were not invented by Microsoft, such as Acid3, which focuses on DOM and JavaScript compliance.
Internet Explorer 9 still isn't standards compliant
For example, despite publishing scores that suggest Internet Explorer 9 will pass all of the W3C tests it submitted related to SVG (scalable vector graphics) on its website, Microsoft's data "doesn't actually show SVG compliance as such," noted "Haavard," a blogger who works at Opera.
Haavard cited CodeDread, a site detailing real world SVG support in different browsers. "As you can see, IE9 still does poorly compared to other browsers. So while Microsoft's own page would give you the impression that IE9 has excellent SVG support, that is not the reality," Haavard wrote.
CodeDread's tests, based on the official SVG Test Suite, assigned shipping versions of Safari, Chrome and Opera "A" scores, while Firefox got a "C" and Internet Explorer 8 earned an "F," failing every test. Microsoft's IE9 Previews have only inched up from "F" to "D" over the past year, despite passing with flying colors the limited set of SVG tests that Microsoft invented for the W3C.
Comments
"The WHATWG was formed in response to the slow development of web standards monitored by the World Wide Web Consortium (W3C)" - Wikipedia
"Chris Wilson of Microsoft was invited but did not join" - Wikipedia
The less Consortium and Micro are involved the better for the language.
Titan10
Truly, truly awesome. Microsoft wouldn't even flinch to say that...because IE9 is 32% more awesome than the standard.
This one seems to make sense if taken in context. I hope W3C can continue to expand this "Official HTML5 Test Suite" to include all HTML5 technologies that have been officially ratified.
http://forums.appleinsider.com/showthread.php?t=107290
Now AI are getting their nickers in a twist when someone else publishes an inadequate appraisal of a technical matter - ha ha - what goes around...
Are they just confused as to where things are going and just clinging on to everything, but not really doing a great job at anything ether?
Sounds to me like Silverlight has had it's day and it's time to move on - well it didn't even have it's day did it??????
The less Consortium and Micro are involved the better for the language
Yeah, right. Leaving out the biggest web browser vendor will surely do wonders for interoperability. Some people also would like to forget that it was Microsoft who sponsored thousands of unit tests for CSS 2.1 (and now CSS 3.0) which uncovered that web-darlings like Gecko and WebKit had buggy and incomplete implementations (which they silently fixed in the meantime). Microsoft is doing a lot of the "boring" grunt-work which Apple, Google and Mozilla couldn't be arsed with obviously.
I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly?
That's what IE9 is for.
I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly?
Ar
Dance for us, Monkey Boy!
I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly?
Since when did Microsoft ever do anything 'properly'? It has always mislead people. It has always played dishonestly.
This is just one example in a long long line...
Since when did Microsoft ever do anything 'properly'? It has always mislead people. It has always played dishonestly.
This is just one example in a long long line...
I remember when we all used Macs that ran on Power PC chips like the G3,4 and 5's. Ah, those were the days. Those Macs outperformed their PC counterparts by massive amounts. We know this to be true because Apple told us it was. Those chips had 'velocity engines' which made them much faster where it counts. Guess who told us that?
We don't need devices that can render Flash on websites either.
Oh yes, open standards are a good thing - Flash is bad because it isn't open - forget DLNA and use Apples AirPlay which is - er, closed - closed is the new open!!!, of course!
*cough*
Its not really clear. If they DID NOT, then there is just another AI link bait freak show blog post.
The dramatic reversal of Microsoft's browser performance in the W3C's HTML5 test suite appears to align with the company's recently stated intent to switch gears in pushing HTML5 web standards over its own Silverlight proprietary web-replacement plugin, in large part an effort to reach Apple's installed base of iOS users who can't install Silverlight.
How exactly does IE web standards compliance help reach Apple's installed base of iOS users? This makes no sense. Maybe I'm missing something.
Yeah, right. Leaving out the biggest web browser vendor will surely do wonders for interoperability. Some people also would like to forget that it was Microsoft who sponsored thousands of unit tests for CSS 2.1 (and now CSS 3.0) which uncovered that web-darlings like Gecko and WebKit had buggy and incomplete implementations (which they silently fixed in the meantime). Microsoft is doing a lot of the "boring" grunt-work which Apple, Google and Mozilla couldn't be arsed with obviously.
IE isn't the majority of browsers anymore, we finally have viable competition and innovation. But otherwise you are correct we need Microsoft input here as well.
IE isn't the majority of browsers anymore, we finally have viable competition and innovation. But otherwise you are correct we need Microsoft input here as well.
You did read the part on the page you referenced that stated:
'W3Schools is a website for people with an interest for web technologies. These people are more interested in using alternative browsers than the average user. The average user tends to use Internet Explorer, since it comes preinstalled with Windows. Most do not seek out other browsers.
These facts indicate that the browser figures above are not 100% realistic. Other web sites have statistics showing that Internet Explorer is used by at least 80% of the users.
Anyway, our data, collected from W3Schools' log-files, over a five year period, clearly shows the long and medium-term trends'.
.... Other web sites have statistics showing that Internet Explorer is used by at least 80% of the users.
...
Which other sites would those be?
Which other sites would those be?
I have no idea. I just republished the statement that was on the w3schools site which I guess is their own disclaimer. Maybe Mircosoft's site has 80%.