W3C publishes "extremely silly" HTML5 test results suggesting win for Internet Explorer 9

Posted:
in Mac Software edited January 2014
A preview version of Microsoft's Internet Explorer 9 browser engine passed more elements of a web standards test than a random selection of other browsers in the Worldwide Web Consortium's "vastly incomplete" HTML5 test suite, creating a news story that W3C members decried as "incorrect information" based on a "rather buggy test results page."



A variety of sources have reported the published results of the W3C test suite without noting that the browser Microsoft submitted for testing on October 28 was its "hot off the press" Internet Explorer Platform Preview 6, introduced to developers at the company's Professional Developers Conference just days earlier. Microsoft's "platform preview" was compared against Apple's currently shipping Safari 5.0.2 (publicly released two months ago), as well a beta version of Google Chrome 7, Mozilla's Firefox 4 beta 6, and an alpha build of Opera 11.



The results, published and amplified by Microsoft in a blog posting, suggest exceptional HTML5 web standards compliance in the company's upcoming Internet Explorer 9 and depict a significant lead over other browsers in web standards performance.



"Bogus," "buggy test results" used in "creating PR events"



The W3C's internal HTML5 test results were copied to a public web site labeled "Official HTML5 Test Suite Conformance Results," and were subsequently picked up by the tech media to falsely suggest Internet Explorer 9 "outperforms other browsers in HTML5 compliance."



The result was then discussed in emails between W3C members, including Anne van Kesteren of Opera, Maciej Stachowiak of Apple, Jonathan Griffin of Mozilla, Mikkel Toudal Kristiansen of Google, and Kris Krueger of Microsoft.



Opera's van Kesteren wrote, "This test suite is vastly incomplete. Publishing unverified results of a vastly incomplete test suite without a big fat warning is extremely silly. Why was this done?"



Stachowiak responded, "It's also strange that the results include alpha/beta/preview versions of most browsers, but the stable version of Safari [rather than the latest nightly build]. Wouldn't be a big deal other than the fact that this rather buggy test results page was labeled as 'Official' and then picked up in the press as authoritative. We should probably be cautious about the chance of creating PR events based on incorrect information."



Google employee Ian Hickson, the author and maintainer of the Acid2 and Acid3 tests and the HTML 5 specification itself, added, "I agree with Anne that it's rather pointless to be publishing results for this test suite. Realistically speaking the test suite isn't even 0.1% complete yet."



Another member of the email thread, identified only as "[email protected]," responded, "Apologies to all. That was just a transmission of a test result to the public list. I did not expect that this result is published immediately." The individual's email address literally means "Miss Understanding" in German, with the placeholder domain name "only for spam."



Microsoft's big push to catch up in HTML5, reach iOS users



Microsoft's currently shipping version of Internet Explorer 8 fails nearly every HTML5, SVG, CSS3, DOM, and JavaScript test across the board. The dramatic reversal of Microsoft's browser performance in the W3C's HTML5 test suite appears to align with the company's recently stated intent to switch gears in pushing HTML5 web standards over its own Silverlight proprietary web-replacement plugin, in large part an effort to reach Apple's installed base of iOS users who can't install Silverlight.



Microsoft has also channeled resources in its money-hemorrhaging Online Services Division to focus Bing web search and maps features on using HTML5 embellishments rather than using Silverlight, an effort the company profiled on stage at Apple's WWDC event this summer. At the time, Apple's Steve Jobs announced that both iOS devices and Safari 5 would gain access to Microsoft's Bing service as a web search option.



The company has also changed its mind about supporting Canvas, an element of HTML5 used to create dynamic, scriptable rendering of 2D shapes and bitmap images within web pages. Canvas, originally created by Apple and contributed royalty free to the HTML5 specification, enables HTML5 developers to create animations and games without needing a plugin helper like Flash or Silverlight.



This spring, Microsoft was promising HTML5 support in IE9 but was still reserving the right to exclude support for Canvas, likely in an effort to protect a market for Silverlight. The company is now including support, albeit in a limited fashion.



Microsoft aces its own tests



However, there's another reason why Microsoft has vaulted from dead last to first place in HTML5 benchmarks: the company has essentially written the W3C's test suite used to generate the scores.



Microsoft has openly contributed thousands of test cases to the W3C pertaining to elements of the HTML5 technology portfolio (which collectively includes CSS, JavaScript and a variety of other components in addition to HTML itself).



Additionally, the test also completely ignores HTML5 features critical to real world web applications, such support for drag and drop, Web Workers, the File API, local storage, and CSS3 transforms and other animations. Further, Microsoft only reports IE's performance against its own tests.



This allows Microsoft's engineers to focus on scoring well on the tests they wrote, while not necessarily competing with other browsers to actually support web standards properly. Internet Explorer certainly does not lead in the performance rankings of independent web standards tests that were not invented by Microsoft, such as Acid3, which focuses on DOM and JavaScript compliance.



Internet Explorer 9 still isn't standards compliant



For example, despite publishing scores that suggest Internet Explorer 9 will pass all of the W3C tests it submitted related to SVG (scalable vector graphics) on its website, Microsoft's data "doesn't actually show SVG compliance as such," noted "Haavard," a blogger who works at Opera.



Haavard cited CodeDread, a site detailing real world SVG support in different browsers. "As you can see, IE9 still does poorly compared to other browsers. So while Microsoft's own page would give you the impression that IE9 has excellent SVG support, that is not the reality," Haavard wrote.



CodeDread's tests, based on the official SVG Test Suite, assigned shipping versions of Safari, Chrome and Opera "A" scores, while Firefox got a "C" and Internet Explorer 8 earned an "F," failing every test. Microsoft's IE9 Previews have only inched up from "F" to "D" over the past year, despite passing with flying colors the limited set of SVG tests that Microsoft invented for the W3C.



«1

Comments

  • Reply 1 of 31
    The HTML 5 has been developed by WHATWG.



    "The WHATWG was formed in response to the slow development of web standards monitored by the World Wide Web Consortium (W3C)" - Wikipedia



    "Chris Wilson of Microsoft was invited but did not join" - Wikipedia





    The less Consortium and Micro are involved the better for the language.





    Titan10
  • Reply 2 of 31
    It would be awesome if IE9 passed 132% of all SVG tests.

    Truly, truly awesome. Microsoft wouldn't even flinch to say that...because IE9 is 32% more awesome than the standard.
  • Reply 3 of 31
    I always take these "HTML5" tests with a grain of salt. It seems someone can swing the results any way they want be including\\excluding certain tests.



    This one seems to make sense if taken in context. I hope W3C can continue to expand this "Official HTML5 Test Suite" to include all HTML5 technologies that have been officially ratified.
  • Reply 4 of 31
    cnocbuicnocbui Posts: 3,613member
    Yea, and AI was more than happy to publish a piece of garbage, masquerading as a scientific assessment, reporting that the display on the Nexus One only had a 16bit display.



    http://forums.appleinsider.com/showthread.php?t=107290



    Now AI are getting their nickers in a twist when someone else publishes an inadequate appraisal of a technical matter - ha ha - what goes around...
  • Reply 5 of 31
    I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly?



    Are they just confused as to where things are going and just clinging on to everything, but not really doing a great job at anything ether?



    Sounds to me like Silverlight has had it's day and it's time to move on - well it didn't even have it's day did it?????? \
  • Reply 6 of 31
    erunnoerunno Posts: 225member
    Quote:
    Originally Posted by Titan10 View Post


    The less Consortium and Micro are involved the better for the language



    Yeah, right. Leaving out the biggest web browser vendor will surely do wonders for interoperability. Some people also would like to forget that it was Microsoft who sponsored thousands of unit tests for CSS 2.1 (and now CSS 3.0) which uncovered that web-darlings like Gecko and WebKit had buggy and incomplete implementations (which they silently fixed in the meantime). Microsoft is doing a lot of the "boring" grunt-work which Apple, Google and Mozilla couldn't be arsed with obviously.
  • Reply 7 of 31
    Quote:
    Originally Posted by macadam212 View Post


    I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly?



    That's what IE9 is for.
  • Reply 8 of 31
    steve-jsteve-j Posts: 320member
    Quote:
    Originally Posted by macadam212 View Post


    I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly?



    Ar \



    Dance for us, Monkey Boy!
  • Reply 9 of 31
    jon tjon t Posts: 131member
    Quote:
    Originally Posted by macadam212 View Post


    I don't really understand Microsoft, if they want to support HTML5 why don't they do it properly? \



    Since when did Microsoft ever do anything 'properly'? It has always mislead people. It has always played dishonestly.



    This is just one example in a long long line...
  • Reply 10 of 31
    The oficial SVG test tested IE preview 4, August 2010. The W3C tested IE preview 6, October 2010. It is quite possible that IE improved a lot in the last 2 months, considering that Microsoft decided to ditch Silverlight and embrace HTML5.
  • Reply 11 of 31
    cnocbuicnocbui Posts: 3,613member
    Quote:
    Originally Posted by Jon T View Post


    Since when did Microsoft ever do anything 'properly'? It has always mislead people. It has always played dishonestly.



    This is just one example in a long long line...





    I remember when we all used Macs that ran on Power PC chips like the G3,4 and 5's. Ah, those were the days. Those Macs outperformed their PC counterparts by massive amounts. We know this to be true because Apple told us it was. Those chips had 'velocity engines' which made them much faster where it counts. Guess who told us that?



    We don't need devices that can render Flash on websites either.



    Oh yes, open standards are a good thing - Flash is bad because it isn't open - forget DLNA and use Apples AirPlay which is - er, closed - closed is the new open!!!, of course!
  • Reply 12 of 31
    gary54gary54 Posts: 169member
    Spin Doctoring. It's the All American Way



    *cough*
  • Reply 13 of 31
    swiftswift Posts: 436member
    This is just the way Microsoft always performs with standards bodies. Remember the MPEG 4 committee, which Microsoft sat on, and then disappeared and announced that they had a codec -- that didn't conform to the standard, wasn't compatible, and only played on Windows machines.
  • Reply 14 of 31
    Did Microsoft leak the test results to the public and call them "Official"?



    Its not really clear. If they DID NOT, then there is just another AI link bait freak show blog post.
  • Reply 15 of 31
    mstonemstone Posts: 11,510member
    Quote:
    Originally Posted by AppleInsider View Post


    The dramatic reversal of Microsoft's browser performance in the W3C's HTML5 test suite appears to align with the company's recently stated intent to switch gears in pushing HTML5 web standards over its own Silverlight proprietary web-replacement plugin, in large part an effort to reach Apple's installed base of iOS users who can't install Silverlight.



    How exactly does IE web standards compliance help reach Apple's installed base of iOS users? This makes no sense. Maybe I'm missing something.
  • Reply 16 of 31
    Quote:
    Originally Posted by Erunno View Post


    Yeah, right. Leaving out the biggest web browser vendor will surely do wonders for interoperability. Some people also would like to forget that it was Microsoft who sponsored thousands of unit tests for CSS 2.1 (and now CSS 3.0) which uncovered that web-darlings like Gecko and WebKit had buggy and incomplete implementations (which they silently fixed in the meantime). Microsoft is doing a lot of the "boring" grunt-work which Apple, Google and Mozilla couldn't be arsed with obviously.



    IE isn't the majority of browsers anymore, we finally have viable competition and innovation. But otherwise you are correct we need Microsoft input here as well.
  • Reply 17 of 31
    mstonemstone Posts: 11,510member
    Quote:
    Originally Posted by Project2501 View Post


    IE isn't the majority of browsers anymore, we finally have viable competition and innovation. But otherwise you are correct we need Microsoft input here as well.



    You did read the part on the page you referenced that stated:



    'W3Schools is a website for people with an interest for web technologies. These people are more interested in using alternative browsers than the average user. The average user tends to use Internet Explorer, since it comes preinstalled with Windows. Most do not seek out other browsers.



    These facts indicate that the browser figures above are not 100% realistic. Other web sites have statistics showing that Internet Explorer is used by at least 80% of the users.



    Anyway, our data, collected from W3Schools' log-files, over a five year period, clearly shows the long and medium-term trends'
    .
  • Reply 18 of 31
    mr. memr. me Posts: 3,221member
    Quote:
    Originally Posted by mstone View Post


    .... Other web sites have statistics showing that Internet Explorer is used by at least 80% of the users.



    ...



    Which other sites would those be?
  • Reply 19 of 31
    mstonemstone Posts: 11,510member
    Quote:
    Originally Posted by Mr. Me View Post


    Which other sites would those be?



    I have no idea. I just republished the statement that was on the w3schools site which I guess is their own disclaimer. Maybe Mircosoft's site has 80%.
  • Reply 20 of 31
    IE is pretty much struggling. Even many Windows users are switching to Firefox or Chrome.
Sign In or Register to comment.