First Apple Silicon Developer Transition Kit benchmarks show Rosetta performance impact

2

Comments

  • Reply 21 of 48
    revenantrevenant Posts: 621member
    seems highly unlikely that apple would show their hand and ship out the zippiest, or even a very zippy device. and I wonder, as these seem to have identifiers, might these developers lose their account as these are not supposed to be benchmarked?
    jony0anonconformistBeatswatto_cobra
  • Reply 22 of 48
    blastdoorblastdoor Posts: 3,570member
    If Geekbench is running through Rosetta then these numbers are actually pretty impressive. 800/1100 = 72% of full performance when running emulated code? That's really impressive. 
    jony0watto_cobra
  • Reply 23 of 48
    revenantrevenant Posts: 621member
    "So the DTK with a two year old iPad chip runs x86_64 code, in emulation, faster than the Surface Pro X runs it natively 😅 Oh boy Qualcomm, what are you even doing?"https://t.co/UAlZiwSsF8 — Steve Troughton-Smith (@stroughtonsmith) ;June 29, 2020
    Fidonet127chiajony0mcdaveBeatswatto_cobra
  • Reply 24 of 48
    hattighattig Posts: 860member
    blastdoor said:
    If Geekbench is running through Rosetta then these numbers are actually pretty impressive. 800/1100 = 72% of full performance when running emulated code? That's really impressive. 
    That's right.

    Also consider that Geekbench is a benchmark - it's a pure x86-64 kernel running in isolation, so it's 100% worst case scenario.

    Most applications will call out to the system to do even the simplest of things - which is pure native.
    rundhvidBeatswatto_cobra
  • Reply 25 of 48
    mr lizardmr lizard Posts: 354member
    tundraboy said:
    We should take these benchmark results seriously because Apple is stupid enough to release a new product that performs worse than the product it replaces.

    /s
    If you want. But then you are making a very conscious and deliberate choice to take seriously a set of benchmarks from a device which:
    - Apple has explicitly stated is not a shipping consumer product 
    - Is currently relying on a translation layer in order to run the benchmark software 
    - Is running an under-clocked processor designed for use in a fan-less battery-powered tablet 

    So you can choose to ignore relevant information if you want, but that’s a few hoops you have to jump
    through to get there. 
    rundhvidwatto_cobra
  • Reply 26 of 48
    razmatazrazmataz Posts: 26member
    lkrupp said:
    macbootx said:
    Developer: “Hey! I gots a good idea. Let’s benchmark this new Apple Silicon before we recompile a single line of code!” 

    ....SMH 

    Websites are probably paying developers to ignore their NDAs. I hope Apple really cracks down on the offenders but unlikely. 

    Be careful what you wish for. Given your comment, how far should this crack down go?

    And what are your thoughts on this website (Apple-In-Cider) which has a very high rate of rumors, leaks, and gladly reports said benchmarks?

    What is being reported here is pretty innocuous and I doubt it'll harm future sales. In fact the results are impressive: not a single issue being reported, straight to benchmarks. This project seems well on its way.

     
     
    watto_cobra
  • Reply 27 of 48
    This is like Snazzy Labs looking at the first developer (non public) release of macOS 11 and nit-picking UI elements. But no, click sitters would never ever do something like that. /s
    watto_cobra
  • Reply 28 of 48
    zimmiezimmie Posts: 651member
    hattig said:
    zimmie said:
    VirtualApple is 100% a Rosetta thing. The reported processor speed is probably just a lie to software to give it an expectation of performance level.
    It's a thin VM/system layer around the translated application, because not everything can be translated. It will trap the issues to do them properly.
    That has nothing to do with what I said.

    I said the processor model name "VirtualApple" is a Rosetta thing. This benchmark is clearly running emulated, no doubt about it.
    hattig said:
    zimmie said:
    VirtualApple is 100% a Rosetta thing. The reported processor speed is probably just a lie to software to give it an expectation of performance level.
    I suspect the clock speed is gained from a syscontrol call so is valid.
    You get the processor model and the processor speed in the same way, frequently at the same time. It's clearly lying about the model, the number of cores, and the features the processor supports, so what makes you think it wouldn't also lie about the speed of the fabricated processor model it is presenting?
    jony0watto_cobra
  • Reply 29 of 48
    mjtomlinmjtomlin Posts: 2,690member
    chaicka said:
    Wondering what if the production A-based Macs coming end of the year actually ship with an A14Z chip instead (since A14 is going to be in this year's iPhone 12s), and that chip includes plentiful of specific-advancements meant for Rosetta 2 and Big Sur as compared to the A12Z in DTK?

     :D 
    netrox said:
    A12Z is literally two generations old already, it's just a binned chip. I imagine the performance will be amazing with A14Z. 



    Macs will not use any A-series SoC. They already mentioned they created a new family of SoCs just for the Mac. While they might share a lot of the same supportive processors; Neural Engine, ISP, DSP, etc... There's a lot that would/could be different. Anything from CPU cores, to GPU, to data bus, I/O controllers (Thunderbolt), etc.

    The A-series SoCs are designed more for efficiency than performance which is necessary for mobile devices (smaller footprint = smaller battery). Anything they design for Macs (and TV and HomePod), is not usually constrained by power, other than laptops, of course.

    Given all of that, I believe they will eventually have a family of 4 distinct Mac SoCs: laptop, desktop, Pro laptop, Pro desktop. And each of these may come in multiple variations; clock speed and core count.
    watto_cobra
  • Reply 30 of 48
    lkrupplkrupp Posts: 10,557member
    razmataz said:
    lkrupp said:
    macbootx said:
    Developer: “Hey! I gots a good idea. Let’s benchmark this new Apple Silicon before we recompile a single line of code!” 

    ....SMH 

    Websites are probably paying developers to ignore their NDAs. I hope Apple really cracks down on the offenders but unlikely. 

    Be careful what you wish for. Given your comment, how far should this crack down go?

    And what are your thoughts on this website (Apple-In-Cider) which has a very high rate of rumors, leaks, and gladly reports said benchmarks?

    What is being reported here is pretty innocuous and I doubt it'll harm future sales. In fact the results are impressive: not a single issue being reported, straight to benchmarks. This project seems well on its way.

     
     
    Well, you are right about one thing, consumers and professionals pay absolutely no attention to what tech forum gadflies say about anything. So yes, this is pretty innocuous. But certain websites like Bloomberg and Gizmodo will run with these benchmarks to imply the new Macs are slower than Intel Macs. Watch for it.
    jony0Beatswatto_cobra
  • Reply 31 of 48
    mdriftmeyermdriftmeyer Posts: 7,503member
    It'll get worse once the lockdown system is opened back up and regular OS X is running full processes/threads. This has a long way to go; hence the two year window.
  • Reply 32 of 48
    mdriftmeyermdriftmeyer Posts: 7,503member
    rcfa said:
    polymnia said:
    I wonder: is it possible that the upcoming Mac-specific Apple Silicon chips might include features specifically designed to accelerate Rosetta 2?

    They can design anything they like now. Maybe smoothing the transition by accelerating Rosetta 2 would be a good move. Perhaps just for the first couple generations. They could drop it once MacOS drops support for Rosetta 2. 
    Whatever would help Rosetta 2 would very likely also help a variety of virtualization solutions. As such, if they were to make such additions to the ARM instruction set, there would be very little incentive to ever remove them again.
    i would however expect Apple to tread very carefully with extending the ARM programming model, particularly in an unilateral move.
    Because it won't ever happen as they don't own the IP.
  • Reply 33 of 48
    It'll get worse once the lockdown system is opened back up and regular OS X is running full processes/threads. This has a long way to go; hence the two year window.
    It has full Metal 2 support, audio support, Gigabit Ethernet, USB 3.0, USB 3.1, APFS, macOS' Window Manager (and so far, everything else macOS supports, except Thunderbolt 3 (which is of course, coming). Plus it is running Rosetta 2.

    So what, pray tell, is "Locked-down" or is not "running full processes/threads"?
    watto_cobra
  • Reply 34 of 48
    rcfa said:
    polymnia said:
    I wonder: is it possible that the upcoming Mac-specific Apple Silicon chips might include features specifically designed to accelerate Rosetta 2?

    They can design anything they like now. Maybe smoothing the transition by accelerating Rosetta 2 would be a good move. Perhaps just for the first couple generations. They could drop it once MacOS drops support for Rosetta 2. 
    Whatever would help Rosetta 2 would very likely also help a variety of virtualization solutions. As such, if they were to make such additions to the ARM instruction set, there would be very little incentive to ever remove them again.
    i would however expect Apple to tread very carefully with extending the ARM programming model, particularly in an unilateral move.
    Because it won't ever happen as they don't own the IP.
    Apple has an "Architectural License" for arm. They can do about anything they want.

    I'm not so sure that they couldn't even resell their Ax series SoCs to other OEMs (or even at retail), if they were so-inclined.
    watto_cobra
  • Reply 35 of 48
    EsquireCatsEsquireCats Posts: 1,268member
    While interesting, there are no conclusions to be drawn about future Apple Silicon products - not even the iPad will still be using this chip.
    It’s entirely plausible that “Apple Silicon” will be more than just a rebranded A-series chip. 
    watto_cobra
  • Reply 36 of 48
    tundraboytundraboy Posts: 1,911member
    macbootx said:
    tundraboy said:
    We should take these benchmark results seriously because Apple is stupid enough to release a new product that performs worse than the product it replaces.

    /s
    But this isn’t a new product. It’s a testing and development platform. 
    Whoosh!  Here that? That's the sound of missing the whole point. :-)
  • Reply 37 of 48
    sflocalsflocal Posts: 6,133member
    netrox said:
    A12Z is literally two generations old already, it's just a binned chip. I imagine the performance will be amazing with A14Z. 


    Since the iMac will not be constrained by batteries, I’m sure Apple has a beefy piece a silicon to use in those models.  Pretty exciting stuff! 
    watto_cobra
  • Reply 38 of 48
    revenant said:
    seems highly unlikely that apple would show their hand and ship out the zippiest, or even a very zippy device. and I wonder, as these seem to have identifiers, might these developers lose their account as these are not supposed to be benchmarked?
    I was looking at that long string and thinking Apple has them all carefully recorded.

    I’d never post a benchmark like this, unless I determined I didn’t care about ever being able to work again, as that’s just asking to have your name be blacklisted.  If only because of that, but really, there’s more: it’s a matter of integrity, as once you’ve done something like this, you have clearly lost it.
    revenantwatto_cobra
  • Reply 39 of 48
    jdb8167jdb8167 Posts: 627member
    hattig said:
    jdb8167 said:
    I’m pretty sure Rosetta only uses the “big” cores which is why these results show up as a 4 core CPU and not 8. You also can’t rely on the reported clock speed. These results are from a kit that isn’t supposed to be benchmarked running an x86-64 translation to Aarch64. They shouldn’t be used to infer anything about native ARM performance. 

    It’s also interesting that the date of the tests were from before WWDC which likely means they came from a developer with very early access. 
    Rosetta is a translator, rather than a runtime emulation. Even the JIT feature will be implemented by translating finalised Intel Code into ARM (when you mark a memory page as executable post-JIT, Rosetta kicks in and does its work - I guess). **HOWEVER** there will be some things that need to be trapped as they are not emulateable, and therefore I suspect Rosetta provides a very thin VM/system layer around the executable. Also binary translators really cannot infer intent, so there is only so much they can do - they will generate code that is less optimal, especially when coming from x86 with all its quirks.

    Yes, these are likely in-Apple or in-close-third-party leaks.
    It looks like Apple Insider got the dates on the Geekbench entries wrong. They are from June 29th not June 20th. So it does look like they from a released DTK. (And someone might be getting a note from Apple very soon since this is expressly prohibited by the NDA.)
    watto_cobra
  • Reply 40 of 48
    jdb8167jdb8167 Posts: 627member

    hattig said:
    zimmie said:
    VirtualApple is 100% a Rosetta thing. The reported processor speed is probably just a lie to software to give it an expectation of performance level.
    It's a thin VM/system layer around the translated application, because not everything can be translated. It will trap the issues to do them properly.
    I suspect the clock speed is gained from a syscontrol call so is valid.
    I’ve seen plenty of examples where the recorded clock speed in a Geekbench entry was wrong. So I will take the clock speed with a bit of skepticism. On the other hand the A12Z CPUs may be actually 2.4 GHz parts that are binned parts rejected for actual iPad Pros. 
    thtwatto_cobra
Sign In or Register to comment.