Alleged iPhone X benchmarks pop up, blows away the Samsung Galaxy S8 in every regard

2»

Comments

  • Reply 21 of 35
    wizard69wizard69 Posts: 12,282member
    EngDev said:
    Repost (posted about this 20 minutes ago):

    Looks good, although, only a ~15% increase in single core performance is a bit disappointing.

    From my understanding Geekbench isn't the best to compare x86 and ARM, so don't go throwing out your Macbooks just yet. There's a reason websites like Anandtech don't use Geekbench.
    Using Geekbench to compare x86 to ARM is completely valid and as useful as any other benchmark.    You get numbers that are useful in certain contexts that requires a bit of intelligence from the person interpreting them.    In many industries it is very common to use benchmarks to compare entirely different architectures.   In the end you tailor the benchmarks to reflect your specific needs.   The real problem with Geekbench is that boiling the numbers for the individual tests down to one number isn't always the most rational approach for a user that has needs in a specific domain.   However you really don't have much of a choice when it comes to a general purpose benchmark aim at the less technical minded.
  • Reply 22 of 35
    wizard69wizard69 Posts: 12,282member

    EngDev said:

    Benchmarking an ARM processor in a phone and comparing the numbers to an x86 processor, like in the MacBook Pro, is perilous because of architectural differences and the optimizations between processor families. The more linear comparison is mobile to mobile -- and ideally on the same operating system.
    Kudos for highlighting that, all to often people assume their phone/tablet is more powerful than their laptop, when there are many other factors that would weigh in.

    It should also be noted that Geekbench will actually pause the benchmark to stop thermal throttling, so everything seen is mostly peak performance and may not be sustained in long term.

    Geekbench inserts a pause (or gap) between each workload to minimize the effect thermal issues have on workload performance. Without this gap, workload that appear later in the benchmark would have lower scores than workload that appear earlier in the benchmark.
    Source: Geekbench
    There aren't another factors to consider, if your cell phone runs a spreadsheet at the same speed you laptop does the performance is equivalent.    I'm not sure why people have the need to dismiss the obvious but ultimately the question becomes how fast is your code executed.   Beyond all of that you do realize that Intel processor regularly throttle in laptops, the big advantage they have though is fan cooling which can put off throttling.

    Now the other factors you may be alluding to might include the possibility of better GPU's or far more RAM.   However RAM is a know factor in performance even in the same model laptop.   The fact that you can get substantially better performance by adding RAM to a laptop doesn't change the fact that they same thing would be observed in a laptop running an ARM based chip.   

    As for Geekbenches imposed pauses, that is actually a very rational way to deal with testing a variety of apps/tasks that users might use on a machine.   If you are a spreadsheet user you don't want to smear the results by over heating the machine just prior to running the spreadsheet benchmarks.  The same thing goes for running a mail app or any other app because the general use case is to run those apps with the machine starting from an idle condition.

    In the end I really don't understand why people get bent out of shape with Geekbench, if the A11 can calculate a result just as fast as a laptop then the performance is equivalent.   It doesn't matter if that laptop is running an ARM based chip, an x86, a PowerPC, a Sparc, or even an old 68000 derivative.   If a stop watch says the task complete in the same amount of time then the performance is equivalent.
    watto_cobraanton zuykov
  • Reply 23 of 35
    melgrossmelgross Posts: 30,166member
    One problem with using commercial software for testing is the question of whether it accounts for special hardware features or not. While this may be ok when testing chips that are sold on the market, it’s not the same thing when comparing phones or tablets - if that company is named Apple.

    the reason is pretty simple. Apple doesn’t sell its chips. So, when testing for something that everyone uses for a test, does the app test using metal, or not? Does it test using any other special chips Apple makes, or not? This isn’t a simple problem.

    way back, when only integer processing was on chip, testing with floating point processors wasn’t done. So, the problem was that the chips that did have one, like the Motorola, didn’t get the advantage from the “standard” tests, as float was measured using regular CPU functions. Was that fair? Not really, but some argued that since most CPUs didn’t have it, it wasn’t fair to test using it. Made no sense to me, but we see that happening today as well.

    so, when Apple first came out with the A7, we saw early comparison testing done with 32 bit apps. That didn’t show most of the advantages of these chips. But some thought it was fair, because it was being compared to other SoCs which were 32 bit. Screwy.

    now Apple has the neural network in the SoC. How does that affect testing? If it can be used to advantage in normal tasks, other than for the specialized processing Apple is using it for (and just maybe for other tasks they didn’t mention) then should testing apps use that in ordinary calculations if the chip doesn’t do,that already, automatically? I think so.
    edited September 2017
  • Reply 24 of 35
    wizard69wizard69 Posts: 12,282member

    melgross said:
    There has been an earlier benchmark circulating. So this is the second.

    but the numbers seem strange. While the single core rating seems about right, though a bit less than I expected, the multi core number is very odd. The reason why the A10x has such good multiprocessing numbers is because it has three cores. And the multi core efficiency from these three cores is very good indeed. But here, we have an impossible situation unless what we’ve been reading about the A11 is wrong, which is possible. 

    According to the leak, and what’s being reported by several people, is that Apple is using an odd 2 high performance core and 4 high efficiency core design. It’s assumed that it works about the same as the A10, with the high and performance cores not working at the same time.

    if that’s true, there’s is simply no way that the multi core number can be equal to double the single core score. But here, it’s way greater than the single core score. So something is wrong. Either people are wrong, and Apple is using a 6 core 3+3 arrangement - same as the A10X, or, they’ve decided to copy the ARM Big/Little arrangement where all cores work when performance is needed. The latter is considered to be a problem, one that Apple avoided in their design.

    it will be interesting to see what’s true in a bit over an hour from now.
    You likely have already seen the news but they are now running all cores at the same time.   This is absolutely wonderful in my mind!    It means that background apps that run all the time, like Mail and probably some network services, will never interfere with the performance you get out of the main core.  Then the other thing here is the ability to run may more threads in an app.    Imagine an IDE such a Playground on the iPad running on one of these chips.    In fact I see this A11 as being huge for the next iPad upgrade, makes me giddy in fact.

    Now what seems to have been completely glossed over by Apple and even the media is the new Apple GPU.   This is where I really want to see benchmarks.   I'm shocked that Apple didn't spend more time on this very important development.   It really seems odd the lack of focus on the SoC in general as this chip is significant in delivering all the goods contained in the new machines and OS.    Not even a mouse peep about the GPU at this point.    Even more so nothing about the Neural engine and how it is integrated with the GPU/CPU complexes.    I had this one possibility in mind the the Neural engine was an adjunct to the GPU but we sure as hell wouldn't know form this debut show.   
  • Reply 25 of 35
    Samsung S8 isn't even the fastest Android device.  There are some phones using the SD821 SOC that is almost as fast.  Goes to show that Samsung bloat does impact performance.  OnePlus 5 is currently the fastest but they've cheated the bench by overclocking when benchmark apps run.  

    I don't think it's important at all because A11 will never run Android and iOS will never run on Snapdragon so it's like comparing Apples to Oranges.

    We should be comparing Kirin to Snapdragon.
    watto_cobra
  • Reply 26 of 35
    melgrossmelgross Posts: 30,166member
    wizard69 said:

    melgross said:
    There has been an earlier benchmark circulating. So this is the second.

    but the numbers seem strange. While the single core rating seems about right, though a bit less than I expected, the multi core number is very odd. The reason why the A10x has such good multiprocessing numbers is because it has three cores. And the multi core efficiency from these three cores is very good indeed. But here, we have an impossible situation unless what we’ve been reading about the A11 is wrong, which is possible. 

    According to the leak, and what’s being reported by several people, is that Apple is using an odd 2 high performance core and 4 high efficiency core design. It’s assumed that it works about the same as the A10, with the high and performance cores not working at the same time.

    if that’s true, there’s is simply no way that the multi core number can be equal to double the single core score. But here, it’s way greater than the single core score. So something is wrong. Either people are wrong, and Apple is using a 6 core 3+3 arrangement - same as the A10X, or, they’ve decided to copy the ARM Big/Little arrangement where all cores work when performance is needed. The latter is considered to be a problem, one that Apple avoided in their design.

    it will be interesting to see what’s true in a bit over an hour from now.
    You likely have already seen the news but they are now running all cores at the same time.   This is absolutely wonderful in my mind!    It means that background apps that run all the time, like Mail and probably some network services, will never interfere with the performance you get out of the main core.  Then the other thing here is the ability to run may more threads in an app.    Imagine an IDE such a Playground on the iPad running on one of these chips.    In fact I see this A11 as being huge for the next iPad upgrade, makes me giddy in fact.

    Now what seems to have been completely glossed over by Apple and even the media is the new Apple GPU.   This is where I really want to see benchmarks.   I'm shocked that Apple didn't spend more time on this very important development.   It really seems odd the lack of focus on the SoC in general as this chip is significant in delivering all the goods contained in the new machines and OS.    Not even a mouse peep about the GPU at this point.    Even more so nothing about the Neural engine and how it is integrated with the GPU/CPU complexes.    I had this one possibility in mind the the Neural engine was an adjunct to the GPU but we sure as hell wouldn't know form this debut show.   
    Yes, I watched the presentation. I hope they do this better than the big/little results I’ve seen in tests on the Android platform. I’d love to know the difference in instruction sets between the performance and the efficiency cores. That’s the biggest problem with the way it’s done over there. The problem is that while it wasn’t locked out, the way Apple did it last year, it wasn’t intended to be used together. Samsung started that so they could advertise “8 cores”, without saying what was what.

    the small cores are lacking some instructions, and run at a lower speed, so everything isn’t equal. It’s harder to divide up the instructions because of that. The scheduler needs to be more complex and therefor, larger.

    so the question is what Apple did here.

    while i was somewhat disappointed at the 25% improvement in CPU performance, as I expected maybe 40% (so much for DEDs insistence that Apple improves these things by 100% each year), I was,more disappointed in the 30% improvement in the GPU. But still, it’s an interesting development. Apple uses 6 cores of Imagination IP. But here, they’re using just 3 cores. So each core is about 110% faster. That very good, unless each core is double the size of the older cores. Then it’s a wash. I was expecting 50% improvement, maybe more.

    as for the neural portion of the SoC, and in June we were led to think this was a stand alone chip, I’m with you. I would have lived to hear more about it. But this is a product intro with the news media there, and likely not the place for technical explanations. The 600 billion calculations per second was something, at least. I wonder how that compares with the vastly more expensive chips coming out of Google and Nvidia, which are intended for major installations (they cost thousands!). Hopefully, they will have a place on their site that explains this more fully. Or will produce a paper that will be read during the next AI conference, since their efforts have come out of the closet.
  • Reply 27 of 35
    melgrossmelgross Posts: 30,166member

    vision33r said:
    Samsung S8 isn't even the fastest Android device.  There are some phones using the SD821 SOC that is almost as fast.  Goes to show that Samsung bloat does impact performance.  OnePlus 5 is currently the fastest but they've cheated the bench by overclocking when benchmark apps run.  

    I don't think it's important at all because A11 will never run Android and iOS will never run on Snapdragon so it's like comparing Apples to Oranges.

    We should be comparing Kirin to Snapdragon.
    Frankly, I don’t care about how any of those chips do, except when a friend waves one of those phone models in front of my face, and I have to patiently that, no, they aren’t faster than my iPhone, and no, they aren’t more technically advanced despite the features which hardly work.
    watto_cobraanton zuykovjony0
  • Reply 28 of 35
    In addition to the weird clock speed, the cache sizes are very implausible. The last Apple processor to have 32k of L1 instruction or data cache was the A6. The A9 and A10 have 3 MB of L2, but this claims 8 MB. A7 through A10 have all had 4 MB of L3, and this says no L3. I think it's safe to assume for now that this is bogus.
  • Reply 29 of 35
    melgrossmelgross Posts: 30,166member
    zimmie said:
    In addition to the weird clock speed, the cache sizes are very implausible. The last Apple processor to have 32k of L1 instruction or data cache was the A6. The A9 and A10 have 3 MB of L2, but this claims 8 MB. A7 through A10 have all had 4 MB of L3, and this says no L3. I think it's safe to assume for now that this is bogus.
    I wouldn’t go that far. A lot of CPUs are now going the large L2, but no L3 route. That’s not enough reason to dismiss this.
  • Reply 30 of 35
    melgross said:
    now Apple has the neural network in the SoC. How does that affect testing?
    Don't get tangled in irrelevant details. It is like saying that you can't compare iOS 9, iOS 10 and iOS 11 performance, because they all MIGHT have slightly different scheduling procedures for the processes. The reality is - if they do, then that becomes part of the performance testing, therefore it SHOULD be measured. If they don't, then why are we even talking about it?

    Where did this notion come from, that A chip needs to be tested separately from the software that runs on it? That makes no sense, as lot of choices that software engineers make while programming something, do affect performance even if the chip is still the same, being it some OS level stuff, or normal user level stuff?
    edited September 2017
  • Reply 31 of 35
    melgross said:
    zimmie said:
    In addition to the weird clock speed, the cache sizes are very implausible. The last Apple processor to have 32k of L1 instruction or data cache was the A6. The A9 and A10 have 3 MB of L2, but this claims 8 MB. A7 through A10 have all had 4 MB of L3, and this says no L3. I think it's safe to assume for now that this is bogus.
    I wouldn’t go that far. A lot of CPUs are now going the large L2, but no L3 route. That’s not enough reason to dismiss this.
    Which makes sense, since ever next layer costs in terms of time it takes to search something in L2, get a miss, and then go to L3 and do the same operation on it, albeit at a slower speed.
  • Reply 32 of 35
    melgrossmelgross Posts: 30,166member
    melgross said:
    now Apple has the neural network in the SoC. How does that affect testing?
    Don't get tangled in irrelevant details. It is like saying that you can't compare iOS 9, iOS 10 and iOS 11 performance, because they all MIGHT have slightly different scheduling procedures for the processes. The reality is - if they do, then that becomes part of the performance testing, therefore it SHOULD be measured. If they don't, then why are we even talking about it?

    Where did this notion come from, that A chip needs to be tested separately from the software that runs on it? That makes no sense, as lot of choices that software engineers make while programming something, do affect performance even if the chip is still the same, being it some OS level stuff, or normal user level stuff?
    It’s not irrelevant. Just because it’s there doesn’t mean the test apps can’t disable it when testing. You don’t know.
  • Reply 33 of 35
    Check out Tom's Guide evaluation of S8 vs iPhone 7  that appears more like a comedy routine. They decide to eschew benchmarks for 'real world' tests of their own devising. Things that we all do every day like opening 15 apps at once as fast as we can. Of course the iP7 did that 4 seconds quicker, but who give a damn anyway. What is most hilarious is that they run it again and even though there a dozens of these type of tests on youtube where the iPhone surges ahead on the second run due to better memory management, they manage to somehow, without filming it get the iPhone to run slower on the second run and take an average. Notwithstanding that this is pure bullshit, even if we allow them to claim a 9 second 'victory' in opening 15 apps we're only talking about 0.7secs per app but it's moot anyway because it's obviously faked.

    I love the final one where it's so close they have to film it in slow motion to try and pretend like it means anything. The long and short of it is that they pronounce the S8 to be the winner here because it got a score of three completely irrelevant 'wins' to the iPhone's single solitary measly little test where it bested their beloved S8.

    And what was this insignificant little test, a very very genuine real world test where a 4K video clip is rendered in two separate apps down to 720p. The score? 1.5 minutes for the iPhone7 vs 9 minutes for the Galaxy. Nine frikken minutes. That's insane, must be all those octocores getting in each others way. Then they laughably speculate that this is likely that the apps weren't optimised for the Snapdragon. Maybe these benchmarks that they say are not 'real world' at the beginning of the video really do mean something. They are like Laurel and Hardy of digital technology showdowns.

    The point is the the iPhone 7 completely destroys, the S8 in a render test so are we to be surprised that the iPhone X never mind the iPhone 8 also does?

    What I find surprising is that in reviews of the Note 8 they never mention that it cannot to 60fps at 4k nor to they mention that it is unable to do 240fps slo mo greater than 720p and that is a big deal. Hell my iPod touch can to 720 at 120fps. Yet they do not hesitate to mention that the Galaxies have 8 cores and more RAM. 

    edited September 2017 tallest skil
  • Reply 34 of 35
    melgross said:
    melgross said:
    now Apple has the neural network in the SoC. How does that affect testing?
    Don't get tangled in irrelevant details. It is like saying that you can't compare iOS 9, iOS 10 and iOS 11 performance, because they all MIGHT have slightly different scheduling procedures for the processes. The reality is - if they do, then that becomes part of the performance testing, therefore it SHOULD be measured. If they don't, then why are we even talking about it?

    Where did this notion come from, that A chip needs to be tested separately from the software that runs on it? That makes no sense, as lot of choices that software engineers make while programming something, do affect performance even if the chip is still the same, being it some OS level stuff, or normal user level stuff?
    It’s not irrelevant. Just because it’s there doesn’t mean the test apps can’t disable it when testing. You don’t know.
    What makes you think that it is not disabled by default? Or are you suggesting that Apple will attempt to use neural engine to compute some arithmetical operations if you do not specify otherwise? That is not how they do things? There is going to be some kind of dedicated set of APIs that let you use that engine.
  • Reply 35 of 35
    melgrossmelgross Posts: 30,166member
    melgross said:
    melgross said:
    now Apple has the neural network in the SoC. How does that affect testing?
    Don't get tangled in irrelevant details. It is like saying that you can't compare iOS 9, iOS 10 and iOS 11 performance, because they all MIGHT have slightly different scheduling procedures for the processes. The reality is - if they do, then that becomes part of the performance testing, therefore it SHOULD be measured. If they don't, then why are we even talking about it?

    Where did this notion come from, that A chip needs to be tested separately from the software that runs on it? That makes no sense, as lot of choices that software engineers make while programming something, do affect performance even if the chip is still the same, being it some OS level stuff, or normal user level stuff?
    It’s not irrelevant. Just because it’s there doesn’t mean the test apps can’t disable it when testing. You don’t know.
    What makes you think that it is not disabled by default? Or are you suggesting that Apple will attempt to use neural engine to compute some arithmetical operations if you do not specify otherwise? That is not how they do things? There is going to be some kind of dedicated set of APIs that let you use that engine.
    Why would it be? Of course Apple has APIs. Neural chips can do a number of things. I doubt Apple would develop a chip that sits idle most of the time. If it would benefit work, Apple will use it. They’re not stupid. They aren’t going to just use it for a small subset of the work the chip does if it can do other things as well.

    you need to remember that it was Apple, way back that first used the gpu for functions other than graphics. It was they that went to the graphics chip companies and asked for more programmable chips. That started everything we see now with graphics cards doing bitcoin mining. That never would have been possible if Apple didn’t start that.

    so don’t state that that’s not how Apple works. You don’t know how Apple works, apparently.
Sign In or Register to comment.