Tested: Thermal conditions in the 2018 i9 MacBook Pro dramatically hampering performance

123578

Comments

  • Reply 81 of 158
    friedmudfriedmud Posts: 149member
    BTW - as a way of comparison...  Here's a quick run of our custom scientific simulation software using 4 cores in my 2017 MBP that with a 3.1GHz i7-7920HQ that can turbo to 4.1 GHz:



    As you can see it looks like it's thermal throttling - but it's actually not.  Those dips in frequency actually correspond to dips in the instruction intensity of our software.  This is software that gets run on the largets computers in the world (literally... later this year we're planning a run that uses all of the new Summit supercomputer at Oak Ridge National Laboratory... which is currently THE fastest computer in the world (according to Top500 anyway))... so it has been incredibly optimized... but even it still has dips and stalls that allow the processor to clock down slightly.

    I would expect something like a rendering program to have even more... it probably has to do plenty of file i/o... at least writing out the frames!  Every time it goes to the filesystem the CPU frequency will drop because the processors don't have enough to do.  This will average out to something less than spectacular looking.
  • Reply 82 of 158
    People surprised about this should get a desktop, like a Mac Pro. This is nothing new. The thinner the body , the more likely the computer will throttle,to maintain temperatures. I suspect there would be even more complaints , if fans on MacBooks made more sound.

    The issue is that the i9 MBP isn't performing as advertised in these tests.  That's a completely different issue from a user choosing the right tool for the job.
    pascal007
  • Reply 83 of 158
    foljsfoljs Posts: 285member
    ahobbit said:
    The real question is, how could this have gotten past Apple QC?
    It's irrelevant for 99% of their customers.
  • Reply 84 of 158
    Mike WuertheleMike Wuerthele Posts: 3,108administrator
    friedmud said:
    For those anoyed with heating issue, just disable Turbo Boost. You can do it using another app like Turbo Boost Switcher or so. On redding there are even some posts reporting better performance with Turbo Boost disabled on certain tasks due to this issue.
    We started looking into this last night. With any luck, we'll get some numbers today on it.
    Mike: can you guys please try my simple program above?  We need more controlled testing on this than hitting "Run" in a rendering application.

    This is important to me because my team is getting ready to buy 5 of these new machines.  We do massively parallel scientific computing and use every single ounce of capability out of our laptops workstations... so if this thing really is thermal throttling then I need to know about it!
    I'll put it on the queue, but there are about a dozen things ahead of it.

    Regarding the I/O, the I/O is controlled by the T2, so there is zero overhead for the CPU. Couple that with the speed of the SSD, and there's still no time for the CPU to slow.
    edited July 19
  • Reply 85 of 158
    croprcropr Posts: 815member
    friedmud said:
    BTW - as a way of comparison...  Here's a quick run of our custom scientific simulation software using 4 cores in my 2017 MBP that with a 3.1GHz i7-7920HQ that can turbo to 4.1 GHz:



    As you can see it looks like it's thermal throttling - but it's actually not.  Those dips in frequency actually correspond to dips in the instruction intensity of our software.  This is software that gets run on the largets computers in the world (literally... later this year we're planning a run that uses all of the new Summit supercomputer at Oak Ridge National Laboratory... which is currently THE fastest computer in the world (according to Top500 anyway))... so it has been incredibly optimized... but even it still has dips and stalls that allow the processor to clock down slightly.

    I would expect something like a rendering program to have even more... it probably has to do plenty of file i/o... at least writing out the frames!  Every time it goes to the filesystem the CPU frequency will drop because the processors don't have enough to do.  This will average out to something less than spectacular looking.
    Of course the application being used has a serious impact.  Video rendering is a CPU bound task.  There is some i/o, but is absolutely peanuts compared to the CPU task, especially with such a fast SSD.   If you check the CPU usage during a video rendering you will notice it immediately
  • Reply 86 of 158
    dewmedewme Posts: 1,488member
    I've simply become distrustful of all forms of "Up To" marketing whether it's on computer performance claims, automobile trade-in values, gas mileage, etc. The advertised boost frequencies of Intel chips look nice on paper, greatly overshadowing the base frequency, and disproportionally tickling the impulse to purchase. But common sense and some basic appreciation for thermodynamics tells you that cramming a beefy 6-core current generation processor into a very restricted enclosure with only small fans and conductive heat sinks/pipes for cooling is going to make those boost numbers a very rare sighting. 
    cgWerks
  • Reply 87 of 158
    seankillseankill Posts: 395member
    frantisek said:
    DuhSesame said:
    ahobbit said:
    The real question is, how could this have gotten past Apple QC?
    if simple tests as the ones done by Lee, AI and others reveal a potential throttling issue where the i9 is unable to maintain the advertised base frequency, even ending up slower in some cases than last year's i7 model, how could Apple not have known about this?

    Does it mean, Apple cares more about specs on paper than real world performance for pro users?

    Didn't Apple bring in-house many pro users to have them help design the next Mac Pro?
    Why were these people not involved in the MacBook Pro i9 testing?  Wouldn't that seem obvious?
    A wasted opportunity?  
    A waste of in-house resources?

    Apple more and more seems poorly managed...
    For all the money and resources they have, they should produce better results - if it is true that they still care about pro users, as they claim.

    Either they don't actually care as much as they claim - or cannot do any better than that.
    Both are very troubling if you are a pro user.
    Again, short-time bursts.  i9 will make things like opening applications or other small actions faster, but not long-term computing.

    The only thing that can provide maximum turbo boost for an i9 can only be those huge gaming laptops.
    Apple got trapped or being visctim of customers desire for powerfull MBP with 32 GB of memory. As is written above, some will get burst benefits and even some benefits from 2 extra cores but for long term benefits it will need eGPU I guess.I can offload some but not all CPU tasks.
    Lord. “Trapped”? Apple apologist much? It amazes me the “Apple can do no wrong” on these forms. 

    Apple has painted themselves in a corner here. The laptop is ultra thin, looks good in presentations; however, the reduced volume kills the room for adequate cooling on the i9. It’s a work machine, how thin do you want it? Thank god Apple doesn’t make trucks, they would be basically cars. Where do you get that the RAM has any tangible effect on this issue? 

    Is it it too much for Apple for users to expect power from a “Pro” laptop? If there were thermal issues, just don’t include the i9. Or, I have a crazy idea (in Apple buildings anyway) take the 2015 size chassis and use the extra room for cooling and battery? There will even be room for more ports. It wouldn’t be the first time Apple fattened up a device, look at the iPhone thicknesses over the years. 

    While Intel gets the blame for the processor itself, you can not pass the blame of thermal issues to Apple. Their engineers are supposed to catch any issues like this and design the issues out or design around them. But I suspect the engineers have become creative designers, they just want it to look good in presentations. 

    Hopefully, there are some ways to fix this issue with software. 
    Going to steer clear of Macs for the foreseeable future. Was really considering an upgrade with the i9 and 32gb. Hopefully my 2012 keeps rolling. 
    cgWerkssingularity
  • Reply 88 of 158
    friedmud said:
    BTW - as a way of comparison...  Here's a quick run of our custom scientific simulation software using 4 cores in my 2017 MBP that with a 3.1GHz i7-7920HQ that can turbo to 4.1 GHz:



    As you can see it looks like it's thermal throttling - but it's actually not.  Those dips in frequency actually correspond to dips in the instruction intensity of our software.  This is software that gets run on the largets computers in the world (literally... later this year we're planning a run that uses all of the new Summit supercomputer at Oak Ridge National Laboratory... which is currently THE fastest computer in the world (according to Top500 anyway))... so it has been incredibly optimized... but even it still has dips and stalls that allow the processor to clock down slightly.

    I would expect something like a rendering program to have even more... it probably has to do plenty of file i/o... at least writing out the frames!  Every time it goes to the filesystem the CPU frequency will drop because the processors don't have enough to do.  This will average out to something less than spectacular looking.
    If it was only the CPU taking a breather, as you say, the task would still be performed faster. Yet, as Lee showed, the task is performed faster (or as fast) using a i7 CPU compared to the i9, so there is a real performance problem when heat-stressed.
  • Reply 89 of 158
    "Once a liar, always a liar" Remember thousand color monitor scandal. Same company, different lie. I am done with Apple. (Although sadly i don't see a better alternative) Sad.
  • Reply 90 of 158
    friedmudfriedmud Posts: 149member
    This is why we used Cinebench. We know that there are no breaks in instruction intensity.

    EDITED: Mike, something was messed up with my view of the forum and I now saw your reply.  Thanks for saying you'll try the simple program.

    Mike - you definitely don't know that - it takes only the tiniest amount of file i/o or thread locking to bring the intensity down to the point where the CPU frequency will come down.

    Here is another example.  Try this one on the i9 as well:

    #include <iostream>
    #include <thread>

    int main()
    {
      double stuff = 1.2;
      int junk = 2;

      unsigned long int iteration = 1;

      while (true)
      {
        stuff *= 1.3;

        junk *= 2;

        if (!(iteration % 1000000000))
        {
          std::cout << "Stalling" <<std::endl;
          std::this_thread::sleep_for(std::chrono::milliseconds(1000));
        }

        iteration++;
      }

      std::cout << stuff << std::endl;
      std::cout << junk << std::endl;

      return 0;
    }

    (compile it with "clang++ -std=c++11 thermal_test.C -o thermal_test")

    Every few seconds it will stall... and the CPU frequency will drop.  Here is what it looks like running on a single core of my MBP:
    If you adjust the milliseconds to make it smaller - the dips will be smaller.  If you adjust the huge number in the if statement to make it smaller it will stall more often and you won't ever reach anywhere close to the max frequency... even if the stalls are tiny.

    These are simple little programs that you can quickly compile and try and then we don't have to wonder what's happening anymore.

    edited July 19
  • Reply 91 of 158
    The overall machine performs much better than the predecessors right?  Enough so to provide a compelling reason for those who need the power to upgrade.  Is the thermal design limiting the potential of the chip?  Seems to be so.  I am just wondering if Apple was looking at what the entire laptop as a whole can bring to the user versus other models...
    andrewj5790cgWerks
  • Reply 92 of 158
    friedmudfriedmud Posts: 149member
    pascal007 said:
    If it was only the CPU taking a breather, as you say, the task would still be performed faster. Yet, as Lee showed, the task is performed faster (or as fast) using a i7 CPU compared to the i9, so there is a real performance problem when heat-stressed.

    If the program is not 100% instruction intensive then a faster processor won't help much.  The speedup will be somewhat limited by whatever is limiting the instruction intensity.  For instance, if there is thread-locking (thread-contention for a shared resource) going on then a bump in MHz will only make you get to the thread lock faster.  That will give you a small speedup - but you'll only get a percentage of the speedup you should because the processors are still waiting.

    Regarding the I/O, the I/O is controlled by the T2, so there is zero overhead for the CPU. Couple that with the speed of the SSD, and there's still no time for the CPU to slow.

    No such thing as zero overhead for i/o.  If the program has to wait for the file to be completely written (flushed) at any point... then the processor will stall.  The T2 and SSD will help minimize things... but processors work on microseconds.  Even just the very slightest stall in the instruction pipeline will bring down the frequency.  Try out that second program I just posted above and play with the parameters to see the impact of even a tiny stall.
    dewme
  • Reply 93 of 158
    Mike WuertheleMike Wuerthele Posts: 3,108administrator
    The overall machine performs much better than the predecessors right?  Enough so to provide a compelling reason for those who need the power to upgrade.  Is the thermal design limiting the potential of the chip?  Seems to be so.  I am just wondering if Apple was looking at what the entire laptop as a whole can bring to the user versus other models...
    1) It performs better, yes. Not much better.
    2) It depends at what price premium you put a small increase in performance.
    3) The thermal design and parameters for controlling the thermal situation do seem to be limiting the potential of the chip.
  • Reply 94 of 158
    dewme said:
    I've simply become distrustful of all forms of "Up To" marketing whether it's on computer performance claims, automobile trade-in values, gas mileage, etc. The advertised boost frequencies of Intel chips look nice on paper, greatly overshadowing the base frequency, and disproportionally tickling the impulse to purchase. But common sense and some basic appreciation for thermodynamics tells you that cramming a beefy 6-core current generation processor into a very restricted enclosure with only small fans and conductive heat sinks/pipes for cooling is going to make those boost numbers a very rare sighting. 

    "...your mileage may vary."

    There probably are no standards for what can be claimed in the "up to" speeds.  I guess they could dial in and optimize the test and the machine to hit a really high peak speed just for the sake of maximizing the "up to" speed they can claim.  I expect that 2.9GHz claim to be pretty accurate, though.  Turbo boost up to 4.8GHz may just be the geek version of "sex sells."

  • Reply 95 of 158
    The overall machine performs much better than the predecessors right?  Enough so to provide a compelling reason for those who need the power to upgrade.  Is the thermal design limiting the potential of the chip?  Seems to be so.  I am just wondering if Apple was looking at what the entire laptop as a whole can bring to the user versus other models...
    1) It performs better, yes. Not much better.
    2) It depends at what price premium you put a small increase in performance.
    3) The thermal design and parameters for controlling the thermal situation do seem to be limiting the potential of the chip.
    4) The marketing is misleading.
  • Reply 96 of 158
    Rayz2016Rayz2016 Posts: 3,926member
    People surprised about this should get a desktop, like a Mac Pro. This is nothing new. The thinner the body , the more likely the computer will throttle,to maintain temperatures. I suspect there would be even more complaints , if fans on MacBooks made more sound.

    The issue is that the i9 MBP isn't performing as advertised in these tests.  That's a completely different issue from a user choosing the right tool for the job.
    Did Apple say that the processor wouldn’t be throttled? Don’t think they did, no. 
  • Reply 97 of 158
    croprcropr Posts: 815member
    Rayz2016 said:
    People surprised about this should get a desktop, like a Mac Pro. This is nothing new. The thinner the body , the more likely the computer will throttle,to maintain temperatures. I suspect there would be even more complaints , if fans on MacBooks made more sound.

    The issue is that the i9 MBP isn't performing as advertised in these tests.  That's a completely different issue from a user choosing the right tool for the job.
    Did Apple say that the processor wouldn’t be throttled? Don’t think they did, no. 
    It is logical that there is throttling for the burst frequency, but it is unacceptable that a there is throttling below the base frequency, which is defined as the frequency at which   the CPU can run irrespective of the load.  

    If these test results are confirmed by others and Apple has no remedy in the short term, I can only have serious doubts that Apple can still be  a trustworthy laptop supplier, who does not deceive the customer by marketing claims it cannot fulfil.

    Apple would better invest more time in building the best qualitative laptop available iso. focusing on thinness, which is, for professional laptop users, not that important.
    edited July 19 muthuk_vanalingamcgWerksavon b7
  • Reply 98 of 158
    seankillseankill Posts: 395member
    cropr said:
    Rayz2016 said:
    People surprised about this should get a desktop, like a Mac Pro. This is nothing new. The thinner the body , the more likely the computer will throttle,to maintain temperatures. I suspect there would be even more complaints , if fans on MacBooks made more sound.

    The issue is that the i9 MBP isn't performing as advertised in these tests.  That's a completely different issue from a user choosing the right tool for the job.
    Did Apple say that the processor wouldn’t be throttled? Don’t think they did, no. 
    It is logical that there is throttling for the burst frequency, but it is unacceptable that a there is throttling below the base frequency, which is defined as the frequency at which   the CPU can run irrespective of the load.  

    If these test results are confirmed by others and Apple has no remedy in the short term, I can only have serious doubts that Apple can still be  a trustworthy laptop supplier, who does not deceive the customer by marketing claims it cannot fulfil.

    Apple would better invest more time in building the best qualitative laptop available iso. focusing on thinness, which is, for professional laptop users, not that important.
    ^This
    It’s only logical. 
    muthuk_vanalingam
  • Reply 99 of 158
    teedeeteedee Posts: 10member
    The i9 CPU averaged at about 2.4-2.8Ghz during throttling. So if we were to disable turbo boost, the CPU should be able to sustain at a consistent 2.9Ghz, improving its performance correct?

    Undervolting the CPU might also help with the performance. Apple could also just release a firmware to limit the turbo boost speed (e.g. to 3.9Ghz) which will reduce heat, which should lead to better performance. Am I right about these?
    edited July 19
  • Reply 100 of 158
    Mike WuertheleMike Wuerthele Posts: 3,108administrator
    teedee said:
    The i9 CPU averaged at about 2.4-2.8Ghz during throttling. So if we were to disable turbo boost, the CPU should be able to sustain at a consistent 2.9Ghz, improving its performance correct?

    Undervolting the CPU might also help with the performance. Apple could also just release a firmware to limit the turbo boost speed (e.g. to 3.9Ghz) which will reduce heat, which should lead to better performance. Am I right about these?
    We talk about the latter in the article.

    As far as the former, we're working on it.
Sign In or Register to comment.