Another test finds HomePod frequency response flat, but results potentially meaningless

12346

Comments

  • Reply 101 of 123
    gatorguygatorguy Posts: 21,095member
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    edited February 2018 mac_128
  • Reply 102 of 123

    FYI:
    I'm gonna be offline for at least few days while I get an brain aneurism pinched off. Please don't be offended if something directed to me goes answered for a while.
    Sorry to hear that.  Take care, and best thoughts!
  • Reply 103 of 123
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
  • Reply 104 of 123
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
    How much do you want to pay, and how long do you want to spend learning how to do it?

    As has been pointed out in this thread, designing speakers is non-trivial. Testing them well is also non-trivial. There is very sophisticated, professional-level software & hardware to do this sort of thing. Some 'audiophile' grade publications produce a series of graphs and charts along with a subjective listening review.

    Room EQ Wizard is popular & free software for doing things at home.

    - Jasen.
  • Reply 105 of 123
    lowededwookie said:

    My point is that not everyone is an audio nerd and listens to music based on the equipment. My ears aren't trained, you're correct on that point but I can tell the difference between a crap unit and a good unit. The fact that my Aiwa didn't sound any different to the $40,000 one clearly states to me that the nuance is so subtle that only a seriously trained person will pick it up but hey they are in such a minority that their expertise is going to be completely useless to me anyway because no matter what they tell me I'm just not going to pick up on it.
    Absolutely correct. IMHO, there is a law of diminishing returns at work. The first $2000 may get you 80% of the quality of a $10,000+ system. At the extreme end you have people paying multiple thousands of dollars for speaker cables. Are we talking 0.001% difference? I'll keep my money, thank you. At that extreme, the temperature or humidity could change the way things sound more than different cables.

    Just as there are people who enjoy cars as a hobby or cooking as a hobby, there are those who enjoy audio. The vast majority of people don't care.


  • Reply 106 of 123
    gatorguygatorguy Posts: 21,095member
    https://finance.yahoo.com/news/readers-weigh-pogues-homepod-listening-test-183253616.html
    David Pogue caught a lot of flak when he reported his own smart-speaker test results. What I hadn't read until now was his response to it all. It's a very good read and helps dispel some of the claims of this that and the other affecting the results. 

    edited February 2018
  • Reply 107 of 123
    jasenj1 said:
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
    How much do you want to pay, and how long do you want to spend learning how to do it?

    As has been pointed out in this thread, designing speakers is non-trivial. Testing them well is also non-trivial. There is very sophisticated, professional-level software & hardware to do this sort of thing. Some 'audiophile' grade publications produce a series of graphs and charts along with a subjective listening review.

    Room EQ Wizard is popular & free software for doing things at home.

    - Jasen.
    Thx for he link, I dloaded it and reading up on it now.  My oldest grandson has some USB mikes -- so I may be able to start at no cost and go from there.
    jasenj1
  • Reply 108 of 123
    gatorguygatorguy Posts: 21,095member
    jasenj1 said:
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
    How much do you want to pay, and how long do you want to spend learning how to do it?

    As has been pointed out in this thread, designing speakers is non-trivial. Testing them well is also non-trivial. There is very sophisticated, professional-level software & hardware to do this sort of thing. Some 'audiophile' grade publications produce a series of graphs and charts along with a subjective listening review.

    Room EQ Wizard is popular & free software for doing things at home.

    - Jasen.
    Thx for he link, I dloaded it and reading up on it now.  My oldest grandson has some USB mikes -- so I may be able to start at no cost and go from there.
    Dick,I  think that's exactly the same software that medical student and Riddit member "WinterCharm"used for his "professional review" of the HomePod. I think you'd find yourself coming up against the same issues as he discovered post-test. It didn't actually end up proving what he thought it was proving.
    https://www.reddit.com/r/audiophile/comments/7wwtqy/apple_homepod_the_audiophile_perspective/
    edited February 2018
  • Reply 109 of 123
    gatorguy said:
    jasenj1 said:
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
    How much do you want to pay, and how long do you want to spend learning how to do it?

    As has been pointed out in this thread, designing speakers is non-trivial. Testing them well is also non-trivial. There is very sophisticated, professional-level software & hardware to do this sort of thing. Some 'audiophile' grade publications produce a series of graphs and charts along with a subjective listening review.

    Room EQ Wizard is popular & free software for doing things at home.

    - Jasen.
    Thx for he link, I dloaded it and reading up on it now.  My oldest grandson has some USB mikes -- so I may be able to start at no cost and go from there.
    Dick,I  think that's exactly the same software that medical student and Riddit member "WinterCharm"used for his "professional review" of the HomePod. I think you'd find yourself coming up against the same issues as he discovered post-test. It didn't actually end up proving what he thought it was proving.
    https://www.reddit.com/r/audiophile/comments/7wwtqy/apple_homepod_the_audiophile_perspective/
    Ugh... I don't want to read thru all that and the contradictory response again...  AFAICT, each side had both good and bad points.

    I have a later version of REW -- V5.19 Beta 8 -- I just want to see if I can visualize what is happening using the Airfoil pseudo stereo. It's worth a try!  When Airplay 2 multiple homePod support arrives I'll test again.
  • Reply 110 of 123
    gatorguygatorguy Posts: 21,095member
    gatorguy said:
    jasenj1 said:
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
    How much do you want to pay, and how long do you want to spend learning how to do it?

    As has been pointed out in this thread, designing speakers is non-trivial. Testing them well is also non-trivial. There is very sophisticated, professional-level software & hardware to do this sort of thing. Some 'audiophile' grade publications produce a series of graphs and charts along with a subjective listening review.

    Room EQ Wizard is popular & free software for doing things at home.

    - Jasen.
    Thx for he link, I dloaded it and reading up on it now.  My oldest grandson has some USB mikes -- so I may be able to start at no cost and go from there.
    Dick,I  think that's exactly the same software that medical student and Riddit member "WinterCharm"used for his "professional review" of the HomePod. I think you'd find yourself coming up against the same issues as he discovered post-test. It didn't actually end up proving what he thought it was proving.
    https://www.reddit.com/r/audiophile/comments/7wwtqy/apple_homepod_the_audiophile_perspective/
    Ugh... I don't want to read thru all that and the contradictory response again...  AFAICT, each side had both good and bad points.

    I have a later version of REW -- V5.19 Beta 8 -- I just want to see if I can visualize what is happening using the Airfoil pseudo stereo. It's worth a try!  When Airplay 2 multiple homePod support arrives I'll test again.
    Oh I don't blame you for not wanting to slog thru that thread. Just wanted to give you a heads-up that you're using essentially the exact software he did in case you didn't want to go thru the whole she-bang and get similar uninsightful results. It looks like a lot of work.
  • Reply 111 of 123
    nhtnht Posts: 4,494member
    gatorguy said:
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    And his point is that testing on a "clean" track is not indicative of off-road performance.  Speaker designs intended for use in the real-world have different performance characteristics than those designed for a chamber.  And yes, many speaker designers DO design for testing in a chamber.  Designers will shoot for a flat anechoic response because that looks good on a spec sheet but can result in speakers that are bright.

    If a vehicle is designed to perform off-road then the testing should be done on a off-road test track.

    If a speaker is designed to perform using reflections off surfaces to provide a richer sound stage then testing should occur in a chamber that actually has reflective surfaces.


  • Reply 112 of 123
    gatorguygatorguy Posts: 21,095member
    nht said:
    gatorguy said:
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    And his point is that testing on a "clean" track is not indicative of off-road performance.  Speaker designs intended for use in the real-world have different performance characteristics than those designed for a chamber.  And yes, many speaker designers DO design for testing in a chamber.

    If a speaker is designed to perform using reflections off surfaces to provide a richer sound stage then testing should occur in a chamber that actually has reflective surfaces.
    Are they really designed differently depending on what tests will be done on them? I've no idea so I'll take your word for it. 

     In any event "your reflective surfaces" could not possibly be the same as "my reflective surfaces" so whatever particular results you have would not be the same as whatever particular result I have would they? It should be obvious that with users of the HomePod getting inconsistent results in their own homes the HomePod is not automatically the best-sounding speaker for everyone. One will claim they're "blown away" while another says too muddy and yet another says lacking mids or too much bass.  Such a huge range of experiences with it,  real buyers have been all over the place in describing the sound. Soli here wasn't impressed nor have been a few others chiming in our forum threads and eslewhere. Others such as Dick Applebaum can't say enough good things about it and it's easy to find posts from those that are in total agreement with him.  

    As for controlled testing if the bass is not well-defined in a controlled environment it would not suddenly be well-defined in an uncontrolled one, correct? Same would apply to mids, etc? That's why the testing for the best-case capabilities if I'm understanding it correctly. But as for sounding great in your home the only way to know for certain is put it in your home, which is where you and I probably agree. And no I'm not making any claims about the sound quality or lack of, or veracity of testing claims or anything else. Simply trying to digest the gist of what's been said. 
    edited February 2018 lorin schultz
  • Reply 113 of 123
    gatorguy said:
    jasenj1 said:
    Can anyone here recommend Mac or iDevice software that can be used to listen to and analyze speakers like the homePod?
    How much do you want to pay, and how long do you want to spend learning how to do it?

    As has been pointed out in this thread, designing speakers is non-trivial. Testing them well is also non-trivial. There is very sophisticated, professional-level software & hardware to do this sort of thing. Some 'audiophile' grade publications produce a series of graphs and charts along with a subjective listening review.

    Room EQ Wizard is popular & free software for doing things at home.

    - Jasen.
    Thx for he link, I dloaded it and reading up on it now.  My oldest grandson has some USB mikes -- so I may be able to start at no cost and go from there.
    Dick,I  think that's exactly the same software that medical student and Riddit member "WinterCharm"used for his "professional review" of the HomePod. I think you'd find yourself coming up against the same issues as he discovered post-test. It didn't actually end up proving what he thought it was proving.
    https://www.reddit.com/r/audiophile/comments/7wwtqy/apple_homepod_the_audiophile_perspective/
    I've been reading the REW docs for about an hour... I don't think I'll go down that path -- it is very complex just to calibrate, I don't have the expertise to understand  or the time/desire to learn -- and I don't think it will give me what I want to know.

    Simply stated, I want to visualize the sounds (I guess that means volume and phase) that are coming out of the tweeters -- especially those tweeters facing nearby rear and side walls -- and how they adjust (map) to the sound source.

    I am sure that the homePod has the compute power to, at least, create and log this data -- maybe even enough bandwidth to transmit the data to a Mac for real-time visualization.

    Knowing Apple, I suspect that they have the tools to do this.
  • Reply 114 of 123
    nhtnht Posts: 4,494member
    gatorguy said:
    nht said:
    gatorguy said:
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    And his point is that testing on a "clean" track is not indicative of off-road performance.  Speaker designs intended for use in the real-world have different performance characteristics than those designed for a chamber.  And yes, many speaker designers DO design for testing in a chamber.  Designers will shoot for a flat anechoic response because that looks good on a spec sheet but can result in speakers that are bright.

    If a vehicle is designed to perform off-road then the testing should be done on a off-road test track.

    If a speaker is designed to perform using reflections off surfaces to provide a richer sound stage then testing should occur in a chamber that actually has reflective surfaces.


    Except that "your reflective surfaces" could not possibly be the same as "my reflective surfaces" so whatever particular results you have would not be the same as whatever particular result I have would they? It should be obvious that with users of the HomePod getting inconsistent results in their own homes the HomePod is not automatically the best-sounding speaker for everyone. Some claim they're "blown away" while another says too muddy and yet another says no mids or too much bass.  Such a huge range of experiences with it.

    As for controlled testing if the bass is not well-defined in a controlled environment it would not suddenly be well-defined in an uncontrolled one, correct? Same would apply to mids, etc? That's why the testing for the best-case capabilities if I'm understanding it correctly. But as for sounding great in your home the only way to know for certain is put it in your home, which is where you and I probably agree. And no I'm not making any claims about the sound quality or lack of, or veracity of testing claims or anything else. Simply trying to digest the gist of what's been said. 
    It doesn't matter that your room is different from my room when the speaker is designed to use reflection and your test environment removes them from the equation.  That speaker will test poorly because of the way you've designed the test and the test is biased to direct firing designs.

    If the design incorporates rear-firing speakers then none of that output appears in the on-axis measurements.  Off-axis measurements will not capture any of the forward firing sound. The results of your "controlled testing" is meaningless and "controlled" doesn't mean necessarily mean no reflective surfaces.

    You can create controlled test environments with reflective surfaces.  Hemi-anechoic chambers do this to more accurately capture real-world sound levels of equipment that sit on concrete floors/roads.  Likewise you can test in chambers with specific levels of absorption or reflectivity.  At the two ends of the spectrum are reverberation rooms and anechoic chambers.  Or you can put specific test surfaces (walls, shelves, corners, floors, cielings, etc) into a anechoic chamber.  

    These would be controlled and repeatable tests more like off-road track testing of vehicles designed for off-road use.
    edited February 2018
  • Reply 115 of 123
    gatorguygatorguy Posts: 21,095member
    nht said:
    gatorguy said:
    nht said:
    gatorguy said:
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    And his point is that testing on a "clean" track is not indicative of off-road performance.  Speaker designs intended for use in the real-world have different performance characteristics than those designed for a chamber.  And yes, many speaker designers DO design for testing in a chamber.  Designers will shoot for a flat anechoic response because that looks good on a spec sheet but can result in speakers that are bright.

    If a vehicle is designed to perform off-road then the testing should be done on a off-road test track.

    If a speaker is designed to perform using reflections off surfaces to provide a richer sound stage then testing should occur in a chamber that actually has reflective surfaces.


    Except that "your reflective surfaces" could not possibly be the same as "my reflective surfaces" so whatever particular results you have would not be the same as whatever particular result I have would they? It should be obvious that with users of the HomePod getting inconsistent results in their own homes the HomePod is not automatically the best-sounding speaker for everyone. Some claim they're "blown away" while another says too muddy and yet another says no mids or too much bass.  Such a huge range of experiences with it.

    As for controlled testing if the bass is not well-defined in a controlled environment it would not suddenly be well-defined in an uncontrolled one, correct? Same would apply to mids, etc? That's why the testing for the best-case capabilities if I'm understanding it correctly. But as for sounding great in your home the only way to know for certain is put it in your home, which is where you and I probably agree. And no I'm not making any claims about the sound quality or lack of, or veracity of testing claims or anything else. Simply trying to digest the gist of what's been said. 
    It doesn't matter that your room is different from my room when the speaker is designed to use reflection and your test environment removes them from the equation.  That speaker will test poorly because of the way you've designed the test and the test is biased to direct firing designs.

    If the design incorporates rear-firing speakers then none of that output appears in the on-axis measurements.  Off-axis measurements will not capture any of the forward firing sound. The results of your "controlled testing" is meaningless and "controlled" doesn't mean necessarily mean no reflective surfaces.

    You can create controlled test environments with reflective surfaces.  Hemi-anechoic chambers do this to more accurately capture real-world sound levels of equipment that sit on concrete floors/roads.  Likewise you can test in chambers with specific levels of absorption or reflectivity.  At the two ends of the spectrum are reverberation rooms and anechoic chambers.  Or you can put specific test surfaces (walls, shelves, corners, floors, cielings, etc) into a anechoic chamber.  

    These would be controlled and repeatable tests more like off-road track testing of vehicles designed for off-road use.
    All sounds quite indefinite. How do you rest for a leather couch with a wood table in front of it compared to a soft fabric one with a metal coffee table. Or a room with no right angles compared to one that is square? You want a cut-and-dried test for the un-testable because it's littered with thousands of variables instead of one with controlled elements in a known repeatable space to set a comparable baseline.  Hmmmm... I wonder how Apple did its testing. Oh, we already know ...
    and it was not your suggested method. 

    According to Jim Dalrymple who was given a tour :smile: 
    "Once testing began, Apple used multiple chambers to study HomePod. The first became one of the largest anechoic chambers in the United States, allowing Apple engineers access to a non-reflective and echo free room to put the speaker's sound through its paces. " 
    (the other two were
    voice/Siri recognition, and the third was a "Noise and Vibration" chamber to detect and help prevent electronic noises and buzzing from the HomePod when it's plugged in but not playing music)

    Hard to believe you missed the story right here at AI with pictures and all.
    http://appleinsider.com/articles/18/02/06/apple-takes-media-on-tour-of-audio-lab-in-run-up-to-homepod-launch


    So you're claiming Apple didn't test the HomePod properly then since they relied on an anechoic chamber to test the sound quality. Gotcha. I'd suggest that rather than Apple "doing it wrong" you might instead consider it's almost certainly you. 

    At the end of the day what YOU hear is like every other sense you have. You like olives or you don't and no one at either end of the spectrum is wrong. 
    edited February 2018 lorin schultz
  • Reply 116 of 123
    nhtnht Posts: 4,494member
    gatorguy said:
    nht said:
    gatorguy said:
    nht said:
    gatorguy said:
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    And his point is that testing on a "clean" track is not indicative of off-road performance.  Speaker designs intended for use in the real-world have different performance characteristics than those designed for a chamber.  And yes, many speaker designers DO design for testing in a chamber.  Designers will shoot for a flat anechoic response because that looks good on a spec sheet but can result in speakers that are bright.

    If a vehicle is designed to perform off-road then the testing should be done on a off-road test track.

    If a speaker is designed to perform using reflections off surfaces to provide a richer sound stage then testing should occur in a chamber that actually has reflective surfaces.


    Except that "your reflective surfaces" could not possibly be the same as "my reflective surfaces" so whatever particular results you have would not be the same as whatever particular result I have would they? It should be obvious that with users of the HomePod getting inconsistent results in their own homes the HomePod is not automatically the best-sounding speaker for everyone. Some claim they're "blown away" while another says too muddy and yet another says no mids or too much bass.  Such a huge range of experiences with it.

    As for controlled testing if the bass is not well-defined in a controlled environment it would not suddenly be well-defined in an uncontrolled one, correct? Same would apply to mids, etc? That's why the testing for the best-case capabilities if I'm understanding it correctly. But as for sounding great in your home the only way to know for certain is put it in your home, which is where you and I probably agree. And no I'm not making any claims about the sound quality or lack of, or veracity of testing claims or anything else. Simply trying to digest the gist of what's been said. 
    It doesn't matter that your room is different from my room when the speaker is designed to use reflection and your test environment removes them from the equation.  That speaker will test poorly because of the way you've designed the test and the test is biased to direct firing designs.

    If the design incorporates rear-firing speakers then none of that output appears in the on-axis measurements.  Off-axis measurements will not capture any of the forward firing sound. The results of your "controlled testing" is meaningless and "controlled" doesn't mean necessarily mean no reflective surfaces.

    You can create controlled test environments with reflective surfaces.  Hemi-anechoic chambers do this to more accurately capture real-world sound levels of equipment that sit on concrete floors/roads.  Likewise you can test in chambers with specific levels of absorption or reflectivity.  At the two ends of the spectrum are reverberation rooms and anechoic chambers.  Or you can put specific test surfaces (walls, shelves, corners, floors, cielings, etc) into a anechoic chamber.  

    These would be controlled and repeatable tests more like off-road track testing of vehicles designed for off-road use.
    All sounds quite indefinite. How do you rest for a leather couch with a wood table in front of it compared to a soft fabric one with a metal coffee table. Or a room with no right angles compared to one that is square? You want a cut-and-dried test for the un-testable because it's littered with thousands of variables instead of one with controlled elements in a known repeatable space to set a comparable baseline.  Hmmmm... I wonder how Apple did its testing. Oh, we already know ...
    and it was not your suggested method. 

    According to Jim Dalrymple who was given a tour :smile: 
    "Once testing began, Apple used multiple chambers to study HomePod. The first became one of the largest anechoic chambers in the United States, allowing Apple engineers access to a non-reflective and echo free room to put the speaker's sound through its paces. " 

    So you're claiming Apple didn't test the HomePod properly then since they relied on an anechoic chamber to test the sound quality. Gotcha. You should let them know where they went wrong.

    At the end of the day what YOU hear is like every other sense you have. You like olives or you don't and no one at either end of the spectrum is wrong. 
    They would want to capture the performance of the individual speakers in the design and that would require an anechoic chamber.   You are lying again by cherry picking the quote.  The relevant quotes are:

    “Anechoic chambers are a standard tool for loud speaker development, but it is especially so for a product like HomePod where we were really interested in the directional behavior,” said Geaves. “Not only how it sounds in one direction, but all directions. That’s a critical component of why HomePod works like it does and enables the system to adapt to the environments the system is placed in.

    and

    "Apple’s custom anechoic chamber in Cupertino used to develop the beam-forming speaker array and high excursion woofer in HomePod"

    Develop, not test for performance.  They would need to be able to accurately measure the 360 output of the device based on their beam forming algorithms without added room reflections.  Then they would also need to test that performance in a controlled test environment with reflective surfaces to confirm their models.

    You tell me how you can test the performance beam forming in a normal speaker testing format in a normal anechoic chamber without reflective surfaces?  Oh wait, you can't.  You've just removed about half the speakers from Apple's design.



    And why so large?



    Now that is one of the isolation chambers but you don't build one of the largest anechoic chamber in the US to test a tiny little HomePod.  You build one of the largest in order to test it with the surfaces you expect to interact with. 

    You constantly lie like this to push the meme that Google is better at everything and Apple products are flawed.  Its tiresome...aren't there google forums for you to hang out in? 
  • Reply 117 of 123
    gatorguygatorguy Posts: 21,095member
    nht said:
    gatorguy said:
    nht said:
    gatorguy said:
    nht said:
    gatorguy said:
    I find the tests performed sans-anechoic chambers to be the best test because I don't know about you but I don't listen to my music in a sterile environment. I want to know what my music is going to sound like in the environment I listen to my music in so the way I see it these tests are more important than "clean" tests.

    It's like saying a car drives so amazingly on a test track. The real world is where it's at. How many test tracks are designed like New Zealand roads? You want to test a car test it on NZ roads and then get back to us how awesome your car is.

    It's the same with music. I had a friend who had a massive $40,000 stereo system and frankly I don't see where the money went. I couldn't tell any noticeable difference to the Aiwa separate component system I was using at the time.
    You didn't read the comment thread before writing that, did you? If you had, you'd know that there's already been extensive discussion of why testing in an anechoic chamber is useful, why publishing tests made in any particular non-anechoic environment are completely useless to anyone but the person in that room, and why the only test that matters in a meaningful way is listening yourself wherever YOU'RE going to use it.

    You can find the details elsewhere in the thread if you're interested, but the Cliffsnotes version is that every single room will have a unique affect on the frequency response of a speaker, and even moving the speaker within the same room will yield a different response. A test conducted in my house would tell you nothing about how it will sound in your house.

    Anechoic testing eliminates the room as an influencing factor so you can see what the speaker is capable of doing, and compare the results to other speakers using a level playing field. It's not supposed to be an indication of what your room will do to the sound because there's no way to test for that.

    The ONLY way to evaluate how a speaker will sound in your environment is to set it up and try it.

    Also, consider the possibility that your not being able to tell the difference between your Aiwa system and your friend's $40K rig may have more to do with your hearing and/or ear training than relative sound quality! :)
    Precisely my point. The tests in "clean" environments aren't going to give me a clear idea of how it's going to sound in MY house either is it? 


    I think the point he was making, and that you didn't understand, is that the "clean" environment can give you a clearer idea of what it's capable of delivering in sound quality given the ideal circumstances. Whether it can reach that level in your own home is a separate even if related issue as you properly noted. 
    And his point is that testing on a "clean" track is not indicative of off-road performance.  Speaker designs intended for use in the real-world have different performance characteristics than those designed for a chamber.  And yes, many speaker designers DO design for testing in a chamber.  Designers will shoot for a flat anechoic response because that looks good on a spec sheet but can result in speakers that are bright.

    If a vehicle is designed to perform off-road then the testing should be done on a off-road test track.

    If a speaker is designed to perform using reflections off surfaces to provide a richer sound stage then testing should occur in a chamber that actually has reflective surfaces.


    Except that "your reflective surfaces" could not possibly be the same as "my reflective surfaces" so whatever particular results you have would not be the same as whatever particular result I have would they? It should be obvious that with users of the HomePod getting inconsistent results in their own homes the HomePod is not automatically the best-sounding speaker for everyone. Some claim they're "blown away" while another says too muddy and yet another says no mids or too much bass.  Such a huge range of experiences with it.

    As for controlled testing if the bass is not well-defined in a controlled environment it would not suddenly be well-defined in an uncontrolled one, correct? Same would apply to mids, etc? That's why the testing for the best-case capabilities if I'm understanding it correctly. But as for sounding great in your home the only way to know for certain is put it in your home, which is where you and I probably agree. And no I'm not making any claims about the sound quality or lack of, or veracity of testing claims or anything else. Simply trying to digest the gist of what's been said. 
    It doesn't matter that your room is different from my room when the speaker is designed to use reflection and your test environment removes them from the equation.  That speaker will test poorly because of the way you've designed the test and the test is biased to direct firing designs.

    If the design incorporates rear-firing speakers then none of that output appears in the on-axis measurements.  Off-axis measurements will not capture any of the forward firing sound. The results of your "controlled testing" is meaningless and "controlled" doesn't mean necessarily mean no reflective surfaces.

    You can create controlled test environments with reflective surfaces.  Hemi-anechoic chambers do this to more accurately capture real-world sound levels of equipment that sit on concrete floors/roads.  Likewise you can test in chambers with specific levels of absorption or reflectivity.  At the two ends of the spectrum are reverberation rooms and anechoic chambers.  Or you can put specific test surfaces (walls, shelves, corners, floors, cielings, etc) into a anechoic chamber.  

    These would be controlled and repeatable tests more like off-road track testing of vehicles designed for off-road use.
    All sounds quite indefinite. How do you rest for a leather couch with a wood table in front of it compared to a soft fabric one with a metal coffee table. Or a room with no right angles compared to one that is square? You want a cut-and-dried test for the un-testable because it's littered with thousands of variables instead of one with controlled elements in a known repeatable space to set a comparable baseline.  Hmmmm... I wonder how Apple did its testing. Oh, we already know ...
    and it was not your suggested method. 

    According to Jim Dalrymple who was given a tour :smile: 
    "Once testing began, Apple used multiple chambers to study HomePod. The first became one of the largest anechoic chambers in the United States, allowing Apple engineers access to a non-reflective and echo free room to put the speaker's sound through its paces. " 

    So you're claiming Apple didn't test the HomePod properly then since they relied on an anechoic chamber to test the sound quality. Gotcha. You should let them know where they went wrong.

    At the end of the day what YOU hear is like every other sense you have. You like olives or you don't and no one at either end of the spectrum is wrong. 
    They would want to capture the performance of the individual speakers in the design and that would require an anechoic chamber.   You are lying again by cherry picking the quote.  The relevant quotes are:

    “Anechoic chambers are a standard tool for loud speaker development, but it is especially so for a product like HomePod where we were really interested in the directional behavior,” said Geaves. “Not only how it sounds in one direction, but all directions. That’s a critical component of why HomePod works like it does and enables the system to adapt to the environments the system is placed in.

    and

    "Apple’s custom anechoic chamber in Cupertino used to develop the beam-forming speaker array and high excursion woofer in HomePod"

    Develop, not test for performance.  They would need to be able to accurately measure the 360 output of the device based on their beam forming algorithms without added room reflections.  Then they would also need to test that performance in a controlled test environment with reflective surfaces to confirm their models.

    You tell me how you can test the performance beam forming in a normal speaker testing format in a normal anechoic chamber without reflective surfaces?  Oh wait, you can't.  You've just removed about half the speakers from Apple's design.



    And why so large?



    Now that is one of the isolation chambers but you don't build one of the largest anechoic chamber in the US to test a tiny little HomePod.  You build one of the largest in order to test it with the surfaces you expect to interact with. 

    You constantly lie like this to push the meme that Google is better at everything and Apple products are flawed.  Its tiresome...aren't there google forums for you to hang out in? 
    LOL. Again with the "lie" stuff when you're out of arguments. I would have expected more from you. If you'd bother reading about why Apple made such a huge anechoic chamber as I did you might find out why. It certainly wasn't for for checking how the sound reflected off the walls when furniture is in the room. Those walls are pretty poor reflectors no matter how much other stuff they put in there.
    If they did. :)

    ...and FWIW you're the one bringing up Google, not me. It's nearly ALWAYS someone other than me bringing them into an Apple conversation uninvited. You should resist the urge to be that guy. 
    edited February 2018
  • Reply 118 of 123

    Ya' know...

    I would expect a speaker manufacturer, Apple included, to test their product in an anechoic chamber to see what it's capable of -- in an ideal environment... Using another car analogue it's like testing a car's engine on a dynamometer.

    But it doesn't stop there, they also test in real-world environments to see if they can adapt/perform adequately in the real world.

    I suspect Apple (and most car manufacturers) do both.

    In the case of the homePod -- I've been playing, all day, with 2 homePods, each sitting atop a McIntosh speaker (that were connected to the old B&O).

    This time tho, I did not try to create an ersatz stereo -- I just let the homePods play the same iTunes  sources au natural -- unadjusted. manually

    They seemed to adjust themselves to the room and the music just fine - and both sounded damn good -- AIR (this is here & now), better than the McIntoshes on the Old B&O.
    edited February 2018
  • Reply 119 of 123
    gatorguygatorguy Posts: 21,095member

    Ya' know...

    I would expect a speaker manufacturer, Apple included, to test their product in an anechoic chamber to see what it's capable of -- in an ideal environment... Using another car analogue it's like testing a car's engine on a dynamometer.

    But it doesn't stop there, they also test in real-world environments to see if they can adapt/perform adequately in the real world.

    I suspect Apple (and most car manufacturers) do both.

    In the case of the homePod -- I've been playing, all day, with 2 homePods, each sitting atop a McIntosh speaker (that were connected to the old B&O).

    This time tho, I did not try to create an ersatz stereo -- I just let the homePods play the same iTunes  sources au natural -- unadjusted. manually

    They seemed to adjust themselves to the room and the music just fine - and both sounded damn good -- AIR (this is here & now), better than the McIntoshes on the Old B&O.
    Dick, I thought the only time the HomePod was actively testing and adjusting sound to the space was when it was initially placed or moved, ie when the accelerometer is triggered, and using only one of the onboard microphones to do so. Am I misunderstanding? You seem to be saying it's constantly adjusting to the space. 
    edited February 2018
  • Reply 120 of 123
    gatorguy said:

    Ya' know...

    I would expect a speaker manufacturer, Apple included, to test their product in an anechoic chamber to see what it's capable of -- in an ideal environment... Using another car analogue it's like testing a car's engine on a dynamometer.

    But it doesn't stop there, they also test in real-world environments to see if they can adapt/perform adequately in the real world.

    I suspect Apple (and most car manufacturers) do both.

    In the case of the homePod -- I've been playing, all day, with 2 homePods, each sitting atop a McIntosh speaker (that were connected to the old B&O).

    This time tho, I did not try to create an ersatz stereo -- I just let the homePods play the same iTunes  sources au natural -- unadjusted. manually

    They seemed to adjust themselves to the room and the music just fine - and both sounded damn good -- AIR (this is here & now), better than the McIntoshes on the Old B&O.
    Dick, I thought the only time the HomePod was actively testing and adjusting sound to the space was when it was initially placed or moved, ie when the accelerometer is triggered, and using only one of the onboard microphones to do so. Am I misunderstanding? You seem to be saying it's constantly adjusting to the space. 

    OK, I'll take a stab at it!

    This from the HomePod Overveiw web page (emphasis mine):

    https://www.apple.com/homepod/

    The new sound of home.

    HomePod is a powerful speaker that sounds amazing and adapts to wherever it’s playing. It’s the ultimate music authority, bringing together Apple Music and Siri to learn your taste in music. It’s also an intelligent home assistant, capable of handling everyday tasks — and controlling your smart home. HomePod takes the listening experience to a whole new level. And that’s just the beginning

    The link is to four 15-second video commercials:  Distortion;  Beat;  Equalizer:  Base.  These videos each illustrate words being played/analyzed/adjusted? by something akin to an instrument  displaying a graphic measurement of the characteristic.  For example the third video displays the word HomePod being adjusted to the music and displayed as a waveform on an oscilloscope.

    A breakthrough speaker all around.

    We completely reimagined how music should sound in the home. HomePod combines Apple-engineered audio technology and advanced software to deliver the highest-fidelity sound throughout the room, anywhere it’s placed. This elegantly designed, compact speaker totally rocks the house.

    There is a cutaway of the HomePod  showing, among other things, the Six-microphone array.   It doesn't explicitly say that the microphones are used to adjust the sound... but the following iMore article does:

    On top of that is a six-microphone array that listens not just for Siri commands, but for the reflection of sound off corners, walls, furniture, and more.

    https://www.imore.com/homepod

    Later on in the Apple HomePod page:

    A8 chip. 
    The biggest brain ever in a speaker.

    An Apple-designed A8 chip powers the most complex audio innovations in HomePod. Like real-time modeling of the woofer mechanics. Buffering that’s even faster than real time. Upmixing of both direct and ambient audio. Beamforming so the microphone can hear you over the music. And advanced echo cancellation. So you get amazing sound without even thinking about it.

    Again, it doesn't explicitly say it is listening and adjusting, but I infer that it does.  I have been playing to the HomePods from iTunes, using the equalizer to change settings.  The adjustments are recognized, but do not take effect for a few seconds  -- exhausting the buffer and then rebuffing and replaying under the new settings.

    It’s easier than easy. It’s automatic.

    Setting up HomePod is quick and magical. Simply plug it in and your iOS device will detect it. Equipped with spatial awareness, HomePod automatically adjusts to give you optimal sound — wherever it’s placed. It can even hear your requests from across the room while loud songs are playing. All you need to do is enjoy your music.
    Again, it doesn't break down the meaning of spatial awareness -- but I infer that it is listening and adjusting as it is playing.


    From another article:

    HomePod doesn't have manual EQ options, will auto-adjust based on analytics says Apple's Eddy Cue

    People buying a HomePod will have to rely on the speaker's own automatic EQ adjustments, as there are no manual controls, Apple's senior VP of Internet Software and Services said at today's Pollstar Live conference for the concert industry.

    The HomePod uses analytics to set levels for each individual song, Eddy Cue said according to Eventellect co-founder Patrick Ryan. The approach could upset people who prefer extra bass or treble in their music.

    Apple has made automatic tuning a focus of the product. It uses microphones and beamforming to gauge the acoustics of a room, correcting for problems within seconds. An accelerometer is used to detect if the speaker has been moved, meaning that it will readjust on its own.

    https://forums.appleinsider.com/discussion/204148/appleinsider.com/articles/18/02/07/homepod-doesnt-have-manual-eq-options-will-auto-adjust-based-on-analytics-says-apples-eddy-cue

    It is unclear whether the analytics are performed before or after the individual song is sent to the HomePod... Or both? 

    Finally, there is a virtual tsunami of of information about how the HomePod does its thing.  It is difficult to separate fact from fiction or supposition (I'm guilty of some of that).  My supposition comes from watching developer videos, reading developer docs, reading patents, reading articles (some very long and very technical) and from having observed Apple for many years.


    Edit:

    And this just in:

    Video: Apple HomePod vs. Sonos One

    Both smart speakers come with six far-field microphones, but the HomePod utilizes them for a whole lot more than just listening for your voice. On HomePod, the microphones are responsible for automatically analyzing the acoustics of the room, allowing the device to tune the tweeters to create a room full of sound, while getting rid of any echo. So if you place the HomePod in a corner, it won't waste power blasting audio into the two walls.

    https://forums.appleinsider.com/discussion/204148/appleinsider.com/articles/18/02/19/video-apple-homepod-vs-sonos-one

    edited February 2018
Sign In or Register to comment.