They still haven’t figured out how to deal with Apple as a special case in the wider tech market.
Why should they deal with Apple, or any company for that matter, as a "special" case?
Frankly, I've found CU's ratings to be quite useful, especially when paired with the user reviews. The fact that my experiences differ on some items, both for the better and the opposite, doesn't invalidate their entire ratings system.
Because, in a misapplied effort to be “impartial,” they often use testing methodologies that directly defeat new technologies.
They tested HomePods in their speaker testing room, which has acoustic wall and ceiling panels that deaden reflected sounds. HomePod actually uses walls and ceilings to actively cancel unwanted acoustics and to create presence in the room. They tested battery life in MacBook Pros by disabling software features that eliminate redundant data downloads to extend battery performance. Defeating these innovative features makes the devices more like the competition in the tests, but it entirely misses the point of innovation.
If they reviewed restaurants, they’d pour ketchup on everything they taste, and claim that makes them “impartial.”
You are misinformed and don't understand sound and sound testing based on what I've read. Proof? Look back at how Apple tested the HomePods.
"Apple’s audio team, does much of their work in ‘anechoic’ chambers built to be totally silent.
The HomePod being tested to see if it produces unwanted noises (Photo: Apple)
A view inside an anechoic chamber at Apple’s Noise and Vibration lab
"These strange rooms are lined with foam to absorb the echoes, meaning they can feel like an ‘infinite cave’ with none of the natural reverberation you expect from a normal space. Some are so quiet you can hear your heart beating whilst standing inside them. Apple has built several massive anechoic chambers built to a size, scale and specification that few of its competitors could afford – if they were even ambitious enough to consider building such epic spaces. These chambers all simulate different rooms. One is set above a strange pit of white foam pyramids, which you walk over using a perilous-feeling metal mesh hanging a few feet in the air."
That's funny, because it would seem that you're the one who doesn't understand what's going on. Anechoic chambers were indeed used by Apple for their own testing of the device, so that they could measure everything coming out of it with great precision. That's not a consumer listening test. They used the chamber to establish the desired baseline output before they then applied the functions in the device that measure, adapt and use the reflective acoustics in a normal room. Apple's testing for the final product happened outside the anechoic chamber, in normal rooms, with walls and ceilings and furniture and windows and drapes and carpets.
They were not using the anechoic chamber to design a device for use in anechoic chambers. HomePod's page at Apple includes a nice video that visualizes what the device does in a room in your home. The text on that page describes how HomePod bounces sounds off walls to achieve the sound field it's intended to create.
Consumer Reports' testing specifically defeated that feature. They weren't putting the HomePod in a testing room to precisely measure frequency response and all that. They put it on a shelf next to all their other speakers in an acoustically dampened room, sat down directly in front of it and listened to some stuff playing on it. It's an unscientific testing method that neither carries out precise measurements like you see in the anechoic chamber pictures you posted, nor does it create a normal home listening environment.
It's like testing cars for power and fuel economy, and removing the turbo from the Honda, because that technology isn't available on the competition, and then proclaiming the Honda isn't any more powerful or fuel efficient than the others. It's brain-dead testing methodology. And just because Honda at some point during development tested that engine without a turbo doesn't mean that Consumer Reports is right to take it off when Honda doesn't even sell that model without one.
They still haven’t figured out how to deal with Apple as a special case in the wider tech market.
Why should they deal with Apple, or any company for that matter, as a "special" case?
Frankly, I've found CU's ratings to be quite useful, especially when paired with the user reviews. The fact that my experiences differ on some items, both for the better and the opposite, doesn't invalidate their entire ratings system.
Because, in a misapplied effort to be “impartial,” they often use testing methodologies that directly defeat new technologies.
They tested HomePods in their speaker testing room, which has acoustic wall and ceiling panels that deaden reflected sounds. HomePod actually uses walls and ceilings to actively cancel unwanted acoustics and to create presence in the room. They tested battery life in MacBook Pros by disabling software features that eliminate redundant data downloads to extend battery performance. Defeating these innovative features makes the devices more like the competition in the tests, but it entirely misses the point of innovation.
If they reviewed restaurants, they’d pour ketchup on everything they taste, and claim that makes them “impartial.”
You are misinformed and don't understand sound and sound testing based on what I've read. Proof? Look back at how Apple tested the HomePods.
"Apple’s audio team, does much of their work in ‘anechoic’ chambers built to be totally silent.
The HomePod being tested to see if it produces unwanted noises (Photo: Apple)
A view inside an anechoic chamber at Apple’s Noise and Vibration lab
"These strange rooms are lined with foam to absorb the echoes, meaning they can feel like an ‘infinite cave’ with none of the natural reverberation you expect from a normal space. Some are so quiet you can hear your heart beating whilst standing inside them. Apple has built several massive anechoic chambers built to a size, scale and specification that few of its competitors could afford – if they were even ambitious enough to consider building such epic spaces. These chambers all simulate different rooms. One is set above a strange pit of white foam pyramids, which you walk over using a perilous-feeling metal mesh hanging a few feet in the air."
That's funny, because it would seem that you're the one who doesn't understand what's going on. Anechoic chambers were indeed used by Apple for their own testing of the device, so that they could measure everything coming out of it with great precision. That's not a consumer listening test. They used the chamber to establish the desired baseline output before they then applied the functions in the device that measure, adapt and use the reflective acoustics in a normal room. Apple's testing for the final product happened outside the anechoic chamber, in normal rooms, with walls and ceilings and furniture and windows and drapes and carpets.
They were not using the anechoic chamber to design a device for use in anechoic chambers. HomePod's page at Apple includes a nice video that visualizes what the device does in a room in your home. The text on that page describes how HomePod bounces sounds off walls to achieve the sound field it's intended to create.
Consumer Reports' testing specifically defeated that feature. They weren't putting the HomePod in a testing room to precisely measure frequency response and all that. They put it on a shelf next to all their other speakers in an acoustically dampened room, sat down directly in front of it and listened to some stuff playing on it. It's an unscientific testing method that neither carries out precise measurements like you see in the anechoic chamber pictures you posted, nor does it create a normal home listening environment.
It's like testing cars for power and fuel economy, and removing the turbo from the Honda, because that technology isn't available on the competition, and then proclaiming the Honda isn't any more powerful or fuel efficient than the others. It's brain-dead testing methodology. And just because Honda at some point during development tested that engine without a turbo doesn't mean that Consumer Reports is right to take it off when Honda doesn't even sell that model without one.
Possible there might be something CS missed, tho I thought that was one of the things Mike had met with them on and came away generally satisfied that the tests were properly done.
This has been a while, though I've found something interesting, Samsung got some of the best people in the audio industry on-board when creating the Galaxy Buds.
Sean Olive is the top figure of Psychoacoustics which he leads the creation of the Harman Target Curve, so far the best sounding reference for earbuds.
If you look at some of Samsung's recent products, their integrated speakers are roughly as capable as Apple's counterpart: Galaxy Tab S5e/S6 compared to iPad Pro, Galaxy Note 10 to iPhone 11s, even Galaxy Home to HomePod, though I have wait to see how the Galaxy Book S performed. It is pretty clear they bought the team to went head-to-head with Apple, which was one of the biggest advantages Apple enjoyed for years.
I'm not sure how AirPods will compete with the Buds as they're not made to be fully in-ear, which is why "Audiophile" b------ so much, but definitely not the best shape for the acoustics. That doesn't mean, however, that it won't sound well, but certainly no match for the best. The battery life simply about makes it bigger, which they're not going for anyway.
Comments
They were not using the anechoic chamber to design a device for use in anechoic chambers. HomePod's page at Apple includes a nice video that visualizes what the device does in a room in your home. The text on that page describes how HomePod bounces sounds off walls to achieve the sound field it's intended to create.
Consumer Reports' testing specifically defeated that feature. They weren't putting the HomePod in a testing room to precisely measure frequency response and all that. They put it on a shelf next to all their other speakers in an acoustically dampened room, sat down directly in front of it and listened to some stuff playing on it. It's an unscientific testing method that neither carries out precise measurements like you see in the anechoic chamber pictures you posted, nor does it create a normal home listening environment.
It's like testing cars for power and fuel economy, and removing the turbo from the Honda, because that technology isn't available on the competition, and then proclaiming the Honda isn't any more powerful or fuel efficient than the others. It's brain-dead testing methodology. And just because Honda at some point during development tested that engine without a turbo doesn't mean that Consumer Reports is right to take it off when Honda doesn't even sell that model without one.
http://www.samsungmobilepress.com/stories/the-secrets-behind-galaxy-buds-premium-audio-performance
Sean Olive is the top figure of Psychoacoustics which he leads the creation of the Harman Target Curve, so far the best sounding reference for earbuds.
If you look at some of Samsung's recent products, their integrated speakers are roughly as capable as Apple's counterpart: Galaxy Tab S5e/S6 compared to iPad Pro, Galaxy Note 10 to iPhone 11s, even Galaxy Home to HomePod, though I have wait to see how the Galaxy Book S performed. It is pretty clear they bought the team to went head-to-head with Apple, which was one of the biggest advantages Apple enjoyed for years.
I'm not sure how AirPods will compete with the Buds as they're not made to be fully in-ear, which is why "Audiophile" b------ so much, but definitely not the best shape for the acoustics. That doesn't mean, however, that it won't sound well, but certainly no match for the best. The battery life simply about makes it bigger, which they're not going for anyway.