ANALYSIS:
As Apple’s HomePod has been out for a week, most of the news, reviews, and early problems have already been discussed ad nauseum. But you might have missed two unusual articles from Yahoo’s David Pogue, whose real world HomePod listening test last Saturday created enough controversy to merit a “readers weigh in” follow-up today. After having praised HomePod’s audio in his initial review, Pogue discovered that a panel of listeners preferred the Sonos One or Google Home Max instead. When he published that finding, some of his readers freaked out, leading him to dive deeper into his testing methodology — and other testers’ results, as well.
My review of the HomePod was pretty straightforward on audio quality: It’s an overly bass-heavy, monaural speaker with recessed mids, ultimately not worthy of its steep asking price. Some real-world reviewers agree with my take, and some don’t. That’s fine — like pocketbooks, audio preferences are somewhat personal. Even before the review was published, my advice was to audition HomePod for yourself.
But it’s worth exploring why there’s been so much difference between the early, universally positive HomePod sonic reviews and the less enthusiastic ones that have followed from members of the public. Specifically, I’d like to fill you in on some details that will help explain why something as simple as a speaker can generate such polarized opinions between “expert” reviewers, “audiophiles,” and regular people.
If you don’t like knowing how review sausage is made, stop reading here, because this is going to get a little messy.
How they tested, or should have tested
Over the course of his articles, Pogue first hinted and then fully explained that Apple required early reviewers to attend a special listening session comparing the HomePod against three competing speakers. Apple selected the tracks, and in an important detail most reviewers probably didn’t realize, played them over different audio connections ranging from Wi-Fi to Bluetooth, Ethernet, and audio line-in. As Pogue notes, reviewers typically left Apple’s listening sessions feeling that HomePod had outperformed its rivals — no shock, given that two were around half or less of HomePod’s price.
But after Pogue disclosed Apple’s speaker shootout, readers questioned it, leading him to run a more controlled comparison of his own. He took the four speakers, picked five revealing songs, assembled a fairly representative panel of listeners, and made sure the input source and volume were kept constant across all the speakers. Contrary to Pogue’s initial assumptions, and his early review’s conclusions, the panel rated HomePod’s overall sonic performance below the Sonos One and Google Home Max.
Because the internet has its fair share of self-proclaimed audiophiles, the second and more neutral round of test results didn’t sit well with some of Pogue’s readers, either. There were questions over the thin curtain that masked the speakers’ identities, suggestions of bias due to listeners’ physical positions relative to the speakers, and complaints that Spotify was the audio source for all the speakers. Pogue was called “grandpaw,” asked to use more randomized methodologies, and told to retest in either sonically silent or visually neutral rooms. The list of reader comments reads much like a series of high school excuses for failing to turn in homework — except nastier.
Here’s how review sausage is made
Although I don’t agree with Pogue’s original claims that “HomePod sounds really, really great” or “amazing,” I applaud him for the courage it took to publicly re-examine the comparison process that led to his initial conclusions. His all-important first impressions were created using a skewed test, and a more neutral methodology would have yielded different results, as his panel’s experiences demonstrated.
Why was there such a difference between the “expert” reviewers’ opinions and Pogue’s real world panel? I’m going to fill you in on a critically important piece of background information that will help you understand the reality behind speaker reviews:
Most journalists know very little about speakers, or audio in general.
Here’s one more key point:
Many journalists, including even legitimate audio experts, have damaged their hearing, or aged past their ears’ sonic prime.
Apple knew this, and it knew that (some of) the people it picked to review the HomePod might feel compelled to disclose their lack of audio experience. That’s why it set up a four-speaker testing room — a step it literally never took with competitors to the Apple TV, Apple Watch, iPad, or iPhone — and walked reviewers through the tests. Apple spared writers the need to do their own comparisons and helped them reach a positive conclusion about HomePod’s audio, presenting it in the best possible light.
These details should help you understand why some of the first HomePod reviewers said that they could have stopped writing about the speaker after a paragraph or so, but felt obliged to write more. Normally, most of these people don’t write about the nitty-gritty of audio products, and they don’t feel comfortable discussing the technical nuances of “great sound.” But in this case, they were being given early access to a new Apple product, knew they had to write full reviews, and did the best jobs they could under the circumstances. That might seem like a somewhat charitable interpretation given how over-the-top some early reviewers were in praising HomePod, but in my experience, it’s largely accurate.
An alternative: Don’t trust (most) audio reviews
If you care about audio — and you certainly should, for $ 349 per speaker — one option would be to seek out opinions from people who have actual audio expertise. Unfortunately, there’s a lot of noise on the internet these days, and true expert opinions are surprisingly hard to find. Assuming you can’t decide for yourself, the best alternative is to find someone with a public track record of audio reviews that match your preferences and experiences, or alternately, someone whose opinions are reliably the opposite of yours.
One thing you shouldn’t do is put faith in people who appear out of nowhere armed with audio graphs purporting to show that a given speaker has “perfect, audiophile-quality sound,” or reviewers who write effusively about the magical melodies produced by every piece of audio gear they test. Reviews that focus heavily on specs or poetic descriptions of sound are usually full of crap — regrettably, many readers confuse pages of numbers and words with genuine expertise. It takes a lot more courage for a writer to produce brief, blunt, and accurate conclusions about a speaker than to convince you it’s going to deliver sonic heaven every time you turn it on.
Unlike his first HomePod review, Pogue’s last article doesn’t attempt to be authoritative on the speaker’s sound. While raising an eyebrow at certain conclusions, it acknowledges a diversity of opinions from everyone, from his panel of listeners to various readers to professional testers at Consumer Reports. That’s a reasonable approach: If you can’t find a great single reviewer whose tastes mirror yours, considering a variety of different opinions is probably the second healthiest way to approach audio purchases.
The best option? Try the product for yourself, and return it if you don’t like it. You’re the best judge of whether something sounds (and works, and looks) great enough to be worthy of your hard-earned cash.
Happy listening!
