You realize that people said pretty much the same thing back in 2007 about handsets right? They were low margin, Apple should just license an OS, etc. Apple went on to demonstrate that it could offer a completely new experience on the phone which consumers would pay for.
If Apple decide to enter the HDTV market it will be because they feel they can differentiate from the existing market, enough to earn a nice margin on top. I've no idea if it will enter the market, but it almost has to have considered it, because at this point there are very few markets that Apple can enter that will materially affect their bottom line.
Apple has the problem of success - it owns the PMP market, it's growing strongly in the handset market, it seems likely to own the tablet market. Where does it go next? What possible business can it invest in that will materially affect the bottom line?
The reason this rumour keeps popping up again and again is because the HDTV market is big enough to make an impact, and fits vaguely into Apple's core competency.
Except that the HDTV market doesn't resemble the PMP, smartphone or tablet markets prior to Apple's entry. Those were all markets where existing products totally sucked and were perceived as totally sucking. (Some might dispute this for smartphones, but, before the iPhone, expect for Blackberrys, which are basically just texting appliances, people weren't rushing out to buy smartphones, and they really didn't do much.)
Existing HDTVs don't totally suck, and aren't perceived to by the general public. It's not a market crying out to be redefined.