The company followed up a few weeks later by tying reviews to the revision of the app, a feature which allowed users to filter out old complaints related to problems that had since been fixed
It's better, but could still be improved. When I post a new release itunes defaults to showing the "most helpful" reviews, since the current version of the app has no reviews. The issue the "most helpful" reviews were from version 1.0 which is almost two years old now and has no baring on the current version of the app. Apple won't remove these ancient reviews, or use any sort of aging so that a helpful review from 2 years ago isn't ranked as as-helpful now.
The other major issue with the review system is that developers still can't respond to customers reviews. Privately would be better than letting devs respond in the review threads so it doesn't turn into a post a review to get support, but letting us privately respond would let us address customers issues or seek clarification on the issue they are having.
Hmmm, now we're less likely to know if an app is not any good.
It seems fine that the system didn't capture positive ratings. It still resulted in lower ratings for apps that weren't as good. Who cares what the average is for all apps as long as the better ones are ranked higher?
Because apps (as well as many other things) should be judged on both a relative and absolute scale. If best app in a particular category was ranked highest by rating, but only rated out at 3 stars, I'd be slightly hesitant to buy.
All arguments about statistical accuracy aside, I think this is a bad move.
The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
Paraphrasing Henrik Ibsen (An Enemy of the People)::
What is true may not be popular.
What is popular may not be true.
As noted by others, we tend to voice our opinion more so when we have a negative experience or opposing perspective. It is not unusual too that there are those who solicit their friends to get good reviews. This fraud has been documented in Amazon book reviews.
One should trust reviews based mainly from reliable sources, those with credibility to weigh in on the matter.
While the current system probably does skew ratings downward, the new system will likewise skew them upward, which is a bit of a disadvantage for apps that have been out for a while (skewed downward) vs. newer apps (skewed upward). However, it's a very difficult problem to solve. Weighting ratings by absolute numbers might help -- for example, a "3-star" app with 1,000 ratings would rank higher than a 3-star app with 10 ratings. Weighting them by how much time they spend on a users iPhone might also help, but raises privacy issues, and tracking usage (which is happening all the time now) is not appropriate. (I think this should have to be disclosed in the description of each app, if it's tracked, how, and what information is used/taken).
But, in the long-run, this is probably the best way to do this, putting all ratings on an equal footing so that everyone who rated did so through the same mechanism.
The biggest problem with the ratings, from a user perspective, is that you can't sort/search the app store based on them.
The current way of doing things ABSOLUTELY skews ratings downward into hell, because why else would you be deleting an app unless you didn't like it and didn't want to use it anymore?
Or you wanted to download something else to try and want to remove some clutter. I did this all the time,especially when the number of screens was lower. didn't mean the app was junk, I just didn't need it all the time and I either had on my computer or could download it again so no worries.
Just because something isn't your favorite doesn't make it bad.
Also, if I use an app, play a game etc more than 3 times a week consistently I consider it my duty to review it. And if an app is bad, especially if it's also overpriced, I will review it for that also. And say what is bad. So not just 'this app sucks' but "the UI is clunky and cluttered making use slow"
Forcing prompts to rate an app you keep using, is still forcing a pop up right when you are trying to do something else. Minimizing that is a good idea.
Maybe Apple should make an App that is all about feedback, rating and sharing things you like. Nintendo has a decent one in their Nintendo Channel for the Wii. It's novel to go and just check out how long you've spent time in any game, and then you're there, so why not rate it? That app could wait until you've used an App for an hour over at least 2 days or something to make sure you gave it a chance (of course you could still go to the AppStore and rate it)
The App could get away with asking some info in the rating (age, interests, what the app is best for) and then make a local profile compared to a generic profile built around the app. It could then make suggestions of other Apps. This also gives a single place for a user to decide whether they want to share info like that or not, just for the purposes of rating.
Sure, only enthusiasts would spend a lot of time there, but I think on fun experience with it and one good suggestion as a reward would bring people back again and again. I know I'd rather look through the Apps I own to rate them rather than try to remember the name, search for it, and then go through the rating process.
Forcing prompts to rate an app you keep using, is still forcing a pop up right when you are trying to do something else. Minimizing that is a good idea.
What might have been a good idea would have been to keep the rate on delete feature and to use the iAd framework to occasionally display a "rate this app" "ad" for apps in use which the user has not rated.
What might have been a good idea would have been to keep the rate on delete feature and to use the iAd framework to occasionally display a "rate this app" "ad" for apps in use which the user has not rated.
Interesting. That sounds like a good idea, but then paid apps don't get rated unless the buyer chooses to use iAd, which may update buyers who paid money to not be bothered.
Perhaps Apple needs to also use an option that is invisible to the user that anonymously tecords the number of times an app is opened and used so that global stats can be listed on the App Store page.
Actually, the statistical effect developers have complained about is called a positive skew, not a negative skew, as referenced in the article. I am surprised that Apple didn't consider this before implementing the rating system. Goes to show how easy it is for any company to miss the effects of survey techniques!
Interesting. That sounds like a good idea, but then paid apps don't get rated unless the buyer chooses to use iAd, which may update buyers who paid money to not be bothered.
Perhaps Apple needs to also use an option that is invisible to the user that anonymously tecords the number of times an app is opened and used so that global stats can be listed on the App Store page.
Well, I was thinking a specific method that could be used by the developer to present a rate this app banner at the developers discretion, without having to actually always have ads in the app. Wouldn't have to be part of iAds but, but a similar mechanism, I guess.
I don't think recording app usage stats, even "anonymously" is appropriate, even though many developers are doing just that now. (Well, I'm not sure how "anonymously they are doing it.)
I don't think recording app usage stats, even "anonymously" is appropriate, even though many developers are doing just that now. (Well, I'm not sure how "anonymously they are doing it.)
How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.
PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used. I have apps that I've never used but still get updates for on a regular basis. This causes extra data to DLed for Apple, the ISPs and me, yet I should just be deleting a 2year old app I don't use, but "out of sight, out of mind" means that I don't even bother hunting these down and removing them. I think Windows 95's Program Manager has the right idea.
As a developer I appreciate this move. From my experience, it hurts free apps harder, because people often download them just to take a look and then may delete them, not necessarily because they are bad, but because they don't need them.
Back in December my son released an app which was calculating the recommended sun exposure time based on actual UV index, which apparently is useful mainly during the summer. He hoped to attract some Australian users but this didn't happen. The majority of the downloads were from the Northern hemisphere. After appearing in the "What's new" category the app was getting a fare number of downloads, and the negative ratings grew first. There were almost no comments to help to identify what customers don't like. With later versions he added an option to send feedback from within the app and got pretty positive feedback from there.
The initial version was free with an in-app purchase to enable the full functionality. After a couple of weeks the full functionality was unlocked and the app was made a paid download, and a separate lite version was released. The lite version followed the same pattern as the first version. Interestingly, the ratings for the paid version were much more generous.
My interpretation is that those who purchased the full version have read the description and made a decision whether they need the app or not, while many downloaded the free version just to take a look and quickly deleted it. The number of the updates suggests that more than 2/3 deleted the lite version.
PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used
You can't just ask customers what they want and then try to give that to them.
It was an obvious terrible idea on paper. Somehow it made it into the OS, where it proved how bad an idea it is. And somehow it lingered through multiple updates anyway. Thank goodness it will be gone?but the truly amazing story is that for now it?s still in 3.2 on iPad! Bizarre.
It?s just that it encourages ratings just from people who don?t like an app. It?s that it encourages ratings from people who don?t even CARE about the app and simply don?t have a use for it! If they download a great app (especially a free one) but it?s not the KIND of app they were after, it gets one star! Want a $1 diving tutor app, but you got a $1 driving tutor instead? Doesn?t matter how good that app is, you?re into diving, not driving. One star. Now multiply this effect by the number of people who randomly grab anything that?s free. Ugh.
We all want a simple answer. Is this app good at what it claims to do? What Apple really needs to do is find a way so that a customer can download any app they want to use without the penalty of paying for it first. Apple would give the user 24hrs to evaluate the app. At the end of the 24hrs, a popup would appear asking them if they want to purchase the app (if its a paid app)? At that time, the user could purchase it or delete it. Let the user decide if an app is good for their purpose, not some stranger.
If the app is free, they can just delete it. No harm, no foul.
I don't want to hi-jack the thread into another subject, but this is going to be even more relevant when iAD ads start. We may see many apps with two versions so that a user has a chance to evaluate an app without paying for it. The user may use the iAD version as a test version to see if they like the app but hate ads. They may delete the iAD version and purchase the paid version. The app store could grow bigger and bigger with this type of logic. It want help the user to see 300,000 apps available when so many have duplicate function because of iAd.
I just think relying on a raw rating system is a flawed system. Let me be the judge of the usefulness of the app. I can tell pretty quickly if its what I want.
From the level of bias I see in forums like AppleInsider I couldn't care less about what some other user thinks about any app or device. All one has to do is look at the user "reviews" on Cnet for example. In the case of the iPhone and iPad there were literally hundreds of user "reviews" before either product was available for purchase, the vast majority of which were strongly negative. We have learned over the years that vendors, their competitors, trolls, and fanboys all have vested interests in promoting or bashing any product. We know that a good number user "reviews" are made by anonymous, paid flacks, both pro and con a product.
So it would suit me just fine this sort of user review system disappeared all together.
Comments
The company followed up a few weeks later by tying reviews to the revision of the app, a feature which allowed users to filter out old complaints related to problems that had since been fixed
It's better, but could still be improved. When I post a new release itunes defaults to showing the "most helpful" reviews, since the current version of the app has no reviews. The issue the "most helpful" reviews were from version 1.0 which is almost two years old now and has no baring on the current version of the app. Apple won't remove these ancient reviews, or use any sort of aging so that a helpful review from 2 years ago isn't ranked as as-helpful now.
The other major issue with the review system is that developers still can't respond to customers reviews. Privately would be better than letting devs respond in the review threads so it doesn't turn into a post a review to get support, but letting us privately respond would let us address customers issues or seek clarification on the issue they are having.
Hmmm, now we're less likely to know if an app is not any good.
It seems fine that the system didn't capture positive ratings. It still resulted in lower ratings for apps that weren't as good. Who cares what the average is for all apps as long as the better ones are ranked higher?
Because apps (as well as many other things) should be judged on both a relative and absolute scale. If best app in a particular category was ranked highest by rating, but only rated out at 3 stars, I'd be slightly hesitant to buy.
All arguments about statistical accuracy aside, I think this is a bad move.
The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
Paraphrasing Henrik Ibsen (An Enemy of the People)::
What is true may not be popular.
What is popular may not be true.
As noted by others, we tend to voice our opinion more so when we have a negative experience or opposing perspective. It is not unusual too that there are those who solicit their friends to get good reviews. This fraud has been documented in Amazon book reviews.
One should trust reviews based mainly from reliable sources, those with credibility to weigh in on the matter.
CGC
But, in the long-run, this is probably the best way to do this, putting all ratings on an equal footing so that everyone who rated did so through the same mechanism.
The biggest problem with the ratings, from a user perspective, is that you can't sort/search the app store based on them.
Finally, Apple has resolved this terrible idea!
The current way of doing things ABSOLUTELY skews ratings downward into hell, because why else would you be deleting an app unless you didn't like it and didn't want to use it anymore?
Or you wanted to download something else to try and want to remove some clutter. I did this all the time,especially when the number of screens was lower. didn't mean the app was junk, I just didn't need it all the time and I either had on my computer or could download it again so no worries.
Just because something isn't your favorite doesn't make it bad.
Also, if I use an app, play a game etc more than 3 times a week consistently I consider it my duty to review it. And if an app is bad, especially if it's also overpriced, I will review it for that also. And say what is bad. So not just 'this app sucks' but "the UI is clunky and cluttered making use slow"
Maybe Apple should make an App that is all about feedback, rating and sharing things you like. Nintendo has a decent one in their Nintendo Channel for the Wii. It's novel to go and just check out how long you've spent time in any game, and then you're there, so why not rate it? That app could wait until you've used an App for an hour over at least 2 days or something to make sure you gave it a chance (of course you could still go to the AppStore and rate it)
The App could get away with asking some info in the rating (age, interests, what the app is best for) and then make a local profile compared to a generic profile built around the app. It could then make suggestions of other Apps. This also gives a single place for a user to decide whether they want to share info like that or not, just for the purposes of rating.
Sure, only enthusiasts would spend a lot of time there, but I think on fun experience with it and one good suggestion as a reward would bring people back again and again. I know I'd rather look through the Apps I own to rate them rather than try to remember the name, search for it, and then go through the rating process.
Forcing prompts to rate an app you keep using, is still forcing a pop up right when you are trying to do something else. Minimizing that is a good idea.
What might have been a good idea would have been to keep the rate on delete feature and to use the iAd framework to occasionally display a "rate this app" "ad" for apps in use which the user has not rated.
What might have been a good idea would have been to keep the rate on delete feature and to use the iAd framework to occasionally display a "rate this app" "ad" for apps in use which the user has not rated.
Interesting. That sounds like a good idea, but then paid apps don't get rated unless the buyer chooses to use iAd, which may update buyers who paid money to not be bothered.
Perhaps Apple needs to also use an option that is invisible to the user that anonymously tecords the number of times an app is opened and used so that global stats can be listed on the App Store page.
Interesting. That sounds like a good idea, but then paid apps don't get rated unless the buyer chooses to use iAd, which may update buyers who paid money to not be bothered.
Perhaps Apple needs to also use an option that is invisible to the user that anonymously tecords the number of times an app is opened and used so that global stats can be listed on the App Store page.
Well, I was thinking a specific method that could be used by the developer to present a rate this app banner at the developers discretion, without having to actually always have ads in the app. Wouldn't have to be part of iAds but, but a similar mechanism, I guess.
I don't think recording app usage stats, even "anonymously" is appropriate, even though many developers are doing just that now. (Well, I'm not sure how "anonymously they are doing it.)
I'm always going to try to take credit for this change as I emailed Apple suggesting the idea a while before they did it.
Was Windows 7 your idea too?
The biggest problem with the ratings, from a user perspective, is that you can't sort/search the app store based on them.
Doesn't Google let you do that?
I don't think recording app usage stats, even "anonymously" is appropriate, even though many developers are doing just that now. (Well, I'm not sure how "anonymously they are doing it.)
How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.
PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used. I have apps that I've never used but still get updates for on a regular basis. This causes extra data to DLed for Apple, the ISPs and me, yet I should just be deleting a 2year old app I don't use, but "out of sight, out of mind" means that I don't even bother hunting these down and removing them. I think Windows 95's Program Manager has the right idea.
Was Windows 7 your idea too?
Those "Windows 7 was my idea" ads have the subconscious tagline of "Windows 7: designed by idiots".
The other obvious answer is, Windows 7 is mostly Apple's ideas, poorly executed :P
Back in December my son released an app which was calculating the recommended sun exposure time based on actual UV index, which apparently is useful mainly during the summer. He hoped to attract some Australian users but this didn't happen. The majority of the downloads were from the Northern hemisphere. After appearing in the "What's new" category the app was getting a fare number of downloads, and the negative ratings grew first. There were almost no comments to help to identify what customers don't like. With later versions he added an option to send feedback from within the app and got pretty positive feedback from there.
The initial version was free with an in-app purchase to enable the full functionality. After a couple of weeks the full functionality was unlocked and the app was made a paid download, and a separate lite version was released. The lite version followed the same pattern as the first version. Interestingly, the ratings for the paid version were much more generous.
My interpretation is that those who purchased the full version have read the description and made a decision whether they need the app or not, while many downloaded the free version just to take a look and quickly deleted it. The number of the updates suggests that more than 2/3 deleted the lite version.
PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used
You can't just ask customers what they want and then try to give that to them.
Those "Windows 7 was my idea" ads have the subconscious tagline of "Windows 7: designed by idiots".
The other obvious answer is, Windows 7 is mostly Apple's ideas, poorly executed :P
Was Windows 7 your idea too?
I always felt it is it should be called "I bitched about what's was wrong is Vista and you mostly fixed it".
It?s just that it encourages ratings just from people who don?t like an app. It?s that it encourages ratings from people who don?t even CARE about the app and simply don?t have a use for it! If they download a great app (especially a free one) but it?s not the KIND of app they were after, it gets one star! Want a $1 diving tutor app, but you got a $1 driving tutor instead? Doesn?t matter how good that app is, you?re into diving, not driving. One star. Now multiply this effect by the number of people who randomly grab anything that?s free. Ugh.
If the app is free, they can just delete it. No harm, no foul.
I don't want to hi-jack the thread into another subject, but this is going to be even more relevant when iAD ads start. We may see many apps with two versions so that a user has a chance to evaluate an app without paying for it. The user may use the iAD version as a test version to see if they like the app but hate ads. They may delete the iAD version and purchase the paid version. The app store could grow bigger and bigger with this type of logic. It want help the user to see 300,000 apps available when so many have duplicate function because of iAd.
I just think relying on a raw rating system is a flawed system. Let me be the judge of the usefulness of the app. I can tell pretty quickly if its what I want.
So it would suit me just fine this sort of user review system disappeared all together.