All arguments about statistical accuracy aside, I think this is a bad move.
The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
I am of the exact same view here, shame they removed it, it dealt with a lot of crap apps effectively.
All arguments about statistical accuracy aside, I think this is a bad move.
The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
This will mostly work, but not always. In my previous post I gave as an example (mostly) summer app released during the winter in northern hemisphere. Many users could have deleted it because they don't need it in December. The summer will come but the ratings will stay. This will also apply to some apps with more specific functionality. A good example here is a client for an online service. Even if the description clearly states what it is supposed to do many will just download it and when they find that an account is needed just delete it with 1 star rating. If you compare this with a client to the same service that may be relevant, but:
1. An alternative client might not exist
2. People are more likely to judge based on their experience with other apps with completely different functionality. In this case the app may look like a crap without being that bad.
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
Let's say they weren't installed an equal number of times - let's say one of the apps - the one with 500 1-star ratings - has 50 times the number of users. Just like that, the number of bad reviews doesn't mean a thing. And I don't recall seeing install counts in the summary when comparing apps.
Of course, now that the negative counts don't mean anything, let's think 'but then look at how many good reviews there are!'. The problem even there is that I've seen people promote their apps by handing out free iTunes coupons in exchange for positive feedback, so I wouldn't count on that being accurate either. But certainly only soliciting feedback in a negative scenario (deletion) and expecting the the result of those reviews (rating) or the number of those reviews to say anything useful is questionable.
How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.
Well, no, because usage doesn't necessarily equate to app quality. Consider 2 apps, A & B.
A. You use this app frequently, but, you don't think it's great, it's just something that happens to require frequent use by it's nature. But, if a better alternative were available, you'd dump it in a second for a new app.
B. You absolutely love this app, it's beautifully designed, no bugs, a joy to work with. But, you don't actually us it that often because it's something you only need for specific circumstances.
Now, based on usage stats, app A, which you have a sort of love-hate relationship with would be highly ranked, and app B, which you think is the most incredible app on your phone would be ranked poorly. Without the context of what these apps are for and how they are used, usage stats are meaningless measures of quality.
I think for ratings to have any meaning, they have to be knowingly provided by users. What would be nice is if you had to, in addition to just tapping a star, actually list, or check off, some pros and cons and maybe add at least a brief review. That's why I was thinking of some relatively unobtrusive way to solicit app reviews from inside the app, as well as actually expanding the optional rate on delete feature. This is why I was thinking it could use functionality from iAd -- rate the app without leaving -- to pop-up a short survey form.
Well, no, because usage doesn't necessarily equate to app quality. Consider 2 apps, A & B.
A. You use this app frequently, but, you don't think it's great, it's just something that happens to require frequent use by it's nature. But, if a better alternative were available, you'd dump it in a second for a new app.
B. You absolutely love this app, it's beautifully designed, no bugs, a joy to work with. But, you don't actually us it that often because it's something you only need for specific circumstances.
Now, based on usage stats, app A, which you have a sort of love-hate relationship with would be highly ranked, and app B, which you think is the most incredible app on your phone would be ranked poorly. Without the context of what these apps are for and how they are used, usage stats are meaningless measures of quality.
I think for ratings to have any meaning, they have to be knowingly provided by users. What would be nice is if you had to, in addition to just tapping a star, actually list, or check off, some pros and cons and maybe add at least a brief review. That's why I was thinking of some relatively unobtrusive way to solicit app reviews from inside the app, as well as actually expanding the optional rate on delete feature. This is why I was thinking it could use functionality from iAd -- rate the app without leaving -- to pop-up a short survey form.
Those examples accentuate my point. If there is an app that is used often by many, but not so great that leaves a room for growth wide open for other developers, which may get the devs for App A to work harder to make another app or risk losing business.
However, I think site reviews do a great job here already. My first wish is to get Apple to use the FairPlay model it uses for video rentals and allow trial periods of apps to be turned on before you actually commit to buying the app.
Now if only Apple can find a way to stop developers from adding dozens to hundreds of fake, positive reviews for their crappy apps. That is just as annoying.
Those examples accentuate my point. If there is an app that is used often by many, but not so great that leaves a room for growth wide open for other developers, which may get the devs for App A to work harder to make another app or risk losing business.
However, I think site reviews do a great job here already. My first wish is to get Apple to use the FairPlay model it uses for video rentals and allow trial periods of apps to be turned on before you actually commit to buying the app.
I must be confused about what you mean by usage statistics. It seems, as pointed out above, that an app that is launched frequently would get a high rating based on usage stats, regardless of how good it were. Without an explicit rating or review, how would one distinguish between between frequently used apps that are great and frequently used apps that suck but there isn't a viable alternative and infrequently used apps that are fantastic?
Trial periods would be nice, but I don't think we'll see that any time soon. Apple seems to be focusing a lot of their App Store decisions on keeping developers happy (despite the clamor to the contrary on the current Apple is evil for making me use Objective-C threads) and a lot of stuff like lists of top downloaded and top grossing apps benefits successful developers but doesn't really help users find the best apps. And, I don't think most developers want trial periods.
Looks like i'm in the minority. But I still prefer the ability to rate upon deletion. It is the most reliable way to ensure that poor quality apps get low ratings. Sure, there are tradeoffs involved, but the above reason trumps all others for my needs.
How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.
PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used. I have apps that I've never used but still get updates for on a regular basis. This causes extra data to DLed for Apple, the ISPs and me, yet I should just be deleting a 2year old app I don't use, but "out of sight, out of mind" means that I don't even bother hunting these down and removing them. I think Windows 95's Program Manager has the right idea.
This is exactly what I was thinking. And this extra information need not replace the current rating model. It can just be extra information about the app that you could see which may (or may not) help someone choose to buy the app or not... Bad metaphore: the last opened date of a file doesn't tell you how "good" that file is, but is convenient information to track and know sometimes...
Comments
All arguments about statistical accuracy aside, I think this is a bad move.
The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
I am of the exact same view here, shame they removed it, it dealt with a lot of crap apps effectively.
All arguments about statistical accuracy aside, I think this is a bad move.
The purpose of the rating system is not to be statistically accurate, but help users see how other people like or dislike applications, who cares about a bias towards negative ratings?
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
This will mostly work, but not always. In my previous post I gave as an example (mostly) summer app released during the winter in northern hemisphere. Many users could have deleted it because they don't need it in December. The summer will come but the ratings will stay. This will also apply to some apps with more specific functionality. A good example here is a client for an online service. Even if the description clearly states what it is supposed to do many will just download it and when they find that an account is needed just delete it with 1 star rating. If you compare this with a client to the same service that may be relevant, but:
1. An alternative client might not exist
2. People are more likely to judge based on their experience with other apps with completely different functionality. In this case the app may look like a crap without being that bad.
Imagine there are 2 apps with similar functionality. Both are downloaded and installed 1000 times. The first app is horrible and 500 people delete it and rate it one star. The second app is much better, but not perfect, and 75 people delete it rating it 1 star. As it so happens no one takes the time to go into iTunes and rate either one without deleting it.
A new user comes along and wants that functionality, they see the 2 apps. The first has 500 one 1-star ratings, the second has 75 1-star star ratings. I mean really, which one is the user going to try? How is the statical error and bias hurting? Bad apps get a slew of negative ratings for a reason.
Let's say they weren't installed an equal number of times - let's say one of the apps - the one with 500 1-star ratings - has 50 times the number of users. Just like that, the number of bad reviews doesn't mean a thing. And I don't recall seeing install counts in the summary when comparing apps.
Of course, now that the negative counts don't mean anything, let's think 'but then look at how many good reviews there are!'. The problem even there is that I've seen people promote their apps by handing out free iTunes coupons in exchange for positive feedback, so I wouldn't count on that being accurate either. But certainly only soliciting feedback in a negative scenario (deletion) and expecting the the result of those reviews (rating) or the number of those reviews to say anything useful is questionable.
How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.
Well, no, because usage doesn't necessarily equate to app quality. Consider 2 apps, A & B.
A. You use this app frequently, but, you don't think it's great, it's just something that happens to require frequent use by it's nature. But, if a better alternative were available, you'd dump it in a second for a new app.
B. You absolutely love this app, it's beautifully designed, no bugs, a joy to work with. But, you don't actually us it that often because it's something you only need for specific circumstances.
Now, based on usage stats, app A, which you have a sort of love-hate relationship with would be highly ranked, and app B, which you think is the most incredible app on your phone would be ranked poorly. Without the context of what these apps are for and how they are used, usage stats are meaningless measures of quality.
I think for ratings to have any meaning, they have to be knowingly provided by users. What would be nice is if you had to, in addition to just tapping a star, actually list, or check off, some pros and cons and maybe add at least a brief review. That's why I was thinking of some relatively unobtrusive way to solicit app reviews from inside the app, as well as actually expanding the optional rate on delete feature. This is why I was thinking it could use functionality from iAd -- rate the app without leaving -- to pop-up a short survey form.
"I loved it, that's why I'm deleting it."
Well, no, because usage doesn't necessarily equate to app quality. Consider 2 apps, A & B.
A. You use this app frequently, but, you don't think it's great, it's just something that happens to require frequent use by it's nature. But, if a better alternative were available, you'd dump it in a second for a new app.
B. You absolutely love this app, it's beautifully designed, no bugs, a joy to work with. But, you don't actually us it that often because it's something you only need for specific circumstances.
Now, based on usage stats, app A, which you have a sort of love-hate relationship with would be highly ranked, and app B, which you think is the most incredible app on your phone would be ranked poorly. Without the context of what these apps are for and how they are used, usage stats are meaningless measures of quality.
I think for ratings to have any meaning, they have to be knowingly provided by users. What would be nice is if you had to, in addition to just tapping a star, actually list, or check off, some pros and cons and maybe add at least a brief review. That's why I was thinking of some relatively unobtrusive way to solicit app reviews from inside the app, as well as actually expanding the optional rate on delete feature. This is why I was thinking it could use functionality from iAd -- rate the app without leaving -- to pop-up a short survey form.
Those examples accentuate my point. If there is an app that is used often by many, but not so great that leaves a room for growth wide open for other developers, which may get the devs for App A to work harder to make another app or risk losing business.
However, I think site reviews do a great job here already. My first wish is to get Apple to use the FairPlay model it uses for video rentals and allow trial periods of apps to be turned on before you actually commit to buying the app.
Those examples accentuate my point. If there is an app that is used often by many, but not so great that leaves a room for growth wide open for other developers, which may get the devs for App A to work harder to make another app or risk losing business.
However, I think site reviews do a great job here already. My first wish is to get Apple to use the FairPlay model it uses for video rentals and allow trial periods of apps to be turned on before you actually commit to buying the app.
I must be confused about what you mean by usage statistics. It seems, as pointed out above, that an app that is launched frequently would get a high rating based on usage stats, regardless of how good it were. Without an explicit rating or review, how would one distinguish between between frequently used apps that are great and frequently used apps that suck but there isn't a viable alternative and infrequently used apps that are fantastic?
Trial periods would be nice, but I don't think we'll see that any time soon. Apple seems to be focusing a lot of their App Store decisions on keeping developers happy (despite the clamor to the contrary on the current Apple is evil for making me use Objective-C threads) and a lot of stuff like lists of top downloaded and top grossing apps benefits successful developers but doesn't really help users find the best apps. And, I don't think most developers want trial periods.
How about if it was done when you sync your iPhone to iTunes and you get a choice to send your anonymous app usage stats to Apple or not, just like with your anonymous iPhone OS and app crash data? Devs are already monitoring your usage already, there is just no built in API for it.
PS: I'd love for the iPhone OS to monitor my app usage and then have that information listed in iTunes under Apps. I'd love to be able to order my apps like I can with other iTunes content; by usage number, amount of time used, and last used. I have apps that I've never used but still get updates for on a regular basis. This causes extra data to DLed for Apple, the ISPs and me, yet I should just be deleting a 2year old app I don't use, but "out of sight, out of mind" means that I don't even bother hunting these down and removing them. I think Windows 95's Program Manager has the right idea.
This is exactly what I was thinking. And this extra information need not replace the current rating model. It can just be extra information about the app that you could see which may (or may not) help someone choose to buy the app or not... Bad metaphore: the last opened date of a file doesn't tell you how "good" that file is, but is convenient information to track and know sometimes...