Citation: De Langhe, B., Fernbach, P., & Lichtenstein, D. (2016). Navigating by the Stars: Investigating the Actual and Perceived Validity of Online User Ratings. Journal of Consumer Research, 42(6), 817-833. DOI: 10.1093/jcr/ucv047.
I picked up this article because an editor is asking my writing team to consider user perceptions of star ratings as we investigate how users evaluate user reviews.
In short, consumers rely heavily on star rating.
Broad conclusion #5 from the paper: “Consumers… place enormous weight on the average user rating as an indicator of objective quality compared to other cues. They also fail to moderate their reliance on he average user rating when sample size is insufficient. Averages based on small samples and distributions with high variance are treated the same as averages based on large samples and distributions with low variance. (p. 819).”
The “Theoretical Background” section has a “User Ratings and Consumer Quality Inferences” sub-section which elaborates on the existing literature.
The “Consumer Studies” section details their work and offers conclusions:
First, consumers relied most heavily on average user ratings, which was true regardless of whether quality was defined as Consumer Reports quality or generically. Second, consumers did not moderate their reliance on average user ratings depending on whether sample size was sufficient or not. Third, in two of the three studies, consumers also relied on price but much less so than on average user ratings. Finally, consumers did use the number of ratings as a direct indicator of quality (p. 828).
Added Bonus: Product Categorization
I’ve been differentiating amazon.com products as search/experience/credence goods because that was the classification used by the product review literature.
The authors differentiate by type of quality–objective or subjective–being assessed. That seems to largely be the distinction between search and experience goods: Experience goods have more subjective qualities that have to be personally experienced in order to make an informed purchase decision; search goods have more objective qualities to consider before purchase.