The UX of rating systems

How would we design a rating system that captures relevant, useful information while achieving a desirable experience for the user? In a current project I’m working on that provides personalized recommendations of restaurants and dishes, I was tasked to design the rating system. In this short article, I’m going to summarize some of the insights and observations I’ve gathered about different rating methods, their pros, cons, and user behaviors they create. The binary method — affirmation/conformity: When it comes to rating, users tend to go for the extreme — they either had an exceptionally great experience, which they rate 5 stars, or a very poor one and then they would rate it a one rate star. This user behavior led companies like Youtube and Netflix to ditch the stars and shift to the all or nothing rating system, using “like” and “dislike” buttons. Social platforms such as Facebook, Instagram, and Twitter continued this binary approach and used the “Like”, “Love” and “Favorite” buttons — without a negative option — as a way to express affirmation, conformity, and eventually: virality. Medium took this to the next level when they introduced the “Clap” (and came up with a unique design icon for it). That move transforms the user (reader) to being an evaluator, and this is an idea I wanted to experiment with for the restaurants’ app since my intuition was that it will allow the app to capture foodies sentiment more accurately. Clapping on MediumIn the early stages of research for the restaurants’ app I discovered that the most influencing factor for users visiting a restaurant for the first time is getting a recommendation from someone they know, or a foodie they trust and has a like-minded taste and preferences. This insight led me to go for the binary approach using the label “Recommend”. Benefits of this approach include: The binary button should work when it comes to users seeking recommendation/affirmation from friends or fellow users. It requires very quick and easy action from users. On top of these two, this allows the product the ability to match future personalized recommendations for the user based on their own affirmations. Getting granular, informative data from review systems Even though the binary button sounded like a perfect solution based on my case, the client wanted to add more value beyond just providing the number of recommendations. He wanted to harness the wisdom of crowds to help users who wish to visit a restaurant for the time know what to expect. To fulfill this goal, I wanted to understand what parameters were most important for users when choosing to dine in a restaurant they haven’t been to, so feedback from other users would be specific to those parameters. Aside from the obvious menu and price (which the app is already offering), users stated the following, which I grouped into 2 categories: Vibes: fine dining / casual / romantic / groups…; noise level; crowdedness Food: type, quality, and presentation So, what are our options to address this? The UX of the review system depends on the information we want to get Here are some of the main methods being used by popular apps these days that fit into users’ mental models of reviews. The textual review A good example of this type of review is Yelp: After rating with stars, the user is prompted to write a review and is being loosely guided by an example. Cons: User is guided but not obliged to refer to the specific criterias we’re looking for, which often leads to long texts that are not actually relevant. Only a very small percentage of users writes a review versus participating in other methods of feedback Pros: Users get to express their opinions with no interruption, which creates engagement. In addition, Users are prompted to upload photos (optional): Cons: This “interferes” with the design and imagery of the app. Feasibility wise: this is expensive (server costs) and it’s harder to gather this data in a glanceable to understand way, so less feasible for the client to implement and maintain. Pros: A picture is worth a thousand words — it adds credibility and authenticity. Creates engagement Keeping in mind my goal is to get focused feedback about specific parameters and considering the other cons, I preferred to move away from the textual review and only make it an additional option to more focused forms of review: Close-ended questions Those are good for describing something: an issue, a type of something, an attribute. A good example is Google Maps questions that are being shown to users after visiting a place, and have yes/no/not sure options: Scale / range These are good for describing a level or a degree of something: service, product, etc. Airbnb is using this for rating focused categories in the stay experience, so after the user provides a general rating for the stay, he or she is being asked to provide rates for 6 parameters that are crucial for hosts and other users: While this should work well for describin

The UX of rating systems
How would we design a rating system that captures relevant, useful information while achieving a desirable experience for the user? In a current project I’m working on that provides personalized recommendations of restaurants and dishes, I was tasked to design the rating system. In this short article, I’m going to summarize some of the insights and observations I’ve gathered about different rating methods, their pros, cons, and user behaviors they create. The binary method — affirmation/conformity: When it comes to rating, users tend to go for the extreme — they either had an exceptionally great experience, which they rate 5 stars, or a very poor one and then they would rate it a one rate star. This user behavior led companies like Youtube and Netflix to ditch the stars and shift to the all or nothing rating system, using “like” and “dislike” buttons. Social platforms such as Facebook, Instagram, and Twitter continued this binary approach and used the “Like”, “Love” and “Favorite” buttons — without a negative option — as a way to express affirmation, conformity, and eventually: virality. Medium took this to the next level when they introduced the “Clap” (and came up with a unique design icon for it). That move transforms the user (reader) to being an evaluator, and this is an idea I wanted to experiment with for the restaurants’ app since my intuition was that it will allow the app to capture foodies sentiment more accurately. Clapping on MediumIn the early stages of research for the restaurants’ app I discovered that the most influencing factor for users visiting a restaurant for the first time is getting a recommendation from someone they know, or a foodie they trust and has a like-minded taste and preferences. This insight led me to go for the binary approach using the label “Recommend”. Benefits of this approach include: The binary button should work when it comes to users seeking recommendation/affirmation from friends or fellow users. It requires very quick and easy action from users. On top of these two, this allows the product the ability to match future personalized recommendations for the user based on their own affirmations. Getting granular, informative data from review systems Even though the binary button sounded like a perfect solution based on my case, the client wanted to add more value beyond just providing the number of recommendations. He wanted to harness the wisdom of crowds to help users who wish to visit a restaurant for the time know what to expect. To fulfill this goal, I wanted to understand what parameters were most important for users when choosing to dine in a restaurant they haven’t been to, so feedback from other users would be specific to those parameters. Aside from the obvious menu and price (which the app is already offering), users stated the following, which I grouped into 2 categories: Vibes: fine dining / casual / romantic / groups…; noise level; crowdedness Food: type, quality, and presentation So, what are our options to address this? The UX of the review system depends on the information we want to get Here are some of the main methods being used by popular apps these days that fit into users’ mental models of reviews. The textual review A good example of this type of review is Yelp: After rating with stars, the user is prompted to write a review and is being loosely guided by an example. Cons: User is guided but not obliged to refer to the specific criterias we’re looking for, which often leads to long texts that are not actually relevant. Only a very small percentage of users writes a review versus participating in other methods of feedback Pros: Users get to express their opinions with no interruption, which creates engagement. In addition, Users are prompted to upload photos (optional): Cons: This “interferes” with the design and imagery of the app. Feasibility wise: this is expensive (server costs) and it’s harder to gather this data in a glanceable to understand way, so less feasible for the client to implement and maintain. Pros: A picture is worth a thousand words — it adds credibility and authenticity. Creates engagement Keeping in mind my goal is to get focused feedback about specific parameters and considering the other cons, I preferred to move away from the textual review and only make it an additional option to more focused forms of review: Close-ended questions Those are good for describing something: an issue, a type of something, an attribute. A good example is Google Maps questions that are being shown to users after visiting a place, and have yes/no/not sure options: Scale / range These are good for describing a level or a degree of something: service, product, etc. Airbnb is using this for rating focused categories in the stay experience, so after the user provides a general rating for the stay, he or she is being asked to provide rates for 6 parameters that are crucial for hosts and other users: While this should work well for describing things like crowdedness, noise level, and tastiness of the food, we go back to users’ tendency to being extreme, especially when they don’t have any guidance about the meaning of different rates. Why not both? An interesting finding from Airbnb, made me think that a mixed-method (scale + close-ended question) is also a great option that works well for certain information. Airbnb not only asks users to rate specific elements, but also give specific reasoning from a list of possible ones, plus provides the option to specify in the users own words: Uber is using a similar approach: If a user rates a ride under 5 starts, he’s prompted to choose the specific issue that causes him to not think it was a perfect ride (or a compliment, if it’s a 5 start ride): This mixed approach is easy for the user and specific enough for providing valuable and relatively accurate information. Long story short Applying those insights into practical prototyping, I’ve decided to test a feedback system that combines different rating approaches: A binary “Recommend” button — “would you recommend this place to a friend?” Close-ended question: for information about restaurant vibe and food type Scale question + close-ended options: for information about the quality of food Scale questions: for crowdedness and noise level. Optional text review + picture upload — for the foodies who like to express themselves beyond those, and still provide valuable information. The UX of rating systems was originally published in UX Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.