I like it... I like it not: Evaluating User Ratings Noise in Recommender Systems

“A common approach to handle digital information overload is to offer users a personalized access to information. Recommender Systems (RS), for instance, automatically suggest new content that should comply with the user’s taste. In the RS literature, these predictions of user preferences are typically obtained by means of approaches such as collaborative filtering – i.e. taking into account other users rating history in order to model the taste of peers – or content-based – i.e. using existing content descriptions to uncover relations between items. Regardless of the approach, these personalized services share a common concern:modeling the user’s taste. Therefore, such systems need to somehow capture likes and dislikes in order to model or infer the user’s preferences.

User preferences can be captured via either implicit or explicit user feedback. In the implicit approach [12], user preferences are inferred by observing consumption patterns. However, modeling user preferences on the basis of implicit feedback has a major limitation: the underlying assumption is that the amount of time that users spend accessing a given content is directly proportional to how much they like it. Consequently, explicit feedback is the favored approach for gathering information on user preferences. Although this approach adds a burden on the users and different users might respond differently to incentives [6], it is generally accepted that explicit data is more reliable in most situations.

The preferred method for capturing explicit preference information from users consists of rating questionnaires [1], where users are asked to provide feedback – via a value point on a fixed scale – on how much they like some content. Typically, scales range from 0 or 1 to 5 or 10 and are quantized to integer values.”