Title:

Identifying top rated products

United States Patent 8666830

Abstract:

The top rated products in a product category are identified by computing a score for each product in the category. The score is computed by calculating a time decay factor for each product category. The time decay factor is utilized to decay the ratings of each product as a function of time. A relevance value is also computed for each product. The relevance value is utilized to modify the ratings for each product in view of the recency of the ratings. The score may be computed for each product using the ratings for the product, the computed time decay factor, and/or the relevance value. The computed scores are utilized to rank the products in the product category.

Inventors:

Dillard, Logan L. (Seattle, WA, US)

Sonne, Samuel S. (Seattle, WA, US)

Fox, Eric B. (Seattle, WA, US)

Dicker, Russell A. (Seattle, WA, US)

Mongrain, Scott A. (Seattle, WA, US)

Sonne, Samuel S. (Seattle, WA, US)

Fox, Eric B. (Seattle, WA, US)

Dicker, Russell A. (Seattle, WA, US)

Mongrain, Scott A. (Seattle, WA, US)

Application Number:

13/075793

Publication Date:

03/04/2014

Filing Date:

03/30/2011

Export Citation:

Assignee:

Amazon Technologies, Inc. (Reno, NV, US)

Primary Class:

Other Classes:

705/27.1

International Classes:

Field of Search:

705/26.1, 705/27.1

View Patent Images:

US Patent References:

20040225577 | System and method for measuring rating reliability through rater prescience | November, 2004 | Robinson | 705/26 |

Primary Examiner:

Zurita, James

Attorney, Agent or Firm:

Hope Baldauff, LLC

Claims:

What is claimed is:

1. A system for ranking, computing and displaying a likely next rating for a product, the system comprising: a merchant system comprising one or more application servers; and an online shopping module executing on the one or more application servers configured to maintain data identifying a plurality of products, to maintain data indicating which of a plurality of categories each of the products is assigned to, to receive and maintain ratings for each of the products, and to compute a time decay factor for each product category, compute a relevance value for each product in each product category, compute a likely next rating score for each of the products in each product category based upon one or more of the time decay factor and the relevance value, and to cause the likely next rating score for at least one of the products to be displayed.

2. The system of claim 1, wherein the online shopping module is further configured to cause an average rating for a product to be displayed in conjunction with the likely next rating score.

3. The system of claim 1, wherein compute a time decay factor for each product category comprises computing a time decay factor for each category that minimizes a prediction error for the ratings for each product in each product category.

4. The system of claim 3, wherein compute a relevance value for each product in each product category comprises: computing a current rating velocity for each product; computing an average rating velocity for each product; and computing the relevance value as a ratio of the average rating velocity and the current rating velocity.

5. The system of claim 3, wherein compute a relevance value for each product in a product category comprises: computing an average rating velocity for each product; determining an expected number of ratings for each product during a previous time period based on the average rating velocity; determining an amount of time taken for each product to actually receive the expected number of ratings; and computing the relevance value for each product as a ratio of the amount of time during the previous time period and the amount of time taken for the product to receive the expected number of ratings.

6. The system of claim 3, wherein compute a likely next rating score for each of the products in the product category further comprises: computing a prior probability distribution for each product category based upon ratings of the products in the product category; decaying the ratings for each of the products using a time decay factor; modifying the decayed ratings using the relevance value for the product; computing a posterior probability distribution for each product in each category based upon the prior probability distribution for the category and the ratings for each product as decayed by the time decay factor and modified by the relevance value; and computing the likely next rating score for each of the products in each product category by computing a predicted next rating for each product in each product category based upon the posterior probability distribution for each product.

7. The system of claim 6, wherein computing a prior probability distribution for each product category comprises computing a prior probability distribution for each category that minimizes the prediction error for the ratings for each product in the product category.

8. A computer-implemented method for ranking, computing and displaying a likely next rating for a product, the method comprising performing computer-implemented operations for: maintaining data identifying a plurality of products by way of one or more computing systems; maintaining data by way of one or more computing systems indicating which of a plurality of categories each of the plurality of products is assigned to; receiving and maintaining ratings for each of the products; computing a time decay factor for each product category; computing a relevance value for each product in each product category; computing, by way of one or more systems, a likely next rating score for each of the products in each product category based upon one or more of the time decay factor and the relevance value; and presenting the likely next rating score for at least one of the products.

9. The computer-implemented method of claim 8, further comprising presenting an average rating for a product in conjunction with the likely next rating score.

10. The computer-implemented method of claim 8, wherein computing a time decay factor for each product category comprises computing a time decay factor for each category that minimizes a prediction error for the ratings for each product in each product category.

11. The computer-implemented method of claim 10, wherein computing a relevance value for each product in each product category comprises: computing a current rating velocity for each product; computing an average rating velocity for each product; and computing the relevance value as a ratio of the average rating velocity and the current rating velocity.

12. The computer-implemented method of claim 10, wherein computing a relevance value for each product in a product category comprises: computing an average rating velocity for each product; determining an expected number of ratings for each product during a previous time period based on the average rating velocity; determining an amount of time taken for each product to actually receive the expected number of ratings; and computing the relevance value for each product as a ratio of the amount of time during the previous time period and the amount of time taken for the product to receive the expected number of ratings.

13. The computer-implemented method of claim 10, wherein computing a likely next rating score for each of the products in the product category further comprises: computing a prior probability distribution for each product category based upon ratings of the products in the product category; decaying the ratings for each of the products using a time decay factor; modifying the decayed ratings using the relevance value for the product; computing a posterior probability distribution for each product in each category based upon the prior probability distribution for the category and the ratings for each product as decayed by the time decay factor and modified by the relevance value; and computing the likely next rating score for each of the products in each product category by computing a predicted next rating for each product in each product category based upon the posterior probability distribution for each product.

14. The computer-implemented method of claim 13, wherein computing a prior probability distribution for each product category comprises computing a prior probability distribution for each category that minimizes the prediction error for the ratings for each product in the product category.

15. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to: maintain data identifying a plurality of products; maintain data indicating which of a plurality of categories each of the plurality of products is assigned to; maintain ratings for each of the products; compute a time decay factor for each product category; compute a relevance value for each product in each product category; compute a likely next rating score for each of the products in each product category based upon one or more of the time decay factor and the relevance value; and present the likely next rating score for at least one of the products.

16. The non-transitory computer-readable storage medium of claim 15, having further computer-executable instructions stored thereupon which, when executed by the computer, cause the computer to present an average rating for a product in conjunction with the likely next rating score.

17. The non-transitory computer-readable storage medium of claim 15, wherein compute a time decay factor for each product category comprises computing a time decay factor for each category that minimizes a prediction error for the ratings for each product in each product category.

18. The non-transitory computer-readable storage medium of claim 17, wherein compute a relevance value for each product in each product category comprises: computing a current rating velocity for each product; computing an average rating velocity for each product; and computing the relevance value as a ratio of the average rating velocity and the current rating velocity.

19. The non-transitory computer-readable storage medium of claim 17, wherein compute a relevance value for each product in a product category comprises: computing an average rating velocity for each product; determining an expected number of ratings for each product during a previous time period based on the average rating velocity; determining an amount of time taken for each product to actually receive the expected number of ratings; and computing the relevance value for each product as a ratio of the amount of time during the previous time period and the amount of time taken for the product to receive the expected number of ratings.

20. The non-transitory computer-readable storage medium of claim 17, wherein compute a likely next rating score for each of the products in the product category further comprises: computing a prior probability distribution for each product category based upon ratings of the products in the product category; decaying the ratings for each of the products using a time decay factor; modifying the decayed ratings using the relevance value for the product; computing a posterior probability distribution for each product in each category based upon the prior probability distribution for the category and the ratings for each product as decayed by the time decay factor and modified by the relevance value; and computing the likely next rating score for each of the products in each product category by computing a predicted next rating for each product in each product category based upon the posterior probability distribution for each product.

21. The non-transitory computer-readable storage medium of claim 20, wherein compute a prior probability distribution for each product category comprises computing a prior probability distribution for each category that minimizes the prediction error for the ratings for each product in the product category.

1. A system for ranking, computing and displaying a likely next rating for a product, the system comprising: a merchant system comprising one or more application servers; and an online shopping module executing on the one or more application servers configured to maintain data identifying a plurality of products, to maintain data indicating which of a plurality of categories each of the products is assigned to, to receive and maintain ratings for each of the products, and to compute a time decay factor for each product category, compute a relevance value for each product in each product category, compute a likely next rating score for each of the products in each product category based upon one or more of the time decay factor and the relevance value, and to cause the likely next rating score for at least one of the products to be displayed.

2. The system of claim 1, wherein the online shopping module is further configured to cause an average rating for a product to be displayed in conjunction with the likely next rating score.

3. The system of claim 1, wherein compute a time decay factor for each product category comprises computing a time decay factor for each category that minimizes a prediction error for the ratings for each product in each product category.

4. The system of claim 3, wherein compute a relevance value for each product in each product category comprises: computing a current rating velocity for each product; computing an average rating velocity for each product; and computing the relevance value as a ratio of the average rating velocity and the current rating velocity.

5. The system of claim 3, wherein compute a relevance value for each product in a product category comprises: computing an average rating velocity for each product; determining an expected number of ratings for each product during a previous time period based on the average rating velocity; determining an amount of time taken for each product to actually receive the expected number of ratings; and computing the relevance value for each product as a ratio of the amount of time during the previous time period and the amount of time taken for the product to receive the expected number of ratings.

6. The system of claim 3, wherein compute a likely next rating score for each of the products in the product category further comprises: computing a prior probability distribution for each product category based upon ratings of the products in the product category; decaying the ratings for each of the products using a time decay factor; modifying the decayed ratings using the relevance value for the product; computing a posterior probability distribution for each product in each category based upon the prior probability distribution for the category and the ratings for each product as decayed by the time decay factor and modified by the relevance value; and computing the likely next rating score for each of the products in each product category by computing a predicted next rating for each product in each product category based upon the posterior probability distribution for each product.

7. The system of claim 6, wherein computing a prior probability distribution for each product category comprises computing a prior probability distribution for each category that minimizes the prediction error for the ratings for each product in the product category.

8. A computer-implemented method for ranking, computing and displaying a likely next rating for a product, the method comprising performing computer-implemented operations for: maintaining data identifying a plurality of products by way of one or more computing systems; maintaining data by way of one or more computing systems indicating which of a plurality of categories each of the plurality of products is assigned to; receiving and maintaining ratings for each of the products; computing a time decay factor for each product category; computing a relevance value for each product in each product category; computing, by way of one or more systems, a likely next rating score for each of the products in each product category based upon one or more of the time decay factor and the relevance value; and presenting the likely next rating score for at least one of the products.

9. The computer-implemented method of claim 8, further comprising presenting an average rating for a product in conjunction with the likely next rating score.

10. The computer-implemented method of claim 8, wherein computing a time decay factor for each product category comprises computing a time decay factor for each category that minimizes a prediction error for the ratings for each product in each product category.

11. The computer-implemented method of claim 10, wherein computing a relevance value for each product in each product category comprises: computing a current rating velocity for each product; computing an average rating velocity for each product; and computing the relevance value as a ratio of the average rating velocity and the current rating velocity.

12. The computer-implemented method of claim 10, wherein computing a relevance value for each product in a product category comprises: computing an average rating velocity for each product; determining an expected number of ratings for each product during a previous time period based on the average rating velocity; determining an amount of time taken for each product to actually receive the expected number of ratings; and computing the relevance value for each product as a ratio of the amount of time during the previous time period and the amount of time taken for the product to receive the expected number of ratings.

13. The computer-implemented method of claim 10, wherein computing a likely next rating score for each of the products in the product category further comprises: computing a prior probability distribution for each product category based upon ratings of the products in the product category; decaying the ratings for each of the products using a time decay factor; modifying the decayed ratings using the relevance value for the product; computing a posterior probability distribution for each product in each category based upon the prior probability distribution for the category and the ratings for each product as decayed by the time decay factor and modified by the relevance value; and computing the likely next rating score for each of the products in each product category by computing a predicted next rating for each product in each product category based upon the posterior probability distribution for each product.

14. The computer-implemented method of claim 13, wherein computing a prior probability distribution for each product category comprises computing a prior probability distribution for each category that minimizes the prediction error for the ratings for each product in the product category.

15. A non-transitory computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by a computer, cause the computer to: maintain data identifying a plurality of products; maintain data indicating which of a plurality of categories each of the plurality of products is assigned to; maintain ratings for each of the products; compute a time decay factor for each product category; compute a relevance value for each product in each product category; compute a likely next rating score for each of the products in each product category based upon one or more of the time decay factor and the relevance value; and present the likely next rating score for at least one of the products.

16. The non-transitory computer-readable storage medium of claim 15, having further computer-executable instructions stored thereupon which, when executed by the computer, cause the computer to present an average rating for a product in conjunction with the likely next rating score.

17. The non-transitory computer-readable storage medium of claim 15, wherein compute a time decay factor for each product category comprises computing a time decay factor for each category that minimizes a prediction error for the ratings for each product in each product category.

18. The non-transitory computer-readable storage medium of claim 17, wherein compute a relevance value for each product in each product category comprises: computing a current rating velocity for each product; computing an average rating velocity for each product; and computing the relevance value as a ratio of the average rating velocity and the current rating velocity.

19. The non-transitory computer-readable storage medium of claim 17, wherein compute a relevance value for each product in a product category comprises: computing an average rating velocity for each product; determining an expected number of ratings for each product during a previous time period based on the average rating velocity; determining an amount of time taken for each product to actually receive the expected number of ratings; and computing the relevance value for each product as a ratio of the amount of time during the previous time period and the amount of time taken for the product to receive the expected number of ratings.

20. The non-transitory computer-readable storage medium of claim 17, wherein compute a likely next rating score for each of the products in the product category further comprises: computing a prior probability distribution for each product category based upon ratings of the products in the product category; decaying the ratings for each of the products using a time decay factor; modifying the decayed ratings using the relevance value for the product; computing a posterior probability distribution for each product in each category based upon the prior probability distribution for the category and the ratings for each product as decayed by the time decay factor and modified by the relevance value; and computing the likely next rating score for each of the products in each product category by computing a predicted next rating for each product in each product category based upon the posterior probability distribution for each product.

21. The non-transitory computer-readable storage medium of claim 20, wherein compute a prior probability distribution for each product category comprises computing a prior probability distribution for each category that minimizes the prediction error for the ratings for each product in the product category.

Description:

An online merchant may provide a large number of products for sale to customers over the Internet. Customers may access information regarding the available products through a web site or other information service provided by the online merchant. Product information may include information needed by a customer to identify and purchase a particular product, such as a description of the product, an image of the product, product specifications, the price and availability of the product, and a mechanism for initiating a purchase of the product.

In order to enhance the online shopping experience and to allow customers to fully research a particular product, an online merchant might also provide a mechanism to allow customers to provide quantitative ratings for products. For instance, an appropriate user interface might be provided through which customers can provide a quantitative rating for each product, such as rating the product between one and five “stars” or providing a numerical rating of the product between one and 100.

The average quantitative rating received for each product is commonly displayed with the product information. In this manner, potential purchasers of each product are provided with a concise indicator of the average rating provided by other customers for each product. It can be difficult, however, for customers to accurately identify the products that are top rated in a category using an average quantitative rating for products in the category. One reason for this is that an average rating does not take into account the number of ratings received for each product. As a result, a product with a single 5 star rating might be rated higher than another product with one thousand ratings that has a 4.5 star average. Accordingly, identifying top rated products based upon average quantitative ratings may be confusing and even potentially misleading to customers.

It is with respect to these and other considerations that the disclosure made herein is presented.

FIG. 1 is a system diagram showing aspects of one illustrative operating environment for implementing the concepts and technologies disclosed herein for identifying top rated products;

FIG. 2 is a functional block diagram showing aspects of one mechanism for identifying top rated products, according to one embodiment disclosed herein;

FIG. 3 is a flow diagram showing one illustrative routine for identifying top rated products, according to one embodiment disclosed herein;

FIGS. 4A-4B are flow diagrams showing one illustrative routine disclosed herein in embodiments for computing a prior probability distribution and a time decay factor for use in identifying top rated products;

FIG. 5 is a flow diagram showing one illustrative routine disclosed herein in embodiments for modifying a prior probability distribution based upon the number of products in a product category;

FIG. 6 is a flow diagram showing one illustrative routine for computing a score for a product for use in identifying top rated products according to one embodiment disclosed herein;

FIGS. 7A-7B are flow diagrams showing several illustrative routines for modifying product ratings based upon product relevance according to one embodiment disclosed herein;

FIG. 8 is a flow diagram showing one illustrative routine for computing a score for a product assigned to multiple product categories according to one embodiment disclosed herein;

FIG. 9 is a flow diagram showing one illustrative routine for computing and displaying a likely next rating score for a product according to one embodiment disclosed herein;

FIG. 10 is a flow diagram showing one illustrative routine for identifying and addressing rating discrepancies according to one embodiment disclosed herein;

FIG. 11 is a flow diagram showing one illustrative routine for ranking products based upon scores computed for sets of products according to one embodiment disclosed herein;

FIG. 12 is a flow diagram showing one illustrative routine for identifying top rated products by attributes other than product ratings, according to one embodiment disclosed herein; and

FIG. 13 is a computer architecture diagram showing one illustrative computer hardware architecture for use in computing devices configured to implement the concepts and technologies disclosed herein in one embodiment.

The following detailed description is directed to technologies for identifying top rated products. As discussed briefly above, previous mechanisms for identifying top rated products utilize average product ratings in order to identify top rated products in a product category. Products having the highest average rating are identified as the top rated products. The average rating, however, does not take into account the number of ratings that have been provided for each product. As a result, a product with a single 5 star rating might be rated higher than another product with one thousand ratings that has a 4.5 star average. Using the average rating to identify top rated products also ignores the expectations of what distribution of ratings is normal for a product in a given category. This measure may be important in determining the number of ratings needed in order to have confidence that the actual ratings for a product are truly indicative of the product's quality.

Using the average rating for products to determine top rated products also ignores the reality that customers' opinions of products often change over time. This can be the result of a change in the product itself, which might render the previous ratings invalid, or a change in customers' expectations or perceptions of a product. For example, the best digital camera from five years ago is likely to be inferior to models available today. Even though an older digital camera received excellent ratings in the past, those ratings likely do not accurately reflect customers' current opinions.

In order to address the issues described, and others, the concepts and technologies disclosed herein may identify the top rated products in a product category as a function of the value, quantity, and recency of product ratings. Data identifying the top rated products can then be displayed to a customer on an appropriate web page, through a user interface widget, or in another manner. Data identifying the top rated products might also be utilized to sort search results or in another fashion.

According to one embodiment disclosed herein, a score (which may be referred to herein as a “likely next rating score”) is computed for each product in a product category. The score is utilized to rank the products in the product category against one another to identify the top rated products in the product category. In one embodiment, the score is computed by establishing a prior probability distribution of quantitative ratings of products in the product category. The prior probability distribution is computed in one embodiment by selecting a prior probability distribution that minimizes a prediction error for each product in the product category.

The prior probability distribution for a category and the ratings for each product are then utilized to compute a posterior probability distribution for each product. A predicted next rating for each product is then computed based upon the posterior probability distribution computed for each product. The predicted next rating for a product is utilized as the score for ranking each product against other products in the same product category.

When computing the posterior probability distribution for a product, the ratings for the product might be decayed utilizing a time decay factor. The time decay factor is utilized to decay each rating as a function of time. For instance, an exponential function might be applied to decay the weight of each rating over time. The time decay factor may be computed by selecting a time decay factor that minimizes a prediction error for the ratings of each product in the product category. The time decay factor might also be regularized in embodiments by increasing the prediction error inversely to the magnitude of the time decay factor.

The actual ratings for each product might also be modified based upon a relevance value for the corresponding product. The relevance value is utilized to modify the ratings in view of the recency of the ratings received for the product. In one embodiment, the relevance value is calculated by computing a current rating velocity and an average rating velocity for a product. The current rating velocity is the number of ratings received for the product in a recent time period. The average rating velocity is the average number of ratings received per time period for the product during its lifetime. In this embodiment, the relevance value is the ratio of the average rating velocity and the current rating velocity.

In another embodiment, the relevance value is calculated by computing the average rating velocity for the product. An expected number of ratings for the product during a previous time period is then computed based upon the average rating velocity. An amount of time necessary for the product to actually receive the expected number of ratings is then determined. The relevance value is then computed as a ratio of the amount of time during the previous time period and the amount of time actually taken for the product to receive the expected number of ratings. For instance, if the average rating velocity of a product is four ratings per month, then the product would be expected to receive four ratings during the previous month. If it actually took the product six weeks to obtain four ratings then the relevance value for the product is ⅔ (four weeks divided by six weeks). The magnitude of the relevance value might be restricted in various embodiments.

In other embodiments, the score for each product may be exposed to customers in lieu of or in addition to an average product rating. The computed scores might also be utilized for other back-end calculations. For example, the score for a product might be compared to the average rating for the product to determine if a discrepancy exists. If a discrepancy exists, appropriate action might be taken such as notifying customers, performing an analysis of the ratings for the product to determine the cause of the discrepancy, suspending sales of a product, or notifying a product vendor.

In another embodiment, products are not ranked according to product category. In this embodiment, an entire product catalog might be treated as a single set of products with a single prior probability distribution and time decay factor. In other embodiments, a product catalog is divided into divisions, referred to herein as “sets,” that do not necessarily align with product categories. For instance, products might be assigned to sets according to the type of specialty store that would sell each product. A prior probability distribution and time decay factor may then be computed for each set and utilized to compute a score for each product. The products might then be ranked against other products in the same set, category, or other sub-division of products.

In another embodiment, attributes other than a general product rating can be utilized to generate a product ranking. For instance, values for product attributes like performance, ease of use, and others can be utilized to generate a product ranking showing the top ranked products by attribute. Additional details regarding this process and others will be described below.

It should be appreciated that the subject matter presented herein may be implemented as a computer process, a computer-controlled apparatus, a computing system, or an article of manufacture, such as a computer-readable storage medium. These and various other features will become apparent from a reading of the following disclosure and a review of the associated drawings.

While the subject matter described herein is presented in the general context of program modules that execute on one or more computing devices, those skilled in the art will recognize that other implementations may be performed in combination with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types.

Those skilled in the art will appreciate that the subject matter described herein may be practiced on or in conjunction with other computer system configurations beyond those described below, including multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, handheld computers, personal digital assistants, electronic book readers, wireless telephone devices, special-purposed hardware devices, network appliances, or the like. The embodiments described herein may also be practiced in distributed computing environments, where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

In the following detailed description, references are made to the accompanying drawings that form a part hereof, and that show, by way of illustration, specific embodiments or examples. The drawings herein are not drawn to scale. Like numerals represent like elements throughout the several figures.

FIG. 1 and the following description are intended to provide a brief, general description of a suitable computing environment in which the embodiments described herein may be implemented. In particular, FIG. 1 is a system and network diagram that shows an illustrative operating environment **100** including several software components for identifying top rated products.

The environment **100** includes a customer **102** that uses a customer computer **104** to access a merchant system **108** across a network **106**. The customer **102** may be an individual or entity that desires to purchase one or more products from the online merchant that operates the merchant system **108**. The customer computer **104** may be a personal computer (“PC”), a desktop workstation, a laptop, a tablet computer, a notebook, a personal digital assistant (“PDA”), an electronic-book reader, a game console, a set-top box, a consumer electronics device, a server computer, or any other computing device capable of connecting to the network **106** and communicating with the merchant system **108**.

The network **106** may be a local-area network (“LAN”), a wide-area network (“WAN”), the Internet, or any other networking topology known in the art that connects the customer computer **104** to the merchant system **108**. The merchant system **108** may include a number of application servers **110** that provide various online shopping services to the customer computer **104** over the network **106**. The customer **102** may use a client application executing on the customer computer **104** to access and utilize the online shopping services provided by the application servers **110**.

In one embodiment the client application is a web browser application, such as the MOZILLA® FIREFOX® web browser from MOZILLA FOUNDATION of Mountain View, Calif. The web browser application exchanges data with the application servers **110** in the merchant system **108** using the hypertext transfer protocol (“HTTP”) over the network **106**. The client application might also be a stand-alone client application configured for communicating with the application servers **110**. The client application might also utilize any number of communication methods known in the art to communicate with the merchant system **108** and/or the application servers **110** across the network **106**, including remote procedure calls, SOAP-based web services, remote file access, proprietary client-server architectures, and the like.

The application servers **110** may execute a number of modules in order to provide the online shopping services to the customer computer **104**. The modules may execute on a single application server **110** or in parallel across multiple application servers in the merchant system **108**. In addition, each module may consist of a number of subcomponents executing on different application servers **110** or other computing devices in the merchant system **108**. The modules may be implemented as software, hardware, or any combination of the two.

According to one embodiment, an online shopping module **112** executes on the application servers **110**. The online shopping module **112** provides functionality for the customer **102** to browse and purchase products available from the online merchant that operates the merchant system **108**. For instance, the online shopping module **112** may retrieve information regarding a particular product offered for sale by the online merchant from a product database **114**, generate a product page **116** containing the product information, and transmit the product page **116** over the network **106** to the client application executing on the client computer **104** for display to the customer **102**. Information regarding each product offered for sale might be stored in a product database **114** maintained by the merchant system **108**. The products represented in the product database **114** may be organized by categories.

The product page **116** transmitted by the application servers **110** to the client computer **104** may be a web page consisting of hypertext markup language (“HTML”), extensible markup language (“XML”), and/or JAVASCRIPT object notation (“JSON”) that contains the product information along with instructions regarding how the product details are to be displayed on the customer computer **104**, for example. It will be appreciated that any number of methods and technologies may be utilized that allow the online shopping module **112** to encode the product information in the product page **116** and send the product page **116** to the customer computer **104** for display to the customer **102**. Other information transmitted from the online shopping module **112** to the client computer **104** may be similarly formatted.

According to embodiments, the online shopping module **112** is also configured to provide functionality for allowing the customer **102** to provide a product rating **118**. A product rating **118** is a quantitative rating of a product. For instance, the online shopping module **112** may be configured to allow the customer **102** to provide a rating of one to five “stars” for each product. Other scales might also be utilized. The online shopping module **112** stores the product ratings **118** in a ratings database **120** maintained by the merchant system **108**. For example, the merchant system **108** might maintain the ratings database **120** using an appropriate database technology or other type of data storage system available to the online shopping module **112**.

As will be described in greater detail below, the online shopping module **112** operates in conjunction with a ranking module **122** to identify top rated products in various product categories and to provide a product ranking **126** identifying the top rated products to the customer computer **104**. The top rated products are those products in a given product category that have been top rated by customers. Various mechanisms are disclosed herein for identifying the top rated products in a product category.

As will be discussed in greater detail below, the top rated products in each product category can be identified in a manner that takes into account the value and quantity of product ratings and their recency. A product ranking **126** identifying the top rated products can then be displayed to the customer **102** directly, on an appropriate product page **116**, through a user interface widget presented on the product page **116** or on another page, or in another manner. The product ranking **126** might also be utilized by other components to sort search results or perform other functions. The product ranking **126** showing the top rated products in a particular category might also be utilized in other ways not disclosed herein.

As will also be described in greater detail below, the ranking module **122** utilizes the product ratings **118** stored in the ratings database **120** to compute a score for each product in each product category. The score is then utilized to rank order the products and to generate the product ranking **126**. The ranking module **122** provides the product ranking **126** to the online shopping module **112** for inclusion on the product page **116**, for direct transmission to the client computer **104**, or for use in another fashion. It should be appreciated that the functionality disclosed herein as being performed by the ranking module **122** might also be performed directly by the online shopping module **112** or by another component. Additional details regarding the embodiments presented herein for identifying top rated products will be provided below with regard to FIGS. 2-13.

Turning now to FIG. 2, additional details will be provided regarding the functionality provided by the merchant system **108** for identifying top rated products. In particular, FIG. 2 is a functional block diagram showing aspects of one mechanism disclosed herein for identifying top rated products.

As discussed briefly above, the product database **114** stores data identifying a number of products **202**A-**202**B offered for sale by the merchant operating the merchant system **108**. The products might be organized into product categories. In the example shown in FIG. 2, the products **202**A-**202**B are members of the product category **204**.

As also briefly described above, the online shopping module **112** provides functionality for customers of the merchant system **110** to provide a product rating **118** for the products offered for sale. For instance, in the example shown in FIG. 2, the product **202**A has received the ratings **118**A-**118**N and the product **202**B has received the ratings **118**B-**118**NN (the ratings **118**A-**118**N and **118**B-**118**NN may be referred to herein singularly as a “rating **118**” or collectively as “the ratings **118**”). The ratings **118** may be stored in the ratings database **120** in the manner described above. It should be appreciated that although a limited number of products **202**, ratings **118**, and product categories **204** have been illustrated in FIG. 2, the merchant system **108** might maintain many more such products **202**, ratings **118**, and categories **204**.

As also briefly described above, the embodiments disclosed herein provide a mechanism for generating a product ranking **126** that identifies the top rated products in a particular product category **204**. In order to generate the product ranking **126**, a score **206**A-**206**B is computed for each of the products **202**A-**202**B, respectively, in a product category **204**. The scores **206**A-**206**B (which may be referred to herein singularly as a “score **206**” or collectively as “the scores **206**”) are then provided to a ranking process **208**. The ranking process **208** numerically ranks the scores **206** to generate the product ranking **126**. As discussed above with regard to FIG. 1, the product ranking **126** might be provided directly to a customer, provided on a product page **116**, provided by way of a user interface widget, or provided in another manner. The product ranking **126** might also be utilized to inform or sort search results and in other ways.

In order to compute the scores **206** for products **202** in a given product category **204**, a prior probability distribution (“PPD”) **210** of ratings **118** for products **202** in the product category **204** is computed. According to one embodiment, the PPD **210** is computed to minimize the prediction error for the ratings **118** for each product **202** in the product category **204**. Details regarding one illustrative mechanism for computing the PPD **210** will be provided below with regard to FIGS. 4A-4B. As will be described in greater detail below with regard to FIG. 5, the PPD **210** might also be modified based upon the number of products in the category **204**.

Once the PPD **210** has been computed for the category **204**, a score **206** is computed for each product **202** in the product category **204**. A score **206**A is computed for a particular product **202**A by computing a posterior probability distribution **212**A for the product **202**A. The posterior probability distribution **212**A is based upon the PPD **210** for the product category **204** and the actual ratings **118**A-**118**N for the product **202**A.

According to one embodiment, the ratings **118**A-**118**N for each product **202** are decayed using a time decay factor **214** for the product category **204**. The time decay factor **214** is utilized to decay the ratings **118** for products **202** in a product category **204** as a function of time. For instance, an exponential function might be applied to decay the weight of each rating **118**A-**118**N over time. The time decay factor **214** may be computed by selecting a time decay factor **214** that minimizes a prediction error for the ratings **118** of each product **202** in the product category **204**. The time decay factor **214** might also be regularized in embodiments by increasing the prediction error as an inverse function of the magnitude of the time decay factor **214**. Details regarding one mechanism disclosed herein for computing the time decay factor **214** for each product **202** will be described below with regard to FIGS. 4A-4B.

The ratings **118**A-**118**N for a product **202**A might also be influenced by a relevance value **216**A. The relevance value **216**A is utilized to modify the magnitude of the ratings **118**A-**118**N for a product in view of the recency of the ratings **118**A-**118**N received for the product **202**A. Details regarding several mechanisms disclosed herein for computing the relevance value **216**A are provided below with regard to FIGS. 7A-7B.

Once the posterior probability distribution **212**A has been computed for a product **202**A, a score **206**A can be generated for the product **202**A. A score **206**B is computed for another product **202**B in a similar fashion using the posterior probability distribution **212**B for that product **212**B, the time decay factor **214**, and the relevance value **216**B for the product **202**B.

The score **206** for each of the products **202** is determined by computing a predicted next rating for each product. The predicted next rating for each product **202** is a prediction of what the next rating **118** for each product will be that is based on the posterior probability distribution **212** for each product, as influenced by the time decay factor **214** and the relevance value **216** for the product. Additional details regarding the computation of the score **206** for each product **202** will be provided below with regard to FIG. 6. Once the scores **206** have been computed for each product in the product category **204**, the scores **206** are provided to the ranking process **208**, which rank orders the scores **206** to generate the product ranking **126**. Additional details regarding this process, and others, will be presented below with regard to FIGS. 3-13.

Turning now to FIG. 3, additional details will be provided regarding the embodiments described herein for identifying top rated products. It should be appreciated that the logical operations described herein are implemented (1) as a sequence of computer implemented acts or program modules running on a computing system and/or (2) as interconnected machine logic circuits or circuit modules within the computing system. The implementation is a matter of choice dependent on the performance and other requirements of the computing system. Accordingly, the logical operations described herein with reference to the various FIGURES are referred to variously as operations, structural devices, acts, or modules. These operations, structural devices, acts, and modules may be implemented in software, in firmware, in special purpose digital logic, and any combination thereof. It should also be appreciated that more or fewer operations may be performed than shown in the figures and described herein. These operations may also be performed in parallel, or in a different order than those described herein.

FIG. 3 is a flow diagram showing one illustrative routine **300** for identifying top rated products, according to one embodiment disclosed herein. In the routine **300** shown in FIG. 3, the top rated products for a single product category **204** are identified. It should be appreciated that the routine **300** might be utilized in a similar fashion to identify the top rated products in multiple categories **204**.

The routine **300** begins at operation **302**, where the PPD **210** and a time decay factor **214** are computed for the product category **204**. One illustrative mechanism for computing the PPD **210** and the time decay factor **214** for a product category **204** will be described below with regard to FIGS. 4A-4B. It should be appreciated that other mechanisms might also be utilized. From operation **302**, the routine **300** proceeds to operation **304**.

At operation **304**, a score **206** is computed for each of the products **202** in the product category **204**. The score **206** is computed based, in part, upon the PPD **210** for the category **204**, the actual ratings **118** for each product **202**, and the time decay factor **214**. The score **206** might also be influenced by the relevance value **216** for each product **202**. One mechanism for computing the score **206** will be described below with regard to FIG. 6. Additional details regarding the embodiments disclosed herein for modifying the score **206** for each product **202** using the relevance value **216** will be provided below with regard to FIGS. 7A-7B. From operation **304**, the routine **300** proceeds to operation **306**.

At operation **306**, the ranking process **208** is utilized to rank order the scores **206** for each of the products **202** in the product category **204**. The output of the ranking process **208** is the product ranking **126**. Once the product ranking **126** has been generated, the routine **300** proceeds to operation **308** where the product ranking **126** is displayed, such as to a customer **102** of the merchant system **108**. As discussed above, the product ranking **126** might also be utilized in other ways. From operation **308**, the routine **300** proceeds to operation **310**, where it ends.

FIGS. 4A-4B are flow diagrams showing one illustrative routine **400** disclosed herein in embodiments for computing a PPD **210** and a time decay factor **214** for a product category **204** for use in identifying top rated products. The PPD **210** represents a histogram of the average ratings **118** for products in the product category **204**. The time decay factor **214** is computed for a product category **204** and utilized to decay the ratings **118** for each product **202** in the category **204** as a function of time. Each product **202** in the product category **204** begins with a ratings histogram equivalent to the PPD **210**. In the routine **400** shown in FIGS. 4A-4B, a hill-climbing algorithm is utilized to identify a PPD **210** and a time decay factor **214** for a product category **204** that minimize a prediction error for the ratings of each product in the product category. It should be appreciated, however, that other mechanisms might also be utilized for computing a PPD **210** and a time decay factor **214** for the product category **204**.

In one implementation, the effect of the ratings **118** changing over time is addressed by applying an exponential time decay to the weight that each rating **118** has. In this example, the time decay factor **214** represents a “half life” for the exponential time decay. For example, if a product category **204** is determined to have a rating half life of one year, a rating **118** on a product **202** in that category **204** that is one year old will be worth one half of a rating **118** submitted today. A rating **118** from two years ago would be worth one quarter of a rating **118** submitted today, and so on. In the embodiment shown in FIGS. 4A-4B, the PPD **210** and the time decay factor **214** are computed at the same time. In other embodiments, these factors might be computed separately. These values might also be assigned to each product category **204** manually.

The routine **400** begins at operation **402** where an initial prior probability distribution is specified for the product category **204**. The initial prior probability distribution sets a baseline for the hill-climbing algorithm and may be specified arbitrarily, based upon an estimate of the prior probability distribution **210** for the product category **204**, or in another fashion. Once the initial prior probability distribution has been specified, the routine **400** proceeds from operation **402** to operation **404**.

At operation **404**, an initial time decay factor is specified for the product category **204**. The initial time decay factor sets a baseline for the hill-climbing algorithm and may be specified arbitrarily, based upon an estimate of the time decay factor **214** for the product category **204**, or in another fashion. Once the initial time decay factor has been specified, the routine **400** proceeds from operation **404** to operation **406**.

At operation **406**, a variable is initialized that is utilized to keep track of the current product in the product category **204**. The variable is initially set to the first product in the product category **204**. From operation **406**, the routine **400** proceeds to operation **408**, where the actual ratings for the product category **204** are modified to reflect the risk averse nature of customers. This accounts for the fact that in a well-rated category, such as digital single lens reflex cameras, a new product with zero ratings will have a relatively high prior probability distribution **210**. However, when consumers view a camera with zero ratings, they typically view the product as having a certain amount of risk associated with its purchase because no product ratings have actually been received.

In one embodiment, a certain amount of noise is added to the actual ratings for the product in order to reflect the risk averse nature of customers. For instance, a certain number of reviews having an average of a 3 star rating might be added to the actual ratings for the product. Because customer reviews tend toward high ratings, this serves to reduce the prior probability distribution **210** to reflect the risk averseness of customers. It should be appreciated that this operation is optional and that this operation might be performed using other mechanisms.

From operation **408**, the routine **400** proceeds to operation **410**, where another variable is initialized that is utilized to keep track of the current rating under consideration. This variable is initialized at operation **410** to the first rating for the current product in the product category **204**. From operation **410**, the routine **400** proceeds to operation **412**.

At operation **412**, the current rating is decayed utilizing the current time decay factor. As discussed above, for instance, if the current time decay factor is one year, the current rating will be decayed by one half if the rating is one year old in one embodiment. Once the current rating has been decayed by the current time decay factor, the decayed rating is added to the prior probability distribution. The routine **400** then proceeds from operation **412** to operation **414**.

At operation **414**, a predicted next rating is computed for the current product. In order to compute the predicted next rating, the average rating from the initial prior probability distribution is calculated. For instance, the initial prior probability distribution for a category **204** might be two 1 star ratings, three 2 star ratings, four 3 star ratings, three 4 star ratings, and one 5 star rating. In this case the predicted next rating is computed by applying the appropriate weights to the ratings in the initial prior probability distribution and taking the average of the sum of the weighted values. In the example given above, the predicted next rating for the product is 2.846. (i.e. (2*1+3*2+4*3+3*4+1*5)/13 ratings=2.846). Once the predicted next rating for the current product and current rating has been computed, the routine **400** proceeds to operation **416**.

At operation **416**, a prediction error is computed for the current product and the current rating by subtracting the predicted next rating from the actual current rating for the current product. In the example given above, the prediction error is 0.154 if the current rating for the current product is 3 stars (i.e. 3−2.846=0.154). The prediction error for the current rating and the current product is added to any previous prediction error computed for other ratings in the same category using the same initial prior probability distribution and time decay factor. From operation **416**, the routine **400** proceeds to operation **418**.

At operation **418**, a determination is made as to whether there are any more ratings **118** for the current product **202**. If so, the routine **400** proceeds from operation **418** to operation **420** where the variable utilized to identify the current rating **118** is incremented to identify the next rating **118** for the current product **202**. From operation **420**, the routine **400** proceeds to operation **412**, where the next rating is decayed and the prediction error is updated in the manner described above. This process continues until there are no more ratings **118** for the current product **202**.

When there are no remaining ratings **118** for the current product **202**, the routine **400** proceeds from operation **418** to operation **422**. At operation **422**, a determination is made as to whether there are more products **202** in the product category **204**. If so, the routine **400** proceeds from operation **422** to operation **424**.

At operation **424**, the variable utilized to identify the current product **202** is incremented to identify the next product **202** in the product category **204**. The routine **400** then proceeds from operation **424** to operation **408**, where the operations described above are repeated in order to update the prediction error in view of the ratings **118** for the next product **202**.

If there are no remaining products **202** in the product category **204** to be processed, the routine **400** proceeds from operation **422** to operation **425**, shown in FIG. 4B. At operation **425**, regularization penalties might be added to the predicted error. For instance, in one embodiment the current time decay factor is regularized. Regularization of the time decay factor refers to a process of increasing the prediction error computed for a particular time decay factor as an inverse function of the magnitude of the time decay factor. In one embodiment, the smaller the time decay factor, the greater the increase that is applied to the computed prediction error. It should be appreciated that the regularization of the time decay factor is optional and may not be performed in all embodiments. Additionally, regularization might also be performed on the magnitude of the prior probability distribution. From operation **425**, the routine **400** proceeds to operation **426**.

At operation **426**, the current prior probability distribution, time decay factor, and the computed prediction error are stored for later comparison to the prediction error computed for other prior probability distributions and time decay factors. The routine **400** then proceeds to operation **428**, where a determination is made as to whether other prior probability distributions and/or time decay factors remain to be tested. If so, the routine **400** proceeds from operation **428** to operation **430** where the initial prior probability distribution and/or time decay factor are modified. For instance, the initial prior probability distribution and/or time decay factor may be incremented. The prediction error is also reset to zero at operation **430**. The routine **400** then proceeds from operation **430** to operation **406** where the prediction error for the modified prior probability distribution and time decay factor is computed in the manner described above.

If no additional prior probability distributions or time decay factors remain, the routine **400** proceeds from operation **428** to operation **432**. At operation **432**, the prior probability distribution and time decay factor that minimize the prediction error for all of the ratings **118** for all of the products **202** in the product category **204** are selected as the prior probability distribution **210** and time decay factor **214** for the product category **204**. The routine **400** then proceeds to operation **434**, where it ends. It should be appreciated that the process illustrated in FIGS. 4A-4B for computing the prior probability distribution **210** and time decay factor **214** for a product category **204** is merely illustrative and that other mechanisms might be utilized with the embodiments disclosed herein.

FIG. 5 is a flow diagram showing one illustrative routine **500** disclosed herein for modifying a prior probability distribution **210** for a product category **204** based upon the number of products **202** in the product category **204**. The routine **500** shown in FIG. 5 might be utilized to modify the computed prior probability distribution **210** for product categories **204** having a small number of products **202** and/or a small number of ratings **118**. For instance, if a product category **204** only has one 5 star rating **118**, the predicted next rating for all products in the category would be 5 stars, which may be unrealistic. The routine **500** shown in FIG. 5 is utilized to provide a more realistic prior probability distribution **210** for product categories **204** that have few products **202** and/or ratings **118**.

The routine **500** begins at operation **502**, where the prior probability distribution **210** for the product category **204** is computed in the manner described above with regard to FIGS. 4A-4B. Once the prior probability distribution **210** has been computed, the routine **500** proceeds to operation **504** where a default prior probability distribution is computed for the product category **204**. In one embodiment, the default prior probability distribution **210** is computed as the average of all of the ratings **118** for all of the products **202** in all of the product categories **204**. In other embodiments, the default prior probability distribution **210** is computed in another manner or specified manually. As will be described in greater detail below, the prior probability distribution **210** is skewed toward the default prior probability distribution **210** depending upon the number of products **202** in the product category **204**.

Once the default prior probability distribution **210** has been computed or assigned at operation **504**, the routine **500** proceeds to operation **506**. At operation **506**, a weight is applied to the computed prior probability distribution **210** that is based upon the actual number of products in the product category **204**. In one embodiment, the weight is one (i.e. no weight) when the product category **204** has two hundred or more products. If the product category **204** has fewer than two hundred products, the weight is increased inversely to the number of products **202** in the category. In this manner, a product category **204** with only one product would have a higher weight than a product category **204** with 150 products. It should be appreciated that the examples given above are merely illustrative and that the weight given to the computed prior probability distribution **210** might be varied in other ways. Additionally, in other embodiments, an appropriate weight might be applied to the default prior probability distribution **210** for a category rather than to the computed prior probability distribution **210**.

From operation **506** the routine **500** proceeds to operation **508**, where an average is calculated of the weighted computed prior probability distribution **210** and the default prior probability distribution **210** that was computed at operations **454** and **456**. The calculated average is utilized as the prior probability distribution **210** for the product category **204** at operation **510**. The routine **500** then proceeds to operation **512**, where it ends.

It should be appreciated that the mechanism illustrated in FIG. 5 causes the prior probability distribution **210** for a product category **204** with very few products **202** and/or ratings **118** to skew highly toward the default prior probability distribution **210**. For product categories **204** with a larger number of products **202** and/or ratings **118**, the process shown in FIG. 5 will have little effect on the prior probability distribution **210**.

FIG. 6 is a flow diagram showing one illustrative routine **600** for computing a score **206** for a product **202** for use in identifying top rated products according to one embodiment disclosed herein. The routine **600** begins at operation **602**, where the prior probability distribution **210** for the product category **204** is retrieved. The time decay factor **214** is also retrieved at operation **602**. From operation **602**, the routine **600** proceeds to operation **604**.

At operation **604**, the ratings **118** for the product **202** for which a score **206** is being computed are identified. Once the ratings **118** have been identified, the routine **600** proceeds to operation **606** where each rating **118** is decayed using the time decay factor **214** for the product category **204** in the manner described above. Once the ratings **118** have been decayed, the routine **600** proceeds to operation **608** where the ratings are modified based upon the current relevance of the product using the relevance value **216** in one embodiment. Several illustrative mechanisms will be described below with reference to FIGS. 7A-7B for computing the relevance value **216** and modifying the ratings for the current product. From operation **608**, the routine **600** proceeds to operation **610**.

At operation **610**, the modified product ratings **118** for the current product **202** are added to the prior probability distribution **210** to generate a posterior probability distribution **212** for the product. The routine **600** then proceeds to operation **612**, where a predicted next rating is computed for the current product. The predicted next rating is computed based upon the average rating in the posterior probability distribution **212** for the product **202**. The predicted next rating for the current product **202** is utilized as the score **206** for the product at operation **614**. From operation **614**, the routine **600** proceeds to operation **616**, where it ends.

FIG. 7A is a flow diagram showing one illustrative routine **700** for modifying a posterior probability distribution **212** based upon the relevance of a product **202** according to one embodiment disclosed herein. As discussed above, a relevance value **216** might be computed for each product **202** and utilized to modify the ratings **118** for the product **202** in view of the recency of the ratings **118**. FIG. 7A shows one illustrative process for computing the relevance value **216** and for utilizing the computed relevance value **216** to modify the ratings **118** for the product **202**.

The routine **700** begins at operation **702**, where a current rating velocity is computed for the current product **202**. The current rating velocity is the number of ratings **118** received for the product **202** in a recent time period, such as during the last month. Once the current rating velocity has been computed, the routine **700** proceeds to operation **704**, where an average rating velocity is computed for the product **202**. The average rating velocity is the average number of ratings **118** received per time period for the product during its lifetime. For instance, if a product has been available from the merchant system **108** for one year and has received 48 ratings, the average rating velocity for the product **202** is four (48 ratings/12 months=4 ratings per month on average).

Once the average rating velocity and the current rating velocity have been computed, the routine **700** proceeds from operation **704** to operation **706**. At operation **706**, the relevance value **216** for the product **202** is computed as the ratio of the average rating velocity and the current rating velocity. Continuing the example given above, a product **202** with a current rating velocity of two and an average rating velocity of four would have a relevance value of one-half.

From operation **706**, the routine **700** proceeds to operation **708**, where the ratings **118** for the product **202** are modified by multiplying the ratings **118** by the computed relevance value **216**. In one embodiment, the ratings **118** are multiplied by the relevance value **216** after application of the time decay factor **214** to the ratings **118** in the manner described above. From operation **708**, the routine **700** proceeds to operation **710**, where it ends.

FIG. 7B shows a routine **750** illustrating a second mechanism for computing the relevance value **216** for a product. The routine **750** begins at operation **752**, where the average rating velocity for the product **202** is calculated in the manner described above. The routine **750** then proceeds to operation **754**, where an expected number of ratings for the product **202** during a previous time period is computed based upon the average rating velocity. At operation **756**, an amount of time necessary for the product **202** to actually receive the expected number of ratings **118** is determined.

The routine **750** then proceeds to operation **758**, where the relevance value **216** is computed as a ratio of the amount of time during the previous time period and the amount of time actually taken for the product **202** to receive the expected number of ratings **118**. For instance, if the average rating velocity of a product **202** is four ratings per month, then the product **202** would be expected to receive four ratings **118** during the previous month. If it actually took the product **202** six weeks to obtain four ratings **118** then the relevance value **216** for the product is two-thirds (four weeks divided by six weeks). The magnitude of the relevance value **216** might also be restricted in various embodiments.

From operation **758**, the routine **750** proceeds to operation **760**, where the ratings **118** for the product **202** are modified by multiplying the ratings **118** by the computed relevance value **216**. The ratings **118** may be multiplied by the relevance value **216** after application of the time decay factor **214** to the ratings **118** in the manner described above. From operation **760**, the routine **750** proceeds to operation **762**, where it ends.

FIG. 8 is a flow diagram showing one illustrative routine **800** for computing a score **206** for a product **202** assigned to multiple product categories **204** according to one embodiment disclosed herein. In some implementations of the merchant system **108**, individual products **202** are assigned to multiple product categories **204**. For instance, a handheld mixer might be assigned to a category **204** for electric mixers and a category **204** for kitchen appliances. FIG. 8 illustrates one process for computing a score for a product **202** that is assigned to two or more categories **204** in this manner.

The routine **800** begins at operation **802**, where a variable for storing an indicator of the current product category **204** is initialized to the first product category **204** that the product **202** is assigned to. The routine **800** then proceeds to operation **804**, where a score **206** is computed for the current product **202** with reference to the current product category **204** in the manner described above with regard to FIG. 6. In particular, the prior probability distribution **210** and the time decay factor **214** for the current product category **204** are utilized to compute the score **206** for the product. From operation **804**, the routine **800** proceeds to operation **806**.

At operation **806**, a determination is made as to whether the product **202** is assigned to more categories. If so, the routine **800** proceeds from operation **806** to operation **808** where the variable identifying the current category **204** is incremented to the next category **204** that the product **202** is assigned to. The routine **800** then proceeds from operation **808** to operation **804** where the score **206** is computed for the product **202** in the next category **204**.

If scores **206** have been computed for the product **202** in all of the categories that the product is assigned to, the routine **800** proceeds from operation **806** to operation **810**. At operation **810**, the lowest score computed for the product **202** in all of the product categories **204** to which the product **202** is assigned is utilized as the score **206** for the product. From operation **810**, the routine **800** proceeds to operation **812**, where it ends.

FIG. 9 is a flow diagram showing one illustrative routine **900** for computing and displaying a likely next rating score for a product **202** according to one embodiment disclosed herein. In this embodiment, the scores **206** are displayed to the customer **102**. The scores **206** for each product **202** may be displayed in lieu of or in addition to an average rating for the product **202**. The scores **206** might also be displayed to the customer **102** separately from or in conjunction with the product ranking **126**.

The routine **900** begins at operation **902**, where the prior probability distribution **210** and the time decay factor **214** are computed for a product category **204** in the manner described above with regard to FIGS. 4A-4B. Once the prior probability distribution **210** and the time decay factor **214** have been computed, the routine **900** proceeds to operation **904**.

At operation **904**, a likely next rating score **206** is computed for each product **202** in the product category **204** using the prior probability distribution **210**, the actual product ratings **118**, the time decay factor **214**, and the relevance value **216**. One illustrative routine for computing the score **206** was described above with regard to FIG. 6. Once the score **206** has been calculated for each product, the routine **900** proceeds from operation **904** to operation **906**.

At operation **906**, the computed likely next rating score **206** is displayed to the customer **102**. As discussed above, the score **206** might be included on a product page **116** and might be displayed in conjunction with or separately from an average rating for the product. The score **206** might also be displayed in conjunction with the product ranking **126**. From operation **906**, the routine **900** proceeds to operation **908**, where it ends.

FIG. 10 is a flow diagram showing one illustrative routine **1000** for identifying and addressing rating discrepancies according to one embodiment disclosed herein. In the embodiment shown in FIG. 10, a score **206** is computed for each product **202** in the manner described above, and the score is utilized to identify discrepancies in the ratings **118**. Appropriate action might be taken in response to identifying a discrepancy, such as alerting customers to the discrepancy, subjecting the product ratings **118** to a detailed review in an attempt to determine the cause of the discrepancy, suspending sales of a product, alerting a product vendor or administrator, or taking other actions. In certain implementations, action might only be taken if the discrepancy exceeds a set threshold. It should be appreciated that the scores **206** might also be utilized by the merchant system **108** for other types of back-end computations.

The routine **1000** begins at operation **1002** where the prior probability distribution **210** and the time decay factor **214** are computed for a product category **204** in the manner described above with regard to FIGS. 4A-4B. Once the prior probability distribution **210** and the time decay factor **214** have been computed, the routine **900** proceeds to operation **1004** where a score **206** is computed for each product in the product category **204** using the prior probability distribution **210**, the actual product ratings **118**, the time decay factor **214**, and the relevance value **216**. One illustrative routine for computing the score **206** was described above with regard to FIG. 6. Once the score **206** has been calculated for each product **202**, the routine **1000** proceeds from operation **1004** to operation **1006**.

At operation **1006**, the computed likely next rating score **206** for each product **202** is compared to the average of the ratings **118** for the product **202**. A determination is then made at operation **1008** as to whether a discrepancy exists between the score **206** for the product **202** and the average rating of the product **202**. For instance, a discrepancy might be identified if the score **206** and the average rating for the product **202** are significantly different. If no discrepancy is identified, the routine **1000** proceeds to operation **1012**, where it ends.

If a discrepancy is identified between the score **206** and the average rating for a product, the routine **1000** proceeds from operation **1008** to operation **1010**. At operation **1010**, action is taken in response to detecting the discrepancy. For instance, as mentioned briefly above, customers might be alerted to the discrepancy in an appropriate fashion, the ratings **118** for the product might be subjected to a detailed review and analysis, and/or a product vendor or administrator of the merchant system **108** might be alerted. Other types of actions might also be taken. Once appropriate action has been taken, the routine **1000** proceeds from operation **1010** to operation **1012**, where it ends.

FIG. 11 is a flow diagram showing one illustrative routine **1100** for ranking products based upon scores computed for sets of products according to one embodiment disclosed herein. The embodiments disclosed herein have been primarily described in the context of ranking a set of products **202** within a product category **204**. It should be appreciated, however, that the disclosure provided herein is not limited to ranking products in this manner. For instance, products might be ranked within sub-categories or other types of divisions. In another embodiment, an entire catalog of products might be ranked without reference to the categories that each product is assigned to.

In the embodiment illustrated in FIG. 11, products **202** in a product catalog are grouped into sets for ranking. The term “sets” as used herein refers to a collection of products **202** that do not necessarily align with particular product categories **204**. For example, in one embodiment, the products **202** in a product catalog are divided according to the type of store that would sell a particular product **202**. Electronics-related products might be sold by an electronics store, a variety of home and garden products might be sold by a home and garden store, and books and related items might be sold by a book store, for instance. Other arbitrary and specific groupings of products **202** might also be utilized.

Once the products have been divided into sets as described above, a time decay factor **214** and prior probability distribution **210** might be computed for each set of products **202**. Those values are then utilized when computing the score **206** for each product **202**. Once the scores **206** have been computed, the products **202** might be ranked against other products **202** in the same set, might be ranked against other products **202** in the same category or sub-category, might be ranked against other products **202** in the product catalog, or in another fashion. This process is illustrated in FIG. 11.

The routine **1100** begins at operation **1102**, where products **202** are divided into sets. As discussed above, the sets might correspond to stores or other types of divisions within a product catalog. Once the products **202** have been divided into sets, the routine **1100** proceeds from operation **1102** to operation **1104**.

At operation **1104**, the prior probability distribution **210** and the time decay factor **214** are computed for a set of products in a manner similar to that described above with regard to FIGS. 4A-4B. Once the prior probability distribution **210** and the time decay factor **214** have been computed for the set, the routine **1100** proceeds to operation **1106** where a likely next rating score **206** is computed for each product in the set using the prior probability distribution **210**, the actual product ratings **118**, the time decay factor **214**, and the relevance value **216** for the set. A mechanism similar to that described above with regard to FIG. 6 might be utilized to compute the score **206**. Once the score **206** has been calculated for each product in the set, the routine **1100** proceeds from operation **1106** to operation **1108**.

At operation **1108**, the products in each set are ranked against other products in the same product category **204** using the scores **206** computed at operation **1106**. The routine **1100** then proceeds to operation **1110**, where the top ranked products **202** are then displayed in the manner described above. From operation **1110**, the routine **1100** proceeds to operation **1112**, where it ends.

FIG. 12 is a flow diagram showing one illustrative routine **1200** for identifying top rated products by attributes other than product ratings, according to one embodiment disclosed herein. In the embodiments described above, the score **206** for each product **202** is computed based upon general quantitative customer ratings **118** for each product. It should be appreciated, however, that the score **206** might be computed based upon values associated with other product attributes. For instance, an appropriate mechanism might be provided that allows customers to rate specific attributes of each product separately from an overall rating. For example, a customer might give a blender 5-stars for ease of use, but 2-stars for ease of cleaning. In this manner, the top rated products might be identified and displayed based upon virtually any attribute of a product, such as performance or other specifications of the product, ease of use, lifespan, and others. FIG. 12 describes this process in greater detail.

The routine **1200** begins at operation **1202** where a product attribute to be utilized for generating the score **206** is identified. As discussed briefly above, the attribute might be related to the performance of a product, technical specifications, ease of operation, lifespan, or virtually any other type of product attributes. Once the attribute has been selected, the routine **1200** proceeds to operation **1204**.

At operation **1204**, a time decay factor **214** and prior probability distribution **210** is computed for a product category **204** or set of products **202** utilizing values for the selected attribute. For instance, if the selected attribute relates to ease of use of a product, the customer ratings regarding ease of use might be utilized to compute the prior probability distribution and the time decay factor. Once the prior probability distribution **210** and time decay factor **214** have been computed, the routine **1200** proceeds to operation **1216**.

At operation **1206**, a likely next rating score **206** is computed for each product **202** in the product category **204** or set using the prior probability distribution **210**, the actual values for the specified product attribute, the time decay factor **214**, and the relevance value **216** for the category or set. A mechanism similar to that described above with regard to FIG. 6 might be utilized to compute the score **206**. Once the score **206** has been calculated for each product **202** in the product category **204** or set, the routine **1200** proceeds from operation **1206** to operation **1208**.

At operation **1208**, the products **202** in the category or set are ranked according to the computed scores **206**. The routine **1200** then proceeds to operation **1210**, where a product ranking **126** might be displayed. In this embodiment, the top ranked products are those products that have the highest ratings for the product attribute selected at operation **1202**. From operation **1210**, the routine **1200** proceeds to operation **1212**, where it ends.

FIG. 13 shows an example computer architecture for a computer **1300** capable of executing the software components described herein for identifying top rated products in the manner presented above. The computer architecture **1300** shown in FIG. 13 illustrates a conventional server computer, workstation, desktop computer, laptop, PDA, electronic book reader, digital wireless phone, network appliance, set-top box, or other computing device, and may be utilized to execute any aspects of the software components presented herein described as executing on the application servers **110**, the customer computer **104**, or other computing platform.

The computer **1300** includes a baseboard, or “motherboard,” which is a printed circuit board to which a multitude of components or devices may be connected by way of a system bus or other electrical communication paths. In one illustrative embodiment, one or more central processing units (“CPUs”) **1302** operate in conjunction with a chipset **1304**. The CPUs **1302** are standard programmable processors that perform arithmetic and logical operations necessary for the operation of the computer **1300**.

The CPUs **1302** perform operations by transitioning from one discrete, physical state to the next through the manipulation of switching elements that differentiate between and change these states. Switching elements may generally include electronic circuits that maintain one of two binary states, such as flip-flops, and electronic circuits that provide an output state based on the logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements may be combined to create more complex logic circuits, including registers, adders-subtractors, arithmetic logic units, floating-point units, or the like.

The chipset **1304** provides an interface between the CPUs **1302** and the remainder of the components and devices on the baseboard. The chipset **1304** may provide an interface to a random access memory (“RAM”) **1306**, used as the main memory in the computer **1300**. The chipset **1304** may further provide an interface to a computer-readable storage medium such as a read-only memory (“ROM”) **1308** or non-volatile RAM (“NVRAM”) for storing basic routines that help to startup the computer **1300** and to transfer information between the various components and devices. The ROM **1308** or NVRAM may also store other software components necessary for the operation of the computer **1300** in accordance with the embodiments described herein.

According to various embodiments, the computer **1300** may operate in a networked environment using logical connections to remote computing devices and computer systems through a network, such as a local-area network (“LAN”), a wide-area network (“WAN”), the Internet, or any other networking topology known in the art that connects the computer **1300** to remote computers. The chipset **1304** includes functionality for providing network connectivity through a network interface controller (“NIC”) **1310**, such as a gigabit Ethernet adapter. For example, the NIC **1310** may be capable of connecting the computer **1300** to other computing devices, such as the application servers **110**, the customer computer **104**, a data storage system in the merchant system **108**, and the like, over the network **106** described above in regard to FIG. 1. It should be appreciated that multiple NICs **1310** may be present in the computer **1300**, connecting the computer to other types of networks and remote computer systems.

The computer **1300** may be connected to a mass storage device **1312** that provides non-volatile storage for the computer. The mass storage device **1312** may store system programs, application programs, other program modules, and data, which have been described in greater detail herein. The mass storage device **1312** may be connected to the computer **1300** through a storage controller **1314** connected to the chipset **1304**. The mass storage device **1312** may consist of one or more physical storage units. The storage controller **1314** may interface with the physical storage units through a serial attached SCSI (“SAS”) interface, a serial advanced technology attachment (“SATA”) interface, a FIBRE CHANNEL (“FC”) interface, or other standard interface for physically connecting and transferring data between computers and physical storage devices.

The computer **1300** may store data on the mass storage device **1312** by transforming the physical state of the physical storage units to reflect the information being stored. The specific transformation of physical state may depend on various factors, in different implementations of this description. Examples of such factors may include, but are not limited to, the technology used to implement the physical storage units, whether the mass storage device **1312** is characterized as primary or secondary storage, or the like. For example, the computer **1300** may store information to the mass storage device **1312** by issuing instructions through the storage controller **1314** to alter the magnetic characteristics of a particular location within a magnetic disk drive unit, the reflective or refractive characteristics of a particular location in an optical storage unit, or the electrical characteristics of a particular capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this description. The computer **1300** may further read information from the mass storage device **1312** by detecting the physical states or characteristics of one or more particular locations within the physical storage units.

In addition to the mass storage device **1312** described above, the computer **1300** might have access to other computer-readable media to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable media can be any available media that may be accessed by the computer **1300**, including computer-readable storage media and communications media. Communications media includes transitory signals. Computer-readable storage media includes volatile and non-volatile, removable and non-removable storage media implemented in any method or technology. For example, computer-readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically-erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technology, compact disc ROM (“CD-ROM”), digital versatile disk (“DVD”), high definition DVD (“HD-DVD”), BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information. Computer-readable storage media does not include transitory signals.

The mass storage device **1312** may store an operating system **1316** utilized to control the operation of the computer **1300**. According to one embodiment, the operating system comprises the LINUX operating system. According to another embodiment, the operating system comprises the WINDOWS® SERVER operating system from MICROSOFT Corporation of Redmond, Wash. According to further embodiments, the operating system may comprise the UNIX or SOLARIS operating systems. It should be appreciated that other operating systems may also be utilized. The mass storage device **1312** may store other system or application programs and data utilized by the computer **1300**, such as the online shopping module **112** and/or the ranking module **122**, both of which were described above.

In one embodiment, the mass storage device **1312** or other computer-readable storage media may be encoded with computer-executable instructions that, when loaded into the computer **1300**, transform the computer from a general-purpose computing system into a special-purpose computer capable of implementing the embodiments described herein. These computer-executable instructions transform the computer **1300** by specifying how the CPUs **1302** transition between states, as described above. According to one embodiment, the computer **1300** has access to computer-readable storage media storing computer-executable instructions that, when executed by the computer, perform the various routines and operations described herein.

The computer **1300** may also include an input/output controller **1318** for receiving and processing input from a number of input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus, or other type of input device. Similarly, the input/output controller **1318** may provide output to a display device, such as a computer monitor, a flat-panel display, a digital projector, a printer, a plotter, or other type of output device. It will be appreciated that the computer **1300** may not include all of the components shown in FIG. 13, may include other components that are not explicitly shown in FIG. 13, or may utilize an architecture completely different than that shown in FIG. 13.

Based on the foregoing, it should be appreciated that technologies for identifying top rated products have been presented herein. Although the subject matter presented herein has been described in language specific to computer structural features, methodological acts, and computer readable media, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features, acts, or media described herein. Rather, the specific features, acts, and mediums are disclosed as example forms of implementing the claims.

The subject matter described above is provided by way of illustration only and should not be construed as limiting. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. Various modifications and changes may be made to the subject matter described herein without following the example embodiments and applications illustrated and described, and without departing from the true spirit and scope of the present invention, which is set forth in the following claims.