the collaborative filtering effect of netflix ratings for indie films … · 2015-08-19 · the...

The Collaborative Filtering Effect of Netflix Ratings for Indie Films versus Blockbusters

and Heavy Users versus Casual Users

May 2014

Henry Zhu Tang

Department of Economics

Stanford University

Stanford, CA 94305

[email protected]

Under the direction of

Professor Timothy Bresnahan

ABSTRACT

Collaborative filtering algorithms, whose adoption by online recommendation engines has

markedly increased in recent years, serve to match users with items based on what they have

consumed in the past or the tastes of similar users. Meanwhile, Internet economists and

marketing experts have cited a new phenomenon driven by online platforms called the "Long

Tail," which is a distributional shift towards lesser-known niche products. In this paper, I test

whether an online platform that uses a collaborative filtering algorithm can help match non-mass

market goods with previously un-informed demanders and how this can affect user heterogeneity.

I choose to examine movie ratings made publicly available in the Netflix Prize, an open source

competition to improve the existing algorithm used for Netflix recommendations. Looking at

demand across different movie categories, I find a stronger responsiveness of demand to early

user ratings for indie films, relative to blockbusters. This effect is further magnified for "heavier"

users with greater rating histories. As movie "buffs" with more variety in tastes, these users not

only are greater influencers of demand, but are also more inclined to be influenced by others in

their own demand, in contrast to what one observes on other online platforms.

Keywords: Long Tail, indie, collaborative filtering, recommendations, heavy users, Netflix

Acknowledgments: I would like to thank my advisor, Professor Bresnahan, for being the

inspiration behind my topic, as well as for his constant guidance throughout the thesis research

and writing process. I would also like to thank Professor Marcelo Clerici-Arias for his invaluable

advice on my preliminary research, his input on structuring my thesis, and his recommendations

on statistical resources. This thesis is dedicated to my close family and friends.

mailto:[email protected]

Table of Contents

1. Introduction ………………………………………………………………………………page 1

2. Literature Review ………………………………………………………………………...page 5

2.1 The Film Industry …………………………………………………………….....page 5

2.2 The Long Tail …………………………………………………………………...page 8

2.3 Empirical Studies on Films and the Long Tail .……………………………......page 10

2.4 A Previous Netflix Study ………………………………………………….…..page 12

3. Data & Methodology ……………………………………………………………………page 14

3.1 Description of Data Set ………………………………………………………..page 14

3.2 Study Design …………………………………………………………………..page 20

4. Results …………………………………………………………………………………..page 22

4.1 Regression Models …………………………………………………………….page 22

4.2 User Heterogeneity …………………………………………………………….page 24

5. Conclusion ………………………………………………………………………………page 28

6. Appendix ………………………………………………………………………………..page 31

6.1 Stata Code ………………………………………………………………….......page 31

6.2 Tables .………………………………………………………………………....page 33

6.3 Figures …………………………………………………………………………page 39

7. References ………………………………………………………………………………page 42

Tang 1

1 Introduction

The film industry, a key driver of the global multibillion-dollar entertainment market, has

underdone many changes over the past century, thanks to constant innovations in technology.

Films, like other taste-based goods, can be widely accessed by consumers today thanks to a

revolutionary development: the Internet. Before the turn of the new century, films that had

already gone through their theatrical run were available almost exclusively as video cassettes in

physical stores such as a Blockbuster or Hollywood Video. However, in 1997, Reed Hastings

and Marc Randolph founded Netflix, an online DVD-by-mail retailer that usurped the traditional

brick-and-mortar model. At once, a wider library of titles had become available to consumers

than ever before. Netflix introduced a proprietary recommendation system, powered by a

collaborative filtering algorithm, to select movies to watch for its customers, a feature it

continues to use for its global video streaming service today. This collaborative filtering

algorithm would further highlight indie or niche films that could not be found (or were

prohibitively difficult to find) in stores.

Introduced in 2004 by editor-in-chief of Wired Magazine Chris Anderson, the Long Tail

is an evolution of the original Pareto principle in marketing1. The conventional Long Tail theory

argues that niche products (i.e. those in the tail of the sales distribution) gain increasingly greater

market share thanks to online innovations such as search queries and collaborative filtering, so

that the top-selling items make up less and less of the sales distribution. With the shift to online

(versus offline) sales, more efficient supply chain management has reduced the need to carry in-

store inventories for retailers, and search costs have been greatly reduced for consumers.

1 According to the Pareto principle, 20 percent of all the products generate 80 percent of all the revenues/sales.

Tang 2

Collaborative filtering is a method invented and developed over the past two decades to

recommend items to consumers online, matching their preferences based on their past browsing

or purchase histories and their similarities with other consumers or users. It is used for the

recommendation engines of many notable Internet services today, including Pandora, Amazon,

YouTube, Reddit, Spotify, and iTunes.

There has been much literature in economics and marketing analyzing both the Long Tail

and the box office performance of films, but little research has been done specifically on the

Long Tail distribution for films in online markets. A frequently cited example of the Long Tail,

Netflix has become widely known for using a collaborative filtering algorithm for the explicit

recommendations it provides users, based on each user's ratings and how similar a user is to

others in the network, among other factors. Netflix has helped revolutionize the movie industry

by offering a platform for niche movies to thrive. Given that product variety2 on Netflix has

always been increasing (with some of its products naturally being lesser-known), overall demand,

as measured by the number of ratings given, for niche titles should be increasing as well.

Taking a step from the past definitions of the Long Tail, I choose to classify the Long

Tail of the Netflix library as "indie" films with limited budgets (implying a constraint on

marketing). These movies were marginalized previously by the theatrical market's high fixed

costs, unable to afford the advertising or screens to meet consumers who would presumably be

interested if they had been informed about the movies. Thanks to the new capacity to distribute

films via the Internet, the rules of the old system no longer apply; movies do not face these same

constraints. I assert that the indies in the Long Tail can benefit due to one of two reasons:

2 The number of distinct movies.

Tang 3

1) There had previously been fragmented demand; indies were for fragments of consumers

with odd tastes but went unfound. Collaborative filters instead have helped connect

supply to demand in many different "communities."

2) There exists a group of "heavy" users or frequent reviewers (more closely representative

of movie "critics" or "buffs" who have a large variety of tastes) that could not find all

these movies until now. Collaborative filters offer recommendations to this group of

users who find them to be valuable.

After my analysis, I find the second theory to be more compelling.

In analyzing the demand distribution and Long Tail in particular, I determine what a

rating on Netflix actually represents. The ratings users have given are for movies they have

presumably seen; even if the score is not favorable, the rating itself can show Netflix the types of

movies this user watches. Since users are generally aware that their ratings help connect movies

with other users (and with themselves), ratings can serve as implicit endorsements or

recommendations. At the very least, ratings serve as signals of demand and perceived quality or

enjoyment.

I hypothesize that there would be an "indie" effect: a movie rating for a niche movie

would be considered more informative and valuable, as Netflix serves to match niche movies to

users who otherwise could not discover them. The rating scores, especially early on, would be a

better leading indicator (or perhaps even influencer) of the overall demand experienced for niche

movies than for blockbusters. I confirm this effect, quantitatively measuring the difference in

responsiveness of demand to ratings for blockbusters versus indies.

Tang 4

As users are matched to lesser-known Long Tail movies that satisfy their tastes, I also

hypothesize that user heterogeneity can increase over time. In other words, user tastes become

more diverse, leading to the formation of sub-communities or different fragments of demanders.

However, I have already said that after breaking down users by "heaviness" or frequency of

rating activity, I find that rather than leading to increased heterogeneity in tastes, Netflix

recommendations serve to especially benefit these heavy users who were previously

underinformed and simply have a greater taste for variety to begin with. Heavy users, not only

are powerful influencers of demand (with their propensity to give more ratings, especially early

after a movie appears on Netflix), but they are also markedly influenced by the ratings of others.

In all, there are three main dichotomies I seek to examine:

1) Blockbusters versus Indies (Hits versus Niches)

2) Frequent Raters versus Infrequent Raters (Influencers of Demand)

3) Movie "Buff" versus Casual Movie Watcher (Likelihood to be Influenced)

To test these hypotheses, I use the training data set of movie ratings made publicly

available during the Netflix Prize, an open competition held in 2006 to improve Netflix's existing

Cinematch collaborative filtering algorithm3. Using additional descriptors such as production

budget from The Numbers movie database, I categorize specific movies within the Netflix data

set into blockbusters, indies, and an intermediate category. I then build linear regression models

of different definitions of demand as the dependent variable and independent variables that

include how well a film is received early on during its run on Netflix. If niche movie ratings are

actually more powerful, I would expect the demand for niche titles to be more responsive to

3 A successful entrant had to improve on the RMSE of the Cinematch algorithm by 10%. Further prizes would be

given for progress beyond this.

Tang 5

ratings than the demand for blockbusters (i.e. the indie effect to hold). I also specify which users

are heavy, offering an absolute as well as a time-varying definition of heaviness.

The remainder of this thesis is organized as follows. Section 2 provides a literature

review of the economics of the film industry, an overview of the Long Tail as defined in the past

and past empirical studies, and a particular study by Tan & Netessine (2009) using the Netflix

Prize data set. Section 3 describes my data sets as well as the methodology I take to test my

hypotheses about the indie effect and increasing user heterogeneity. Section 4 presents an

analysis of my findings from the regression models and a further discussion on the role of heavy

users over time. Section 5 offers conclusions and further thoughts on movies, the Netflix

platform going forward, collaborative filtering, and additional research.

2 Literature Review

2.1 Film Industry

Much economics literature has been published about the film industry, as it continues to

innovate and the dynamics of competition evolve. Eliashberg et al. (2006) note that each of the

three main stages of the film value chain -- production, distribution, and exhibition -- have

greatly changed over time. Before a movie is produced, it must be "green-lit" or go through an

approval process. To better maximize the success rate and minimize risk of failures, marketing

researchers have made much progress in developing box office forecasting models to assess

possible demand. One important consideration is that moviegoers (or movie-watchers) are

heavily influenced by others' opinions and choices. Studios have also tried to cope with risk by

pursuing established movie franchises, making sequels more prevalent and a "safer bet." In turn,

Tang 6

distributors are also investing more than ever into advertising and development, focusing on a

small number of "blockbusters" that nowadays capture an increasingly larger share of attention

and revenues. This can make or break a distributor's overall box-office performance and has led

towards a "winner-take-all" market (e.g., flops are bigger, and the average production cost for a

movie has dramatically increased). While the box office market for big-name movies is still

booming, it has become increasingly more difficult for niche movies with limited budgets to

make it onto theater screens. Thus, movie producers and viewers alike have to turn more towards

ancillary markets such as home video or digital distribution (on-demand streaming), where there

are potentially much greater revenues and profits to be had.

Moul (2005) discusses the issues of when to release a film and how to capitalize on (or

minimize the effect of) word-of-mouth, essential questions in the supply chain of the film

industry. He also looks at the impact of critics as well as the starring cast on movie demand.

Movies can present a unique problem in marketing because they possess traits of both durable

products (i.e. sales come mostly from first-time purchases) and nondurables (i.e. sales mostly

come from repeat purchases). Since branding is conspicuously absent from production and

distribution4, the star power of the cast, and to a lesser extent, the director, play the primary role

in establishing intent-to-view for movies. The presence of a star associated with specific types of

movies can provide a signal on what one can expect or that the studio has put a notable

investment into the movie, and serve as promotional effort or free publicity.

Karniouchina (2011) and Berger, Sorensen, and Rasmussen (2010) discuss whether

movie stars can increase movie demand even through negative consumer buzz, both separately

4 There are a few exceptions, perhaps most prominently of which is Pixar.

Tang 7

finding that this indeed is the case. Using a combination of empirical studies and more rigorous

econometric analysis, Berger, Sorensen, and Rasmussen demonstrate that negative publicity is

more likely to hurt products already with broad awareness, but help products that are relatively

unknown (e.g., those in the Long Tail or considered niche). Meanwhile, Karniouchina finds that

the early buzz generated by stars ultimately has a net positive impact for box office sales, even

for poorly received films, because of the initial boost in revenues.

Eliashberg and Shugan (1997) discuss the role that movie critics play for a movie's box

office performance: whether they act as influencers who are regarded as thought leaders and can

sway consumer sentiment, or as predictors who are merely leading indicators. Ultimately, the

authors find that critics, perhaps contrary to what one would expect, appear to act more like

leading indicators. On Netflix, the closest role to a movie critic is that of "heavy users," who take

the time to rate more movies than the vast majority of the user community. They likely watch

more movies than the average user5, but they also provide more ratings likely because they

believe recommendations to be useful to themselves and others. Since ratings are anonymous,

one would imagine that heavy users are also predictors rather than influencers of demand.

As one can see though, the vast majority of movie economics literature still focuses on

box office performance, rather than newer ancillary markets such as DVDs or video-on-demand.

This is due in large part to the availability or reliability of data, especially given possible

concerns on privacy.

5 One can watch a movie and not rate it on Netflix. One can also rate movies without having watched them through

Netflix.

Tang 8

2.2 The Long Tail

Regarding online platforms, much research has been done on the phenomenon of the

Long Tail. The term, coined by Wired editor-in-chief Chris Anderson (2006), refers to the shift

away from a small number of hits ("mainstream" products) at what he calls the "head of the

demand curve" to a larger number of niches in the "tail." According to Anderson, there are a

number of factors that explain this: the far greater number of niche goods available, the decrease

in distribution and search costs, and "filters" or recommendations that sort through the increased

product variety and drive demand toward the tail. The demand curve thus flattens; there are still

hits and niches, but the hits are relatively less popular and the niches relatively more so

(Appendix: Figure 6). With the costs of reaching consumers lowered, niches can suddenly reach

as huge a potential market as hits can. Thanks further to recommendation systems, supply and

demand are connected more closely than before. Finally, consumers are even able to guide

themselves to what they are looking for by posting reviews or ratings of past products, shaping

recommendations that drive demand down into the niches further. As their interests and tastes

narrow, consumers can form sub-communities of affinity.

Brynjolfsson, Hu, and Smith (2010) go through a number of supply-side and demand-side

drivers that contribute to the increase in product variety and decrease in sales concentration for

the Long Tail. On the supply-side, the cost of stocking products greatly decreases; Internet

retailers only need to add the inventory of a listed product to their centralized warehouses, as

opposed to a brick-and-mortar model where the product must be carried in-stores. On the

demand-side, new technologies such as Google queries have greatly reduced search costs.

Collaborative filters used for personalization and recommendation technologies can

disproportionately help consumers find obscure products they would otherwise not have known,

Tang 9

although they can also lead to disproportionate gains in sales of popular products relative to

niche products due to the amount of existing information available (Fleder and Hosanagar, 2009).

This Superstar or "blockbuster" effect, the opposite of the Long Tail, can also be furthered by

online communities and social networks, which attach themselves to the latest fads and

blockbusters. Brynjolfsson et al. (2011) use a data set for an online and catalog retailer that

allows them to isolate the demand-side and individual-level consumer search behavior.

Ultimately, they find that purchases made through the Internet channel by consumers with prior

Internet experience (i.e. more experienced users) are more skewed towards obscure products, and

the Internet channel exhibits a significantly less concentrated sales distribution than the catalog

channel.

Anderson offers a fairly limited and vague definition of the Long Tail, primarily viewing

it as a principle for the marketing realm. These Long Tail items, or "niches," are characterized by

constraints in either appeal or production, failing to become "mainstream." Anderson argues that

niches can be as economically attractive as hits though, thanks to the Internet, if one is able to

find the requisite demanders. He does not take the step to describe why exactly these niches have

been fixed on the tail of the demand distribution (e.g., separating why they are not mainstream or

commercial between demand and supply sides, and how one would characterize the potential

demanders for these products). Brynjolfsson et al. do offer additional insights on both demand

and supply-side factors as well as the characteristics of demanders for items in the tail; however,

they describe online catalogs or stores where it is likely easier to quantify demand, rather than

subscription-based services such as a Netflix.

There have been detractors as well as proponents of the Long Tail. Elberse and

Oberholzer-Gee (2008) investigate the actual profitability of the Long Tail by looking at Nielsen

Tang 10

sales data for videos/music and data from Quickflix (an Australian equivalent of Netflix). They

find that the tail becomes longer, but it represents a rapidly increasing number of titles that sell

very rarely if ever; success is becoming concentrated in fewer best-sellers at the head of the

distribution. Elberse (2008) directly challenges Anderson's view of the long tail by citing

sociologist William McPhee's theory of exposure, which describes two phenomena of

distribution: natural monopoly and double jeopardy. Natural monopoly refers to the "monopoly"

that hit products have on "light" consumers or those with otherwise marginal participation in the

market. McPhee also notes that the larger the proportion of consumers unfamiliar with a given

alternative, the less likely those who are familiar with it to like it especially. Niche products

therefore have a double disadvantage ("double jeopardy"): first, they are not well-known; second,

even if they are known, it is by people who "know better" and still prefer the popular products.

Because of these reasons, Elberse suggests that retailers should market the most popular products

but broaden their selection with more niche products if they want to cater to "heavier" users.

Lastly, Elberse finds that niche or obscure titles receive, on average, lower ratings (which, as I

found, was the case in Netflix) - this could suggest that niche titles similarly rated as popular

titles are actually of higher quality (assuming that niche titles get lower ratings simply due to a

lack of popularity or misaligned demand).

2.3 Empirical Studies on Movies and the Long Tail

There have been a number of recent empirical studies pertaining to the weight of online

reviews or recommendations. Tucker and Zhang (2011) conduct a field experiment to test

whether popularity information benefits niche products with narrow "appeal" disproportionately,

because the same level of popularity (a possible measure of demand) implies higher quality for

narrow-appeal products than for broad-appeal products. They experiment with shifting a

Tang 11

wedding service website from a "yellow pages" listing with no popularity information to a "best-

seller list" style ranked by number of clicks for each vendor. The definition of "appeal" is based

on the population of the vendor's town. Using a differences-in-differences (DID) method as their

primary approach, they find that narrow-appeal vendors receive more visits than equally popular

broad-appeal vendors after the "treatment" of popularity information, consistent with their

hypothesis.

Dellarocas, Gao, and Narayan (2010) examine the postconsumption propensity to give

movies reviews. They describe two opposing forces in effect: consumers prefer posting reviews

for products less available and successful in the market (Dichter's theory of self-involvement);

however, at the same time, they are also more likely to contribute reviews for products that many

other people have already commented on online. The tension between these forces leads to a U-

shaped relationship between the average propensity to review a movie postconsumption and that

movie's box office revenues: moviegoers are more likely to contribute reviews for very obscure

movies but also for huge blockbusters. Movies falling in the average-performing range would be

least likely then to get consumer reviews. Based on this, niche and blockbuster movies could

both be benefitting from Netflix (while the "middle-of-the-pack" movies see the least returns).

Finally, Zhou and Duan (2012) consider the interaction of a demand-side factor (online

reviews) and a supply-side factor (product variety) for the Long Tail in the context of online

software downloads on CNET. Using a quantile regression technique, they find that the impact

of both positive and negative reviews is weakened as product variety increases. The increase in

product variety also reduces the impact of reviews on popular products more than on niche

products, suggesting that a review or rating for a niche product carries more weight,

strengthening the Long Tail. On Netflix, over time each new rating for a movie naturally carries

Tang 12

less weight; however, this further confirms that niche ratings should be viewed as more

informative of a movie's quality or "likeability."

2.4 Netflix Study

Tan and Netessine (2009) use the Netflix Prize data set in its entirety6 to look at

aggregate as well as individual-level demand on Netflix. They assert that the pre-existing

definition of the Long Tail is too static, and that it implicitly excludes the impact of increasing

product variety. An increase in product variety would likely create demand diversification, but it

could also lead to a conflicting definition of "hits" and "niches." When product variety is large,

the demand for any one product would almost certainly be smaller than when product variety is

small. Furthermore, when the consumer base is large, learning about new products is faster than

when the consumer base is small. The authors address the empirical question of whether

consumers can (and actually do) keep up with discovering "obscure" products as they appear on

the marketplace. They offer a more dynamic definition of the Long Tail that takes into account

increasing product variety as a measure for relative popularity. Demand in their case is the

number of ratings for a movie.

The authors in their analysis test two separate hypotheses:

Hypothesis 1: If popularity is measured in absolute terms, over time, demand for hits will

decrease, while demand for niches will increase.

Hypothesis 2: If popularity is measured in relative terms, over time, demand for hits will

increase, while demand for niches will decrease.

6 Over 100 million ratings submitted by approximately 480,000 users on Netflix from 2000 to 2005.

Tang 13

They analyze the yearly distribution of cumulative demand using a Popularity variable (defined

as the ranking for a particular movie by number of ratings during the time period). Relative

demand and popularity are measured by including a Product Variety variable (defined as the total

number of different movies rated during a period). Tan and Netessine also control for product

rating to ensure that the Long Tail is not simply a manifestation of hit movies deteriorating in

quality over time.

Since movie demand is increasing exponentially, Tan and Netessine use a logarithmically

transformed time series model to examine the dynamics of demand across the distribution.

Equation 1. Logarithmic model of demand on a trend variable. (Tan & Netessine, 2009)

They eventually make the following findings:

1) The top 20 percent of movies constitute approximately 85 percent of total demand,

significantly more than 80 percent, leading them to reject the Long Tail effect.

2) Consumers over time indeed watch more niche movies in absolute terms (Hypothesis 1),

but the rate at which they shift demand from the hits to the niches is considerably lower

than the growth rate of product variety.

3) Normalizing for product variety and measuring popularity in relative terms, consumers

watch more and more hits over time (Hypothesis 2).

4) Consumers who do watch niches tend to be heavy users, who constitute only a small part

of the entire user base. Therefore, hits continue to drive the market.

While Tan and Netessine make interesting conclusions, their results by and large are from

a high-level point of view and do not incorporate more meaningful differences among movies

Tang 14

and users. They view hits and niches according to the existing demand distribution and how it

changes over time, given specific cutoffs (Figure 1).

Figure 1. Distribution of demand or ratings for movies on Netflix over time. (Tan & Netessine, 2009)

Meanwhile, I use a different approach in considering which movies are in the Long Tail, offering

an ex-ante definition based on individual movie characteristics, and how movies benefit directly

from their ratings. I also further observe behavior of subgroups of users, specifically those users

with "heavier" rating activity.

3 Data & Methodology

3.1 Description of Data Set

Like Tan & Netessine, I use the Netflix Prize training data set7, made publicly available

in the company's open competition in 2006 for an improved collaborative filtering algorithm to

7 Netflix provided what they refer to as a training data set and qualifying data set. Participating teams in the Netflix

Prize were to use the qualifying set in the actual competition by predicting rating scores for the entire set while only

being informed of the scores for half of the data. The training set was larger and used only for teams to practice.

Tang 15

predict user ratings for movies in the Netflix library. My data consists of a collection of 17,770

text files (one for each Netflix movie title), with approximately 100 million ratings in total8.

Each observation in the Netflix Prize data set contains several fields: movie ID, user ID, rating

(from 1 to 5, with 5 being the best), and date of rating. Additionally, the Netflix Prize data

provides a movie index file that lists the movie IDs, their corresponding movie titles, and the

years of theatrical release. Since the algorithm is used to give recommendations based only on a

user's past ratings, no other information about the users or movies is made available.

Using OpusData Explorer to search a movie database called The Numbers, I filter out all

its movies by total budget and release date. I then add additional fields to these movies, such as

production budget, genre, box office (domestic & international) figures, MPAA rating,

distributor, date of movie release, and whether the film is a sequel. By further differentiating my

selection of movies by specific characteristics such as budget, I create three categories

(blockbusters, indies, and intermediate). For simplicity, I choose to look only at movies that had

been released in theaters, and all movies chosen were released domestically between 1999 and

2005 (or the approximate time period in which all the Netflix ratings were given), limiting the

Netflix ratings I would actually observe to a much smaller subset. I classify movies with

production budgets9 of over $100 million as "blockbusters," while movies with under $10

million budgets were considered "indie films." For an intermediate category, I select all the

movies that the 20 highest-grossing actors of all-time (according to Box Office Mojo) appeared

in between 1999 and 2005, excluding any that were already in the blockbuster or indie categories.

8 This was still only a fraction of the over 1 billion ratings Netflix already had at this point.

9 Marketing budgets would have been a better measure, but proved more difficult to find.

Tang 16

These intermediate films are an attempt to isolate the word-of-mouth "marketing" associated

with star power, in contrast to the actual marketing campaigns that blockbusters would run.

To analyze the data I had downloaded in raw text format, I selectively import the Netflix

Prize text files for my movies of interest (as identified by their movie IDs), as well as the movie

index file, into Stata ultimately as a single data set. I then download the additional movie data

fields from The Numbers into Excel spreadsheets and import into Stata, merging on movie names

with my aggregate Netflix Prize data set.

For my analysis, I generate the following variables, defined in Table 1 and described

further in my Study Design.

Table 1. Variable Descriptions

Early response The average rating score for a movie within 180 days of its first rating.

Heavy user A user who has given at least 100 ratings.

Nonheavy user A user who has given between 1 and 99 ratings.

Blockbuster Movie with production budget of over $100 million.

Indie Movie with production budget of under $10 million.

Intermediate Movie in which one of the 20 highest-grossing actors of all time has appeared, excluding any already labeled as blockbuster or indie.

Movie age Difference between date of movie rating for observation and date of first movie rating.

Popular genre Action/adventure and comedy (including romantic comedy, excluding black comedy).

Other genre Black comedy, concert/performance, documentary, drama, horror, musical, thriller/suspense, and western.

Demand (1) Total ratings for movie.

Tang 17

Demand (2) Total ratings for movie in particular month, divided by total users who rated anything that month. (Measured after first six months for a movie.)

Heavy demand Total ratings by heavy users for a movie.

Heavy early response

The average rating score by heavy users for a movie within 180 days of its first rating.

As reference, Table 2 lists the averages for the key variables in my models, which become

important to my results by allowing me to compare the relative impact of regressors in

regressions across movie categories.

Table 2. Summary of Means

Variable Overall Blockbuster Indie Intermediate Popular Genre

Other Genre

Rating 3.570 3.672 3.508 3.553 3.542 3.602

Early response 3.356 3.726 3.248 3.411 3.422 3.311

Heavy user - rating score

3.424 3.580 3.356 3.403 3.404 3.444

Heavy user - number of

ratings 126.0 24.0 34.8 67.2 63.9 62.1

Nonheavy user - rating score

3.605 3.689 3.551 3.590 3.573 3.642

Nonheavy user - number of

ratings 26.1 6.5 6.1 13.5 13.9 12.2

Movie age (in number of

days) 1180.1 1133.8 1076.6 1344.8 1213.1 1157.6

Movie-level Demand (1)

37450.1 88196.7 16985.1 53458.1 48751.9 29735.3

Movie-month Demand (2)

0.00501 0.00982 0.00202 0.00744 0.00582 0.00444

Heavy demand 7235.1 13625.0 3739.8 10586.2 9066.9 5990.0


3.235 3.626 3.084 3.346 3.300 3.190

Tang 18

For the blockbuster category, there are over 3.4 million ratings, 39 films rated, and

approximately 420,000 unique users who have given ratings for these films. For the indie

category, there are over 3.5 million ratings, 207 films rated, and approximately 385,000 unique

users who have rated these films. For the intermediate category, there are over 7.5 million ratings,

141 films rated, and over 450,000 unique users who have rated these films. In total, the aggregate

data set has approximately 14.5 million observations, over 470,000 unique users, and 387 films

rated. Blockbusters receive higher rating scores on average, likely because of their mass appeal.

Early response, which only includes the ratings received for a film within the first six months of

its first rating, has an even greater disparity in rating score for blockbusters versus indies.

Though indies are less well-received at first, their average rating greatly increases over time,

confirming that recommendations become better at matching users with these films.

There is a very different breakdown by genre between these three categories of movies

(blockbusters, indies, and intermediate), as illustrated by Table 3. Blockbusters have a much

higher percentage of action/adventure films, which one would expect based on the typically

higher budgets given to these special effects-laden films. On the other hand, comedies tend to

make up much more of the Indie and Intermediate categories because they typically do not

require a large budget. Based on what is most frequently "green-lit" in Hollywood, I categorize

action/adventures and comedies (including romantic comedies or "rom-coms") as the "popular"

genres. Many other genres, including horror, documentary, and thriller/suspense, are not even

present among the Blockbusters.

Tang 19

Table 3. Breakdown of genres for movies across categories

Blockbuster

Percent

Indie Intermediate

Number Percent Number Percent Number Percent

Action 16 41.03 3 1.45 15 10.64

Adventure 15 38.46 2 0.97 12 8.51

Black Comedy - - 2 0.97 4 2.84

Comedy 2 5.13 43 20.77 32 22.70

Concert/Performance - - 2 0.97 - -

Documentary - - 11 5.31 2 1.42

Drama 5 12.82 86 41.55 41 29.08

Horror

- - 10 4.83 4 2.84

Musical - - 2 0.97 - -

Not reported - - 28 13.53 - -

Romantic Comedy 1 2.56 8 3.86 8 5.67

Thriller/Suspense - - 10 4.83 19 13.48

Western - - - - 1 0.71

(missing) - - - - 3 2.13

Total 39 100.00 207 100.00 141 100.00

I also divide users by their user activity (i.e. how many ratings they have given). I

consider the most active users as "heavy users," or those likely to be "movie buffs" as opposed to

casual movie-watchers/raters. Heavy users, despite making up only 4.7 percent of the total users,

are responsible for 19.25 percent of the total ratings. Aside from being more critical in their

ratings than non-heavy users, they also watch different types of movies. Approximately 26.5

percent of all ratings by heavy users are for action/adventure movies, compared to 31 percent for

other users. Heavy users instead watch more comedies (approximately 26.5 percent of their

ratings versus approximately 24.5 percent of non-heavy users' ratings) as well as more movies in

the smaller genres, such as horror, western, musical, and concert/performance, which are

associated with indies rather than blockbusters.

Tang 20

3.2 Study Design

Using my aggregate Netflix data set, I test the following hypotheses:

Hypothesis 1: Collaborative filtering helps match non-mass market movies to demanders.

Hypothesis 2: Collaborative filtering permits more heterogeneity in users or demanders.

Hypothesis 1 is connected to the indie effect, or how movies on the Long Tail of Netflix are

affected. Hypothesis 2 examines the heterogeneity of tastes over time, also taking into account

the dichotomy of users between movie buffs (with a greater variety in tastes) and casual movie-

watchers.

For Hypothesis 1, I build a linear regression model (1) of movie-level "demand" (the total

number of ratings a movie receives10

) on early response and a time-correction variable (i.e. when

the movie was first rated). I test whether the early response for an indie movie is more important

in predicting its total demand than the early response for a blockbuster, given the substantial

marketing that big-budget movies would already enjoy. I also build a linear regression model (2)

with early response and movie age as the independent variables and demand on a movie-month

level as the dependent variable. To measure only the subsequent impact from the early ratings, I

exclude the first 180 days of ratings in this definition of demand. This definition of demand also

normalizes the number of ratings for a movie by the total number of users who have given any

ratings on Netflix that month. In other words, this "demand" considers how popular a movie is

relative to all its peers at a specific time.

10

The number of viewings would likely be a more appropriate measure of demand, but data on this is unavailable. I

use the number of ratings as a proxy for viewings, assuming that users are equally likely to give ratings for movies

they view across movie categories.

Tang 21

Movie-level Model: gfirstratindatenseearlyrespoDemand _210 (1)

Movie-month Model: agemovienseearlyrespoDemand _210 (2)

For Hypothesis 2, I consider whether there is greater user heterogeneity over time or

whether users who do exhibit a greater variety in tastes (the heavier users) are simply benefitting

more. By splitting up users by the extent of their rating activity and classifying them as either

heavy or non-heavy, I look specifically at heavy demand on a movie-level, a slight modification

of Model 1. I also look specifically at heavy early response (the average of only the ratings given

by heavy users within six months of a movie's first rating) to see if heavy early response has a

different effect than early response in general. To see if heavy users are becoming more or less

common, I observe how the breakdown between heavy and non-heavy users evolves over the

time period of the data set, proceeding with the current definition of heavy users and then

offering a more dynamic definition that better reflects which users are heavy year over year.

By introducing a new definition of heavy users as those who give over double the

average number of ratings that all users give in a year, I more accurately identify whether heavier

users have become more or less common in relative terms. This definition allows the possibility

for users to be considered either heavy or not year-to-year. As Table 4 shows, the average rating

score by heavy users goes up markedly (especially for indies), indicating they are either

becoming more lenient or they are getting better matches.

Heavy user A user who has given more than double the number of ratings that users gave on average in a year.

Tang 22

Table 4. Summary of means for ratings by heavy users overall, and in 2000 versus 2005

Overall Blockbuster Indie Intermediate

Rating 3.464 3.617 3.421 3.431

Number of ratings per user

95.98 18.28 24.93 52.76

Rating (in the year 2000) 3.315 3.626 3.024 3.319

Number of ratings per user (in the year 2000)

27.95 2.67 3.13 22.15



104.67 21.34 27.14 56.19

Heavy demand 3925.52 7388.33 1922.58 5892.87

Heavy early response 3.206 3.648 3.035 3.332

Finally, using my second definition of heaviness, I reconstruct heavy demand and heavy early

response, rerunning my regressions with these variables.

4 Results

4.1 Regression Models

Table 5 presents the results from regressing movie-level demand (1) on early response

and the date of a movie's first rating (a time-correction variable), per each category of movie.

Table 5. Movie-level Demand

(1) (2) (3) (4) (5)

Blockbuster Indie Intermediate Popular Genre Other Genre

EARLY_RESPONSE 77,393 16,770 58,974 50,360 30,263

(15,419) (2,449) (6,078) (5,804) (3,594)

DATE_FIRSTRATING -22.57 0.0618 -3.199 1.560 -10.13

(13.95) (3.129) (5.232) (5.989) (4.057)

(Intercept) 153,180 -38,455 -98,359 -147,808 87,361

(202,846) (48,699) (77,893) (88,732) (63,405)

Tang 23

Table 6 shows the results from regressing movie-month demand (2) on early response and movie

age, per each category of movie.

Table 6. Movie-month Demand

(1) (2) (3) (4) (5)


EARLY_RESPONSE 0.00721 0.00196 0.00521 0.00397 0.00416

(0.000512) (6.92e-05) (0.000337) (0.000281) (0.000140)

MOVIE_AGE -6.72e-06 -2.18e-06 -8.28e-06 -6.13e-06 -4.23e-06

(6.06e-07) (1.04e-07) (3.37e-07) (3.33e-07) (1.86e-07)

(Intercept) -0.0106 -0.00247 -0.00223 -0.00202 -0.00558

(0.00201) (0.000246) (0.00121) (0.00103) (0.000499)

The movie-level and movie-month demand are, as what one would expect, most responsive to

the early response for blockbusters in absolute terms. However, once I take into account the

average level of demand (movie-level or movie-month level) for each category that I

summarized in Table 2, I see that demand responds relatively more to early response for indies

than for blockbusters, confirming an indie effect. For instance, an increase in early response by a

rating score of one results in a 98.73 percent increase in total movie demand for the average indie,

compared to a 87.75 percent increase for the average blockbuster.

There are several caveats I wish to address about the results for these models. Although

ratings are more likely to suffer from reviewer bias11

with a smaller sample of ratings, the

reviewers who do rate these niche titles are more likely to be heavy users. One would presume

that these heavy users are more likely than non-heavy users to give their own ratings, free of

external biases. I am also not measuring the score of subsequent ratings, but simply the number

of ratings (in total, or per month). The results for the Intermediate category also prove to be too

inconclusive for me to make any meaningful comments. One issue with the selection of the films

11

The first or average rating tends to dictate what the following ratings will be.

Tang 24

for this category is likely that the films are bounded both above and below (by budget). There is

also the possibility that some of these highest-grossing actors I looked at do not actually have the

star power to draw in audiences, but instead have had extended careers absent much fanfare (or

minimal roles in a series of blockbusters). Big-name directors or producers (or even studios) can

also play a part in users' intent-to-view, which I do not account for.

Finally, there is also not as significant a difference between the relative changes in

demand for movies of "popular" genres versus other genres as there is for blockbusters versus

indies, so this effect is not nearly as interesting. The difficulty with this could be that comedies,

which are clearly a popular genre of films for studios to produce, are technically indies for the

most part because of their inexpensive budgets. To produce a more accurate hits versus niches

dichotomy using genre, I could have factored in marketing budget12

and whether the movie is

produced by a major studio.

4.2 User Heterogeneity

Tables 7 and 8 presents the results from regressing heavy demand on early response and

regressing heavy demand on heavy early response, respectively.

Table 7. Heavy Demand (1)

(1) (2) (3) (4) (5)


EARLY_RESPONSE 7,500 2,943 7,142 6,556 4,649

(1,369) (401.3) (760.1) (676.1) (563.0)

(Intercept) -14,317 -5,751 -13,844 -13,366 -9,363

(5,150) (1,329) (2,634) (2,353) (1,899)

12

As mentioned prior, marketing budget numbers tend to be far more difficult to find than overall budgets, however.

Tang 25


(1) (2) (3) (4) (5)


EARLY_RESPONSE_HEAVY 7,996 3,114 6,966 6,328 4,958

(1,280) (355.2) (699.4) (634.5) (487.7)

(Intercept) -15,372 -5,825 -12,677 -11,867 -9,751

(4,689) (1,127) (2,371) (2,140) (1,589)

I find that heavy demand for indies is more responsive to early ratings (i.e. the relative changes

are greater) than it is for blockbusters, and that the difference in responsiveness is greater than it

was for overall demand for indies and blockbusters. An increase in early response by a rating

score of one implies 2943 more ratings by heavy users for an indie, compared to the 3739 total

ratings by heavy users (on average) per indie, or a 78.7 percent increase in heavy demand.

Meanwhile, the same increase in early response implies only a 55.0 percent increase in heavy

demand for blockbusters.

This disparity in responsiveness of demand between indies and blockbusters is clearly

greater when I look only at heavy users than before when I included all users. Therefore, heavy

users (who, as we have seen, watch a large number of indies and films of less popular genres)

actually seem to be influenced more by how others have rated indies than blockbusters. A higher

score, even if the ratings are from non-heavy users, is more valuable for these "long tail" films in

the decision-making process of heavy users on what to watch. Contrary to the theory that casual

users would benefit most from the ratings associated with recommendations, the "movie buff"-

type users actually find the ratings more meaningful. Thus, they not only give ratings the most

frequently, thereby influencing others, they also use ratings the most in deciding whether to

"demand" an unwatched movie. Casual users more likely are influenced by outside sources such

as advertising, so they naturally gravitate more to blockbusters regardless of Netflix rating. The

Tang 26

effect remains very similar if early response is restricted to only ratings given in the first six

months by heavy users, as one would expect. Ratings are given anonymously, though heavy

users are likely to make up many of the first reviewers for any new movies.

When looking at user heterogeneity over time, it is difficult to isolate different fragments

of tastes and whether the changes in user tastes can be attributed to the Netflix algorithm itself or

outside forces instead (e.g., changes in content). For example, the composition of ratings by

genre does not change much over time. Instead, I have just seen how heavy users with a greater

variety in tastes are benefitting, which I consider a more plausible phenomenon than movies

being reached to more fragments or "communities" of demanders.

In 2000, there are 6845 total users who gave ratings, 1285 of which are "heavy users"

according to the absolute definition I previously offered. In 2005, there are 424,737 total users,

22,084 of which are heavy users. At first glance, it appears more casual users had been joining

Netflix, and early adopters of the Netflix service are much more likely to become heavy users

because they keep getting recommended good matches and using the service. However, this

definition does not lend a fair comparison over time, since 2005 would likely have newer users

who simply have not had enough time to give that many ratings while the users in 2000 very

likely had not given over 100 ratings until sometime after 2000.

Using my year-over-year relative definition of heavy users instead, I find that in 2000,

only 131 out of the 6845 total users or 1.91 percent are considered heavy (overall users giving on

average 11.99 ratings that year). In 2005, users give on average 41.21 ratings; 8405 of the

424,737 total users or 1.98 percent are heavy. Thus, even as there is an influx of total users and

users are giving many more ratings on average, the proportion of heavy users stays relatively the

Tang 27

same. Heavy users continue to make up only a small proportion of the total users, but naturally

their ratings take up a much larger proportion of total ratings.

I run through the same models as before, with heavy demand and heavy early response

altered to only include ratings by this year-over-year group of heavy users. Tables 9 and 10

present the results from regressing heavy demand (2) on early response and regressing heavy

demand (2) on heavy early response, respectively.


(1) (2) (3) (4) (5)


EARLY_RESPONSE 4,077 1,515 3,754 3,391 2,533

(944.0) (229.3) (500.8) (414.1) (343.2)

(Intercept) -7,853 -2,991 -6,962 -6,711 -5,142

(3,552) (759.4) (1,735) (1,441) (1,158)


(1) (2) (3) (4) (5)



(886.6) (185.4) (417.9) (361.5) (268.5)

(Intercept) -7,985 -2,316 -5,187 -5,970 -3,851

(3,269) (582.2) (1,416) (1,220) (868.0)

Based on these results, heavy users are still reacting much more strongly to early response for

indies than for blockbusters, even under my new definition for heaviness. For indies, there is a

78.8 percent increase in heavy demand associated with an increase in rating score of one for

early response; for blockbusters, there is only a 55.2 percent increase.

Tang 28

5 Conclusion

Companies and marketplaces residing on the Internet preserve large banks of private or

proprietary information on inventory and customers; Netflix is no exception. There is no

generally available industry-wide data on the performance of movies via online distributors,

explaining why economic experiments on this market are much less frequent than for box office

performance. However, this thesis uses one of the rare publicly available data sets on movie-

watching behavior through online channels, in an attempt to verify beliefs about the types of

movies that benefit from electronic commerce and search/recommendation tools such as

collaborative filtering. Netflix, while having released only a fraction of its total data, still is an

excellent example of an Internet service that uses collaborative filtering and a testing ground for

which types of items belong to the Long Tail.

I ultimately find that positive early feedback benefits to a greater degree "indie" films that

fall under my definition of the Long Tail, as compared to blockbusters with presumably much

larger marketing budgets. This has direct implications for Netflix or other online movie

platforms, as well as content providers (such as movie studios and television networks). Given

the increasing difficulty in getting movies exhibited in theaters, more studios outside of the Big

Six (Disney, Warner Bros., Universal, Columbia, Paramount, and 20th Century Fox) will be able

to green-light and produce films with the thought of ancillary markets such as Netflix in mind.

Meanwhile, an ongoing challenge for Netflix will be to continually license diverse and lesser-

known content that caters to the tastes of its subscribers. By filtering these programs correctly to

the proper demanders, Netflix can even turn niches ("long tail" films) into hits on its platform,

fueled by positive buzz or word-of-mouth.

Tang 29

Perhaps the most intriguing inference from my results is that heavy users are apparently

more influenced by ratings than casual users are. Since all of us as consumers have been

preconditioned by our experiences with other online services such as Yelp or Angie's List, this

finding seems counterintuitive at first. On Yelp, for example, the casual user is almost certainly

the one benefiting the most, thanks to all the reviews provided by heavy users or "foodies." On

Netflix though, the heavy user benefits the most in using other users' ratings as information for

his or her movie-watching decision process. Of course, Netflix also differs from the above

mentioned platforms because its ratings are purely anonymous and absent of written reviews.

One cannot filter ratings by "heavy" or "elite" users, though my models indicated that heavy

early response did not affect heavy demand much differently than aggregate early response did.

By the first definition of heavy users (with an absolute cutoff), there is a relatively small

group of users who have contributed to a relatively large proportion of total ratings. Under the

stricter second definition (that measures heaviness relative to a moving yearly average of user

activity), even as users give more ratings on average over time, a similar fraction of users

remains heavy year-over-year. I would claim that these users remain dedicated to giving such a

great quantity of ratings (and users in general are giving more ratings each year) because they

find existing ratings for movies valuable and informative. By proceeding to use these ratings as a

basis to watch more movies, they then give more ratings as well to improve their matches and so

that other users would be informed. This positive feedback loop could help explain why Netflix

retention rates have been so high, even as global subscribers are increasing at a rapid rate.

With the advent of big data, analysts have speculated that Netflix could use the data it

holds on user behavior to strategically change its content and pricing in the future. Since indies

almost always cost less to license than blockbusters (especially if using my definitions), it would

Tang 30

be beneficial to Netflix to license and recommend more "Long Tail"-type content for its users to

watch. As Benjamin Shiller (2013) shows, Netflix data scientists can also use the rating

behaviors of its subscribers (in additional to other information such as web browsing and

demographics) for first-degree price discrimination, leading to higher economic profits. Heavy

users ("movie buffs") not only watch more movies, but also find the Netflix ratings themselves

more useful in guiding them to appropriate movies to watch, as we have seen. For now though,

Netflix remains adamant that its pricing policy will remain the same.

Further studies of interest using the Netflix Prize data set in combination with additional

movie databases such as The Numbers could include box office numbers (an entirely different

market) and how strongly correlated they are with viewership in ancillary markets such as DVDs

or video streaming, how sequels or remakes perform (e.g., whether they have a return on

investment comparable to the original film), and how the timing of a movie release on Netflix

affects its demand (e.g., any seasonality patterns). One could further break down the films by

MPAA rating, Rottentomatoes aggregate rating, or specific genres to see which "types" of films

benefit more or become more prevalent over time, and whether heavy users behave differently

across these different categories.

For further additional research, it would be interesting to look at another popular service

(e.g., Amazon) that uses a collaborative filter for its recommendations but with one key

difference: ratings are not anonymous. This could have an impact on several variables, including

the number of ratings per user or item (especially in comparison to views or "purchases"), the

impact of early ratings relative to more recent ratings, and the relative impact or influence of a

heavy user's rating versus a casual user's rating.

Tang 31

6 Appendix

6.1 Stata Code

Importing Netflix Prize data from text files:

Generating variables:

Tang 32

Constructing models:

Tang 33

6.2 Tables

Table 1. Variable Descriptions

Early response The average rating score for a movie within 180 days of its first rating.

Heavy user A user who has given at least 100 ratings.

Nonheavy user A user who has given between 1 and 99 ratings.

Blockbuster Movie with production budget of over $100 million.

Indie Movie with production budget of under $10 million.

Intermediate Movie in which one of the 20 highest-grossing actors of all time has appeared, excluding any already labeled as blockbuster or indie.

Movie age Difference between date of movie rating for observation and date of first movie rating.

Popular genre Action/adventure and comedy (including romantic comedy, excluding black comedy).

Other genre Black comedy, concert/performance, documentary, drama, horror, musical, thriller/suspense, and western.

Demand (1) Total ratings for movie.

Demand (2) Total ratings for movie in particular month, divided by total users who rated anything that month. (Measured after first six months for a movie.)

Heavy demand Total ratings by heavy users for a movie.


The average rating score by heavy users for a movie within 180 days of its first rating.

Tang 34

Table 2. Summary of Means

Variable Overall Blockbuster Indie Intermediate Popular Genre

Other Genre

Rating 3.570 3.672 3.508 3.553 3.542 3.602

Early response 3.356 3.726 3.248 3.411 3.422 3.311

Heavy user - rating score

3.424 3.580 3.356 3.403 3.404 3.444

Heavy user - number of

ratings 126.0 24.0 34.8 67.2 63.9 62.1

Nonheavy user - rating score

3.605 3.689 3.551 3.590 3.573 3.642

Nonheavy user - number of

ratings 26.1 6.5 6.1 13.5 13.9 12.2

Movie age (in number of

days) 1180.1 1133.8 1076.6 1344.8 1213.1 1157.6

Movie-level Demand (1)

37450.1 88196.7 16985.1 53458.1 48751.9 29735.3

Movie-month Demand (2)

0.00501 0.00982 0.00202 0.00744 0.00582 0.00444

Heavy demand 7235.1 13625.0 3739.8 10586.2 9066.9 5990.0


3.235 3.626 3.084 3.346 3.300 3.190

Tang 35

Table 3. Breakdown of genres for movies across categories

Blockbuster

Percent

Indie Intermediate

Number Percent Number Percent Number Percent

Action 16 41.03 3 1.45 15 10.64

Adventure 15 38.46 2 0.97 12 8.51

Black Comedy - - 2 0.97 4 2.84

Comedy 2 5.13 43 20.77 32 22.70

Concert/Performance - - 2 0.97 - -

Documentary - - 11 5.31 2 1.42

Drama 5 12.82 86 41.55 41 29.08

Horror

- - 10 4.83 4 2.84

Musical - - 2 0.97 - -

Not reported - - 28 13.53 - -

Romantic Comedy 1 2.56 8 3.86 8 5.67

Thriller/Suspense - - 10 4.83 19 13.48

Western - - - - 1 0.71

(missing) - - - - 3 2.13

Total 39 100.00 207 100.00 141 100.00

Table 4. Summary of means for ratings by heavy users overall, and in 2000 versus 2005

Overall Blockbuster Indie Intermediate

Rating 3.464 3.617 3.421 3.431

Number of ratings per user

95.98 18.28 24.93 52.76



27.95 2.67 3.13 22.15



104.67 21.34 27.14 56.19

Heavy demand 3925.52 7388.33 1922.58 5892.87

Heavy early response 3.206 3.648 3.035 3.332

Tang 36

Table 5. Movie-level Demand

(1) (2) (3) (4) (5)


EARLY_RESPONSE 77,393 16,770 58,974 50,360 30,263

(15,419) (2,449) (6,078) (5,804) (3,594)

DATE_FIRSTRATING -22.57 0.0618 -3.199 1.560 -10.13

(13.95) (3.129) (5.232) (5.989) (4.057)

(Intercept) 153,180 -38,455 -98,359 -147,808 87,361

(202,846) (48,699) (77,893) (88,732) (63,405)

Table 6. Movie-month Demand

(1) (2) (3) (4) (5)


EARLY_RESPONSE 0.00721 0.00196 0.00521 0.00397 0.00416

(0.000512) (6.92e-05) (0.000337) (0.000281) (0.000140)

MOVIE_AGE -6.72e-06 -2.18e-06 -8.28e-06 -6.13e-06 -4.23e-06

(6.06e-07) (1.04e-07) (3.37e-07) (3.33e-07) (1.86e-07)

(Intercept) -0.0106 -0.00247 -0.00223 -0.00202 -0.00558

(0.00201) (0.000246) (0.00121) (0.00103) (0.000499)


(1) (2) (3) (4) (5)


EARLY_RESPONSE 7,500 2,943 7,142 6,556 4,649

(1,369) (401.3) (760.1) (676.1) (563.0)

(Intercept) -14,317 -5,751 -13,844 -13,366 -9,363

(5,150) (1,329) (2,634) (2,353) (1,899)


(1) (2) (3) (4) (5)



(1,280) (355.2) (699.4) (634.5) (487.7)

(Intercept) -15,372 -5,825 -12,677 -11,867 -9,751

(4,689) (1,127) (2,371) (2,140) (1,589)

Tang 37


(1) (2) (3) (4) (5)


EARLY_RESPONSE 4,077 1,515 3,754 3,391 2,533

(944.0) (229.3) (500.8) (414.1) (343.2)

(Intercept) -7,853 -2,991 -6,962 -6,711 -5,142

(3,552) (759.4) (1,735) (1,441) (1,158)


(1) (2) (3) (4) (5)



(886.6) (185.4) (417.9) (361.5) (268.5)

(Intercept) -7,985 -2,316 -5,187 -5,970 -3,851

(3,269) (582.2) (1,416) (1,220) (868.0)

Table 11. Breakdown of Netflix ratings by genre for blockbusters, indies, and

intermediate

Blockbusters (by ratings)

Tang 38

Indies (by ratings)

Intermediate (by ratings)

Tang 39

6.3 Figures

Figure 1. Distribution of demand or ratings for movies on Netflix over time. (Tan & Netessine, 2009)

Figure 2. Sample search query on OpusData.

Tang 40

Figure 3. Descriptive statistics for aggregate Netflix Prize data set. (Tan & Netessine, 2009)

Figure 4. Histogram of Netflix ratings from 2000 to 2005. (Tan & Netessine, 2009)

Figure 5. The exponential rise in movie ratings, and linear increase in number of movies being rated. (Tan &

Netessine, 2009)

Tang 41

Figure 6. Three forces of the Long Tail. (Anderson, 2006)

Tang 42

7 References

Anderson, Chris. 2006. The Long Tail: Why the Future of Business Is Selling Less of More.

New York: Hyperion.

Berger, Jonah, Alan Sorensen, and Scott J. Rasmussen. 2010. "Positive Effects of Negative

Publicity: When Negative Reviews Increase Sales," Marketing Science, September 29(5):

pp. 815-827.

Brynjolfsson, Erik, Yu (Jeffrey) Hu, and Duncan Simester. 2011. "Goodbye Pareto Principle,

Hello Long Tail: The Effect of Search Costs on the Concentration of Product Sales,"

Management Science, August 57(8): pp. 1373-1386.

Brynjolfsson, Erik, Yu (Jeffrey) Hu, and Michael D. Smith. 2006. "From Niches to Riches: The

Anatomy of the Long Tail," Sloan Management Review, July 47(4): pp. 67–71.

Brynjolfsson, Erik, Yu (Jeffrey) Hu, and Michael D. Smith. 2010. "Long Tails vs. Superstars:

The Effect of Information Technology on Product Variety and Sales Concentration

Patterns," Information Systems Research, December 21(4): pp. 736-747.

Cieply, Michael. 2013. "Hollywood Wants Numbers on the Digital Box Office." Last updated

09/2013. http://www.nytimes.com/2013/09/16/business/media/movie-industry-wants-to-

get-a-handle-on-the-digital-box-office.html, accessed September 20, 2013.

Dellarocas, Chrysanthos, Guodong (Gordon) Gao, and Ritu Narayan. 2010. "Are Consumers

More Likely to Contribute Online Reviews for Hit or Niche Products?", Journal of

Management Information Systems, September 27(2): pp. 127-157.

Elberse, Anita. 2008. "Should You Invest in the Long Tail?", Harvard Business Review, July

(86): pp. 88-96.

Elberse, Anita and Oberholzer-Gee, Felix. 2008. "Superstars and Underdogs: An Examination of

the Long Tail Phenomenon in Video Sales," MSI Reports: Working Paper Series, July (4):

pp. 49-72.

Eliashberg, Jehoshua, Anita Elberse, and Mark A.A.M. Leenders. 2006. "The Motion Picture

Industry: Critical Issues in Practice, Current Research, and New Research Directions,"

Marketing Science, November 25(6): pp. 638-661.

Eliashberg, Jehoshua and Shugan, Steven. 1997. "Film Critics: Influencers or Predictors?",

Journal of Marketing, April 61(2): pp. 68-78.

Fleder, Daniel and Hosanagar, Kartik. 2009. "Culture's Next Rise or Fall: The Impact of

Recommender Systems on Sales Diversity," Management Science 55(5): pp. 697-712.

http://www.nytimes.com/2013/09/16/business/media/movie-industry-wants-to-%09get-a-handle-on-the-digital-box-office.html

http://www.nytimes.com/2013/09/16/business/media/movie-industry-wants-to-%09get-a-handle-on-the-digital-box-office.html

Tang 43

Karniouchina, Ekaterina. 2011. "Impact of Star and Movie Buzz on Motion Picture Distribution

and Box Office Revenue," Intern J. of Research in Marketing, (28): pp. 62-74.

Moul, Charles. 2005. A Concise Handbook of Movie Industry Economics. New York: Cambridge

University Press.

Netflix, Inc. 2009. "Netflix Prize." Last updated 2009. http://www.netflixprize.com, accessed

September 10, 2014.

Shih, Willy, Stephen Kaufman, and David Spinola. 2009. "Netflix," Harvard Business School.

Case No. 607-138, April.

Shiller, Benjamin. 2013. "First Degree Price Discrimination Using Big Data," Brandeis

University. Working Paper No. 58, August.

Tan, Tom and Netessine, Serguei. 2009. "Is Tom Cruise Threatened? Using Netflix Prize Data to

Examine the Long Tail of Electronic Commerce,” Wharton Business School. Working

Paper No. 1361, September.

Tucker, Catherine and Zhang, Juanjuan. 2011. "How Does Popularity Information Affect

Choices? A Field Experiment," Management Science, May 57(5): pp. 828-842.

Vanderbilt, Tom. 2013. "The Science Behind the Netflix Algorithms That Decide What You'll

Watch Next." Last updated 08/2013.

http://www.wired.com/underwire/2013/08/qq_netflix-algorithm/, accessed September 10,

2013.

Zhou, Wenqi and Duan, Wenjing. 2012. "Online User Reviews, Product Variety, and the Long

Tail: An Empirical Investigation on Online Software Downloads," Electronic Commerce

and Research Applications, May 11(3): pp. 275-289.

http://www.netflixprize.com/

http://www.wired.com/underwire/2013/08/qq_netflix-algorithm/

the collaborative filtering effect of netflix ratings for indie films … · 2015-08-19 · the...

Documents