Netflix Revisited: Running Out of DVDs to Rate, and Other Observations

Been a while since I revisited Netflix, but it's time. This installment will review two areas: Long Waits, and Broken Ratings and Recommendations.

1. More on Long Waits
I've talked about Long Waits before, but this is going to cover different ground. If you recall from the previous article, I spoke of the various DVD titles that had become "Very Long Wait" status in my Netflix Queue. As time went on and the waits stayed as waits, I decided to remove everything from my Queue except for the items marked as "Short Wait," "Long Wait," or "Very Long Wait."

One DVD in particular I was interested was Hands on a Hard Body (hereinafter referred to as HOAHB), a documentary that I'd never heard of before becoming a Netflix customer. Indeed: I heard about it because Netflix recommended it to me. And yet, Netflix would not rent it to me --- it was marked as "Very Long Wait." Occasionally I would go to Netflix's details page for this DVD and read up on the reviews. Customers loved it. "This movie is hilarious," said one. "One of the great documentaries," said another. "Real people in a real town, willing to suffer for a truck they really need," said yet another. "This movie is suspenseful, touching, funny, and sad. Low production values, but high emotions." Ok sounds great. Unfortunately, Netflix wouldn't send it to me.

Month after month after month went by. Collapsing the Queue down to seven or eight titles did the trick: suddenly, a "Long Wait" or "Very Long Wait" DVD title would become available and arrive in the mail. Yet HOAHB remained elusive. Eventually my Queue got down to two titles: Heavenly Creatures, a film Peter Jackson did before taking on The Lord of the Rings, and HOAHB. Creatures eventually arrived (and was pretty disappointing). Now, all that remained on my Queue was HOAHB.

More days passed. More weeks passed.

I refused to add anything more to my Queue. This was war. By gum, Netflix was going to send me HOAHB or else. More time passed. Nothing changed. My usage of Netflix shriveled to nothing.

Netflix was going to win this battle, and I wasn't happy about it. As someone must've once said, "If you can't beat 'em, buy 'em."

So that's what I did. I removed HOAHB from my Queue, went over to eBay, found a half-dozen brand-new, still-in-shrinkwrap copies of the HOAHB DVD, and bought the cheapest one.

When my eBay-purchased HOAHB DVD arrived, and I watched it. It wasn't side-splittingly funny, but it was okay. Was it worth the long wait? No. Do I regret buying it? Not really. I'm going to sell it back on eBay. (If you want to buy it from me directly, just email me an offer... first offer over $6 + $2 for US-only shipping gets it: brian at nettle dot com).

Of course, this gave me an idea, which turned into a humor piece over at Denounce Newswire. But this idea isn't just imaginary or fictional: I know of real cases where employees at companies are sharing their DVDs with each other in a kind of free borrowing system modeled after Netflix. (In fact, a group of employees at one company calls their little service "Workflix" which I think is a brilliant name.)

I'm not sure if these employees are sharing their own personal DVDs or sharing their Netflix rentals or both. I suspect both. In fact, think about it: imagine a bunch of employees each sign up for Netflix. Then they compare and keep track of their Queues in a sort of "Group Queue". That way, they theoretically increase their chances of beating the "Long Wait" syndrome: maybe a DVD title that's "Long Wait" on most Queues within the group will show up as "Available Now" on one of them. That's all the group would need: once the disk arrives, they can share it amongst themselves. A classing gaming of the system.

2. Breaking The Netflix Ratings System
I mentioned back in April (see that article's section 2, entitled "The Rating Marathon") that I was starting to rate more and more movies, in response to Netflix's "Rate More!" challenge.

That was then (April 2003).

 

This is now (July 2003).

How'd I rate fifteen thousand three hundred movies at Netflix? Well, it required some cleverness that's for sure. First, I haven't seen all those movies --- but then, I don't plan on ever seeing most of them. With Netflix's "Not Interested" and "No Opinion" rating options, I was able to rate huge gobs of Netflix's inventory. How'd I find it all? That was tricky. You can't browse Netflix's library: it doesn't show you everything, not by a long shot. If you traverse the whole genre tree, and I mean the big tree, accessible from the "See All Genres" link, you still won't see every Netflix title. Instead, you have to search for all kinds of things --- simple Hollywood keywords like "war", "funny", "she", "love", "battle", "over", "on", "with", and so on, as well as "1" through "9" --- that'll get you a ton of titles to rate.

Now here's the thing: as I mentioned back in April, I eventually reached 8000 DVDs rated, and yet I found Netflix's much-touted Recommendations engine still recommending things that didn't make any sense, including titles I'd indicated I was not interested in. Well, it only got worse on the climb from 8000 to 15300.

Soon, I was seeing empty genres. And soon if I clicked on "Rate More!" I began seeing this:

Now, this actually isn't true much of the time. Netflix is regularly adding new titles to its inventory, and every couple of days I stumble on gobs of new titles I've not rated yet. So while I will see the above error message, "There are no more movies to rate," often all I have to do is start clicking down the genre links on the left nav, and sure enough, scrolling to the bottom of the genre pages shows me stuff like this:

Even worse, I was seeing this when I clicked on the "Recommendations" link in the left navigation column:

Note the message: "In order to get recomendations you need to rate more movies. The more movies you rate, the more specific and helpful your recommendations will be."

Before you laugh and say, "well duh, you rated every damn movie in the store so you can't expect Netflix to recommend movies you've not yet rated!" it's not that easy. All along the way, from rating 259 movies all the way up to rating 15300 movies, I never had a sense that Netflix was doing that great a job recommending things to me. I won't say it was a complete failure --- for a while there it turned into the kind of reminder service I had always wanted it to be: it presented titles to me that I'd forgotten about but definitely wanted to see. But even at the halfway point (7500 titles rated), it couldn't recommend anything interesting to me. It was around here that things seemed to start breaking down. One would think that the recommendations engine would have an IQ of 250 from having such a huge amount of data on-hand to provide intelligent recommendations to the customer. Nope. I never had the impression there was much sophistication in the recommendations. Now, of course, having rated pretty much all of the inventory at Netflix, the recommendations flat-out don't work at all. it's understandable, in one sense, but in another, it's disappointing.

Understandable, in that I've exhausted Netflix's inventory of possible titles it thinks I've not seen. Disappointing, though, in the sense that here I am, a frequent renter, active customer, obviously taken a great interest in the company and the service and the DVD inventory, and it can't figure out something imaginative to recommend to me? I mean, start recommending movies I've rated highly. Maybe I wanna see 'em again. Even things I've rented. I woulnd't mind. Heck, I might just rent 'em again! Whoda thunk it possible? (In fact, I've already rented some titles multiple times at Netflix --- something every Netflix customer is bound to do once they've been with the company for 8 months or more, I suspect.... I mean: surely you've rented the same movie more than once at Blockbuster? Even if several years intervened between rentals?)

Continued in Part Eleven...

Click Here for a list of previous installments in Nettle's ongoing series on Netflix.

Posted by brian at July 10, 2003 03:59 PM