Netflix and Qwikster

What one of the biggest blunders in group decision making can teach us about the power of independent feedback

Photo by freestocks on Unsplash

In 2011, Reed Hastings, CEO of Netflix, was facing a big decision about what to do about their DVD rental and streaming businesses. It was clear the two had become separate businesses, with distinct strategies and trajectories. Separately, the trailblazing streaming business was starting to face competition, as services like Hulu entered the market.

Netflix needed to change their strategy.

In a blog post, Hastings announced their solution: They would split Netflix into two independent businesses. The streaming business would remain Netflix but the DVD rental business would now be rebranded as Qwikster.

It was a disaster. And, with the benefit of hindsight, it is obvious why.

Customers who previously could head to one website as their one stop shop for streaming and DVD services would now have to navigate two completely separate ones, complexifying a previously seamless experience.

Netflix had fantastic brand recognition, associated with great service and innovation. Now they were asking customers to learn a wholly new brand. Qwikster would have to earn name recognition from scratch.

It’s confusing as to why, after building such brand equity, Netflix would choose to toss the name aside.

Making matters worse, prior to the split, the subscription price for both services was $10 a month total. Now, with the split, customers would pay $7.99 for the DVD rental service under the Qwikster banner but would also have to pay an additional $7.99 for the Netflix streaming service (the same price that Hulu was charging). That meant they were now paying $15.98 for services they were previously paying $10 for.

That’s a 60% price hike. Adding insult to injury, no new features came with the increased price.

Needless to say, this did not go over well. As Wired magazine bluntly put it,

“…it's impossible to see how the split itself benefits customers. The price and plan changes that flustered many of them months ago remain in place, but the company now directs them to two web sites with two search indexes, two completely separate sets of recommendations, two entries on their credit card statements, and so forth.”

It was such a disaster that it even earned an SNL lampoon.

The debacle caused Netflix to lose 800,000 subscribers and saw its stock lose half its value, including one week where the stocked dropped a whopping 24%.

So the question is, of course, how did a company as smart as Netflix make such a disastrously stupid decision? Was it just that no one in the company saw the response from the market coming?

The answer to that is a resounding no. As Reed Hastings tells it in the “Designing a Culture of Reinvention” episode of the a16z podcast, there were serious doubts about the new strategy among leadership. And Hastings did attempt to get the feedback. He just was unable to hear it because no one expressed their doubts with any force.

If all these smart, successful, accomplished execs thought this was such a bad idea, why didn’t they speak up? Why wasn’t Hastings able to get the feedback?

Hastings, in retrospect, figured out the answer.

“What happened was something more subtle, which is the leading people were too deferential. They were like, “Reed’s been right so many times, I think this is going to be bad, but he must see something.” And they didn’t know that each other kind of all felt the same.”

*****

Groups should be powerful, positive forces in decision making: more minds, more viewpoints, more data. We humans wouldn’t have evolved as social creatures if there were no benefits to doing things in a group. But there is a giant chasm between the logic of two heads are better than one and what actually goes down when groups get down to business. 

There have been so many fascinating – even bizarre – experimental findings by social scientists about the ways groups can go completely off the rails. 

How’s this for just one? Solomon Asch showed people two cards, each with a line drawn on them. The line on the first card was clearly longer than the line on the second card. More than 99% of people identified the line on the first card as longer when they were asked about the cards alone. But if you put those people at a table with a bunch of people in cahoots with the experimenter and all of those people say the line on the second card is longer before the participant gets give their answer, 36.8% of people will disobey what their own eyes are telling them and agree with the confederates that the line on the second card is longer.

Decades of social science backs up and expands on Asch’s findings. The way we generally elicit feedback in group settings reduces the chances for us to hear what other people actually think. And that reduces decision quality.

Garold Stasser and William Titus demonstrated this reduction in decision quality with groups tasked with deciding which of three candidates was best suited to be student body president. The researchers created dossiers containing positive and negative attributes of each candidate. Based on all the information, Candidate A was designed to be the most favorable.

Each member of these groups reviewed the dossier in advance, indicated their private preference, and then the groups caucused to choose their preference.

For some groups, each individual member got a complete dossier on all three candidates. That meant each participant had a complete picture of each candidate, regardless of what was shared in the group discussion.

For some other groups, the researchers gave each member an incomplete dossier. The participants were informed that the dossiers contained some common information, but the remainder was scattered among the members in the group. For these groups, all of the information about each candidate was available to the everyone in the group, but only if each member shared everything they knew in the group discussion.

That meant that some groups independently and individually had all the information they needed to choose the best candidate, while other groups only collectively had what they needed.

That difference had a huge impact on decision quality.

In groups where each member got the complete dossier, Candidate A was, as expected, the preference of most members individually as well as the preferred candidate once these groups caucused. In fact, 98% of the groups chose Candidate A.

When the information about the three candidates was distributed among participants, most of the members of these groups did not make Candidate A their private preference. The surprising thing was that when they discussed the candidates with the group, their decisions didn’t get any better. Why? Because participants didn’t share the information in their individual dossiers, even though they were told each member in the group had some exclusive information about each candidate.

Instead, they quickly settled on a candidate based on which one received a plurality of votes in their private preferences. Once the group saw a consensus forming, members with divergent information were unlikely to share it. The researchers also found that the group discussion did nothing to correct the inaccuracies in the way the participants viewed each candidate prior to the group convening.

These groups ended up showing no preference for Candidate A over the other candidates after caucusing. Discussing as a group did nothing to improve decision quality.

*****

The way we generally elicit feedback in group settings reduces the chances we’ll hear what other people actually think. It could be because an influential person in the room sways people to their side, or they think that influential person is wrong but don’t want to disagree with the boss or expert. It could be that once people sense consensus building they don’t want to rock the boat. It could simply be groupthink.

Whatever the dynamic, discovering people’s opinions in a group setting is generally bad for decision accuracy, often to disastrous effect like in the Netflix case. The good news is there is a simple solution to all these woes: Elicit feedback independently and asynchronously.

Groups that are working independently and asynchronously are called nominal groups, just a group or team that, for some part of the decision making process, is working independently. The best decisions come out of groups that toggle between group discussion and nominal group work.

By getting feedback independently, no one on the team knows what anyone else believes before offering their opinion. It gives people a chance to offer up a more complete picture of what they think and why they are thinking it.

Most importantly, they don’t know what the big dog thinks or if there is consensus on the team which quarantines them from getting contaminated by the group. This makes it more likely that people will actually give you their honest opinion.

In retrospect, Hastings realized this was his error.

“And in hindsight, if we had said all execs write down, “What’s your level of confidence in this move, disaster to genius,” you know it would have come up twenty people thinking it’s going to be bad and then the strength of the co-feeling would have realized, “No, we’re right. Reed is wrong.” Okay? And probably would have stopped it.

And so the key thing there is we said is in the future all major decisions we gotta have everybody write down what they think, you know positive 10 to negative 10, and why, and we just do that in a Google sheet but any shared medium is fine. And then everyone knows what everyone else thinks…and so that little device has helped us avoid the chaos and catastrophe.”

*****

The error that Hastings made, trusting people to speak up in group settings, is a common one. We intuit that many heads are better than one. We think if we ask for people’s opinions they will give them to us. But they don’t. Sometimes because they just don’t want to speak up. Sometimes because they convince themselves they are wrong. Either way, eliciting opinions in a group will not get you the feedback you need.

The solution Netflix implemented, elicit feedback independently, is not so common. But it is strongly backed by science.

Even for brainstorming tasks, the domain where working in groups feels the most #creative, nominal groups have been found to markedly outperform real groups in the number of ideas produced as well as the quality of those ideas. 

A well-known implementation of the use of nominal groups to improve decision quality is the Delphi method. Burcu Gurcay, Barbara Mellers, and Jonathan Baron of the University of Pennsylvania explained the method in their study of its effectiveness: “Group members begin with independent judgments. These judgments and rationales are then given to all group members by a moderator who ensures anonymity. After considering others’ judgments, members can revise their opinions, and the process is iterated until consensus is reached.”

These researches found that, in groups using this method, “Discussion resulted in more accurate estimates and greater consensus relative to independent estimates.”

You don’t need a decision-making group to be “more than the sum of its parts.” You just need all of its parts to be fulling functioning. As Reed Hastings found out through hard experience, groups can unintentionally silence the individual members.

If you want to get the most from a decision group, what’s most important is what you do BEFORE the group weighs in.

Some of my best stuff comes from my readers. One reader, Chris Neumann, sent along this great material on Qwikster, including pointing me to the a16z podcast and the SNL skit. Thank you, Chris!