In October 2006, Netflix announced a competition to significantly improve the recommendation system currently used at the popular movie rental site. Teams and individuals were competing for a $1 million prize. Last week, an article in the New York Times reported that two finalist teams have been selected and a winner will be declared in September.
There was no shortage of entries for this competition, so what was it that contributed to the success of these two teams? When pressed, these teams did not credit some brilliant insight, rather they pointed to the power of teamwork and collaboration. The article notes:
“…the formula for success was to bring together people with complimentary skills and combine different methods of problem-solving.”
One of the leading teams is from AT&T Research and is comprised of statisticians, machine learning experts and computer engineers from four different countries. The other leading team is actually an conglomeration of over 30 people called the Ensemble, the result of team combinations as the contest progressed. It turns out, as more and more teams combined their best ideas, the solution got better and better. One of the contestants, David Weiss, says:
“the surprise was that the collaborative approach works so well, that trying all the algorithms, coding them putting the together far exceeded our expectations.”
And Weiss might add that the combined solution worked better than any one team’s approach (with the possible exception of AT&T Research) and certainly better than any one individual’s approach.
This kind of competition, and the incredible innovation it is driving for Netflix is the tangible result of several trends that we, in the leadership development community, must be aware of and adapt to:
- Technology enables greater collaboration. The kind of collaboration and teamwork required to conduct a competition like this is only possible because of the rapid changes in technology. It is much easier and simple now for teams to work remotely and share information in real-time. As Web 2.0 tools become more ubiquitous, the successful organizations will be the ones that are able to rapidly build high performing teams in a way that is faster and more effective than the competition.
- Diversity of thought and perspective leads to the best solution. In the past, diversity has been viewed by many in our field as a compliance issue. What the Netflix competition highlights is that diversity of thought and perspective is a crucial ingredient for any team or organization that is attempting to solve problems and find optimal solutions. This idea, although not new, is thoroughly explored in the book The Difference by Scott Page.
- Successful companies manage talent both internally and externally. Traditional talent management is about attracting and developing talent within the organization, but that is starting to change with the advent of crowdsourcing. When you think about it, what Netflix has accomplished here is the successful outsourcing of an incredibly complex research and development problem. From a talent management perspective, we need to give Netflix credit for attracting and motivating some of the best minds in the world to work on their problem, and the net investment was only $1 million. In return Netflix has gained two great solutions and a lot of free publicity. Motivating talent outside of your organization to solve difficult and complex problems is a growing trend and a major shift in how organizations have traditionally viewed leadership and talent management.
This takes the customer suggestion-box to a whole new level. I wrote a blog post about what John Psarouthakis calls the “Productivity Swap Bubble.” He argues that much of the productivity of companies is being shifted to customers. They now do (especially in terms of customer service) what we used to expect from employees. This Netflix effort to get external teams to solve a Netflix problem seems to me to be an example of productivity swap. It doesn’t get recorded in GNP. What does this mean for measuring our economy?