JACKSONVILLE - The Browns had a tough decision again for the franchise QB spot and this time I think they got it right with Mayfield. I think some of the other NFL GM's were trying to coax them into a mistake with leaked information about how great the other guys were, but they all had warts. Mayfield struck me as almost a modern day, collegiate version of Jim McMahon, and that spells winner.
from behavioralscientist.org
http://behavioralscientist.org/how-to-identify-talent-five-lessons-from-the-nfl-draft/How to Identify Talent: Five Lessons from the NFL DraftCade MasseyApril 23, 2018
This week the Cleveland Browns will decide which, if any, of the top quarterback prospects to acquire with the first pick in the NFL draft. There are four leading contenders, and the team has been evaluating them in detail for more than a year. They will pay the young man over $30 million, and his success or failure will drive the team's fortunes for years. No pressure. Never mind the sobering 52 percent success rate for picking between two players in the NFL draft.
Few industries invest as much into each "hire" as professional sports. I have seen this firsthand, working closely with NFL teams for more than 10 years and more recently with professional basketball and baseball teams. Their issues parallel those I see in nonsports organizations, from multinational corporations to graduate school admissions and urban nonprofits. The challenge of finding the next generation of talent is both ubiquitous and vexing. Successful track records are rare.
Yet, if there is one consistent yet underappreciated principle for making good hires, it is that process beats technology. It turns out that best practices in hiring have much in common with what psychologists have preached for decades.
More independence is often the biggest improvement an organization can easily make in their hiring process.
Why is it so hard to draft or hire well? One reason it is that it is so difficult to be consistent. A draft pick is the product of a year's work, by dozens of people, an outcome negotiated from diverse perspectives to satisfy diverse preferences. The process is overseen by owners and general managers subject to the shifting sentiment of outside forces. And the actors are constantly changing, with scouts and coaches moving between teams each year. This is hardly a recipe for consistency—and not that different from many nonsports organizations.
That is why a structured decision process is especially valuable. The best teams have philosophies and policies that firmly guide the draft process. There are few organizations capable of doing that consistently, and it's a real advantage for those who can.
There is no silver bullet. In sports or out, there is no test or technology that reliably distinguishes star from bust, despite all our advances and many promises otherwise. How should we do it then? What can we learn from how NFL teams draft? Here are five universal prescriptions.
#1: Understand Your Goal
I once worked for an NFL team that was much more successful drafting players one side of the ball than the other. One year we surveyed the staff about the kind of player they wanted to acquire at a particular position. The position was on the side they struggled with, and we discovered there was zero consensus on the type of player to pursue. Given the range of opinions, the group average was meaningless. No wonder they had trouble drafting the right guys!
To paraphrase Lewis Carroll, if you don't know where you're going, any road will get you there. An easy way to flub a hire from the beginning is to skip asking exactly what it is you're looking for. People often don't understand their decision objectives, but the most successful sports teams are clear about their goal and don't stray from the principles and attributes they've established.
#2: Keep Your Judges Apart
NFL teams have dozens of scouts and a complicated method of getting multiple looks at their prospects. Even marginal prospects are seen by more than one scout, and the top guys might be evaluated by a dozen different people. In some teams a scout will know the previous scouts' opinions before forming his or her own while other teams keep scouts blind to previous evaluations. This is a key difference.
Replicability is a referendum on your process.
People are impressionable. When they are exposed to other's opinions before forming their own, they tend to anchor on the existing view. More generally, the "wisdom of the crowd" depends on the independence of the respective opinions within the crowd. Yet that independence is easily compromised, in many ways—anchoring, of course, but also common backgrounds, training, friends, etc. You may think you're getting 5-6 opinions when effectively you're getting only 1.5!
A good hiring process explicitly pushes against these compromising factors: Don't let people talk to each other or see other's opinions before providing their own, expose the candidate to judges in different ways and at different points in time, and bring people with different perspectives into the process. More independence is often the biggest improvement an organization can easily make in their hiring process.
#3: Break the Candidate Into Parts …NFL teams break players down in minute detail. There are physical qualities: For a quarterback it includes, for example, arm strength, mobility, and release. And then there is their "makeup," or the intangibles considered critical for success. Ideally even these intangibles are broken into component parts, such as work ethic and selflessness. This decomposition means players are explicitly evaluated on many distinct dimensions.
This involves work. It's much easier to give one, global evaluation—like or dislike, hire or reject. These overarching evaluations are natural and efficient, but unfortunately, they are often biased. For a more reliable evaluation, you need to break the objective into component parts and evaluate them separately. Daniel Kahneman recognized the importance of this step years ago when designing performance evaluations for the Israeli military and has advocated for them since.
#4: … and Bring Them Back Together MechanicallyAn NFL executive once mused about what would happen if their draft board fell down (teams generally sort players by placing nametags on a physical board). After painstakingly ranking their prospects for months, would they be able to put them back up in the same order? His answer was no, and that troubled him. The logic is simple: If your hiring rankings aren't perfectly replicable, you have noise in the process; if you have noise in the process, you're more likely to take bad candidates and pass on good ones. Replicability is a referendum on your process.
The single best way to reduce noise is to aggregate opinions mechanically. This can mean finalizing a scout's opinion of a prospect by merely summing his or her component grades for an overall score. At the team level it can mean summarizing the group's collective opinion by simply averaging scouts' opinions. At the very least this approach provides a more systematic starting point for a group discussion. This sounds boring, but that's kind of the point.
Hiring is best thought of as a forecasting process, and the only way to improve forecasts is to map them against results and refine the process over time.
Letting a spreadsheet—or a model—summarize your judgment may seem like an abdication, but we've known for decades that simple linear models outperform intuitive judgment. This isn't because evaluators don't have expertise but rather because they apply their expertise inconsistently. Hiring decisions shouldn't depend on the loudest voice in the room or how many strong candidates were seen earlier that day. The best way to ensure this is to aggregate mechanically. It also happens to be a big time-saver, freeing you to focus on the issues humans handle better than machines.
#5: Keep ScoreNFL scouts are always looking for a guy with "it"—the elusive quality that seems to distinguish those who excel in the pros from those who struggle. Eyes light up when scouts talk about a player they believe has "it." Trouble is we don't know how reliable these beliefs are. On one hand, intuition based on years of repeated exposure and pattern recognition can be quite good. On the other, unless that experience comes with clear and regular feedback it can be misleading.
Of course, the rest of us are no different than scouts in this regard. We've all been animated by the sense we've just seen the next star in our field. The trick is to capture those judgments and track them over time to learn how predictive they are. This applies to all judgments. Hiring is best thought of as a forecasting process, and the only way to improve forecasts is to map them against results and refine the process over time. By doing this, teams learn which tests matter most, which questions are more informative, and which scouts are best with each position on the field.
Many organizations don't bother keeping score, because the payoff is delayed; it can be years before enough data arrive for reliable insights. The kind of foresight and humility required is exactly what distinguishes good management.
At the moment, the defending champions in all three major professional sports—the Eagles, Astros, and Warriors—are unapologetic devotees of "Moneyball" in some form. Moneyball isn't about a model or a technology but rather a decision-making process. The tenets of that process suggest steps any organization can take to improve their hiring. They are deceptively easy. The challenge isn't understanding them; it's sticking to them amid the drama of high-pressure hires. Now, back to (ahem) the Browns.
Sent from my iPhone
No comments:
Post a Comment