Predicting Performance
To the extent that success is predictable, our goal is to predict it. Questions that SportLab's BASELINE system will help answer include...
- Will a player be drafted?
- When will he be drafted?
- Should he be drafted?
- How much money would this player make?
- How long will he play before retiring for any reason?
- What is the likelihood that he will accomplish some milestone?
- What is a reasonable contract (salary and length) for this player?
The actual predictions that are displayed in the main page of BASELINE are customized using a series of short experiments and interviews. This process allows BASELINE to identify team preferences and priorities. As preferences and priorities change, BASELINE will adapt the program to account for new information on the fly. This can be done because rather than attempting to design studies to validate every individual algorithm separately, we have validated a process for creating algorithms that includes both archival validation and a forward oriented feedback loop. This basically implies that the algorithm is first generated and cross-validated using historical data and user input but then adapts to future data by identifying its own mistakes and searching for modifications that will improve its own performance. This design gives the system an amazing degree of flexibility. It's important to remember that the program adapts specifically to your needs, not some outside publication or scouting service.
People have strengths and weaknesses when it comes to decision making. BASELINE allows team decision makers to capitalize on their strengths while avoiding many of their weaknesses. It's simply a tool to help scouts make accurate decisions in a fraction of the time it takes today. Furthermore, while your scouts focus on the most promising candidates, BASELINE will continuously monitor thousands of additional players on the off chance that one of them might be worth a second look.
One of the factors that gives BASELINE an advantage in accuracy is that our method of evaluating players has a very specific purpose. For example, an evaluation of when a player will be drafted does not necessarily imply that he should be drafted at that spot. An analysis of where a player should be drafted is most useful when it is designed to meet the specific preferences and priorities of a given team. A team dedicated to making a Super Bowl run in the upcoming season would be interested in prospect analyses that are fundamentally different from the analyses that would interest a first year expansion team.
Finally, the questions that everybody asks: What about heart? What about character? "What about the intangibles that simply can't be measured? Isn't talent evaluation more art then a science? How can you make an evaluation using only statistics? Don't the scout's already do what you are talking about? Isn't it obvious which players are going to be the most successful?
All these questions deserve answers that we would be happy to provide in a one-on-one phone call or e-mail conversation. But here let us simply note that BASELINE does account for heart, character, and other intangibles to the extent that these characteristics predict player success. BASELINE doesn't just use sterile statistics. It uses both objective (height, weight) and subjective (recruiter's evaluations, interviews) information and it combines all data in a statistically optimal fashion. BASELINE does not replace a scout's expertise, but it will enhance a scout's performance by 36% or more. The value of an extra impact player or two per draft may initially sound like a small improvement, but over multiple seasons it has a dramatic impact on any roster. Reducing cognitive biases in your decision making will help your team avoid embarrassing mistakes that have plagued NFL decision makers for decades.