Predictive scoring is a method that uses historical outcomes to estimate how likely a person, company, or deal is to succeed, expressed as a score that can be ranked, explained, and later checked against what actually happened.
How predictive scoring works
A predictive score answers one question: given what we know today, how likely is a good outcome later? The method borrows from predictive analytics (Hofman and colleagues, 2021, set out how it differs from explanatory modeling) and follows the same five steps whether the subject is a candidate, a founder, or a company.
- Define the outcome. Pick something you can observe later: still in role and rated strong after 12 months, raised a follow-on round within 24 months, hit plan in year one after acquisition.
- Collect past cases with known outcomes. Each case needs the information that was available at decision time, and only that.
- Find which inputs separated good outcomes from bad ones. This produces a scoring model and its weighting.
- Score new cases. Each gets a number, usually a probability or a rank. Sorted, the scores become a ranked shortlist of who to see first.
- Check against reality. As outcomes arrive, compare them to the scores and adjust.
Step 2 hides the biggest trap. If a case record includes anything learned after the decision, such as a later funding round, the model will look accurate in testing and fail in use. This is called leakage.
Why predictive scoring matters
The case for formal prediction is old and well tested. A 2023 review by Neumann and colleagues sums up several large meta-analyses across fields: combining information with a fixed rule gives more valid predictions of human performance than combining it in someone's head. For job performance, mechanical predictions were found to be about 50 percent more valid. Part of the reason is consistency: people do not apply the same weights from one applicant to the next.
The ceiling is also real. In personnel selection, Sackett and colleagues (2022) found that earlier validity estimates for hiring methods had been inflated by overcorrecting for range restriction, and revised most of them down by 0.10 to 0.20 points. Structured interviews came out as the top-ranked method. Even the best predictors of job performance leave most of the variance unexplained. A predictive score narrows the odds. It does not remove them.
For a venture fund doing founder scoring, a buyout firm, or a hiring team, that is still worth having. A score that is right a bit more often than unaided judgment, applied to every case, changes who gets the next meeting.
Worked example: predicting first-year success in sales hiring
Brightwater, a fictional software company, has hired 60 account executives in four years. It defines success as "still employed and at or above 80 percent of quota at month 12." Of the 60 hires, 27 met that bar.
Looking only at what was known at offer time, the team finds two inputs that separated the groups: candidates who had sold to the same buyer type before succeeded 15 times out of 22, and candidates whose work-sample call scored 3 or higher succeeded 19 times out of 30. Candidates with neither succeeded 4 times out of 21.
A new applicant, Candidate B, has sold to the same buyer and scored a 4 on the work sample. The model puts Candidate B in the group where most hires met the bar. The hiring manager still runs the final interview, and twelve months later the result goes back into the data as one more case. That feedback loop is what ties predictive scoring to quality of hire.
Sixty cases is a small sample, and the team should treat the model as a guide that will sharpen as more outcomes come in.
Predictive scoring vs rules-based scoring
| Predictive scoring | Rules-based scoring | |
|---|---|---|
| Where the weights come from | Learned from past outcomes | Set by people based on experience |
| Data needed | Past cases with known results | A clear rubric, no history required |
| Strength | Finds patterns people miss or overrate | Easy to explain and fast to set up |
| Weakness | Repeats past bias; needs volume | Encodes assumptions nobody tested |
| Typical use | Mature pipelines with years of outcomes | New funds, new roles, first hires |
Many teams start rules-based and move to predictive as outcomes pile up. Sales teams have done this for years with lead scoring, where conversion data arrives in weeks instead of years.
Common mistakes
- Leakage. Using information that was not available at decision time.
- Only learning from yeses. If you never see how rejected candidates or passed-on companies turned out, the model cannot learn what it missed.
- Treating a score as a probability without checking. A score of 0.8 should come true about 80 percent of the time. Testing that is model calibration.
- Training on biased history. If past decisions favored one group, the model will learn that preference. Run a bias audit on outcomes by group before relying on it.
- Never retraining. Markets, roles, and theses change. A model built on 2021 deals will misread 2026 ones.
How ScoringFactory approaches prediction
ScoringFactory learns from a team's own past decisions, the founders it backed and the ones it passed on, the hires that worked and the ones that did not, and applies that bar to new cases. Each score ties back to the record it came from, so a partner or hiring manager can see why a case ranked where it did and decide for themselves. See the flow.
Frequently asked questions
What is predictive scoring?
Predictive scoring estimates how likely a person, company, or deal is to reach a defined good outcome, based on how similar past cases turned out. The result is a score or probability used to rank new cases. It is checked over time by comparing scores to real results, and adjusted when the two drift apart.
What data do you need to build a predictive score?
You need past cases with known outcomes and the information that was available when each decision was made. Include cases you rejected or passed on if you can, since a model trained only on yeses cannot learn what it missed. A few dozen cases can show a rough pattern; hundreds make the estimate far more stable.
How accurate are predictive hiring or investing models?
Useful but limited. Research on hiring shows even the best methods, such as structured interviews, predict only part of later job performance. In investing, outcomes take years and depend on markets no model sees. Expect a predictive score to improve the odds over unaided judgment, and measure that against your own outcomes instead of trusting a vendor claim.
Is predictive scoring the same as a scoring model?
No. A scoring model is any structured way of turning inputs into a score, including one whose weights were set by hand. Predictive scoring is a scoring model whose weights were learned from past outcomes and whose output is meant to estimate the chance of a future result. Every predictive score uses a model, but not every model predicts.
Sources
- Sackett et al. (2022), Revisiting meta-analytic estimates of validity in personnel selection, Journal of Applied Psychology (APA)
- Hofman et al. (2021), Integrating explanation and prediction in computational social science, Nature
- Neumann, Niessen, Hurks and Meijer (2023), Holistic and mechanical combination in psychological assessment: why algorithms are underutilized and what is needed to increase their use, International Journal of Selection and Assessment (Wiley)