Vendor scoring is the practice of rating suppliers and service providers against fixed, weighted criteria such as cost, capability, reliability, security and risk, so that selection and renewal decisions rest on comparable evidence instead of the last sales call.
How vendor scoring works
Vendor scoring is a form of supplier evaluation, and a systematic review by Chai and Ngai (2020) of supplier selection research from 2013 to 2018 treats it as a decision problem with many criteria and catalogs the techniques used to solve it. The usual method in practice is a weighted sum, one of the simplest tools in multiple-criteria decision analysis (Cinelli and colleagues, 2020).
- Set knockouts. Write the conditions a vendor must meet to be considered at all: a security certification, data residency, a contract term. These knockout criteria are pass or fail, not points.
- Choose 4 to 7 criteria. Typical ones are total cost, capability against requirements, reliability (uptime, delivery record), support, security and financial stability.
- Agree weights before seeing bids. Weights should sum to 100 percent.
- Define each score level. Use a rubric that says what a 1, 3 and 5 look like for every criterion, so two evaluators give the same vendor the same score.
- Score with evidence. Each score points to a document: a reference call, a test result, a contract clause.
- Compute and discuss.
vendor score = sum of (weight x criterion score). The total starts the discussion; it does not end it.
Why vendor scoring matters
Without a fixed method, vendor choices drift toward the best demo or the lowest headline price. Scoring forces the team to agree on what matters before the pitches start, which is the hard part.
Security has made this more pressing. NIST's SP 800-161 Rev. 1 on cybersecurity supply chain risk management treats suppliers as part of an organization's own risk, and recommends assessing them before and during the relationship. For private equity firms, vendor scoring shows up twice: in due diligence, where a target's dependence on one critical supplier is a risk to price in, and in portfolio operations, where firms standardize how portfolio companies choose and renew vendors.
Commercial teams use the same method when they buy tools for their own scoring, such as data providers for GTM scoring or sequencing software for outbound scoring.
Worked example: choosing a payroll provider
Brightwater Clinics, a fictional PE-backed group of 30 dental practices, is replacing its payroll provider. All three finalists pass the knockouts (SOC 2 report, US data hosting). The team scores them on a 1 to 5 scale.
| Criterion | Weight | Vendor X | Vendor Y | Vendor Z |
|---|---|---|---|---|
| Total three-year cost | 30% | 5 | 3 | 4 |
| Multi-state payroll capability | 25% | 2 | 5 | 4 |
| Reliability and references | 20% | 3 | 4 | 4 |
| Support | 15% | 3 | 4 | 3 |
| Financial stability | 10% | 4 | 5 | 3 |
| Weighted score | 100% | 3.45 | 4.05 | 3.75 |
Vendor X is cheapest but weak where Brightwater most needs strength: it operates in four states. Vendor Y wins on the total, and the team checks the result against intuition before signing. They also note that Vendor Z's support score rests on a single reference call, a thin piece of evidence, and record that next to the score.
Vendor scoring vs vendor risk assessment
| Vendor scoring | Vendor risk assessment | |
|---|---|---|
| Question | Which vendor is the best choice? | Is this vendor safe enough to use? |
| When | Selection and renewal | Before onboarding, then on a schedule |
| Output | A ranked comparison, often a scorecard | A risk rating and a list of required controls |
| Owner | Procurement and the business team | Security, legal or compliance |
The two connect. A risk assessment usually supplies the knockouts and the security criterion inside the vendor score. Folding all risk into a weighted score is a mistake, because a high price score can then outweigh a serious security gap.
Common vendor scoring mistakes
- Setting weights after the demos. Weights chosen once a favourite exists tend to favour it.
- Weighting risks that should be gates. If a missing control would stop the deal, make it a knockout.
- Undefined score levels. "Good support" means different things to different people. Write the rubric.
- Scoring on vendor claims. Ask for proof: references, trial results, contract terms.
- No record of why. At renewal, nobody remembers the reasons. Keep the evidence with the score so the decision has auditability.
- Never rescoring incumbents. Score the current vendor at renewal on the same sheet as challengers.
How ScoringFactory approaches it
ScoringFactory does not score vendors. The discipline it applies to companies and candidates is the same one good vendor scoring uses: a bar agreed in advance, scores tied to the record behind them, and a team that makes the final call. If you want that discipline on a target market or an open role, contact the founders.
Frequently asked questions
What is a vendor scorecard?
A vendor scorecard is a table that lists the criteria a team uses to judge suppliers, the weight of each criterion, and each vendor's score with the evidence behind it. It is used to compare finalists during selection and to review an incumbent's performance at renewal. A good scorecard defines what each score level means.
What criteria should you use to evaluate vendors?
Start with pass or fail requirements such as security certifications and data location. Then score four to seven criteria that matter for the purchase, commonly total cost, capability against your requirements, reliability, support quality, security and the vendor's financial stability. Drop any criterion that would not change your decision.
How do you weight vendor risk?
Split risk in two. Anything that would stop the deal, such as a missing security control, becomes a knockout and is not weighted at all. Remaining risk, such as financial stability or concentration, gets a weight like any other criterion. NIST SP 800-161 is a useful reference for supply chain security risks.
How often should vendors be rescored?
Score at selection, then again at each renewal and after any major incident, such as an outage or a breach. For critical suppliers, many teams also run a lighter annual review. Use the same criteria and weights each time so scores can be compared across years.
Sources
- SP 800-161 Rev. 1, Cybersecurity Supply Chain Risk Management Practices for Systems and Organizations, NIST, 2024
- Chai and Ngai (2020), Decision-making techniques in supplier selection: recent accomplishments and what lies ahead, Expert Systems with Applications (Elsevier)
- Cinelli, Kadziński, Gonzalez and Słowiński (2020), How to support the application of multiple criteria decision analysis? Let us start with a comprehensive taxonomy, Omega (Elsevier)