Bias mitigation

Where bias enters a scoring process

NIST's SP 1270, Towards a Standard for Identifying and Managing Bias in Artificial Intelligence, sorts bias into three groups. The split is useful for any scoring process, automated or not.

• Systemic bias. Built into institutions and history. If a fund's past investments came mostly from one network, a model trained on them learns that network as a signal.
• Statistical and computational bias. Comes from the data and the method: unrepresentative samples, missing data for some groups, or a variable that stands in for a protected trait (a proxy).
• Human cognitive bias. Comes from the people scoring: first impressions, similarity to oneself, anchoring on the first number heard.

Mitigation has to address all three. Fixing the model does little if the criteria themselves reward a background rather than a skill.

Bias mitigation methods, stage by stage

1. Before scoring: fix the criteria. Every criterion should connect to the work or the deal. Drop ones that mostly measure pedigree. Write score levels in a rubric before anyone is reviewed.
2. During scoring: structure the judgment. Ask every candidate the same questions in a structured interview, score each answer before discussing it, and compare candidates one question at a time. Daniel Kahneman and Olivier Sibony give the reason for scoring before discussion: once the most senior person speaks, the rest drift toward that view, so judgments should be made independently and only then combined (McKinsey Quarterly, 2021). Hide information that is irrelevant to the criteria, such as photos.
3. For models: check inputs for proxies. Test whether any input predicts group membership. Postcode, school and some hobbies often do.
4. After scoring: measure outcomes by group. Compare selection rates. A large gap is adverse impact and calls for a look at which criterion causes it. An independent bias audit does this formally.
5. Repeat. Pools, markets and models change. Recheck on a schedule.

Worked example: a fund's first-meeting rate

Northwind Ventures, a fictional seed fund, reviews a year of inbound pitches. It splits them by how they arrived.

The partners then rescore a random sample of 50 cold pitches against the written thesis, with the intro source hidden. Eleven score as well as the median warm-intro pitch. The intro itself had been doing much of the work of the score. Northwind changes its process: every pitch is scored for thesis fit before anyone sees who sent it, and the source is recorded but not scored. The partners still decide who to meet; they now see the cold pitches that clear the bar.

Bias mitigation vs bias audit

An audit without mitigation produces a report. Mitigation without an audit has no way to show it worked. Both belong in AI governance.

Common mistakes

• Removing the protected field and stopping there. Proxies carry the same information.
• Adjusting scores by group. In US employment this can itself be unlawful: Title VII, as amended in 1991, bars adjusting scores or using different cutoff scores on the basis of race, color, religion, sex or national origin. Fix the criteria instead. This is not legal advice.
• Relying on one-off bias training. A single workshop does not change how scores are given.
• Measuring only the final decision. Bias often enters at the first screen, where most people are dropped.
• Scoring without evidence. A score that cites nothing is hard to check for bias. Evidence-based scoring makes each point traceable. For models, explainable AI does the same job: it shows which inputs drove each score, which is how a proxy gets caught.

How ScoringFactory approaches it

ScoringFactory scores every founder, company or candidate against the same written bar and cites each score to the record, so a team can see what drove it and question it. It ranks; it does not reject, and the team decides. Because it learns from a team's past decisions, it also reflects their patterns, so outcomes should be checked by group. See the trust page for current practices.