Human-in-the-loop

What human-in-the-loop means in practice

In scoring and AI decisions, human in the loop means one thing: the system recommends, a person decides. Four conditions make the loop real.

1. The person sees the recommendation before it takes effect. Not a weekly report of what already happened.
2. The person sees the reasons. Without explanations, review becomes guessing.
3. The person has authority and time to disagree. An override that needs three approvals will not happen.
4. The decision and any override are recorded. That record is what gives the process auditability.

Human-in-the-loop vs human-on-the-loop vs human-out-of-the-loop

Many teams mix them: a system may advance strong applicants with a person on the loop, but no applicant is rejected without a person in it.

What GDPR and the EU AI Act say about human oversight

A short summary, not legal advice.

• GDPR Article 22. Article 22 of the General Data Protection Regulation gives people the right not to be subject to a decision based solely on automated processing, including profiling, that produces legal effects or similarly significant effects on them. Exceptions exist (contract necessity, authorisation by law, explicit consent). Under the contract and consent exceptions, the controller must still offer safeguards that include at least the right to obtain human intervention, to express a point of view and to contest the decision. A law that authorises such decisions must set out its own safeguards.
• EU AI Act Article 14. Article 14 of the EU AI Act, Regulation (EU) 2024/1689, requires high-risk AI systems, a category that includes recruitment and selection tools, to be designed so people can oversee them effectively. The people assigned must be able to understand the system's limits, stay aware of the tendency to over-trust its output (automation bias), interpret the output correctly, decide not to use it or override it, and stop the system. Article 26 then requires deployers to assign that oversight to people with the competence, training and authority to do it. For recruitment tools these duties apply from 2 December 2027: a 2026 amendment, Regulation (EU) 2026/1744, moved the original August 2026 date.

Neither rule is satisfied by a person who clicks approve without looking. The point is that the human could have decided otherwise.

Worked example: a screening loop for 400 applicants

Ledgerly, a fictional Series A fintech, receives 400 applications for a product designer role and uses AI candidate scoring to rank them. The talent lead sets the loop up like this:

1. The tool ranks all 400 and shows reasons for each rank. It never rejects anyone.
2. The hiring manager reviews the top 40 and a sample of 20 from ranks 41 to 400, chosen at random.
3. In the sample, the manager finds 3 strong candidates ranked low because their portfolios were links the tool could not read. All 3 move to interview, and the team fixes the intake form.
4. Every advance, rejection and override is logged with the reviewer's name.

The random sample is what made the loop work. Reviewing only the top of the list would have confirmed the tool's view and missed its blind spot.

Why it matters for investment teams

Venture and private equity firms already run a human loop at the end: the investment committee. The gap is earlier, where automated deal screening may decide which companies a partner ever sees. A firm that lets a score silently drop companies has removed the human from the step that matters most for sourcing. The fix is the same as in hiring: rank rather than reject, show reasons, and sample below the cutoff.

Common mistakes

• Rubber-stamping. A reviewer approving hundreds of outputs an hour is a formality, not oversight.
• Reviewing only the top. Errors that bury good candidates or companies never surface.
• No reasons shown. The reviewer cannot disagree with a number.
• Overrides that cost the reviewer. If disagreeing means paperwork or blame, people stop disagreeing.
• Not tracking override rates. Zero overrides over months usually means nobody is looking. Track it as part of AI governance.

How ScoringFactory approaches it

ScoringFactory is built for the human in the loop. It ranks founders, companies and candidates against the team's own bar and cites every score to the record, so reviewers can check and disagree. It never rejects anyone or makes an investment or hiring decision; the team makes the call. Read more on the trust page.