Every score needs a receipt

Most screening tools give you a number and a shrug. A candidate is a 74. Another is an 81. The list is sorted, the recruiter moves on, and nobody can say what the difference actually was.

That is not a hiring problem. It is a product problem. A score without evidence is a guess wearing a suit.

A score is a claim

When a system says a candidate has strong technical depth, it is making a claim about a person. Claims need support. In a technical screen, the support usually exists: a line in a resume, a query that returned the right rows, a design answer that named the trade-off, a test case that passed after three failed attempts.

The job of the grading system is to point at that support, not to replace it with a vibe.

At boson, every score carries four things:

• Reasoning in plain language.
• Evidence linked to the source.
• Confidence, because a model's certainty is not the same as correctness.
• Provenance, meaning the prompt and model version that produced the result.

If any of those is missing, the score is harder to defend to a hiring manager, a candidate, or an auditor.

Evidence should be verifiable, not plausible

A quote that sounds like it came from the resume is not evidence. It is a paraphrase with good manners.

Resume quotes should be checked against the source text. If the model produces a sentence that is not there, the quote should be dropped and the event logged. The same idea applies to code: cite the line range and the run, not a summary of what the code probably did. For process signals, cite the event and its timestamp.

This is a small engineering decision with a large trust payoff. It means a recruiter can click a score and land on the exact thing that caused it.

Confidence is part of the answer

LLM grading is not deterministic in the way a test case is. Running the same prompt once and treating the output as a fact hides that.

A better pattern is multi-sample judging: run the grader more than once, report the median score, and keep the spread. A wide spread is information. It says the evidence is thin, the rubric is ambiguous, or the answer sits between two levels.

That is exactly when a human should look. A run with high spread or missing evidence should be flagged for review, not quietly pushed into the shortlist.

Keep the human in the loop

Evidence-backed grading does not remove the recruiter. It gives them a better place to start.

Overrides should require a reason. The reason, the original score, the new score, the reviewer and the timestamp should all be stored. That audit trail protects the candidate and the company. It also makes the system teachable: a cluster of overrides is a signal that a rubric or prompt needs work.

The product rule is simple: steps can be required, but outcomes are scores and flags. Nobody is rejected automatically.

A practical checklist

If you are evaluating a screening tool, ask:

1. Can I open any score and see the evidence behind it?
2. Is the evidence verified against the source, or just generated next to it?
3. Does the tool report confidence, or pretend every score is equally certain?
4. Can a human override, and is the override logged?
5. Can I see which model and prompt version produced the result?
6. Does the candidate know what is being evaluated and why?

A shortlist is only useful if you can explain it. The receipt is the product.