“Culture fit” is not a hiring criterion, because nobody has written it down
A criterion that cannot be stated precisely cannot be assessed consistently. In practice it resolves to familiarity.
I removed "culture fit" from our interview rubric, and the objection I heard most often was that it captures something real that the other criteria miss. I agree that interviewers experience something real. I have never seen anyone write down what it is.
That is the whole argument. A criterion that cannot be stated precisely cannot be assessed consistently, cannot be calibrated across interviewers, and cannot be defended when two candidates receive different scores for the same behaviour.
What it resolves to
In the panels I have audited, "culture fit" resolves to familiarity: the candidate reminded the interviewer of themselves, or of someone who has worked out well. This is not malice and it is not usually conscious. It is what an undefined criterion does under time pressure.
The cost is not only fairness, though that is sufficient. It is that the criterion systematically selects against the people you claim to be hiring — anyone whose background makes them unfamiliar to the current panel.
The replacement
We replaced it with two written criteria: how the candidate handles disagreement with someone more senior, and whether they have changed their mind about something significant in their field. Both are behavioural, both produce evidence you can quote in a debrief, and both can be failed by someone the panel likes.
That last property is the test of a real criterion. If nobody the panel enjoyed talking to has ever failed it, it is not a criterion.