Exactly. Managers should insist that peer-reviewers give specific attributes or behaviors.

Also, "cultural fit" questions can be casually ageist as hell. I was asked once (at 40 or so), "We like to skateboard around the office for fun. What do you like to do?" I sincerely believe it was (for the 20-something interviewer) an innocent, almost routine interview question. I diffused the question by saying that sounded like a blast but I prefer Nerf Blaster wars or pranking people by hijacking their wireless mouse or whatever. I got the job offer but I've been on other interviews where I aced the manager's and technical interviews but got shut down by a barely college graduate who clearly didn't think they'd have to work with "their Dad." Not much you can do about that.

By saying 'not a good fit' requires additional details to be valid you are actually acknowledging that 'not a good fit' is not a valid reason to not hire someone, those additional details are the valid reason to not hire someone.

When people say "not a good fit" they mean "cultural fit" ie, not related to technical skill, which is what the context of the parent post was implying. Maybe that's not so ubiquitous, but I've come across a technical person not be able to point out exactly what they don't like about another technical person's skills.