Alignment is often reduced to first-order norms: theft is bad, insults are inappropriate. But humans in a social world do more than recognize a rule; they track who enforces it, how severe the reaction is, and when society looks away. Using the NormReact dataset — 450 norm-violation scenarios with multi-perspective annotation — the study compares six language models against human judgment.
Models are not bad at detecting norm violations, but they show a systematic bias in predicting social reactions: they predict punishment where humans mostly expect inaction, restraint, or tolerance. And the larger the social distance, the worse the model's fit with humans — the model's social world is harsher than the real one.
If a model's social world-model is overly punitive, strict safety policies may amplify that very bias; alignment with fixed rules is not enough, and calibration against real human reactions is needed. Of course, the dataset, though multi-perspective, is still a limited sample of cultures — and 'human judgment' is no absolute ground truth either.