Invisible bias: effects on evaluation

An online research talk hosted by RIKEN AIP on invisible biases in computer vision evaluation: factors correlated with model outcomes that are not visible in the data yet systematically skew what benchmarks measure. Drawing on three recent studies, the talk covers bias introduced by camera acquisition and processing parameters, data leakage where evaluation images reappear in training data and inflate reported performance, and gender bias in vision-language models driven by demographic priors. It frames bias as a general threat to the construct validity of benchmarks, models, and visual representations, not only a fairness concern.