This paper systematically investigates fairness in medical imaging AI benchmarking competitions, revealing significant biases in dataset composition and critical flaws in data accessibility, licensing, and documentation. The findings highlight a disconnect between leaderboard success and clinically meaningful AI, urging for improved transparency and reusability standards.