This paper systematically investigates fairness in medical imaging AI benchmarking competitions, revealing significant biases in dataset composition and critical flaws in data accessibility, licensing, and documentation. The findings highlight a disconnect between leaderboard success and clinically meaningful AI, urging for improved transparency and reusability standards.

This paper introduces a benchmark for evaluating long context reasoning capabilities of WebAgents through sequentially dependent subtasks that require retrieval and application of information from extended interaction histories. It observes a dramatic performance degradation as context length increases and proposes an implicit RAG approach for modest improvements.