This paper explores the significant limitations in evaluating Explainable AI (XAI) methods, particularly focusing on the scarcity of robust human-grounded evaluations and the challenges posed by evolving data streams with concept drift. It highlights the need for more systematic and interdisciplinary approaches to assess the effectiveness and trustworthiness of AI explanations.