This paper investigates how real-world shortcuts manifest across different layers of MedCLIP, a medical Vision-Language Model, and its vision encoder. By attaching linear probes and analyzing calibration and layer-wise confidence, the study reveals that despite high performance, the model is vulnerable to shortcuts, underscoring the critical need for high-quality and well-annotated datasets in medical AI.