This paper introduces an explainable AI framework comparing Quantum Boltzmann Machines (QBMs) and Classical Boltzmann Machines (CBMs) to enhance transparency in decision-making. By leveraging quantum principles for richer latent representations and focused feature attributions, the framework demonstrates improved accuracy and clearer identification of key influential features, advancing trustworthy AI systems.