This paper provides a comprehensive tutorial on enhancing Reinforcement Learning (RL) for wireless networks using Large Language Models (LLMs). It proposes a taxonomy for LLM roles in RL (state perceiver, reward designer, decision-maker, generator) and showcases their application in various wireless scenarios to address classical RL's limitations in generalization, interpretability, and sample efficiency.