Highlights: What are the main findings? The paper presents a unified overview of RL-based lane-changing systems by linking maneuver taxonomy, algorithm families, simulators, and evaluation metrics into one structured framework. A comparative analysis shows that no single RL paradigm suits all driving scenarios, revealing distinct trade-offs among value-based, policy-based, actor–critic, model-based, and hybrid methods. What are the implications of the main findings? The findings highlight the need for integrated designs that combine RL with safety layers (e.g., MPC/CBFs), cooperative multi-agent decision-making, and explainable mechanisms to ensure trustworthy deployment. The results motivate the development of standardized evaluation benchmarks and simulation-to-real adaptation strategies to improve robustness and real-world applicability of RL-based lane-changing systems. Autonomous lane-changing is one of the most critical and complex tasks in automated driving. Recent progress in reinforcement learning (RL) has shown strong potential to help autonomous vehicles (AVs) make safe and flexible lane-change decisions in real time under uncertain traffic conditions. In the current studies, there is a lack of a common structure that links RL algorithms, simulation tools, and performance evaluation methods. This paper presents a detailed examination of RL-based lane-changing systems in AVs, tracing their development from early rule-based models to modern learning-based approaches. It introduces a clear classification of lane-changing types—discretionary, mandatory, cooperative, and emergency—and connects each to the most suitable RL methods, including value-based, policy-based, actor–critic, model-based, and hybrid algorithms. Each method is examined for its performance, safety, and computational demands. Furthermore, it reviews major simulation environments, such as SUMO, CARLA, and SMARTS, and summarizes key evaluation measures related to safety, efficiency, comfort, and real-time performance. The comparison shows open research challenges, including model adaptation, safety assurance, and transfer from simulation to real-world driving. Finally, it outlines promising directions for future work, such as cooperative decision-making, safe and explainable RL, and lightweight models for real-time use. This review provides a clear foundation and practical guide for developing reliable and understandable RL-based lane-changing systems for future intelligent transportation.