This current paper deals with the investigation of the variation of the hardness of the rail steel material in the heat-affected zone (HAZ) of a rail joint made by thermit welding (TW). The tested rail is a normal R260 type rail steel category; its production date is 1977, the rolling mill was Diosgyor (Hungary), rail profile is MÁV 48.5. This rail has not been in railway track before the test. The authors performed hardness tests on the rail head’s surface even after the rail welding, as well. After the welding and hardness tests, the rail joint was cut with +/–200 mm by a rail cutter and transported to the laboratory. Water jet cutting was applied to shape six longitudinal direction slices with five vertical cutting lines from the rail piece’s head. The slices’ length was 400 mm, the width of these slices was approx. 10 mm; the TW rail joint was in the mid-point of the slices. Micro-Vickers (HV10) hardness tests were executed on these slices, in the –150…+150 mm interval lengthwise and in the 3, 6, and 10 mm depth points below the rail head’s top surface. As a result, the authors received a very detailed hardness functions of the HAZ of rail joint made by TW. These variation functions were compared to the official Elektrothermit’s SoW-5 hardness tests’ results. It can be concluded that the variation of the hardness of rail steel in the area of the HAZ correlated with the Elektrothermit’s results; however, there were some critical points where significant differences were able to be found. The highest deviation was concluded in 50 mm distance measured from the axis of welded rail joint. The authors gave possible valuable explanations for these phenomena.