In this paper interpolative and non-interpolative fuzzy rule based machine learning systems are investigated by using simulation results. The investigation focuses mainly on two objectives: to compare the efficiency of the inference techniques combined with different numerical optimization methods for solving machine learning problems and to discover the difference between the properties of systems applying interpolative and non-interpolative inference techniques. © 2010 IEEE.