Although the applications of fuzzy cognitive maps cover a wide variety of scientific and engineering fields, the deep theoretical analysis of their behaviour has a much more weaker literature. In this chapter, first we introduce a more compact description of the updating rule, which makes the further mathematical investigations easier. Then several examples of different qualitative behaviour are shown, highlighting the importance of the strict mathematical discussion of this problem. Finally, the main mathematical factors regarding the behaviour of fuzzy cognitive maps and its extensions will be introduced.