Fuzzy cognitive maps (FCMs) are recurrent neural networks applied for modelling complex systems and structures. In this method, the system is represented by a weighted, directed digraph, where the nodes of the network represent the main characteristics of the modelled system, while the weighted and directed edges correspond to the direction and strength of causal relationships between these factors. The FCM based decision making is based on the so-called activation values of the nodes, which represents the state of the system. These activation values are determined by an iteration, which may lead to an equilibrium point (fixed point), but limit cycles or chaotic behaviour may also occur. In this paper, the dynamics of fuzzy cognitive maps with hyperbolic tangent threshold function is mathematically discussed. Theoretical conditions are provided for the existence and uniqueness of fixed points. Moreover, the stability of fixed points and their basins of attractions are also analysed. The results presented here give insight into the special symmetric nature of hyperbolic FCMs.