When conducting natural communication in addition to perform verbal communication, non-verbal communication such as gestural information should also be understood. By the word "understand" we mean not only the recognition of one action, but also grasp the meaning of the gesture itself. Therefore, in order to understand the meaning of an action, in this paper we propose emotional model along with the gesture recognition technique. First we discuss the gesture recognition method using iPhone camera by applying steady state genetic algorithm, spiking neural network and self organizing map. After that we use the gesture recognition result as an input data for the emotional model.