Unsupervised learning methods play an essential role in many deep learning approaches because the training of complex models with several parameters is an extremely data-hungry process. The execution of such a training process in a fully supervised manner requires numerous labeled examples. Since the labeling of the training samples is very time-consuming, learning approaches that require less or no labeled examples are sought. Unsupervised learning can be used to extract meaningful information on the structure and hierarchies in the data, relying only on the data samples without any ground truth provided. The extracted knowledge representation can be used as a basis for a deep model that requires less labeled examples, as it already has a good understanding of the hidden nature of the data and should be only fine-tuned for the specific task. The trend for deep learning applications most likely leads to substituting as much portion of supervised learning methods with unsupervised learning as possible. Regarding this consideration, our survey aims to give a brief description of the unsupervised clustering methods that can be leveraged in case of deep learning applications.