The approximation methods of mathematics are widely used in theory and practice for several problems. In the framework of the paper a novel tensor-product based approach for representation of neural networks (NNs) is proposed. The NNs in this case stand for local models based on which a more complex parameter varying model can numerically be reconstructed and reduced using the higher order singular value decomposition (HOSVD). The HOSVD as well as the tensor-product based representation of NNs will be discussed in detail. © 2011 IEEE.