The Multiple-Input Multiple-Output (MIMO) system can provide improved spectral efficiency and energy performance. However, the computational demand faced by conventional signal recognition techniques has significantly increased due to the growing number of antennas and higher-order modulations. To overcome these challenges, deep learning approaches are adopted as they offer versatility, nonlinear modelling capabilities, and parallel computation efficiency for large-scale MIMO detection. Therefore, a deep network for channel estimation and massive MIMO detection is developed to reduce computational complexity issues. Initially, a channel estimation scheme is developed to enhance the channel capacity of the MIMO system. It correlates the transmitted and received signals using a confusion matrix. The proposed Modified Squid Game Optimizer (MSGO) is employed for channel state estimation. Based on the obtained channel state information, MIMO detection is performed within the communication system. Here, Multiuser Interference Cancellation (MIC)-based iterative sequential detection is initially conducted. Then, massive MIMO detection is performed using the Dilated Adaptive Recurrent Neural Network with Attention Mechanism (DARNN-AM) through learnable parameters. Moreover, to further optimize the detection performance by fine-tuning the attributes of DARNN-AM, the MSGO is utilized. The proposed network performs multi-segment mapping across multiple constellation points with different modulation schemes. The effectiveness of the proposed deep learning-based MIMO detection system is evaluated by comparing it with existing techniques and algorithms to validate its superior performance.