Differential equations govern the dynamics of many physical systems, including mass–spring and mass–spring–damper systems. We propose a fully connected Gudermannian-activated neural network (FCGNN) trained with a hybrid global–local optimizer: a genetic algorithm for a global search, followed by an active-set method for rapid local refinement. The Gudermannian activation smoothly links trigonometric and hyperbolic behaviors, enabling a single network to capture both oscillatory (underdamped/forced) and exponential (overdamped) responses with improved expressivity over standard activation functions. We also studied the effect of L1 and L2 regularization on generalization. Using canonical vibration benchmarks, Monte Carlo trials quantify the robustness of the initialization and noise. The FCGNN’s predictions closely match the analytical solution and outperform physics-informed neural networks, achieving higher accuracy with compact architectures and reduced training effort. The novelty of the model is in the integration of the gudernmannian activation function with a hybrid GA–ASM optimization and independent regularization analyses, along with the Monte Carlo simulations, which yield an accurate solver for oscillatory systems.