The prediction of terrain elevation values is a key task when it comes to off-road dynamics and inertial data estimation. A reliable elevation map can help in the estimation of future vehicle states and thus extend the response time window for autonomous navigation and control. We trained a deep learning model that is able to successfully predict top-down terrain depth maps in an off-road setting using a lightweight monocular depth estimation network. The labels were generated using a custom preprocessing algorithm to aid single image depth model training. Unlike other elevation estimation algorithms, our work can predict terrain variation from a higher camera setting without the use of a multi-sensor system. The network is also shown to work outside of the training data domain. The code will be available at https://www.github.com/norbertmarko/terrain-depth.