This work proposes a hybrid unsupervised and supervised learning method to pre-train models applied in earth observation downstream tasks where only a handful of labels denoting very general semantic concepts are available. We combine a contrastive approach to pre-train models with a pretext task to predict spatially coarse elevation maps, which are commonly available worldwide. The intuition behind this is that there is generally some correlation between elevation and targets in many remote sensing tasks, allowing the model to pre-learn useful representations. We assess the performance of our approach on a segmentation downstream task on labels gathering many possible subclasses (pixel level classification of farmlands vs. other) and an image binary classification task derived from the former on a dataset in the north-east of Colombia. In both cases, we pretrain our models with 39K unlabeled images, fine-tune the downstream task only with 80 labeled images, and test it with 2944 labeled images. Our experiments show that our methods, GLCNet+Elevation for segmentation and SimCLR+Elevation for classification, outperform their counterparts without the elevation pretext task in terms of accuracy and macro-average F1, which supports the notion that including additional information correlated to targets in downstream tasks can lead to improved performance.