Abstract In this case study, rubble masonry walls from the Sierra Nevada de Santa Marta in Northern Colombia were investigated. We applied a combined object‐based image analysis (OBIA), photogrammetry, and petrography method to characterize and determine the provenance of the materials used in three targeted terraces of the archaeological site of La Palma. In situ data acquisition included detailed photographic records and petrographic descriptions of a selected section of masonry walls, as well as mapping of local geological units. Photographs were processed using OBIA. The data obtained allowed characterization and systematization of the constructive elements, via parameters that include architectural function within the construction (headers, stretchers, and wedges), roundness, and lithotype. This methodology enabled us to identify the source of the lithotypes from local rock outcrops (metagranodiorite and schist) as well as rock boulders and cobbles from the nearby riverbeds. The results suggest that pre‐Hispanic communities had knowledge of the properties of the geological resources available in the region, which they used to build masonry that has endured more than five centuries. Vertical differences identified in segments of different terrace walls suggest changes in the constructive process and therefore may serve to determine a construction chronology.