Abstract This work evaluates the performance of a Deep Learning technique for classification of challenging hyperspectral images of the interior of fruits and vegetables when they are combined. Some of these samples have low contrast, similar colour features, and their skins or characteristic shapes are lost when cut to expose their interiors. We implemented a two-dimensional convolutional neural network for this classification task and compared their results against the technique of support vector machines. We randomly selected a group of 13 hyperspectral images from a public database containing information of the interior of 42 fruits and vegetables. Using parts of these 13 selected images, we constructed three artificial hyperspectral images merging these parts differently. We applied the two proposed techniques over the three of them. The comparison of the classification results shows that the two-dimensional convolutional neural network over-performs the support vector machine in all three composite images. The two-dimensional convolutional neural network exceeded 98% classification accuracy in all of them. These results show that the two-dimensional convolutional network benefits from the spatial and spectral data in the images obtaining proper levels of classification even in samples mixed in complex contexts, as it can occur in the food or pharmaceutical industries.