This work presents a framework for coffee maturity classification from multispectral image data based on Convolutional Neural Networks (CNNs). The system leverages the use of multispectral image acquisition systems that generate large amounts of data, by taking advantage of the ability of CNNs to extract meaningful patterns from very high-dimensional data. We validated the use of five different popular CNN architectures on the classification of cherry coffee fruits according to their ripening stage. The different models were trained on a training dataset balanced in different ways, which resulted in a top accuracy higher than 98% when applied to the classification of 600 coffee fruits in 5 different stages of ripening. This work has the potential of providing the farmer with a high-quality, optimized, accurate and viable method for classifying coffee fruits. In order to foster future research in this area, the data used in this work, which was acquired with a custom-developed multispectral image acquisition system, have been released.