This paper presents a system to detect the rust disease in sugarcane crops using deep-learning neural networks for visible range images. Three deep-learning neural networks are addressed: one formed by adding different layers in a convolutional neural network from scratch, and the remaining two using learning transfer for two pre-trained models: Inception V3 and InceptionResNet V2 network. These networks are evaluated under different strategies to deal with the data overfitting problem. Finally, adversary attacks were analyzed to identify the robustness of the proposed models.