A coupled deep learning approach for coded aperture design and single-pixel measurements classification is proposed. A whole neural network is trained to simultaneously optimize the binary sensing matrix of a single-pixel camera (SPC) and the parameters of a classification network, considering the constraints imposed by the compressive architecture. Then, new single-pixel measurements can be acquired and classified with the learned parameters. This method avoids the reconstruction process while maintaining classification reliability. In particular, two network architectures were proposed, one learns re-projected measurements to the image size, and the other extracts small features directly from the compressive measurements. They were simulated using two image data sets and a test-bed implementation. The first network beats in around 10% the accuracy reached by the state-of-the-art methods. A 2x increase in computing time is achieved with the second proposed net.