Hand gesture recognition is an area of human-computer interaction that helps to recognize accustomed hand gestures to establish a communication interface between people and computers. sEMG-, camera-, and impedance-based hand gesture solutions are robust orientations but contain drawbacks such as user comfort, light sensitivity, and easy installation, as appropriate. Accelerometer data are a complementary focus for hand gestures and require only to be positioned over the object to provide positional information. This paper performs a hand gesture classification with a deep convolutional neural network from time-frequency domain transformed RGB (Red, Green, Blue) images of the X, Y, and Z accelerometer signals. We propose to compare GoogLeNet and ResNet pre-trained networks using the RGB approach of the accelerometer signals. Our database included 22 people containing 6 gestures with a record of 2000 samples acquired. We have found that the best classification rate was with GoogLeNet, which reached a general accuracy of 91.9% and more than 96% with some specific gestures. Furthermore, we have evidenced that knock-knock gestures possess user singularities that do not allow our classifier to generalize its recognition. For future work, we propose testing our RGB wavelet architecture with smartwatch accelerometer data to test its capacity for wrist gesture recognition.