This paper presents a technique for the classification of images and recognition of vowels and numbers from 0 to 5 of Colombian Sign Language (CSL). This work contains six stages: Data set construction, pre-processing, feature extraction, sampling, classification and reporting result. The classification stage is done by using Support Vector Machines (SVM) with Kernel RBF and K-Nearest Neighbor (KNN), after applying cross-validation of 5-folds and the data is divided with different percentages of training set and test set. With the dataset and the sampling of 80%-20% the best results were for SVM with precision performance measures, Recall and F1-Score was obtained 70%, 69%, 69% respectively.