Feature extraction is one of the most challenging tasks in the design of a classification system. In this work we present a novel floating evaluation and search algorithm focused on weak features. In classification problem with a high number of weak features an exhaustive feature selection protocol is calculation cost prohibitive, so in our approach a floating method is proposed with restricted feature subset evaluation. Our proposal considerably decreases the calculation costs of feature search compared with conventional bottom-up, top-down and floating techniques, as well with other recent techniques, without reducing the classification performance. The proposed methodology was tested for 7-class facial expression recognition and the results show the viability of the approach for multiclass problems with weak features.