This article proposes STEP, a novel object-based similarity matrix, for assessing both geometric and thematic accuracies of remote-sensing image classification. In contrast to the traditional error matrix, STEP uses samples of classified and reference objects rather than counts of pixels. Moreover, STEP provides four (4) similarity metrics for characterization of classified objects compared with reference objects: (i) shape similarity (S); (ii) theme similarity (T); (iii) edge similarity (E); and (iv) position similarity (P). Individual objects' similarity metrics are grouped by thematic class and expressed in the integrated STEP similarity matrix. The proposed approach is illustrated using both a hypothetical classification and a real urban land-cover classification obtained from high spatial resolution orthoimagery. Results show that the STEP indices and matrices are able to express meaningful information about thematic and geometric accuracies of object-based image classifications. It also yields area weighted aggregated-by-class error matrices that allow for calculating overall accuracy metrics.