Introduction: We propose a novel approach for the assessment of the similarity of retinal vessel segmentation images that is based on linking the standard performance metrics of a segmentation algorithm, with the actual structural properties of the images through the fractal dimension.Method: We apply our methodology to compare the vascularity extracted by automatic segmentation against manually segmented images.Results: We demonstrate that the strong correlation between the standard metrics and fractal dimension is preserved regardless of the size of the subimages analyzed.Discussion or Conclusion: We show that the fractal dimension is correlated to the segmentation algorithm’s performance and therefore it can be used as a comparison metric.