In 2015, as members and advisors of the ministry of education of Colombia, the authors developed an academic ranking with a multidimensional approach (MIDE), with the objective of measuring the quality performance of higher education institutions. Throughout the course of design and implementation of the model, they had to overcome different challenges, from issues of information and choice of indicators to the actual process of disclosure to stakeholders. In this article, they present five main challenges that they faced and the solutions they adopted for the design and implementation of the ranking.