Cellulose-based aerogels hold the potential to become a cost-effective bio-based solution for thermal insulation in buildings. Low thermal conductivities (<0.025 W·m−1·K−1) are achieved through a decrease in gaseous phase contribution, exploiting the Knudsen effect. However, several challenges need to be overcome: production energy demand and cost, moisture sensitivity, flammability, and thermal stability. Herein, a description and discussion of current trends and challenges in cellulose aerogel research for thermal insulation are presented, gathered from studies reported within the last five years. The text is divided into three main sections: (i) an overview of thermal performance of cellulose aerogels, (ii) an identification of challenges and possible solutions for cellulose aerogel thermal insulation, and (iii) a brief description of cellulose/silica aerogels.