The monitoring of people in health centers and geriatric homes is performed by rehabilitation professionals who manually evaluate the surveillance cameras for identifying one person's position, and his physical condition. However, this task is tedious and demands the full attention of the rehabilitation staff because patients with neurological conditions need special care or in some cases the 24/7 monitoring. On the other hand, the use of artificial intelligence in the detection of objects and people through images or videos has presented a great performance. This article presents a methodology based on deep learning for the detection and monitoring of people in closed and open environments using video. The proposed method is non-invasive, low-cost, and evaluates the physical activity and inactivity of people in real-time. Preliminary results in public databases present outstanding results in the monitoring and estimation of caloric expenditure in people in indoor and outdoor spaces.