Smart building controls learn occupant behavior to reduce energy use
Overview of the RL-based controller © Amirreza Heidari, 2022 EPFL
Occupant behavior, a highly stochastic and complex phenomenon, is unique in each building and remains a challenge for energy-efficient operation of building systems. Current building controls, which rely on the hard-coded expert knowledge, either ignore or over-simplify the expert knowledge and follow an energy-intensive approach to ensure occupant comfort. Recent study by Amirreza Heidari, a PhD student at ICE co-supervised by Prof. François Maréchal from IPESE, indicates that Reinforcement Learning, an aritificial intelligence method, can enable building controls to learn and adapt to the occupant behavior and provide a significant energy reduction while preserving comfort and health of people.
Current control systems, called expert-based controls, rely on the hard-coded knowledge of the experts. They effectively deal with problems that can be mathematically described and programmed to the controller by domain experts, such as heat loss from building envelopes, but they are limited to expert knowledge. Occupant behavior is a highly stochastic and complex phenomenon and is unique in each building, which makes it hard to deal with for the experts. In programming conventional expert-based controls, occupant behavior is either ignored or over-simplified, which has resulted in a gap between what is provided by building systems and what is actually needed by occupants. To bridge this gap, the recent study of Amirreza Heidari investigates the learning-based building controls that can perceive, learn, and adapt to the occupant's behavior to save energy. Reinforcement Learning is used to provide the ability to learn and adaptiveness to the controller. The learning-based controls can autonomously learn the control policy, taking into account the unique occupant behavior in each specific building, without any prior knowledge by the experts.
The current control of water heating systems in residential buildings is totally detached from occupant behavior and follows a conservative and energy-intensive operation. Given the significant share of water heating in the energy use of buildings, and the limited research on integrating occupant behavior in hot water systems control, the first part of this study proposes a novel occupant-centric control framework for heat pump water heating systems. This framework learns the hot water use behavior of occupants and accordingly schedules the heating cycles of the hot water tank to save energy without violating the occupant comfort and hygiene aspects. The second part further extends the framework by proposing an occupant-solar-centric control framework for a solar-assisted hot water and space heating system. This framework learns the hot water use behavior of occupants, as well as fluctuating solar energy production, and accordingly adjusts the indoor air temperature and schedules the heating cycles of the tank to save energy without violating occupant comfort and hygiene aspects. Real-world weather conditions and hot water use behavior of occupants were monitored in 3 residential case studies in Switzerland and used to evaluate the performance of the proposed frameworks. Results of this study indicate the significant energy reduction potential of integrating occupant behavior into building controls. Results are published as two papers in the Applied Energy journal.