Nowadays machine learning and reinforcement learning methods have found a large number of applications in the energy systems domain. Large datasets with historic data on energy consumption, generation and pricing make it possible to create advanced models and algorithms to optimize monitoring, prediction and control tasks.
In the RENergetic project, AI methods are used for forecasting and analyzing energy consumption and generation in the heating and electricity domains. Reinforcement learning is used to optimize the charging process for a group of electric vehicles.
Forecasting of energy generation and consumption is a key feature of the RENergetic platform. Global and domain-specific optimizers use the output of the prediction models to reduce carbon dioxide emissions and increase autarky. In combination with anomaly detection algorithms and interactive dashboards, energy managers get a much better insight into the current and future operation of the energy island.
The RENergetic system uses a hierarchical optimization process with variable time resolution consisting of two levels. The global optimizer plans the distribution of the energy between the different domains (e.g. heating of buildings, electric vehicle charging, etc.) and sets parameters of the devices that connect multiple domains (i.e., devices that generate energy in one domain and consume it from another domain (e.g., a heat pump consuming electricity to heat a building). Based on these decisions domain-specific tools optimize the devices that belong to a particular domain (e.g. an electric vehicle charging station), possibly taking additional domain specific objectives into account. Although global optimization utilizes linear programming methods, the domain-specific optimization problem can be solved by any method, which is most suitable for the specifics of the application area. Thus, RENergetic solution supports the coupling of non-learning approaches with machine learning methods.
Electric vehicle demand response is the domain-specific algorithm that optimizes the charging of a group of electric vehicles. Using the AI technique of reinforcement learning, it intelligently plans charging periods for each charging station following a specific objective (e.g. sum of charging powers should be lower than a limit imposed by the global optimizer) and respecting the preferences of the users (e.g. deadline when charging needs to be finished).
Reinforcement learning exploits historical data to learn from past experiences. This allows a controller to be trained without the need for detailed models of the controlled systems or user behavior. Such an approach is more scalable than creating specific models for every new environment or location.
Reinforcement learning (and other AI) methods often require a lot of computational resources. To address this, we designed new compact state-action representations that scale linearly with system capacity and coordination horizon, as well as computationally efficient linear cost functions.
