How to forecast energy demand with machine learning
Energy consumption and energy demand change over time. Monitoring this change over time results in time series that can be leveraged to understand patterns and to predict future behaviors. Microsoft Azure can help accurately predict peak demand for energy products and services to provide a competitive advantage for businesses.
This solution is based on Azure managed services: Azure Stream Analytics, Event Hubs, Azure Machine Learning, Azure SQL Database, Data Factory and Power BI. These services run in a highly available, vetted and compliant environment, allowing you to focus on the solution, rather than the environment in which they run.
Energy demand forecasting with machine learning
Architecture diagram:
Using Azure services as Machine Learning in a solution that forecasts energy demand.
Workflow:
Time series data can be stored in various formats, depending on its source. The data can be stored as files in Azure Data Lake Storage or in table form in Azure Synapse or Azure SQL Database.
Reading:
Azure Machine Learning (ML) can be plugged in and read from such sources. Ingesting time series data into Azure Machine Learning allows automated machine learning (AutoML) to preprocess the data and train and register a model.
The first step within AutoML is the setup and preprocessing of the time series data. In this step, the data provided for training is prepared. The data drives the following expected features and configurations:
- Attribution of missing values
- Vacation and date and time feature engineering
- Delays and gradual windows
- Variable source cross validation
During the training phase, AutoML leverages the previously processed dataset to train, select and explain the best forecasting model.
Model training:
A wide range of Machine Learning models can be used, such as classical forecasting, deep neural networks and regression models.
Model evaluation:
Model evaluation allows AutoML to evaluate the performance of each trained model and allows the best performing model to be selected for implementation.
Explanation:
AutoML provides the ability to provide the explanation of the selected model. This allows you to better understand what features drive the model's results.
The best performing model is registered in Azure Machine Learning with AutoML, making it available for deployment.
Deployment:
The model registered in Azure Machine Learning can be deployed, providing a live connection point that can be exposed for inference.
Deployment can be done using Azure Kubernetes Service, while running a Kubernetes managed cluster where containers are deployed from images that are stored in Azure Container Registry. Alternatively, Azure Container Instances can be used instead of AKS.
Inference:
once the model is implemented, inference of new data can be performed using the available connection point. Near real-time and batch predictions are supported. Inference results can be stored as documents in Azure Data Lake Storage or in table form in Azure Synapse or Azure SQL Database.
Visualization:
Stored model results can be consumed via user interfaces, such as Power BI dashboards or via custom web applications. The results can be written to a storage option in file or tabular format, and then Azure Cognitive Search indexes them appropriately. The model runs as batch inference and stores the results in the appropriate data store.
What did you think? If you have any questions or think we can help your organization, please do not hesitate to contact us. We will be glad to help you😊