smart grid research papers download

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 24 July 2024

Optimizing solar power efficiency in smart grids using hybrid machine learning models for accurate energy generation prediction

• Muhammad Shoaib Bhutta   ORCID: orcid.org/0000-0001-5176-3129 1 ,
• Yang Li   ORCID: orcid.org/0009-0006-4043-039X 1 ,
• Muhammad Abubakar   ORCID: orcid.org/0000-0002-7694-8389 2 ,
• Fahad M. Almasoudi   ORCID: orcid.org/0000-0002-8585-7286 3 ,
• Khaled Saleem S. Alatawi   ORCID: orcid.org/0000-0003-2504-9364 3 ,
• Mohammad R. Altimania   ORCID: orcid.org/0000-0002-2566-2356 3 &
• Maged Al-Barashi   ORCID: orcid.org/0000-0001-8907-4310 4  

Scientific Reports volume  14 , Article number:  17101 ( 2024 ) Cite this article

584 Accesses

1 Altmetric

Metrics details

• Electrical and electronic engineering
• Energy grids and networks
• Power stations

The fourth energy revolution is characterized by the incorporation of renewable energy supplies into intelligent networks. As the world is shifting towards cleaner energy sources, there is a need for efficient and reliable methods to predict the output of renewable energy plants. Hybrid machine learning modified models are emerging as a promising solution for energy generation prediction. Renewable energy generation plants, such as solar, biogas, hydropower plants, wind farms, etc. are becoming increasingly popular due to their environmental benefits. However, their output can be highly variable and dependent on weather conditions, making integrating them into the existing energy grid challenging. Smart grids with artificial intelligent systems have the potential to solve this challenge by using real-time data to optimize energy production and distribution. Although by incorporating sensors, analytics, and automation, these grids can manage energy demand and supply more efficiently, reducing carbon emissions, increase energy security, and improve access to electricity in remote areas. However, this research aims to enhance the efficiency of solar power generation systems in a smart grid context using machine learning hybrid models such as Hybrid Convolutional-Recurrence Net (HCRN), Hybrid Convolutional-LSTM Net (HCLN), and Hybrid Convolutional-GRU Net (HCGRN). For this purpose, this study considers various parameters of a solar plant such as power production (MWh), irradiance or plane of array (POA), and performance ratio (PR). The HCLN model demonstrates superior accuracy with the RMSE values of 0.012027 for MWh, 0.013734 for POA and 0.003055 for PR, along with the lowest MAE values of 0.069523 for MWh, 0.082813 for POA, and 0.042815 for PR. The obtained results suggest that the proposed machine learning models can effectively enhance the efficiency of solar power generation systems by accurately predicting the required measurements.

Similar content being viewed by others

An analysis of case studies for advancing photovoltaic power forecasting through multi-scale fusion techniques

Deep learning based optimal energy management for photovoltaic and battery energy storage integrated home micro-grid system

Fault detection and diagnosis of grid-connected photovoltaic systems using energy valley optimizer based lightweight CNN and wavelet transform

Introduction.

Recent advancements in artificial intelligence (AI) and the Internet of Things (IoT) have spurred innovative approaches in various domains. In the field of service recommendation and Quality of Service (QoS) prediction, where traditional methods encounter accuracy limitations, the introduction of a two-stream deep learning model incorporating user and service graphs has demonstrated improved QoS prediction accuracy 1 . Exploring the intersection of deep learning and defense mechanisms in Digital Twins and Deep-Learning-as-a-Service Computing Systems, the study emphasizes the use of deep learning algorithms for data analysis to address security challenges in service computing systems, highlighting the crucial role of Digital Twins in enhancing network defense capabilities 2 . In the domain of pattern recognition, a proposed Attention-Driven Framework for Unsupervised Pedestrian Re-identification aims to address challenges posed by variations in pose, occlusion, and lighting conditions 3 . The study underscores the importance of efficient sampling strategies and introduces a clustering optimization approach to enhance the performance of Convolutional Neural Networks (CNN) in pedestrian re-identification 3 . Exploring the integration of intrusion detection, Deep Learning (DL), and Digital Twins for city network security, a model utilizing Deep Neural Network (DNN) is presented to enhance network security defense systems, incorporating a trust model based on Keyed-Hashing-based Self-Synchronization (KHSS) 2 .

Addressing the challenges of inconsistent file sizes in fog radio access networks (F-RANs), the investigation into joint edge caching and content recommendation proposes a Double Deep Q-Network (DDQN) based distributed edge caching algorithm, demonstrating increased net profit and caching efficiency 4 . In the context of IoT and edge computing, an intelligent model is introduced for supporting edge migration of virtual function chains, focusing on exploiting computational power at the edge and supporting demanding services through features like auto-healing and Quality of Service monitoring 5 .

Recent advances in artificial intelligence (AI) and the Internet of Things (IoT) have led to innovative solutions across various domains. Specifically in aquaculture, the control of water quality in Recirculating Aquaculture Systems (RAS) is critical for the survival and growth of aquatic organisms. To overcome the limitations of conventional methods in water quality prediction, a hybrid deep learning framework is introduced, integrating Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU), and Attention mechanisms to enhance the efficiency and accuracy of water quality prediction 6 . In exploring power systems, particularly in power line outage identification, Graph Convolutional Networks (GCNs) are applied to frame the problem as a graph signal classification challenge. Utilizing spatial and spectral GCN architectures, the study incorporates graph shift operators and frequency filters for effective convolution, successfully classifying abnormal signal patterns and demonstrating efficacy in power line outage identification 7 . Addressing limitations in graph convolutional networks, Automatic Graph Convolutional Networks (AutoGCN) are introduced to capture the entire spectrum of graph signals, autonomously updating filter bandwidth and avoiding the low-pass filter approach. This innovative approach, grounded in graph spectral theory and spatially localized, outperforms baseline methods, providing a more comprehensive understanding of graph-structured data 8 .

So, all the above discussion about the technological landscape necessitates the development of smart grids, which are intricate systems designed to facilitate this integration. Smart grids are complex infrastructures that provide the seamless integration of renewable energy resources into the existing power grid. Hybrid models of machine learning, such as Convolutional Neural Network-Recurrent Neural Network (CNN-RNN), Convolutional Neural Network-Gated Recurrent Unit (CNN-GRU), and Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM), have shown a great deal of promise in the area of anticipating the demand for energy and optimizing the amount of energy that is used by smart grids 9 . In addition, these models could potentially reduce our reliance on fuels and enhance the long-term sustainability of our energy system by maximizing the generation and storage of renewable energy 10 . Machine learning based models have been utilized to forecast the production of energy and address short term electric power demands 11 , 12 , 13 . Additionally advanced machine learning techniques, like the CNN RNN CNN GRU and CNN LSTM have been utilized to forecast traffic patterns 14 , 15 . In the context of smart grids, deep learning can be used to enable demand response by predicting energy demand, and machine learning models may be used to improve the accuracy of these predictions 12 . Furthermore, deep learning has the potential to support demand response by incorporating reinforcement learning. These techniques can also aid in the real time management of the power grid, aiming to boost the share of renewable energy sources in the grid's energy mix while reducing its carbon footprint. The integration and management of renewable energy sources may be improved via the use of machine learning and deep learning models in smart grids. This will ultimately result in energy systems that are more effective and sustainable. For instance, a hybrid CNN-LSTM model has been proposed for the prediction of the price of electricity 16 , a novel hybrid model combining Random Forest and LSTM has been developed for the forecasting of traffic flow 17 , and a hybrid model that combines CNN and Autoregressive Integrated Moving Average (ARIMA) has been developed for the forecasting of the electricity demand 18 .

In addition, a technique that combines CNN and SVR has been suggested to predict wind speed 19 . The hybrid machine learning models that are being used possess the capability to enable the seamless integration of renewable energy sources into smart grids, thereby supporting the global shift towards sustainable energy systems. Researchers have employed a mix of advanced machine learning techniques like CNN RNN, CNN GRU and CNN LSTM to forecast various factors such as electricity prices, traffic patterns, wind power output in the short term, electricity demand fluctuations, wind speeds, the health of wind turbines, long term electricity demand trends, short term load predictions and residential energy usage 16 , 17 , 18 , 19 , 20 , 21 , 22 , 23 , 24 . These models can take into account the data's temporal dependencies as well as their non-linear correlations. For short-term load forecasting, a hybrid CNN model that incorporates residual learning and an attention mechanism has been suggested 23 . Forecasting energy usage has also been accomplished with the assistance of multi-hybrid ensemble learning 25 . These models have the potential to increase the accuracy of forecasts and to assist in the incorporation of renewable energy resources into smart grids. Wind turbine monitoring and malfunction diagnosis is another area in which they may be of use 21 . Current research projects have focused on constructing these hybrid machine-learning models for different applications relating to energy, such as predicting renewable energy, monitoring the state of wind turbines, and forecasting the power demand. For instance, research 21 , 24 , and 26 have all applied CNN-based models to estimate the condition of wind turbines, household energy consumption, and wind speed forecasts, respectively.

In the meanwhile, research 27 , 28 , and 29 have employed hybrid models that combine LSTM with other methodologies to obtain accurate forecasts of power demand and load. These hybrid models have the potential to find applications in smart grids, which are an important aspect of the fourth industrial revolution. This is due to its ability to facilitate the efficient integration of renewable energy sources into the power grid, hence optimizing energy use. A technique based on frequency modulation is used in 30 to evaluate and determine the most effective way to coordinate energy storage devices. An updated gravitational search algorithm is used to construct a dual-stage optimization strategy to optimize the economic operation of energy storage integration 31 . This was done to maximize the effectiveness of the optimization. The authors of reference 32 put out a proposition for a voltage stability index that incorporates a unique single-port equivalent and is based on the distinctive attributes and enduring sensitivity of components. Using methods from machine learning, the authors of 33 examined the operational efficiency of large-scale solar power facilities.

Also, in 34 , Machine learning algorithms perform better than statistical techniques in accurately estimating solar power output. The Coarse Tree model has the lowest Root Mean Square Error (RMSE) when utilized combined with a Maximum Power Point Tracking (MPPT) controller. However, Rational Quadratic Gaussian Process Regression (RQGPR) provides the lowest Root Mean Square Error (RMSE) without involving the application of a Maximum Power Point Tracking (MPPT) controller. Studies in 35 , 36 , 37 , 38 utilizes sophisticated deep learning and machine learning models, including GRU, LSTM, SVM, and SNN, to predict the operating conditions of power grids. This research demonstrates notable improvements by training these models with synthetic data and their real-time practicality. Furthermore, it emphasizes the improved accuracy of a Deep Neural Network (DNN) model compared to Bagged Tree and ARIMA models in predicting solar power generation. The DNN model exhibits the lowest RMSE and MSE values, both with and without the MPPT controller. All of these studies highlight the significance of optimizing energy storage and renewable energy systems in smart grids through the application of sophisticated machine learning models to improve the effectiveness and dependability of these systems, as well as to facilitate the incorporation of renewable energy sources during the fourth industrial revolution. These applications may be made smarter by using more advanced hybrid machine learning models, which are successful in predicting power demand and short-term load forecasting. These models can also be used to make these applications.

In this study, the novelties and the main contribution are discussed as follows:

The hybrid models are carefully designed by altering their structures to incorporate the useful features of both convolutional and recurrent neural networks.

The significant implications of these modifications are demonstrated by the real-time data gathered from a solar plant, which surpasses the prediction capability of individual models.

The study demonstrates a significant improvement in accuracy rates and a notable decrease in error values. The primary achievement of this study is the progress made in predictive modeling for renewable energy systems, representing a significant enhancement in this area. Additionally, the study has refined hybrid machine learning models, specifically HCRN, HCGRN, and HCLN, which were designed to overcome the limitations of traditional models when dealing with complex datasets and intricate patterns. Through the integration of supplementary techniques and the optimization of algorithms, these sophisticated hybrid models exhibit exceptional performance by outperforming simple base models. The study emphasizes the enhanced effectiveness and accuracy attained by these hybrid models, specifically in tasks necessitating complex pattern identification and sequence manipulation. Moreover, the study highlights the significance of utilizing these advanced models as a primary focus in deep learning research and development, specifically for tasks involving sequential data and image processing.

Section " Proposed frame work/methodology " is the description of the proposed framework/methodology that they have developed for analyzing the real-time data parameters acquired from the solar plant. The framework involves creating mathematical models and utilizing customized machine learning algorithms to study the data and recognize patterns and trends. Section " A comparative analysis between developed HCRN, HCGRN, HCLN models, and the basic state of the art machine learning models " is a case study that discusses the data parameters acquired from a solar plant. Various visualization methods are employed to analyze the data, such as box plots, pair plots, heat maps and histograms. The methods mentioned are utilized to spot patterns and tendencies, in the data with the goal of enhancing the efficiency of the facility. In section " Case study ", the results of the analysis of the real-time data parameters acquired from the solar plant using the proposed framework/methodology are discussed in terms of patterns and trends identified in the data and how these can be used to improve the solar plant’s performance. Lastly, section " Results and discussion " provides concluding remarks about the study summarizes the key findings discusses the limitations of the study, and provides suggestions for future research.

Proposed frame work/methodology

Figure  1 shows the overall methodology of this research paper, which depicts the comprehensive framework of the study. A real-time data collected from a solar plant, comprising three essential parameters: power production (MWh), the plane of array (POA), and Performance ratio (PR). To develop the predictive models, 80% of the collected data is utilized for training, while the remaining 20% is employed for testing and validation process. The study employs three hybrid models: Hybrid Convolutional-Recurrence Net (HCRN), Hybrid Convolutional-GRU Net (HCGRN), and Hybrid Convolutional-LSTM Net (HCLN). These models are the modified improved versions of CNN-RNN, CNN-GRU, and CNN-LSTM respectively which is the novelty of this research work. These modifications are made by changing the hyperparameters and layer structures of the basic models to create new models. The data is split into two parts: the predicted output of the parameters and the projected errors RMSE and MAE generated by these modified hybrid models. The results indicate that Hybrid Convolutional-LSTM Net (HCLN) outperformed the other two hybrid models, with lower RMSE and MAE values compared to Hybrid Convolutional-Recurrence Net (HCRN) and Hybrid Convolutional-GRU Net (HCGRN).

A framework for identifying the optimal predictive model for solar plants.

To begin the process, the first dataset is imported, and the data's quality is assessed before proceeding to the validation stage to find the independence of the data. Figure  2 illustrates the validation process, which involves checking whether the imported data is in numerical form and meets the requirements of time series analysis. Reliable and accurate data is crucial, so any data in the form of strings, containing negative numbers or outliers, is filtered out. If the data is confirmed to be accurate, the process moves on to the next phase. However, if the data does not meet the validation criteria, it undergoes another round of filtering, as depicted in Fig.  2 . The flowchart describes the methodology of this research study starting with the importation of the dataset, which is then immediately followed by a stringent validation stage. If the data fails the validation, it goes through a process of recycling; otherwise, it is transferred to the modified codes and continues to its final destination. After the dataset has been verified effectively, it is then split up into several subsets, such as testing and training. After that, the data from these subsets is used as input for the updated models, which include the Hybrid Convolutional-GRU Net (HCGRN), the Hybrid Convolutional-LSTM Net (HCLN), and the Hybrid Convolutional-Recurrence Net (HCRN). These models were developed to improve the system's capacity for accurate prediction. The dataset is run through these models, which then produces findings that may be used for prediction. A thorough comparison of prediction outcomes, taking into account important metrics such as root mean squared error (RMSE) and mean absolute error (MAE) helps to identify the model that displays greater performance. This solid technique guarantees that the study will use the most accurate prediction model possible.

Flowchart illustrating the step, by step process of generating predicted values and visualizing errors.

Convolutional neural networks (CNNs) have proven to be highly effective, in tasks involving the analysis of time series data, such, as signal processing, speech recognition and image interpretation. Convolutional layers, known for their weight matrices serve as the principle of Convolutional Neural Networks (CNNs) and are utilized to execute convolutions on the input data. The data provided includes multiple data points linked with specific timestamps. The models’ layers are crafted to uncover insights and details, from this time sensitive data. The basic setup of a Convolutional Neural Network (CNN) is depicted in Fig.  3 .

Convolution neural network structure.

In the Convolutional Layer, the input sequence is subjected to convolutional operations involving a series of filters, resulting in the generation of feature outputs. By applying a sliding mechanism to the filter throughout the input sequence and calculating the dot product between the filter and the relevant subsequence of the input, the corresponding output can be obtained. The non-linearity of the model is achieved by the application of non-linear activation functions to the outputs of the components within each layer. As a result of this phenomenon, the Convolutional Neural Network (CNN) can effectively acquire and comprehend complex patterns within the dataset. To effectively decrease the dimensionality of spatial data while preserving crucial information, a pooling layer may be used to down sample the outputs of the layer. This can be achieved by the utilization of functions such as max pooling or average pooling. One possible approach to mitigate overfitting is the use of a regularization method known as dropout, which may be implemented on the output of the pooling layer. To promote the acquisition of more intricate features, the technique of dropout is used, whereby a fraction of the activation values are randomly set to zero over the whole of the training phase. Convolutional layers produce a vector, which is subsequently fed into one or more fully connected layers tasked with performing classification or regression processes.

Recurrent Neural Networks (RNNs) are renowned for their ability to proficiently handle time-series and sequential data. The architecture of the RNN cell comprises a succession of interconnected ‘memory cells’ as illustrated in Fig.  4 a 29 . These memory cells exhibit a sequential connection, resembling a chain-like structure. At discrete time intervals, the input undergoes processing by the initial memory cell, leading to alterations in its internal state and the subsequent transmission of information to the subsequent memory cell. The output of each memory cell depends on both its internal state and the input it receives at a given time step. A distinctive feature of recurrent neural networks (RNNs) is their unique property which is the output of a memory cell at each time step is recursively fed into the subsequent memory cell in the sequence. Using this feedback mechanism, the network adeptly stores and retrieves past inputs, facilitating the prediction of future inputs. Equations ( 1 ) and ( 2 ) provide a mathematical representation of RNN internal structure.

( a ) Basic RNN structure 29 . ( b) Basic GRU structure 29 . ( c) Basic LSTM structure 29 .

The hidden state of RNN at time t−1, computed from the input at time t−1 and the previous hidden state at time t−2, is denoted by H t-1 in the given equation. Input at time t is denoted by the vector X t , which may be a visual feature. Both the input at time t and the previous hidden state can have an effect on the current hidden state, which is controlled by the weight matrices P h and P x . Before applying the sigmoid function (σ), which modifies the output based on the input values and the last hidden state. The bias vector B a is multiplied by the products of P h and H t-1 and P x and X t . The sigmoid function is used to determine how much the current input and the previous hidden state matter when calculating the current hidden state. \({P}_{o}\) is also a weight matrix, but it calculates how much the current hidden state contributes to the output at time t. Before applying the tanh activation function, which scales the output to a value between -1 and 1, the bias vector \({B}_{o}\) is added to the product of \({P}_{o}\) and \({H}_{t-1}\) . By using the present concealed state and the matrix of weights \({P}_{o}\) , the output at time t is denoted by \({Y}_{t}\) .

Gated Recurrent Units, commonly referred to as GRUs, stand as a distinct variant of recurrent neural networks, initially developed as a substitute for the long-term memory (LSTM) approach 29 . The internal configuration, as depicted in Fig.  4 b 29 , mirrors that of an LSTM while employing fewer parameters. GRUs, similar to LSTM, include a set of memory cells that are specifically engineered to store information for a duration of time. In contrast to Long Short-Term Memory (LSTM) models, Gated Recurrent Units (GRUs) include a gating mechanism that effectively controls the flow of information inside the memory cells.

Similarly, a GRU encompasses two crucial gates: one is reset gate and the other is update gate. The initial gate, known as reset gate, dictates the degree of retention of prior memory states. Subsequently, the update gate, the second gate, assumes the role of determining the preservation of the previous state and the assimilation of fresh information from the current input. Before calculating these gates, the input at each time step undergoes a series of linear transformations. The collective transformations are responsible for determining the states of the gates. Once they are formed, the update and reset gates work together to alter the state of the current memory cell. The resultant state is formed by a combination of its previous condition and the present input. The resultant output of the Gated Recurrent Unit (GRU) is fundamentally governed by the condition of the memory cell at each sequential time step. This result demonstrates usefulness in both predicting and classificatory tasks. To have a thorough comprehension of the internal mechanisms of the Gated Recurrent Unit, one may consult Eqs.  3 – 6 , shown below:

At time step t, the update gate \({R}_{t}\) determines the proportion (ranging between 0 and 1) of the prior hidden state to retain and update with fresh data. The sigmoid function (σ), denoted as Sigma, compresses inputs into a range of 0 to 1. The weight matrix \({W}_{rh}\) pertains to the previous hidden state \({H}_{t-1}\) influencing the amount by which it should persist. \({H}_{t-1}\) is a vector capturing the preceding GRU output. Another weight matrix \({W}_{rh}\) relates to the current input \({X}_{t}\) , modulating its contribution to updating the hidden state. \({X}_{t}\) is a vector embodying novel information destined for integration into the hidden state.

The internal structure of the Long Short-Term Memory (LSTM), a subtype of recurrent neural network (RNN), is visualized in Fig.  4 c 29 . This design is adept at capturing extensive temporal relationships within sequential data and mitigating the issue of vanishing gradients. The internal configuration of the Long Short-Term Memory (LSTM) network is distinguished by an array of memory cells, each equipped with a specific ensemble of gate mechanisms that oversee the flow of information within the cell. Comprising three distinct types of input, output, and forget gates, the internal structure of the Long Short-Term Memory (LSTM) network leverages linear transformations to process input data at each successive time step. These transformations calculate the activation levels of the gates. The forget gate determines the exclusion of certain information from the prior memory state, while the input gate dictates the assimilation of new data into the current memory state. Concurrently, the output gate regulates the selection of information to be extracted from the current memory state. The Long Short-Term Memory (LSTM) cell's inner state undergoes updates by combining the previous state with fresh information, overseen by the forget and input gates. Subsequently, the output gate decides the pertinent information for transmission to the subsequent cell in the sequence. LSTMs have showcased their efficacy in handling prolonged dependencies across a spectrum of sequential tasks, encompassing natural language processing, speech recognition, and time-series forecasting. Equation  7 – 12 delineates the mathematical representation portraying the internal architecture of the Long Short-Term Memory Unit (LSTM).

At the current time step, denoted as t the components of the Long Short-Term Memory (LSTM) architecture interact to manage information flow. The forget gate, labeled as \({F}_{t}\) , decides the relevance of the previous cell state, considering both the input \({X}_{t}\) and the preceding hidden state \({\text{H}}_{\text{t}-1}\) , determining what to retain or discard. Simultaneously, the "input gate," denoted as \({I}_{t}\) , assesses the extent of alteration that the incoming cell state should undergo, factoring in the variables \({X}_{t}\) and the preceding hidden state \({\text{H}}_{\text{t}-1}\) . The current cell state, \({\text{C}}_{\text{t}}\) , is then updated using the outcome of the forget gate \({F}_{t}\) , the input gate \({I}_{t}\) , and a new potential value, \({\text{C}}^{\prime}_{{\text{t}}}\) , computed from \({X}_{t}\) and \({\text{H}}_{\text{t}-1}\) . This process unfolds against the backdrop of the prior cell state, \({\text{C}}_{\text{t}-1}\) . The candidate value \({{\text{C}}^{^{\prime}}}_{\text{t}}\) is derived by applying the hyperbolic tangent to the linear combination of \({X}_{t}\) and \({\text{H}}_{{{\text{t - }}1}}\) . The "output gate," labeled as \({O}_{t}\) , intervenes to establish the portion of present state of the cell that should be conveyed as updated hidden state, \({H}_{t}\) , factoring in \({X}_{t}\) and the preceding hidden state \({\text{H}}_{\text{t}-1}\) . Ultimately, the current hidden state, \({H}_{t}\) , emerges through the element-wise multiplication of the output gate \({O}_{t}\) and the hyperbolic tangent of the current cell state \({C}_{t}\) . This intricate interplay enables LSTMs to effectively manage and process sequential data.

Hybrid convolutional-GRU net (HCGRN)

A neural network model for sequence data processing is shown in Fig.  5 a. The model has layers for extracting features, modifying sequences, and predicting outputs. Convolutional operations on input data are performed by two one-dimensional convolutional layers, conv1d and conv1d_1. These provide output tensors with forms (None, 3, 256) and (2, 128). After the convolutional layers, max_pooling1d reduces the spatial dimensions of the output. Its tensor shape is (None, 1, 128). A single-dimensional tensor with 128 elements is produced by the flattening layer, which may be fed to subsequent recurrent layers. A repeat vector layer copies this tensor 30 times, creating an output shape of (None, 30, 128) for the recurrent neural network (RNN) layers. The model has three GRU layers: gru, gru_1, and gru_2. Each layer analyses 30-length sequences and contains 100 hidden states. Dropout layers after GRU layers prevent overfitting during training. The model's temporal linkages are improved with a bidirectional layer that handles sequence data forward and backward. Another dropout layer precedes the heavy layers. The "dense" layer has 100 units and is completely connected. Next is "dense_1", a single-unit dense layer that outputs regression tasks. The model has 459,073 trainable parameters and no non-trainable parameters.

( a) Layer structure of hybrid convolutional-GRU Net (HCGRN), ( b) Hybrid convolutional-LSTM Net (HCLN), ( c) Hybrid convolutional-recurrence net (HCRN).

Hybrid convolutional-LSTM net (HCLN)

Figure  5 b shows the architecture of a neural network model for sequential data analysis. The system has layers that extract distinctive features, transform sequences, and predict potential outcomes. Two one-dimensional convolutional layers, conv1d and conv1d_1, convolve the input data. It produces output tensors of dimensions (None, 3, 256) and (None, 2, 128). After that, max_pooling1d decreases the spatial dimensions of the output from the previous layer to (None, 1, 128). To prepare the output for the next recurrent layer, the flattening layer converts it into a one-dimensional tensor of 128 elements. An output shape of (None, 30, 128) is suitable for input into Long Short-Term Memory (LSTM) layers after a repeat vector layer duplicates this tensor 30 times with a single dimension. The model has three LSTM layers, lstm, lstm_1, and lstm_2—that handle 30-length sequences with 100 hidden states. Dropout layers after each LSTM layer prevent model overfitting during training. This model better captures temporal links by handling sequence data in both directions using a bidirectional layer. Another dropout layer precedes the heavy layers. The "dense" layer has 100 units and is completely connected. Next, "dense_1" has one unit and is the regression-specific output layer. The model has 579,129 trainable parameters and 0 non-trainable parameters.

Hybrid convolutional-recurrence net (HCRN)

The layers, output forms, and number of parameters for each layer in a neural network model are listed in Fig.  5 c. The model starts with two one-dimensional convolutional layers, conv1d, and conv1d_1, that extract input data properties. Output tensor sizes are (None, 3, 256) and (None, 2, 128) for these layers. A max-pooling layer, max_pooling1d, reduces the spatial dimensions of the previous layer's output to (None, 1, 128). The flattened layer creates a one-dimensional tensor of 128 elements from the output. A repeat vector layer copies this tensor 30 times with a single dimension, creating an output shape of (None, 30, 128) for the recurrent neural network (RNN) layers. The model uses three Simple RNN layers: simple rnn, simple_rnn_1, and simple_rnn_2. Each layer analyses 30-length sequences and contains 100 hidden states. Dropout layers after each Simple RNN layer stochastically change input units to zero during training to prevent overfitting. A bidirectional layer evaluates sequence data forward and backward to enhance the model's temporal linkages. Before the thick layers, another dropout layer is added. The "dense" layer has 100 units and is completely connected. Next is "dense_1", a single-unit dense layer that outputs regression jobs. The model has 213,957 trainable parameters and no untrainable parameters.

A comparative analysis between developed HCRN, HCGRN, HCLN models, and the basic state of the art machine learning models

A comparative analysis between the developed HCRN, HCGRN, and HCLN models and other state-of-the-art machine learning models reveals notable distinctions in their capabilities and applications. While simple basic models like CNN, RNN, GRU, and LSTM utilize a combination of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) to extract features and handle sequential data, advanced hybrid models such as HCGRN, HCLN, and HCRN further enhance efficiency by incorporating additional methods and refining algorithms. While basic models may excel at simpler tasks, they often face challenges when dealing with complex datasets, whereas sophisticated models demonstrate superior performance, particularly in managing intricate patterns and sequences. However, the increased sophistication of models like HCGRN, HCLN, and HCRN necessitates more significant computational resources and longer training periods. Moreover, while basic models offer simplicity and ease of use, advanced hybrid models provide greater customization possibilities and are well-suited for tasks requiring high accuracy and intricate pattern recognition, representing the forefront of research and development in deep learning for sequential data.

The comparison Table 1 shows some differences between basic hybrid machine learning models like CNN-RNN, CNN-GRU, and CNN-LSTM and more complex ones like HCGRN, HCLN, and HCRN. For feature extraction, simple models use Convolutional Neural Networks (CNN), and for handling sequential data, they use basic Recurrent Neural Networks (RNN), GRU, or LSTM. These simple models are easy to use and offer a clear method, but they may not be able to capture long-range relationships in data and can't handle complex patterns well. Instead, more advanced models use more complex types of RNN, GRU or LSTM, along with maybe some other methods for improving performance. However, advanced models typically require more computational resources, extensive training, and a deeper understanding of advanced deep learning concepts. The choice between these models hinges on the specific requirements of the problem at hand, with simple models sufficing for basic tasks and advanced models excelling in complex and challenging applications.

Data collected for this study is from a large-scale solar plant. Details discussion of the plant is given in subsection " Capacity/Structure of solar plant " and data collection, analysis and visualization are provided in subsection " Data collection process " and " Data analysis and visualization " respectively.

Capacity/Structure of Solar Plant

The current solar plant was developed in different phases and each phase utilizes different equipment. The further details of this solar plant have been presented in Table 2 . The first phase of this project had an installation capacity of 100 MW and was completed in 2015. It consisted of more than 400,000 solar panels from various manufacturers, including JA Solar, Jinko, and Canadian Solar companies. These panels were mounted on fixed-tilt structures. The inverters from Huawei and SMA companies were used to convert the DC power generated by the solar panels into AC power which later can be used by the main grid. These inverters were connected to the solar panels in a centralized architecture which allows better control and monitoring of the overall system. To manage the flow of power from the solar plant to the grid, the project used a substation that contained transformers from ABB company.

The substation also contained other electrical equipment such as switchgear, protection relays, and control systems 26 . The second phase of the project had an installation capacity of 300 MW and was completed in 2017. It used more than 1.1 million solar panels from Trina Solar and Canadian Solar companies, which were mounted on single-axis structures. The third phase of the project was completed in 2019 and had an installed capacity of 600 MW. It used more than 2.4 million solar panels from different manufacturers, including Canadian Solar, Trina Solar, and Jinko, which were also mounted on fixed-tilt structures. The solar power plant was a significant achievement for Pakistan and helped the country to further reduce its dependence on fossil fuels. The scale of the project and size demonstrate the potential for large-scale solar projects in the country and the use of multiple manufacturers and suppliers shows that Pakistan is diversifying its sources of renewable energy. Figure  6 shows the fundamental structure of the solar plant.

Fundamental framework and key components of solar power plants.

Data collection process

To gather data from a solar power plant, a diverse range of sensors and monitoring devices must be used as depicted in Fig.  7 . These sensors are used for the measurement of several factors, including daily power production, array orientation, and performance ratio. Energy meters are used to gather daily data on the quantity of electricity produced at the solar power plant. These meters quantify the aggregate amount of energy generated by the plant. Analyzing the data allows for monitoring the plant's performance and identifying any potential issues or malfunctions. Weather sensors mounted on solar panels can identify the plane of the array, which refers to the angle of the solar panels' positioning. These figures are crucial for optimizing energy production and achieving peak performance at the facility. The performance ratio of the plant may be calculated by dividing the actual energy production by the projected energy output.

Data collection process and monitoring system of solar plant.

This quantifies the plant's operational efficiency. The inverter monitoring devices are used to measure the DC voltage and current output of the solar panels and to collect the data. The plant operators have the capability to optimize the functioning of the solar power plant and ensure that it is generating the maximum quantity of energy.

Data analysis and visualization

Data visualization is a multifaceted procedure that entails comprehending the data, the objective of the representation, the intended viewers, and the accessible tools and design ideas. By taking into account these variables, you may generate impactful and significant visual representations that effectively and properly convey your facts.

The correlation heat maps in Fig.  8 a and the density heat map in Fig.  8 b illustrate the correlation coefficients between three characteristics of a solar plant's performance during a year. These parameters are power output (MWh), plane of array (POA), and performance ratio (PR). The correlation coefficient is a statistical measure of the strength of the relationship between two variables with a range of negative one to positive one. Here negative one indicates a perfect negative correlation, zero indicates no correlation and a positive one indicates a perfect positive correlation. The numbers shown in the heat map indicate the correlation coefficients among these three parameters. A correlation coefficient of positive one in the center of the map represents a perfect positive relation between POA and itself. The color scale on the right side of the heat map displays the range of correlation coefficients. The lowest value of -0.4 indicates a mild negative relation while the maximum value of 1.0 indicates a perfect positive correlation. The heat map observations indicate a moderate positive relation between MWh (megawatt hours) and POA (plane of the array), a weak negative correlation between MWh and PR (performance ratio), and a moderate negative correlation between POA and PR. These correlations may provide valuable insights into the overall efficiency of the solar plant and facilitate the identification of areas that can be improved.

( a) Correlation Heat Map, ( b) Density Heat Map.

A histogram is a useful tool for analyzing the features of a dataset by giving information about its shape, center, and dispersion. It can detect data patterns like skewness or normal distribution, show outliers or uncommon characteristics, and compute statistical measures like mean, median, and standard deviation. In general, a histogram represents the data distribution in a clear and simple manner, making it easier to compare various datasets and acquire insights that may not be immediately evident from raw data. Figure  8 shows the data parameters solar power generation in (MWh), plane of array (POA) and performance ratio (PR) on the x-axis represents range values, divided into a set of bins, and the Y-axis represents the frequency of occurrence for each bin. In Fig.  9 it can be seen clearly that the data used in this study is symmetrical, almost equally distributed, and with a minimum outlier which indicates that the data set is predictable and consistent to draw valid conclusions.

A quantitative analysis and distribution patterns, advanced histograms depicting key data parameters in solar power plant.

Innovation and improvements

In this research, notable advancements and innovations were achieved by elevating the conventional CNN-LSTM, CNN-GRU, and CNN-RNN models to a new echelon through the introduction of hybrid architectures. These enhanced models, specifically the hybrid CNN-LSTM, hybrid CNN-GRU, and hybrid CNN-RNN, were meticulously crafted by modifying their structures to amalgamate the strengths of convolutional and recurrent neural networks. The transformative impact of these modifications was assessed through the extraction of results derived from real-time data obtained from a solar plant. Significantly, the results exceeded the predictive capabilities of individual models, demonstrating a significant increase in accuracy rates and a noticeable decrease in error values. This is a significant achievement. Through the presentation of newly developed and enhanced hybrid models that demonstrate higher performance in forecasting energy output in solar plants, this study represents an important improvement in this field. As a result, it contributes to the development of predictive modeling in renewable energy systems. In the next section " Case study ", graphical results from the modified models are discussed with charts and tables.

Results and discussion

The comparison between actual loss and validation loss for the prediction of the MWh parameter using the Hybrid Convolutional-GRU Net (HCGRN) has been shown in Fig.  10 a. The actual loss which represents the loss on the training dataset and exhibits a consistent downward trend as the number of epochs increases. Starting at 1.9785 it gradually decreases to 0.0841 by the 99th epoch. This declining pattern signifies that the model is proficiently assimilating knowledge from the training dataset, resulting in a reduction of error when forecasting the MWh parameter. On the other hand, the validation loss starts at 2.0746 and varies until it reaches 0.0468 in the last epoch.

( a ) Actual loss vs. Validation loss, ( b ) MAE loss vs. validation MAE loss and ( c ) MSE loss vs. validation MSE loss of MWh through HCGRN.

Figure  10 b is the comparison between actual MAE loss and validation MAE loss for MWh. The actual MAE loss on the training dataset shows a consistent trend of reduction as the number of epochs increases. Starting at 1.3593, it steadily decreases to 0.2139 by the 99th epoch. This shows that the model is efficiently acquiring knowledge from the training dataset, as the MAE loss decreases significantly. In contrast, the validation MAE loss, which assesses the model's performance starts at 1.4264, fluctuates, and ends at 0.1204 in the final epoch. These fluctuations suggest that the model's performance on unseen data varies during training.

Figure  10 c is the actual MSE loss and validation MSE loss of the MWh parameter. The actual MSE loss, representing the mean squared error on the training dataset, steadily decreases as the number of epochs increases. Starting at 1.9785, it consistently reduces to 0.0841 by the 99th epoch. This implies that the model is proficiently acquiring knowledge and refining its predictive capabilities through training data, as evidenced by a substantial reduction in the Mean Squared Error (MSE) loss. Similarly, the validation MSE loss begins at 2.0746, fluctuates and ends at 0.0468 in the final epoch. These fluctuations suggest that the model's performance on unseen data varies during training.

In the context of optimizing the POA parameter prediction through Hybrid Convolutional-GRU Net (HCGRN), it is crucial to examine the dynamics of actual loss and validation loss over epochs. The training process in Fig.  11 a unfolds with notable trends in both losses. Initially, at epoch 0, the actual loss stands at 2.14545989, while the validation loss is slightly higher at 2.456800461, indicating a disparity between training and generalization. However, as training progresses, these losses steadily converge. By epoch 3, there is a substantial drop in both actual loss (0.357102871) and validation loss (0.050259765), signifying improved model performance and its ability to generalize well. Throughout subsequent epochs, the actual loss and validation loss maintain a relatively close relationship, underscoring the robustness of the HCGRN model in maintaining low prediction errors. Notably, both losses exhibit fluctuations but generally remain aligned, indicating that the model doesn't overfit the training data. Towards the end of the training process, at epoch 99, the actual loss is 0.083061673, and the validation loss is 0.056649759, demonstrating a successful optimization of the POA parameter prediction model through HCGRN.

( a ) Actual loss vs. Validation loss, ( b ) MAE loss vs. validation MAE loss and ( c ) MSE Loss vs. validation MSE loss of POA through HCGRN.

In Fig.  11 b, a comparison is shown between the actual MAE loss and the validation MAE loss across numerous epochs. The actual MAE loss, starting at 1.41453886 at epoch 0, it consistently decreased, eventually reaching 0.214511037 at epoch 99. This signifies that the model is becoming more adept at minimizing prediction errors during training. In contrast, a similar pattern was shown by the validation MAE loss, which measures the model's performance on untested data. It started at epoch 0 at 1.550364137 and decreased steadily, ending at epoch 99 at 0.165685818. The fact that the validation and actual MAE losses synchronize shows that the HCGRN model avoided overfitting by successfully extending its predictions to previously unobserved data. Figure  11 c of the Hybrid Convolutional-GRU Net (HCGRN) displays a comparison between the actual Mean Squared Error (MSE) loss and the validation MSE loss for the POA parameter.

At epoch 0, the actual mean squared error (MSE) loss was 2.14545989, whereas the validation MSE loss was 2.456800461. During the training process, it was seen that both losses consistently decreased, indicating that the HCGRN model successfully learned from the data. During epoch 3, there was a notable decrease in both the actual and validation mean squared error (MSE) losses. The actual MSE loss was 0.357102871, while the validation MSE loss was 0.050259765. These data suggest that the model is improving in accuracy. The tendency persisted consistently throughout the training procedure, with the real and validation losses maintaining an accurate alignment. The convergence of these loss curves indicates that the HCGRN effectively optimized the prediction of the POA parameter, showcasing its potential for accurate forecasting in real-world situations.

In Fig.  12 a, analyzing the performance of the Hybrid Convolutional-GRU Net (HCGRN) for predicting the PR parameter, a comparison between the actual loss and the validation loss across various epochs is conducted. The actual loss, which represents the model's error during training, shows a decreasing trend throughout the training process. Starting at 1.129761338 during the initial epoch, it consistently diminished, reaching 0.025812617 at epoch 99. The observed decline in performance suggests that the HCGRN model is effectively acquiring information from the given training data, as shown by its gradual reduction of errors. Likewise, the validation loss, which evaluates the model's performance on data it hasn't seen before, displayed a synchronized pattern. Starting at 0.586367011 in the first epoch, it progressively decreased to 0.01203534 by the 99th epoch. The correlation between the actual and validation loss indicates that the HCGRN model has the capability to make accurate predictions on new data, demonstrating its ability to prevent overfitting on the training dataset.

( a ) Actual loss vs. Validation loss, ( b ) MAE loss vs. Validation MAE loss and ( c ) MSE loss vs. Validation MSE loss of PR through HCGRN.

Figure  12 b depicts a comparison between the empirical Mean Absolute Error (MAE) loss and the validation MAE loss over many epochs. The actual MAE loss represents the error incurred by the model during its training phase, while the validation MAE loss assesses the performance of the model on unseen data. The actual MAE loss started at a relatively high value of 1.034575105 during the initial epoch and exhibited a consistent downward trend. It gradually decreased with each subsequent epoch, ultimately reaching a low value of 0.104480341 at epoch 99. This consistent reduction demonstrates that the HCGRN model is effectively learning and minimizing errors during the training process, indicating its ability to capture patterns in the training data. Simultaneously, the validation MAE loss, which assesses the model's ability to generalize to unknown data, also exhibited a synchronized pattern. The value started at 0.753925204 on the first epoch and gradually declined to 0.085371122 by the 99th epoch. The observed consistency between the mean absolute error (MAE) losses of the real and validation data indicates that the HCGRN model is not affected by overfitting and has the capability to provide accurate predictions for unseen data.

The Mean Squared Error (MSE) loss and the validation MSE loss for the PR parameter in the Hybrid Convolutional-GRU Net (HCGRN) model can be shown in Fig.  12 c. At the beginning of the training process, the mean squared error (MSE) loss is much larger, namely at 1.129761338. This suggests a significant difference between the model's predictions and the actual target values in the training data. Nevertheless, as the training advanced, the mean squared error (MSE) loss consistently dropped, ultimately achieving a much lower value of 0.025812617 by the 99th epoch. The constant decline seen indicates that the HCGRN model successfully reduced the squared errors between its predictions and the actual training data, demonstrating its strong learning ability.

Simultaneously, the validation mean squared error (MSE) loss, which evaluates the model's accuracy on new data, showed a same trend. The value started at 0.586367011 in the first epoch and consistently decreased, finally reaching 0.01203534 by epoch 99. The convergence found in the real and validation Mean Square Error (MSE) losses suggests that the HCGRN model does not suffer from overfitting and has the capacity to make correct predictions for data that was not included in the training set.

Similarly in Fig.  13 a the training of Hybrid Convolutional-LSTM Net (HCLN) model for prediction the MWh parameter, the comparison between actual loss and validation loss across epochs offers significant information into the efficiency and modification capabilities of the model. At the start, at epoch 0, the true loss was 1.95187974, although the validation loss was marginally higher at 2.034267902. This suggests that the model's performance on the training data was not ideal, since there was a notable difference between its predictions and the actual MWh values. As the training progressed, both the actual and validation loss continually reduced. As the model iteratively processed the data, the actual loss consistently decreased to 0.076261148 at the 99th epoch. The significant reduction indicates that the HCLN model successfully enhanced its capacity to reduce discrepancies between its forecasts and the real MWh values during the training phase. Simultaneously, the validation loss exhibited a similar downward trajectory, ultimately reaching a value of 0.044133689 at epoch 99. The convergence of both the actual and validation loss indicates that the HCLN model effectively learnt from the training data and shown excellent generalization skills by reaching low loss values on unseen validation data.

( a ) Actual loss vs. Validation loss, ( b ) MAE loss vs. Validation MAE loss and ( c ) MSE loss vs. Validation MSE loss of MWh through HCLN.

Figure  13 b displays a comparison examination of the Actual Mean Absolute Error (MAE) loss and validation MAE loss throughout epochs, providing vital insights into the model's performance and its ability to generalize to unseen data. Initially, at epoch 0, the observed real loss is 1.350197196, but the validation loss is slightly greater which is equal to 1.412231326. At epoch 99, the actual loss dropped significantly to 0.202950507, demonstrating that the model has greatly improved its capacity to reduce errors when predicting the MWh parameter on the training data. It is important to mention that the validation loss closely followed the real loss throughout the training process. At epoch 99, the validation loss decreased to 0.12600483, which nearly matched the actual loss. The convergence of the real and validation losses indicates that the HCLN model effectively learnt from the training data and exhibited robust generalization capabilities by reaching minimal loss values on unseen validation data.

The comparison of the Mean Squared Error (MSE) loss and the validation MSE loss of the MWh parameter in Fig.  13 c during the training of the Hybrid Convolutional-LSTM Net (HCLN) model provides valuable insights into the model's performance and its ability to generalize. During the initial training phase (epoch 0), the mean squared error (MSE) loss is 1.95187974, whereas the validation MSE loss is 2.034267902. The first loss values indicate that the model is encountering difficulty in accurately capturing the fundamental patterns in the training data, resulting in some significant errors in both the training and validation sets. During the course of the training, as each epoch passed, both the mean squared error (MSE) losses for the real data and the validation data consistently decreased. This trend indicates that the model is learning from the training data and gradually improving its performance in terms of minimizing the mean squared error. By the end of the training process, at epoch 99, the actual MSE loss had decreased significantly to 0.076261148, and the validation MSE loss also reached a low value of 0.044133689. This convergence of the two loss values suggests that the HCLN model learned from the training data and demonstrated strong generalization capabilities by achieving similarly low MSE values on the unseen validation data.

Figure  14 a is the actual loss to the validation loss of the POA parameter during the training of the Hybrid Convolutional-LSTM Net (HCLN). At the beginning of training (epoch 0), the actual loss is 2.147356033, while the validation loss is 2.503918409. These initial loss values indicate that the model has a relatively high error rate on both, training as well as validation datasets, suggesting that the model's performance needs improvement. As training proceeds through subsequent epochs, both the actual and validation losses exhibit a consistent decreasing trend. This graph indicates that the model is learning from the training data, since the error is decreasing. By the end of the training process, at epoch 99, the actual loss had decreased significantly to 0.077837422, and the validation loss had also decreased to 0.065957859. The convergence of the actual and validation loss functions signifies that the model has effectively captured the inherent data patterns, allowing it to produce accurate predictions, not only on the training dataset but also on previously unseen validation data.

( a ) Actual loss vs. Validation loss, ( b ) MAE loss vs. Validation MAE loss and ( c ) MSE loss vs. Validation MSE Loss of POA through HCLN.

In Fig.  14 b, at the beginning of training (epoch 0), the actual MAE loss is 1.414865255, and the validation MAE loss is 1.565488696. These initial losses indicate that the model has a relatively high error rate on both the training and validation datasets. As training starts through subsequent epochs, both the actual and validation MAE losses consistently decrease. By the end of the training process, at epoch 99, the actual MAE loss has significantly decreased to 0.204322264, and the validation MAE loss has also reduced to 0.198790282. The alignment between the actual and validation MAE losses signifies that the model has effectively acquired the inherent data patterns, enabling it to generate accurate predictions, not only within the confines of the training dataset but also when applied to previously unseen validation data.

Figure  14 c is the comparison of the actual Mean Squared Error (MSE) loss versus the validation MSE loss for the POA parameter during the training of the Hybrid Convolutional-LSTM Net (HCLN) gives significant information about the model's accuracy and capacity to generalize. At the beginning of training (epoch 0), the actual MSE loss is 2.147356033, while the validation MSE loss is 2.503918409. These initial losses indicate a relatively high error rate on both the training and validation datasets, suggesting that the model has substantial room for improvement. As the training starts through subsequent epochs, both the actual and validation MSE losses consistently decreased. This pattern shows that the model is improving its performance as a result of being exposed to training data. By the end of the training process, at epoch 99, the actual MSE loss has significantly decreased to 0.077837422, and the validation MSE loss has also reduced to 0.065957859. This convergence of the actual and validation MSE losses highlights the model's successful learning of the underlying patterns in the data and its capacity to make accurate predictions, not only on the training dataset but also on previously unseen validation data.

Figure  15 a is the actual loss versus the validation loss for the PR through training the Hybrid Convolutional-LSTM Net (HCLN). At the outset of training (epoch 0), the actual loss was 1.183701396, and the validation loss is 0.638390422.

( a ) Actual loss vs Validation Loss, ( b ) MAE Loss vs Validation MAE Loss and ( c ) MSE loss vs. Validation MSE Loss of PR through HCLN.

These initial losses indicate that the model's performance on the training dataset is relatively higher compared to the validation dataset. Such a discrepancy is common in the early stages of training, as the model has yet to fully adapt to the data's underlying patterns. As training progresses, the actual loss consistently decreases with each epoch, suggesting that the model is learning from the training data and optimizing its parameters effectively. Meanwhile, the validation loss also shows a similar decreasing trend. This alignment between actual and validation losses indicates that the model is generalizing well to unseen data, which is a positive sign of its robustness. By the end of the training process, at epoch 99, both the actual loss and validation loss had converged to similar values, with the actual loss at 0.026409347 and the validation loss at 0.01381046. This convergence signifies that the model had effectively learned the underlying patterns in the data and can make accurate predictions, not only on the training data but also on unseen validation data.

Similarly, Fig.  15 b is the actual Mean Absolute Error (MAE) loss versus the validation MAE loss for the PR through training the Hybrid Convolutional-LSTM Net (HCLN). At the beginning of training (epoch 0), the actual MAE loss is 1.059277177, while the validation MAE loss is 0.787348688. These initial losses showed that the model performed better on the training data than on the validation data, which is a common occurrence as the model starts learning from the training set. As training proceeds, both the actual MAE loss and validation MAE loss consistently decrease with each epoch. This decrease indicates that the model is efficiently acquiring the fundamental patterns in the data and is improving its performance on both the training and validation datasets. The congruence between the actual and validation losses is an encouraging indication, indicating that the model is effectively extrapolating to unfamiliar data. At epoch 99, the training process reached a point where both the actual MAE loss and validation MAE loss had converged to identical values. The real MAE loss is 0.10242752 and the validation MAE loss is 0.094468474. The convergence demonstrates that the model has effectively understood the essential data patterns, allowing it to make accurate predictions, not just on the training dataset but also on unseen validation data.

Figure  15 c displays the plot of the Mean Squared Error (MSE) loss compared to the validation MSE loss for the parameter PR during the training of the Hybrid Convolutional-LSTM Net (HCLN). During the first stage of training (epoch 0), the mean squared error (MSE) loss is 1.183701396 for the real data, but the validation data has an MSE loss of 0.638390422. The first discrepancy suggests that the model's performance is superior on the training data compared to the validation data, which is a typical occurrence when the model starts to learn from the training set. During the training process, the mean squared error (MSE) loss for both the real data and the validation data continuously decreases with each epoch. The declining pattern indicates that the model is successfully acquiring the fundamental patterns in the data and improving its performance on both the training and validation datasets. The convergence of the real and validation losses indicates that the model is effectively generalizing to unseen data. At epoch 99, the real mean squared error (MSE) loss and the validation MSE loss both reach identical levels, indicating convergence towards the conclusion of the training procedure. The current mean squared error (MSE) loss is 0.026409347, whereas the validation MSE loss is 0.01381046. The convergence showcases the model's proficiency in understanding the underlying data patterns, allowing it to provide accurate predictions not only on the training data but also on the previously unknown validation data.

In Fig.  16 a the evaluation of the MWh parameter through the Hybrid Convolutional-Recurrence Net (HCRN), it is evident that there are significant differences between the Actual Loss and Validation Loss over the course of training. The Actual Loss, which represents the model's performance on the training data, decreases steadily as the number of epochs increases. Commencing at 0.3257 and progressively diminishing to 0.0385, this trend signifies the model's effective learning from the training dataset and performance enhancement. In contrast, the validation Loss, evaluating the model's generalization to unseen data, exhibits a more oscillatory pattern. Initiating at 0.1145, it initially decreases but subsequently displays fluctuations during training. This fluctuation suggests the possibility of overfitting, since the model's consistent performance on the validation data is not sustained. Significantly, there are occurrences when the validation loss experiences sudden increases, namely around epoch 58, reaching a value of 0.1421, which suggests possible problems with the model's ability to generalize. In general, the comparison between Actual Loss and Validation Loss indicates that the model is effectively acquiring knowledge from the training data. However, there is still potential for enhancing its ability to apply this knowledge to unfamiliar data.

( a ) Actual loss vs Validation Loss, ( b ) MAE Loss vs Validation MAE Loss and ( c ) MSE loss vs. Validation MSE Loss of MWh through HCRN.

In the same way, Fig.  16 b presents the mean absolute error (MAE) loss and validation MAE Loss for the MWh parameter estimated by the Hybrid Convolutional-Recurrence Net (HCRN) model. The MAE loss, which measures the average absolute difference between the predicted and actual values on the training data, steadily decreases from 0.3858 to 0.1357 as the number of epochs increases. This indicates that the model is effectively minimizing errors and improving its accuracy on the training data. While the validation MAE loss which evaluates the model's performance on unseen data, also demonstrates a decreasing trend from 0.2800 to 0.2109 but exhibits more fluctuations compared to the MAE Loss. Initially, the validation MAE loss is higher than the MAE loss, indicating that the model struggles with generalization. However, as training progresses, the gap between the two losses narrows, suggesting that the model is learning to predict better over time.

Figure  16 c depicts the comparative analysis between the Mean Squared Error (MSE) Loss and the Validation MSE Loss concerning the MWh parameter determined by the Hybrid Convolutional-Recurrence Net (HCRN) model. The MSE Loss, representing the average squared difference between the predicted and actual values on the training data, decreases progressively from 0.3257 to 0.0385 as the number of epochs increases. This steady decline indicates that the model is effectively minimizing the errors and improving its predictive accuracy on the training dataset. In contrast, the Validation MSE Loss, which assesses the model's performance on unseen data, follows a similar decreasing trend but shows more fluctuation compared to the MSE Loss. Initially, the Validation MSE Loss is higher than the MSE Loss, suggesting that the model struggles with generalization, which is not uncommon during early training epochs. However, as training progresses, the Validation MSE Loss gradually converges toward the MSE Loss, indicating an improvement in the model's ability to generalize. Hence, the decreasing trends in both MSE Loss and Validation MSE Loss signify that the model is learning and enhancing its performance over time. The fluctuations in Validation MSE Loss suggest that there may be some initial overfitting, but the model eventually learns to generalize better.

The comparative analysis of the Mean Squared Error (MSE) Loss and Validation MSE Loss for the MWh parameter during the training of the Hybrid Convolutional-Recurrence Net (HCRN) provides insightful observations into the model's performance and its generalization capability, as shown in Fig.  17 a. At the outset of training, specifically in epoch 0, the actual loss records 1.03837204, while the validation loss is notably lower at 0.049138479. This initial contrast indicates the model's superior performance on the training dataset, a common occurrence as it begins to grasp the training-specific data patterns. With the progression of training, both actual loss and validation loss exhibit consistent reduction in each subsequent epoch. This diminishing trend signifies the model's effective acquisition of underlying data patterns, resulting in enhanced performance on both the training and validation datasets. The convergence of these loss metrics strongly suggests the model's proficiency in generalizing to previously unseen data. By the conclusion of the training process, at epoch 99, the actual loss and validation loss have both reached comparable values of 0.080057055 and 0.076434441, respectively. This convergence underscores the model's successful mastery of underlying data patterns, enabling accurate predictions not only on the training dataset but also on novel validation data.

( a ) Actual loss vs Validation Loss, ( b ) MAE Loss vs Validation MAE Loss and ( c ) MSE loss vs. Validation MSE Loss of POA through HCRN.

Figure  17 b offers a comparative analysis of the Mean Absolute Error (MAE) losses, both actual and validation, pertaining to the MWh parameter during the training of the Hybrid Convolutional-Recurrence Net (HCRN). This illustration provides valuable insights into the model's performance and its ability to generalize to previously unencountered data. At the commencement of training, precisely in epoch 0, the actual MAE loss is recorded at 0.862085044, while the validation MAE loss is notably lower at 0.123599067. The substantial gap observed at this stage strongly implies that the model is exhibiting signs of overfitting to the training data, resulting in a considerably lower MAE for the training dataset compared to the validation dataset. Overfitting transpires when a model excessively molds itself to the training data, impairing its capacity to effectively generalize to novel, unobserved data. Nevertheless, as the training regimen advances, both actual and validation MAE losses consistently ameliorate with the progression of epochs, signifying that the model is progressively assimilating the latent data patterns and improving its predictive prowess. The consistent diminishing of these loss metrics indicates the model's diminishing overfitting tendencies and its increasing capacity for generalization. By the culmination of the training process, occurring at epoch 99, both the actual and validation MAE losses have reached congruent values of 0.211874843 and 0.201278687, respectively. This convergence underlines the model's triumphant acquisition of the underlying data patterns, endowing it with the capability to render precise predictions not only on the training dataset but also on hitherto unexplored validation data.

Figure  17 c elucidates a comparative analysis involving the Mean Squared Error (MSE) losses, encompassing both actual and validation aspects, pertaining to the MWh parameter. This analysis is conducted within the context of training the Hybrid Convolutional-Recurrence Net (HCRN), offering invaluable insights into the model's performance and its adeptness at generalization. Commencing with the initial stages of training, specifically during epoch 0, the actual MSE loss registers a notably higher value, specifically 1.03837204, in contrast to the validation MSE loss, which stands at 0.049138479. This pronounced disparity serves as a significant indicator that the model, during its inception, is prone to overfitting to the training data. This results in a substantially lower MSE when applied to the training dataset as opposed to the validation dataset.

Overfitting is a recurrent issue where the model excessively focuses on the noise and intricacies inherent within the training data, thus impeding its ability to effectively generalize its predictions to unexplored data. Nevertheless, as the training regime progresses, both the actual and validation MSE losses consistently record declines with each passing epoch. This trajectory highlights the model's ongoing process of actively acquiring the underlying data patterns and simultaneously improving its forecasting abilities. The simultaneous reduction in both loss measures indicates that the model is effectively reducing overfitting and improving its ability to adapt. At epoch 99, after completing the training procedure, the actual mean squared error (MSE) loss and validation MSE loss show a convergence towards remarkably similar values. The current mean squared error (MSE) loss is 0.080057055, whereas the validation MSE loss is 0.076434441. The convergence indicates that the model has successfully identified the underlying data patterns and is effective at providing accurate predictions. This ability goes beyond the scope of the training dataset, demonstrating its usefulness in accurately predicting unknown validation data.

Figure  18 a illustrates a comparison of loss metrics, including both actual and validation, especially within the PR parameter domain. This study occurs during the training phase of the Hybrid Convolutional-Recurrence Net (HCRN), offering deep insights into the model's performance and its ability to successfully generalize the new data. Starting from the beginning, both the real loss and validation loss begin at rather high levels. The first loss for the real data is 0.325732052, whereas the validation data begins with a loss of 0.114503279. These early points indicate a clear restriction in the model's ability to provide accurate predictions for the PR parameter. As the training process progresses, both loss indicators consistently show a lower trend. This gradual decline suggests that the model is actively identifying and absorbing the underlying data patterns. Moreover, throughout the whole training process, the real loss and validation loss closely resemble each other. This alignment serves as a strong signal that the model avoids overfitting to the training data, as seen by the agreement between the validation loss and the actual loss. The progressive decrease seen in both loss metrics highlights the model's effectiveness in both acquiring information from the training data and later using this knowledge to make correct predictions when tested on unknown validation data. As the training process nears its end, namely around epoch 99, both the real loss and validation loss converge towards lower values that are equivalent to each other. The loss value reaches 0.038496502, and the validation loss stabilizes at 0.089028001. This convergence demonstrates the model's successful incorporation of the underlying data patterns and its intrinsic capacity to provide accurate predictions for the PR parameter.

( a ) Actual loss vs Validation Loss, ( b ) MAE Loss vs Validation MAE Loss and ( c ) MSE loss vs. Validation MSE Loss of PR through HCRN.

Figure  18 b displays the Mean Absolute Error (MAE) loss for the PR parameter in the training process of the Hybrid Convolutional-Recurrence Net (HCRN) model. This loss includes both the real and validation parts. This comparison provides a deep perspective for understanding the effectiveness of the model and its ability to extrapolate to new data that has not been seen before. At the nascent stage of the training process (epoch 0), the actual MAE loss manifests a relatively higher value, registering at 0.385757178, in contrast to the slightly lower figure of 0.279971421 observed in the validation MAE loss. This initial disparity proffers an indication that the model might be moderately susceptible to overfitting in the initial phases of training, as it exhibits superior performance concerning the training data when juxtaposed with its performance on the concealed validation data. Nonetheless, with the maturation of the training process, both the actual MAE loss and validation MAE loss consistently witness a decline, denoting that the model is in the process of honing its capability to formulate more precise predictions concerning the PR parameter.

Notably, these loss metrics closely shadow each other, delineating the model's congruence in terms of performance on the validation data as compared to its performance on the training data. This alignment emphasizes the idea that the model, instead of being affected by overfitting, is actively focused on understanding and incorporating important patterns in the data. As the training trip nears its completion (at epoch 99), both the real MAE loss and validation MAE loss converge towards similar reduced levels. More precisely, the mean absolute error (MAE) loss reaches a value of 0.13570945, whilst the validation MAE loss stabilizes at 0.210953534. This convergence demonstrates the model's successful incorporation of the underlying data patterns, confirming its ability to provide accurate predictions about the PR parameter.

In Fig.  18 c, the contrast between the genuine Mean Squared Error (MSE) loss and the validation MSE loss, both pertinent to the PR parameter, unfolds an array of salient insights into the performance of the Hybrid Convolutional-Recurrence Net (HCRN) during its training phase, particularly regarding its capacity to generalize effectively. At the onset of the training process, precisely at epoch 0, the actual MSE loss registers a relatively higher value of 0.325732052, while the validation MSE loss concurrently manifests a lower figure of 0.114503279. This initial divergence hints at the possibility of a moderate degree of overfitting, where the model's performance appears notably superior when gauged against the training data in comparison to its performance with respect to the concealed validation data. However, as the training regimen advances, both the actual MSE loss and validation MSE loss consistently exhibit a diminishing trend. This observation denotes the model's ongoing refinement in formulating increasingly accurate predictions concerning the PR parameter. Significantly, this downward trajectory of the losses underscores the model's active learning from the training data and its remarkable proficiency in generalizing its acquired knowledge to the validation data. The harmonization observed between the training and validation losses stands as an encouraging indicator, suggesting that the model is not afflicted by a conspicuous overfitting dilemma. As the training approaches its culmination, precisely at epoch 99, both the actual MSE loss and validation MSE loss converge to lower values. More precisely, the mean squared error (MSE) loss reaches a convergence point of 0.038496502, whereas the validation MSE loss stabilizes at 0.089028001. This convergence clearly demonstrates the model's ability to accurately capture the underlying data patterns, hence confirming its capability to provide accurate predictions about the PR parameter.

Figure  19 presents the findings in a graphical format. The y-axis represents energy values, ranging from 0 to 700 MWh. The x-axis represents the number of days, ranging from 0 to 400. The first 40 days are allocated for test results, shown by the red line on the graph. The blue line represents the MWh estimates made by HCGRN, offering a visual representation of the model's forecasting skills. Similarly, the black line depicts the MWh forecasts obtained via HCLN, demonstrating an alternate method of prediction.

Graphical representation of MWh testing and predictive performance using enhanced HCGRN, HCLN, and HCRN models.

Finally, the pink line represents the MWh estimations produced by HCRN, providing an additional viewpoint on energy forecasts. This graph is a useful tool for evaluating the precision and efficiency of several prediction models over a period of time. The HCGRN, HCLN, and HCRN columns of the dataset are individually analyzed and their statistical measurements are reported in Table 3 . The HCGRN column exhibits a range of data points, with the lowest value estimated to be around 85.26 and the highest value estimated to be about 574.38. The average value for this model is almost equal to 463.71. The column data for HCLN model ranges from a minimum value of roughly 113.08 to a highest value of around 627.37 and has a mean value of approximately 481.71.

The column for HCRN model includes values ranging from around 79.06 to almost 592.90, with a median value of approximately 467.89. These statistical metrics provide an understanding of the characteristics of the dataset giving insights on the data's range and central tendency.

Figure  20 is the graphical representation of POA testing and predictive performance through HCGRN, HCLN, and HCRN Models, the y-axis measures the Plane of Array (POA) ranging from 0 to 8, while the x-axis represents the number of days spanning from 0 to 400. The initial 40 days are dedicated to test results, indicated by the red line. Subsequently, the blue line charts the POA predictions generated by the HCGRN model, the black line illustrates predictions from the HCLN model, and the magenta line displays predictions produced by the HCRN model. This graph provides a clear visual overview of how these various models forecast the Plane of Array values over time, aiding in the assessment and comparison of their predictive capabilities.

Graphical representation of POA testing and predictive performance using enhanced HCGRN, HCLN, and HCRN models.

For each of the three neural network models, the Table 4 provides daily predictions of some parameter (presumably the Point of Arrival, or POA) for a PV system. HCGRN (Hybrid Convolutional-GRU Net): The maximum POA prediction in this model is approximately 7.15, the minimum is around 2.47, and the mean (average) POA prediction is roughly 6.13. HCLN (Hybrid Convolutional-LSTM Net): For this model, the maximum POA prediction is about 7.55, the minimum is around 1.04, and the mean POA prediction is approximately 6.12. HCRN (Hybrid Convolutional-Recurrence Net): In this model, the maximum POA prediction is about 7.41, the minimum is approximately 0.70, and the mean POA prediction is around 6.34. These statistics provide insights into the performance and variability of each neural network architecture in predicting the POA parameter for the PV system over the 404-day period. It seems that HCRN had the lowest minimum forecasts while HCLN had the greatest maximum predictions. For each model, the mean values provide an overall impression of the central tendency of the forecasts. To find out which model performed the best or if their performance differs noticeably depending on the scenario and for this further investigation would be required. Table 4 displays the mean, minimum, and maximum values for each of the three prediction models (HCRN, HCLN, and HCGRN) according to the provided dataset.

Figure  21 depicts a visual representation illustrating the testing and prediction performance of the parameter PR by the models HCGRN, HCLN and HCRN. The Performance Ratio (PR) values range from 60 to 90 on the y axis, while the x axis represents the number of days from 0 to 400. The initial forty days are allocated for test results denoted by the red line. Additionally, PR forecasts derived from the HCGRN model are represented by the blue line, those from the HCLN model by the black line and predictions from the HCRN model by the magenta line.

Graphical representation of PR testing and predictive performance using enhanced HCGRN, HCLN, and HCRN models.

This graph facilitates a direct comparison of performance and prediction accuracy across these various models. Furthermore, it offers a comprehensive visual depiction of how these models forecast Performance Ratio (PR) values over time. Table 5 presents the statistics for these models namely, HCGRN, HCLN, and HCRN. In the Maximum Value column, HCGRN achieved a peak of 85.64155, HCLN reached 87.13593 and HCRN hit 87.01641 as their highest value. Conversely, in the Minimum Value column, HCGRN scored a minimum of 70.31578, HCLN at 61.31177, and HCRN at 61.3588. The Mean Value column indicates the average values over a specific period, approximately 75.836 for HCGRN, around 75.875 for HCLN, and about 75.970 for HCRN.

These data points provide insights into the performance and variability of each model. Table 5 represents the highest, lowest, and average values of the datasets derived from the prediction models.

These results section also provides a tabular form that outlines the computational load of each hybrid machine-learning model. Table 6 presents data on the duration and memory consumption of three different models: HCRN, HCGRN, and HCLN. The HCRN model stands out for its very quick time elapsed of 120.1 s, while also using a comparatively minimal amount of memory resources, namely 0.8 Mbytes. By contrast, the HCGRN model has a little longer processing time of 128.5 s, while also significantly increasing memory use to 0.9 Mbytes. Conversely, the HCLN model has the highest computing demands, taking 189.4 s to complete and using 0.9 Mbytes of memory resources. This table provides a concise summary of the computational expense associated with each hybrid machine-learning model, making it easier to compare and analyze different models based on their computational efficiency.

The prediction performance comparison of the models HCGRN, HCLN, and HCRN across three parameters (MWh, POA, PR) using the error metrics MSE and MAE reveals distinct performance differences. HCLN consistently exceeds the other models, obtaining the lowest MSE values of 0.012027 for MWh, 0.013734 for POA, and 0.003055 for PR, as well as the lowest MAE values of 0.069523 for MWh, 0.082813 for POA, and 0.042815 for PR. HCGRN shows moderate performance, with MSE values of 0.012259 for MWh, 0.014046 for POA, and 0.003170 for PR, and MAE values of 0.082857 for MWh, 0.085446 for POA, and 0.043418 for PR. Furthermore, HCRN generally performs the least efficient, having the highest MSE values of 0.012613 for MWh, 0.014500 for POA, and 0.003287 for PR, as well as the highest MAE values of 0.084669 for MWh, 0.085291 for POA, and 0.043226 for PR.

Upon careful examination and analysis of the results obtained from these models, it has been determined that they possess certain limitations. There are numerous hurdles involved in using these systems for real-time data prediction purposes. Firstly, these models require considerable computational resources, which may restrict their practicality in real-time applications that require prompt decision-making. Moreover, the longer duration of training required for these advanced models might affect their ability to provide real-time predictions, hence impacting their practical applicability. Likewise, the complexity linked to the implementation and deployment of these models presents additional challenges, particularly in situations with limited resources where simplicity and efficiency are of the highest priority. In order to maximize the capabilities of advanced hybrid models for real-time prediction tasks, it is essential to address these issues in the future.

The integration of multiple machine learning models, such as the Hybrid Convolutional-GRU Net (HCGRN), Hybrid Convolutional-LSTM Net (HCLN), and Hybrid Convolutional-Recurrence Net (HCRN), provided a promising prospect for accurate energy production forecasts. The models demonstrated outstanding predictive efficiency via comprehensive assessments of important characteristics of a solar plant, such as electricity output (MWh), plane of array (POA), and performance ratio (PR). The models provided an opportunity to greatly improve the efficiency of solar power generating systems, with an average output of MWh ranging from around 463.71 to 592.90. The examination of POA values further shown their capacity to consistently forecast values within a range of 1.04 to 7.55 (HCLN), 0.70 to 7.41 (HCRN), and 2.47 to 7.15 (HCGRN). The models HCGRN, HCLN, and HCRN showed great potential for smart grids. They were able to achieve high intended PR outcomes, with maximum values of 87.13593, 87.01641, and 85.64155, while still retaining relatively low minimum values of 61.31177, 61.3588, and 70.31578. Therefore, these results demonstrated that HCLN performed better and highlighted the ability to effectively simplify energy demand management and accelerate the shift towards a cleaner and more sustainable energy environment. We will propose future research directions to further and broaden the use of these promising models, using their revolutionary capacities in the field of integrating renewable energy.

Data availability

The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.

Abbreviations

Artificial intelligence

Autoregressive integrated moving average

Automatic graph convolutional networks

Convolutional neural networks

Convolutional neural network-recurrent neural network

Convolutional Neural Network-Gated Recurrent Unit

Convolutional neural network-long short-term memory

Deep learning

Deep neural network

Double deep Q-network

Fog radio access network

Gated Recurrent Unit

Graph convolutional network

Hybrid convolutional-recurrence net

Hybrid convolutional-LSTM net

Hybrid convolutional-GRU net

Internet of Things

Keyed-hashing-based self-synchronization

Long short-term memory

Maximum power point tracking

Mega-watt hour

Plane of array

Performance Ratio

Quality of Service

Root mean square error

Recurrent neural network

Rational quadratic Gaussian process regression

Recirculating aquaculture systems

References:

Zhang, P., Huang, W., Chen, Y. & Zhou, M. Predicting Quality of Services based on a two-stream deep learning model with user and service graphs. IEEE Trans. Serv. Comput. 16 , 4060 (2023).

Article   Google Scholar  

Lv, Z., Chen, D., Cao, B., Song, H. & Lv, H. Secure deep learning in defense in deep-learning-as-a-service computing systems in digital twins. IEEE Trans. Comput . (2021).

Wang, X., Sun, Z., Chehri, A., Jeon, G. & Song, Y. a novel attention-driven framework for unsupervised pedestrian re-identification with clustering optimization. Pattern Recognit. 146 , 110045 (2024).

Yan, J. et al. Double Deep Q-Network based joint edge caching and content recommendation with inconsistent file sizes in fog-RANs. IEEE Trans. Vehicular Technol. (2023).

Tsakanikas, V., Dagiuklas, T., Iqbal, M., Wang, X. & Mumtaz, S. An intelligent model for supporting edge migration for virtual function chains in next generation Internet of Things. Sci. Rep. (2023).

Yang, J. et al. Prediction and control of water quality in Recirculating Aquaculture System based on hybrid neural network. Eng. Appl. Artif. Intell. 121 , 106002 (2023).

He, J. & Cheng, M. Graph convolutional neural networks for power line outage identification. 25th International Conference on Pattern Recognition (ICPR) (2021).

Wu, Z., Pan, S., Long, G., Jiang, J. & Zhang, C. Beyond low-pass filtering: Graph convolutional networks with automatic filtering. IEEE Trans. Knowl. Data Eng. (2022).

Amin, M. R. & Wollenberg, B. F. Toward a smart grid: Power delivery for the 21st century. IEEE Power Energy Mag. 3 (5), 34–41. https://doi.org/10.1109/MPAE.2005.1500134 (2005).

Hamidian, M. & Khorram, E. Smart grid architecture and applications: A review. Renew. Sustain. Energy Rev. 99 , 43–58. https://doi.org/10.1016/j.rser.2018.10.027 (2019).

Patnaik, S. & Singh, V. K. Predicting renewable energy generation: A review of machine learning models. Renew. Sustain. Energy Rev. 107 , 407–438. https://doi.org/10.1016/j.rser.2019.02.011 (2019).

Abubakar, M., Che, Y., Faheem, M., Bhutta, M. S. & Mudasar, A. Q. Intelligent modeling and optimization of solar plant production integration in the smart grid using machine learning models. Adv. Energy Sustain. Res. 5 , 2300160 (2024).

Sun, L., Ren, H., Yang, S. & Zhao, X. Short-term electricity demand forecasting based on machine learning: A review and evaluation. Renew. Sustain. Energy Rev. 81 , 1548–1568. https://doi.org/10.1016/j.rser.2017.06.068 (2018).

Du, Y. & Wang, J. A novel hybrid model of convolutional neural network and recurrent neural network for traffic flow prediction. Expert Syst. Appl. 96 , 246–255. https://doi.org/10.1016/j.eswa.2018.01.013 (2018).

Qi, L. & Zhang, Y. A novel hybrid model combining convolutional neural network and long short-term memory for traffic flow prediction. Neurocomputing 291 , 87–93. https://doi.org/10.1016/j.neucom.2018.02.014 (2018).

Xu, X., Li, Y., Wang, D. & Wang, C. A hybrid CNN-LSTM model for electricity price prediction. Appl. Energy 242 , 728–738. https://doi.org/10.1016/j.apenergy.2019.03.160 (2019).

Article   ADS   CAS   Google Scholar  

Shen, Y., Zhang, Q. & Zhang, X. A novel hybrid model combining random forest and long short-term memory for traffic flow forecasting (2019).

Xu, Z., Sun, Y. & Liu, S. A hybrid model combining convolutional neural network and autoregressive integrated moving average for electricity demand forecasting. Appl. Energy 242 , 673–686. https://doi.org/10.1016/j.apenergy.2019.03.090 (2019).

Tavana, M., Abbasi, A. & Ebrahimzadeh, A. A hybrid CNN-SVR approach for wind speed forecasting. Renew. Energy 116 , 105–116. https://doi.org/10.1016/j.renene.2017.09.011 (2018).

Yin, X. & Wu, J. Short-term wind power forecasting using a novel hybrid model. Appl. Energy 240 , 371–384. https://doi.org/10.1016/j.apenergy.2019.02.089 (2019).

Article   CAS   Google Scholar  

Khosravi, A., Nahavandi, S., Creighton, D. & Swiegers, G. Wind turbine condition monitoring and fault diagnosis using convolutional neural networks and signal processing techniques. Energy 157 , 796–807. https://doi.org/10.1016/j.energy.2018.05.192 (2018).

Zafar, A. et al. Machine learning autoencoder-based parameters prediction for solar power generation systems in smart grid. IET Smart Grid. 7 , 328 (2024).

Xue, Y., Zhang, X., Zeng, Z. & Sun, X. Short-term load forecasting using convolutional neural network with residual learning and attention mechanism. Int. J. Electr. Power Energy Syst. 107 , 172–180. https://doi.org/10.1016/j.ijepes.2018.11.047 (2019).

Zhang, G., Guo, Y. & Huang, B. A novel hybrid model for residential energy consumption prediction using optimized deep neural network and LSSVM. Energy 174 , 361–370. https://doi.org/10.1016/j.energy.2019.02.060 (2019).

Wu, J. & Zhang, W. Energy consumption forecasting with multi-hybrid ensemble learning. Energy 181 , 969–985. https://doi.org/10.1016/j.energy.2019.06.156 (2019).

Liao, H., Zhang, L., Chen, Q. & Sun, Y. A hybrid model combining convolutional neural network and extreme gradient boosting for wind speed forecasting. Renew. Energy 146 , 2476–2488. https://doi.org/10.1016/j.renene.2019.07.044 (2020).

Ren, Z., Wei, Y. & Wu, J. Hybrid model based on CNN and LSTM for predicting residential electricity demand. J. Clean. Prod. 220 , 914–926. https://doi.org/10.1016/j.jclepro.2019.02.132 (2019).

Feng, Y., Huang, B. & Zhou, Z. Short-term load forecasting based on deep residual network and LSTM. Energies 12 (5), 906. https://doi.org/10.3390/en12050906 (2019).

Feng, Y., Huang, B., Zhou, Z. & Li, X. A novel hybrid model based on long short-term memory and extreme learning machine for short-term load forecasting. Appl. Energy 236 , 1333–1350. https://doi.org/10.1016/j.apenergy.2018.12.051 (2019).

Bhutta, M. S. et al. Neuro-fuzzy based high-voltage DC model to optimize frequency stability of an offshore wind farm. Processes 11 (7), 2049 (2023).

Nazir, M. S. et al. Optimized economic operation of energy storage integration using improved gravitational search algorithm and dual stage optimization. J. Energy Storage 50 , 104591 (2022).

Bhutta, M. S. et al. Voltage Stability Index using new single-port equivalent based on component peculiarity and sensitivity persistence. Processes 9 (10), 1849 (2021).

Abubakar, M., Che, Y., Ivascu, L., Almasoudi, F. M. & Jamil, I. Performance analysis of energy production of large-scale solar plants based on artificial intelligence (machine learning) technique. Processes 10 (9), 1843 (2022).

Mukherjee, D., Chakraborty, S., Guchhait, P. K. & Bhunia, J. Machine learning based solar power generation forecasting with and without MPPT controller. In 2020 IEEE 1st International Conference for Convergence in Engineering (ICCE), Kolkata, India 44–48 (2020). https://doi.org/10.1109/ICCE50343.2020.9290685 .

Mukherjee, D., Chakraborty, S., Ghosh, S. & Mishra, R. K. Application of deep learning for power system state forecasting. Int. Trans. Electr. Energy Syst. 31 (9), e12901. https://doi.org/10.1002/2050-7038.12901 (2021).

Mukherjee, D., Chakraborty, S. & Ghosh, S. Power system state forecasting using machine learning techniques. Electr. Eng. 104 (1), 283–305. https://doi.org/10.1007/s00202-021-01328-z (2022).

Mukherjee, D., Chakraborty, S. & Ghosh, S. Deep learning-based multilabel classification for locational detection of false data injection attack in smart grids. Electr. Eng. 104 (1), 259–282. https://doi.org/10.1007/s00202-021-01278-6 (2022).

Mukherjee, D., Chakraborty, S. & Abdelaziz, A. Y. Deep learning assisted prediction for generation of power from solar PV. In 2022 IEEE 10th Region 10 Humanitarian Technology Conference (R10-HTC), Hyderabad, India 382–386 (2022). https://doi.org/10.1109/R10-HTC54060.2022.9929559 .

Akhtar, U. & Mahmood, T. A novel hybrid CNN-SVR model for electricity demand forecasting. Energy 170 , 511–521. https://doi.org/10.1016/j.energy.2019.06.107 (2019).

Download references

Acknowledgements

This work was supported by the Department of Education of Guangxi Autonomous Region under grant number 2023KY0826.

Author information

Authors and affiliations.

School of Automobile Engineering, Guilin University of Aerospace Technology, Guilin, 541004, China

Muhammad Shoaib Bhutta & Yang Li

Key Laboratory of Smart Grid of Ministry of Education, Tianjin University, Tianjin, 300072, China

Muhammad Abubakar

Department of Electrical Engineering, Faculty of Engineering, University of Tabuk, 47913, Tabuk, Saudi Arabia

Fahad M. Almasoudi, Khaled Saleem S. Alatawi & Mohammad R. Altimania

School of Aeronautics and Astronautics, Guilin University of Aerospace Technology, Guilin, 541004, China

Maged Al-Barashi

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, M.S.B. and Y.L.; methodology, M.S.B.; software, M.A.; validation, M.S.B., M.A. and M.R.A.; formal analysis, M.A.B.; investigation, Y.L.; resources, M.S.B.; data curation, M.A.; writing original draft preparation, M.S.B.; writing review and editing, K.S.S.A.; visualization, F.M.A.; supervision, K.S.S.A.; project administration, M.R.A.; funding acquisition, F.M.A. and K.S.S.A.

Corresponding author

Correspondence to Muhammad Shoaib Bhutta .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Bhutta, M.S., Li, Y., Abubakar, M. et al. Optimizing solar power efficiency in smart grids using hybrid machine learning models for accurate energy generation prediction. Sci Rep 14 , 17101 (2024). https://doi.org/10.1038/s41598-024-68030-5

Download citation

Received : 18 March 2024

Accepted : 18 July 2024

Published : 24 July 2024

DOI : https://doi.org/10.1038/s41598-024-68030-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Renewable energy resources
• Hybrid machine learning models
• Smart grids
• Energy generation prediction
• Solar power generation

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

• IEEE Xplore Digital Library
• IEEE Standards
• IEEE Spectrum

• Xplore ">Content in IEEE Xplore
• IEEE Smart Grid Research

Features and Benefits

IEEE Smart Grid Research is a collaborative effort that leverages the work of more than 100 industry experts and pioneers in the field to share a view of the future of smart grid as far as 2030 and beyond. These insights are essential to business planning, research, and reference.

It provides detailed long-term and short-term projections of smart grid industry evolution, technology challenges and opportunities, and areas requiring additional research to help you prepare for what is to come.

• Communications
• Control Systems
• Vehicle Technology
• Long-term vision document
• Reference Model*
• Consumer Awareness
• Cyber Security
• Materials will address projections of where the smart grid will evolve to, technology challenges and opportunities, and areas where additional research is needed.
• Potential use cases, application scenarios, and enabling technologies are included to deliver the most complete picture possible of the next generation of technology in each respective smart grid space.
• The insights gained can shape the future of your organization and help you prepare for what's to come.
• Stay ahead of competitors with access to key information that will help drive your goals for the future.
• Learn more about the latest advancements in the industry right from the experts.

* Please note: Vehicle Technology is the only bundle that does not include a Reference Model document.

How to Subscribe

Contact us for more information, request a custom quote.

• IEEE Electronic Library
• IEEE All-Society Periodicals Package
• IEEE Proceedings Order Plans
• IEEE Conference Proceedings Archive
• IEEE Journals Library Plus
• IEEE Journals Archive
• IEEE Conference Library Plus
• IEEE Communications Library
• IEEE Enterprise
• IEEE Power & Energy Library
• IEEE Standards Online Collections
• IEEE Standards Select
• National Electrical Safety Code ® (NESC ® ) 2023
• IEEE Synchrophasor TSS
• IEEE eLearning Library
• IEEE eLearning Library Course Programs
• IEEE English for Technical Professionals™
• IEEE Resource Centers for Institutions
• IEEE | Rutgers Online Mini-MBA for Engineers
• IEEE Leading Technical Teams
• Intensive Wireless Communications Course Series
• Advanced Topics in Wireless Course Series
• IEEE CertifAIEd™ Assessor Training Program
• Mars Space Colony: A Game of Standardization
• Artech House eBooks Library
• IEEE-Wiley eBooks Library
• Manning eBooks Library
• MIT Press eBooks Library
• Now Publishers Foundations and Trends® Technology eBooks
• Packt eBooks Library
• Princeton University Press eBooks Library
• River Publishers eBooks Library
• SAE eBooks Library
• Wiley Data and Cybersecurity eBooks Library
• Wiley Semiconductors eBooks Library
• Wiley Telecommunications eBooks Library
• 2024 eBook Frontlists
• MIT Press Journals Library: Computing & Engineering Collection
• IBM Journal of Research & Development Archive
• Bell Labs Technical Journal Archive
• CABA Smart Grid Study
• IEEE DiscoveryPoint for Communications™
• National Transportation Data & Analytics Solution
• IEEE 5G/6G Innovation Testbed™
• IEEE DataPort ™
• Request a Quote
• Request a Free Organizational Trial
• Request a Subscription for your Organization

If you would like to learn more about the IEEE Xplore Digital Library, please fill out the form below to receive the latest information.

Fields marked with an asterisk are required.

Thank you for your information. We will be contacting you soon.

Please fill out the following information:

See the IEEE Privacy Policy .

Thank you! Your email has been sent.

smart grid Recently Published Documents

Total documents.

• Latest Documents
• Most Cited Documents
• Contributed Authors
• Related Sources
• Related Keywords

A Trust-Based Framework and Deep Learning-Based Attack Detection for Smart Grid Home Area Network

On the scalability of supply cost for demand management in the smart grid., hybrid bypass technique to mitigate leakage current in the grid-tied inverter.

The extensive use of fossil fuel is destroying the balance of nature that could lead to many problems in the forthcoming era. Renewable energy resources are a ray of hope to avoid possible destruction. Smart grid and distributed power generation systems are now mainly built with the help of renewable energy resources. The integration of renewable energy production system with the smart grid and distributed power generation is facing many challenges that include addressing the issue of isolation and power quality. This paper presents a new approach to address the aforementioned issues by proposing a hybrid bypass technique concept to improve the overall performance of the grid-tied inverter in solar power generation. The topology with the proposed technique is presented using traditional H5, oH5 and H6 inverter. Comparison of topologies with literature is carried out to check the feasibility of the method proposed. It is found that the leakage current of all the proposed inverters is 9 mA and total harmonic distortion is almost about 2%. The proposed topology has good efficiency, common mode and differential mode characteristics.

An efficient authenticated key agreement scheme supporting privacy-preservation for smart grid communication

Various demand side management techniques and its role in smart grid–the state of art.

The current lifestyle of humanity relies heavily on energy consumption, thus rendering it an inevitable need. An ever-increasing demand for energy has resulted from the increasing population. Most of this demand is met by the traditional sources that continuously deplete and raise significant environmental issues. The existing power structure of developing nations is aging, unstable, and unfeasible, further prolonging the problem. The existing electricity grid is unstable, vulnerable to blackouts and disruption, has high transmission losses, low quality of power, insufficient electricity supply, and discourages distributed energy sources from being incorporated. Mitigating these problems requires a complete redesign of the system of power distribution. The modernization of the electric grid, i.e., the smart grid, is an emerging combination of different technologies designed to bring about the electrical power grid that is changing dramatically. Demand side management (DSM) allow customers to be more involved in contributors to the power systems to achieve system goals by scheduling their shiftable load. Effective DSM systems require the participation of customers in the system that can be done in a fair system. This paper focuses primarily on techniques of DSM and demand responses (DR), including scheduling approaches and strategies for optimal savings.

A cosh-based smoothing Newton algorithm for the real-time pricing problem in smart grid

Challenges, trends and solutions for communication networks and cyber-security in smart grid..

Abstract: Power grid is one of the most important manifestations of the modern civilization and the engine of it where it is described as a digestive system of the civil life. It is a structure has three main functions: generation, transmission lines, distribution. This concept was appropriate for a century. However, the beginning of the twenty-first century brought dramatic changes on different domains: media, human growth, economic, environmental, political, and technical etc. Smart grid is a sophisticated structure including cyber and physical bodies hence it reinforces the sustainability, the energy management, the capability of integration with microgrids, and exploiting the renewable energy resources. The quantum leap of smart grid is related to the advanced communication networks that deal with the cyber part. Moreover, the communication networks of smart grid offer attractive capabilities such as monitoring, control, and protection at the level of real time. The wireless communication techniques in integration frame are promised solution to compensate the requirements of smart grid designing such as wireless local area networks, worldwide interoperability for microwave access, long term evolution, and narrowband- internet of things. These technologies could provide high capacity, flexibility, low-cost maintenance for smart grid. However, the multi-interfaces in smart grid may exploit by persons or agencies to implement different types of cyber-attacks may lead to dangerous damage. This research paper reviews the up-to-date researches in the field of smart grid to handle the new trends and topics in one frame in order to offer integration vision in this vital section. It concentrates on the section of communication networks the mainstay of smart grid. This paper discusses the challenging and requirements of adopting the wireless communication technologies and delves deeply into literature review to devise and suggest solutions to compensate the impairments efficiently. Moreover, it explores the cyber security that representing the real defiant to implement the concept of smart grid safely.

SMART GRIDS COMO ALTERNATIVA DE DESENVOLVIMENTO URBANO INTELIGENTE E SUSTENTÁVEL EM PARINTINS/AM

Devido à rápida e não planejada urbanização surgem problemas de aclimatação urbana, para os quais alternativas de soluções foram desenvolvidas a partir do conceito de Smart City. Nesta pesquisa, o foco é nos problemas da gestão da produção e da distribuição de energia elétrica em Parintins/AM. Para a problemática energética são utilizados os conceitos de rede elétrica inteligente, Smart Grid, que por meio da tecnologia otimiza os sistemas de produção e distribuição de energia, tornando-os mais eficientes, eficazes e sustentáveis. O objetivo deste trabalho é analisar, com base nas teorias desenvolvidas acerca das Smart Cities e das Smart Grids, quais são os aspectos que devem ser levados em consideração para a gestão inteligente e sustentável da produção e distribuição de energia elétrica em Parintins/AM. O paradigma de pesquisa é o funcionalista, por meio de pesquisa qualitativa, com delineamento descritivo e analítico. Os dados foram levantados por meio de revisão bibliográfica, análise documental e entrevistas semiestruturadas, triangulados e analisados mediante técnica de análise de conteúdo. Os resultados obtidos foram: 1) desenvolvimento científico na área das Smart Cities e Smart Grids, em construção, por meio de estudo empírico no interior do Amazonas; 2) identificação dos fatores impulsionadores e limitantes à implantação de Smart Grids em Parintins/AM.

Regulation Effect of Smart Grid on Green Transformation of Electric Power Enterprises: Based on the Investigation of “Leader” Trap

The 2060 carbon neutral target reflects the long-term equilibrium and stability of production activities and the natural environment. As an important part of Chinese energy structure, the operation and transformation of power enterprises will face higher requirements. Although the rapid development of smart grids provides necessary technical support for power enterprises to build a modern energy system with green power as the core, whether power enterprises can use smart grids to improve their operating performance and environmental performance has yet to be discussed. The differences caused by the heterogeneity of property rights will also have an impact on the green transformation and development of enterprises. This paper selects 25 Chinese power enterprises as the research objects and uses the 2011–2019 enterprise panel data and the data envelopment analysis model to evaluate the operating performance and environmental performance of power enterprises. The results show that the overall fluctuation trend of the total factor productivity index and green total factor productivity index of power enterprises are W-shaped, and technological progress is the main driving force for the improvement of power operating performance and environmental performance; Compared with enterprises with a single power generation method, enterprises with diversified power generation methods performed better in their overall total factor productivity index. After that, text mining and machine learning methods are used to classify the text of the enterprise’s annual report to determine whether the enterprise applies smart grid technology for production and operation activities. Finally, using feasible generalized least squares method (FLGS) and dynamic panel system generalized moment estimation (SYS-GMM) to analyze the impact of smart grid on the operating performance and environmental performance of power enterprises, and the nature of corporate property rights in this process. It is found that smart grids can improve the operating performance and environmental performance of power enterprises; compared with state-owned enterprises, non-state-owned enterprises can achieve better performance in the application of smart grids to improve operating performance and environmental performance. Finally, this study provides corresponding policy recommendations for power enterprises to achieve performance improvement and green transformation development.

Export Citation Format

Share document.

Smart Grid - Open Access Research

Articles, commentaries, journals and more on smart grid.

Take a look at our open access journals covering Smart Grids, read what our authors and editors say on the topic, and browse selected research.

Featured open access journals covering Smart Grids

Explore our two power systems journals,  Protection and Control of Modern Power Systems  and the  Journal of Modern Power Systems and Clean Energy ​​​​​​​. Why not submit your own Smart or Power Grid research to these journals?

• Journal of Modern Power Systems and Clean Energy
• Protection and Control of Modern Power Systems

• Green neighbourhoods in low voltage networks: measuring impact of electric vehicles and photovoltaics on load profiles
• Transactive control: a framework for operating power systems characterized by high penetration of distributed energy resources
• Overview of power electronics technology and applications in power generation transmission and distribution​​​​​​

Read more Smart Grid research  published in the  Journal of Modern Power Systems and Clean Energy . 

• Day-ahead optimal charging/discharging scheduling for electric vehicles in microgrids
• Shifting of research trends in islanding detection method - a comprehensive survey
• New development in relay protection for smart grid ​​​​​​​

Read more Smart Grid research  published in  Protection and Control of Modern Power Systems . 

Exclusive series of Smart Grid commentaries

Read selected commentaries on a broad variety of aspects related to Smart Grid, written by our authors and editors.

Wide area monitoring, protection and control in future smart grid

by Vladimir Terzija and Yutian Liu

Wind power prediction in the smart grid                       

by L. YE and Y. N. Zhao

Integrating high levels of variable renewable energy into electric power systems

by Benjamin Kroposki

From demand response to transactive energy

by Chen-Ching Liu

Find more Smart Grid commentaries here

More Smart Grid research articles

Being a multifaceted topic, research related to Smart Grids is published in a broad range of our open access journals. Find a selection of Smart Grid articles below - all articles are free to read and share.

Submit your Smart Grid manuscript

Are you looking for a journal to submit your own Smart Grid research to? Read our tips on how to find the right journal here: 

Register with us and stay up to date

As a registered user you can:

•  Add article alerts from all SpringerOpen journals

•  Easily manage your article alerts

•  Receive regular news from your preferred subject areas

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

• IEEE Xplore Digital Library
• IEEE Standards
• IEEE Spectrum

Join IEEE Smart Grid for Free!

• White Papers

The Smart Grid describes a next-generation electrical power system that is stypified by the increased use of communications and information technology in the generation, delivery, and consumption of electrical energy worldwide. IEEE Smart Grid hosts a series of white papers on varying aspects of global grid modernization.

Access White Papers

2016 ieee international forum on smart grids for smart cities.