### Summary of the Paper:
**Title:** A Novel Forecasting Model for Solar Power Generation by a Deep
Learning Framework With Data Preprocessing and Postprocessing
**Authors:** Quoc-Thang Phan, Yuan-Kang Wu, Quoc-Dung Phan, Hsin-Yen
Lo
**Published in:** IEEE Transactions on Industry Applications, 2023
---
### **Key Contributions:**
1. **Proposed Model:**
- A deep learning framework (KPCA-XGB-GRU) combining:
- **Data preprocessing:** Feature engineering (Pearson correlation +
XGBoost) and Kernel PCA (KPCA) for dimensionality reduction.
- **Forecast engine:** Three GRU networks trained on time-of-day clusters
(morning, noon, afternoon).
- **Postprocessing:** Error correction using a fourth-degree polynomial
fitting curve with outlier removal.
2. **Input Data:**
- Historical solar power, irradiance, and NWP data (temperature, irradiance,
rainfall, wind speed, air pressure, humidity).
3. **Evaluation Metrics:**
- Normalized Root Mean Squared Error (NRMSE) and Normalized Mean
Absolute Percent Error (NMAPE).
4. **Results:**
- Outperformed benchmark models (ANN, LSTM, XGBoost, single GRU)
with average NMAPE of 2.90% and NRMSE of 4.95% across 10 PV sites in
Taiwan.
- Demonstrated the importance of combining preprocessing and
postprocessing for accuracy.
5. **Innovations:**
- Time-of-day clustering for GRU training.
- Novel error-correction postprocessing method.
- Comprehensive framework integrating feature selection, KPCA, and GRU.
---
### **Methodology Highlights:**
1. **Preprocessing:**
- **Feature Selection:** Pearson correlation removed weakly correlated
features (e.g., rainfall, air pressure); XGBoost ranked remaining features by
importance.
- **Dimensionality Reduction:** KPCA retained nonlinear relationships, with
the first principal component covering 62.67% of data variance.
2. **GRU Model:**
- Three GRUs trained on time-specific data clusters.
- Hyperparameters: 64 hidden nodes, learning rate 0.002, dropout 0.2.
3. **Postprocessing:**
- Error correction via polynomial fitting of daily forecast errors, excluding
outliers beyond one standard deviation.
---
### **Comparative Analysis:**
- **Proposed Model vs. Benchmarks:**
- **ANN:** Poor performance (NMAPE 4.50%).
- **Single GRU (no preprocessing/postprocessing):** Higher errors (NMAPE
3.40%).
- **GRU with Preprocessing Only:** Improved but still inferior to full
framework.
- **Ablation Study:**
- Preprocessing contributed more to accuracy than postprocessing, but the
combination yielded the best results.
---
### **Strengths:**
- **Robustness:** Validated on 10 PV sites with varying data durations (3
months to 2 years).
- **Comprehensiveness:** Integrated feature engineering, nonlinear
dimensionality reduction, and error correction.
- **Practicality:** Demonstrated effectiveness for one-hour-ahead forecasting,
aiding grid stability.
---
### **Limitations & Future Work:**
- **Data Dependency:** Performance may vary with seasonal changes; future
work could include weather classification.
- **Computational Cost:** GRU training time is longer than simpler models
(e.g., XGBoost).
---
### **Conclusion:**
The paper presents a state-of-the-art solar forecasting model by synergizing
advanced preprocessing, GRU-based time clustering, and innovative
postprocessing. Its superior accuracy and methodological rigor make it a
valuable contribution to renewable energy integration and smart grid
management.
**Potential Applications:**
- Grid operators for load balancing.
- Energy traders for market bidding.
- PV plant maintenance planning.
**Suggested Improvements:**
- Extend testing to more diverse climates/seasons.
- Explore hybrid models (e.g., GRU with attention mechanisms).
---
This summary captures the paper’s essence while highlighting its technical
novelty and practical implications. Let me know if you’d like to dive deeper
into any section!
To forecast PV power generation following the paper **"A Novel Forecasting
Model for Solar Power Generation by a Deep Learning Framework With Data
Preprocessing and Postprocessing"**, you need to meticulously replicate its
methodology. Below is a **step-by-step implementation guide**, including
data requirements, preprocessing, model training, and postprocessing.
---
## **Step-by-Step Implementation Plan**
### **1. Data Requirements**
The paper uses the following input data:
- **Historical PV Power** (kW)
- **Historical Solar Irradiance** (W/m²)
- **Numerical Weather Prediction (NWP) Data**:
- Temperature (°C or K)
- Solar Irradiance (W/m²)
- Rainfall (mm)
- Wind Speed (m/s)
- Air Pressure (Pa)
- Humidity (%)
**Ensure your dataset includes these variables.**
---
### **2. Data Preprocessing**
#### **(A) Normalization**
- Apply **Min-Max Scaling** to all features (range [0,1]):
\[
D_{i,n} = \frac{D_i - \min(D_{\text{train}})}{\max(D_{\text{train}}) - \
min(D_{\text{train}})}
\]
#### **(B) Feature Selection**
1. **Pearson Correlation Analysis**
- Remove features with weak correlation to PV power (e.g., `NWP_rain`,
`NWP_press`).
- Keep: `power`, `rad`, `NWP_rad`, `NWP_temp`, `NWP_WS`, `NWP_hum`.
2. **XGBoost Feature Importance**
- Train XGBoost on normalized data.
- Rank features by **F-score** (Fig. 5 in the paper).
- Remove low-importance features (e.g., `NWP_rain`).
#### **(C) Dimensionality Reduction (KPCA)**
- Apply **Kernel PCA** (Gaussian RBF kernel) to reduce noise and extract
nonlinear features.
- Retain principal components covering >60% variance (first PC in the paper
explains 62.67%).
---
### **3. Forecast Engine: GRU with Time-of-Day Clustering**
#### **(A) Time-Based Data Splitting**
- Split data into **three clusters** based on time:
- **Morning:** 6 AM – 10 AM
- **Noon:** 11 AM – 3 PM
- **Afternoon:** 3 PM – 6 PM
#### **(B) GRU Architecture**
For **each time cluster**, train a separate GRU model with:
- **Input:** Historical PV power (t-1, t-2, ..., t-24) + selected NWP features.
- **Structure:**
- **2 GRU layers** (100 nodes each).
- **Batch size:** 16
- **Epochs:** 100
- **Optimizer:** Adam (learning rate = 0.002)
- **Dropout:** 0.2 (to prevent overfitting)
#### **(C) Training Process**
1. **Input Matrix (X):**
- Shape: `[samples, timesteps, features]`
- Example: If using 24 historical steps, `[Link] = (N, 24, 6)` (6 features after
selection).
2. **Output (Y):**
- Next-hour PV power (`t`).
3. **Loss Function:**
- **Mean Squared Error (MSE)** for regression.
---
### **4. Postprocessing: Error Correction**
#### **(A) Calculate Forecast Errors**
For training data, compute:
\[
\text{error}_t = \frac{y_t - \hat{y}_t}{\text{PV Capacity}}
\]
(where \(y_t\) = actual, \(\hat{y}_t\) = predicted).
#### **(B) Fit a Polynomial Curve**
1. **Daily Error Aggregation**
- Collect errors for each hour (e.g., 6 AM errors across all days).
2. **4th-Degree Polynomial Fitting**
- Fit:
\[
\text{error}(t) = at^4 + bt^3 + ct^2 + dt + e
\]
- Remove **outliers** beyond **1 standard deviation**.
3. **Average Curve**
- Compute mean error per hour (Fig. 12 in the paper).
#### **(C) Apply Correction to Forecasts**
\[
\hat{y}_{\text{corrected}} = \hat{y}_t + \text{avg\_error}(t) \times \text{PV
Capacity}
\]
---
### **5. Evaluation Metrics**
Use the same metrics as the paper:
1. **Normalized Mean Absolute Percentage Error (NMAPE):**
\[
\text{NMAPE} = \frac{1}{n} \sum_{i=1}^{n} \frac{|y_i - \hat{y}_i|}{\
text{Capacity}} \times 100
\]
2. **Normalized Root Mean Squared Error (NRMSE):**
\[
\text{NRMSE} = \sqrt{\frac{1}{n} \sum_{i=1}^{n} \frac{(y_i - \
hat{y}_i)^2}{\text{Capacity}^2}} \times 100
\]
---
### **6. Implementation Code (Python Example)**
```python
import numpy as np
import pandas as pd
from [Link] import MinMaxScaler
from [Link] import KernelPCA
from xgboost import XGBRegressor
from [Link] import Sequential
from [Link] import GRU, Dense
from [Link] import mean_absolute_error, mean_squared_error
# 1. Load & Normalize Data
data = pd.read_csv("pv_data.csv")
scaler = MinMaxScaler()
data_scaled = scaler.fit_transform(data)
# 2. Feature Selection (Pearson + XGBoost)
# (Assume features already selected: ['power', 'rad', 'NWP_rad', 'NWP_temp',
'NWP_WS', 'NWP_hum'])
# 3. KPCA
kpca = KernelPCA(kernel="rbf", n_components=3) # Adjust n_components
X_kpca = kpca.fit_transform(data_scaled)
# 4. Time-Based Data Splitting
def split_by_time(data, time_ranges):
# time_ranges = [(6,10), (11,15), (15,18)]
clusters = []
for start, end in time_ranges:
mask = (data['hour'] >= start) & (data['hour'] <= end)
[Link](data[mask])
return clusters
# 5. GRU Model for Each Cluster
def build_gru(input_shape):
model = Sequential([
GRU(100, return_sequences=True, input_shape=input_shape),
GRU(100),
Dense(1)
])
[Link](optimizer="adam", loss="mse")
return model
# 6. Error Correction
def correct_errors(y_pred, error_curve):
return y_pred + error_curve * capacity
# 7. Evaluate
nmape = mean_absolute_error(y_true, y_pred) / capacity * 100
nrmse = [Link](mean_squared_error(y_true, y_pred)) / capacity * 100
```
---
### **7. Expected Results**
- **NMAPE:** ~2.9% (as in the paper).
- **NRMSE:** ~4.95%.
- **Improvement:** The full model (preprocessing + GRU + postprocessing)
should outperform standalone models (ANN, LSTM, etc.).
---
### **Final Notes**
- **Reproducibility:** Ensure your dataset matches the paper’s structure.
- **Hyperparameter Tuning:** Adjust GRU layers, learning rate, and KPCA
components if needed.
- **Validation:** Use a 70-15-15 split (train-val-test).
Would you like help with **data preparation** or **debugging the code**? Let
me know! 🚀
Key Interpretation for Your Presentation:
"The first 2 KPCA components capture 89% of the variance in our solar power
data, allowing us to reduce dimensionality while preserving nonlinear patterns
critical for accurate forecasting."
Key Insight:
KPCA typically shows more balanced variance distribution across
components because it captures nonlinear relationships.
"KPCA's components show clearer power-output clusters (right) compared to
PCA (left), proving its superior ability to disentangle complex solar generation
patterns."
Why KPCA Wins for Solar Data
Presentation Textbox:
"✓ Nonlinear Relationships: KPCA's kernel trick captures cloud-cover effects
and irradiance saturation
✓ Diurnal Patterns: Better separates morning/afternoon generation regimes
✓ Weather Noise Reduction: Maintains signal despite meteorological
variability"
Final Test MAPE: 3.48%
=== Metrics Comparison ===
Training Test
MAPE 6.9218% 6.8087%
NMAPE 0.9528% 1.0712%
NRMSE 1.0960% 1.2883%
RMSE 0.0110 0.0129
MAE 0.0095 0.0107