🔹 Introduction
“Let me now present and analyze the experimental results of our study, where we
compared the baseline model, LightGCN, with our proposed enhancement, UltraGCN.
Our goal here was not just to improve accuracy, but to do so while maintaining or even
reducing computational cost. We evaluated both models across four real-world datasets
using three key metrics: Recall@20, NDCG@20, and Precision@20.”
🔹 Datasets Used
“We selected datasets of varying sparsity and complexity:
● MovieLens-1M, which is relatively dense.
● Gowalla and Yelp2018, representing social interaction networks.
● Amazon-Book, a highly sparse product recommendation dataset.
This variety allowed us to test generalizability across different domains and data densities.”
🔹 LightGCN Results — Baseline
“First, let’s look at the results from LightGCN.”
“We observe that MovieLens-1M shows the best performance across all three metrics. This
is expected, since it's the densest dataset with the most reliable user-item interaction
signals.
In contrast, Amazon-Book, being highly sparse, yields the lowest results. LightGCN, like
most graph-based CF models, struggles in such environments where many nodes (users or
items) have few connections.”
🔹 UltraGCN Results — Our Model
“Now, let’s examine the performance of UltraGCN, our improved model.”
“In every case, UltraGCN outperforms LightGCN—significantly.
Let’s break it down:
● On Gowalla, Recall@20 improves from 0.215 to 0.280, a gain of ~30%. That’s a
strong improvement in a sparse environment.
● On Amazon-Book, despite being the most challenging dataset, we see gains in
Recall, NDCG, and Precision, each increasing by about 20–30%.
● On MovieLens-1M, the improvement is more modest but still meaningful:
○ Recall improves by 9.6%,
○ NDCG by 6.4%, and
○ Precision by 13%.
These results are remarkable considering that UltraGCN also reduces computation thanks
to its use of precomputed similarity matrices instead of multi-layer message passing.”
🔹 Key Observations
“What do these numbers tell us?”
1. UltraGCN generalizes better across datasets.
It adapts well to both dense and sparse graphs by directly embedding high-order similarity
through constraints rather than relying on deep message passing.
2. Precision gains reflect better ranking quality.
Precision@20 increasing from 0.150 to 0.170 on MovieLens-1M means not just more
relevant items, but a higher density of relevance in the top positions — a key goal in practical
recommender systems.
3. Gains are larger in sparse datasets.
This is crucial: in real-world applications, most data is sparse. UltraGCN shines in these
conditions where traditional methods fail — for instance, Recall@20 on Amazon-Book jumps
from 0.190 to 0.230.
4. NDCG gains mean better user satisfaction.
Higher NDCG values show that UltraGCN is not only recommending relevant items but also
placing them higher in the ranking — improving user experience by pushing what matters to
the top.
🔹 Why UltraGCN Performs Better
“Why do we see such consistent improvements?”
There are three major reasons:
1. Constraint-based learning lets the model simulate infinite-depth aggregation
without costly propagation, allowing it to capture long-range relations efficiently.
2. Weighted negative sampling avoids the issue of learning from uninformative
negatives, which improves learning dynamics — especially in sparse data.
3. Regularization and self-loop constraints help preserve uniqueness in
embeddings, mitigating the over-smoothing problem that LightGCN suffers from
when using too many layers.
🔹 Conclusion of Results
“To conclude this section:”
● UltraGCN significantly outperforms LightGCN in Recall, NDCG, and Precision
across all datasets.
● The improvement is most notable on sparse datasets, addressing a key challenge
in collaborative filtering.
● These results confirm that our proposed improvements—particularly
constraint-based loss, similarity precomputation, and better sampling
strategies—lead to faster and more accurate recommendation systems.