On the Effectiveness of Sampled Softmax Loss for Item Recommendation

ACM transactions on office information systems, Journal Year: 2023, Volume and Issue: 42(4), P. 1 - 26

Published: Dec. 13, 2023

The learning objective plays a fundamental role to build recommender system. Most methods routinely adopt either pointwise (e.g., binary cross-entropy) or pairwise BPR) loss train the model parameters, while rarely pay attention softmax loss, which assumes probabilities of all classes sum up 1, due its computational complexity when scaling large datasets intractability for streaming data where complete item space is not always available. sampled (SSM) emerges as an efficient substitute loss. Its special case, InfoNCE has been widely used in self-supervised and exhibited remarkable performance contrastive learning. Nonetheless, limited recommendation work uses SSM objective. Worse still, none them explores properties thoroughly answers “Does suit recommendation?” “What are conceptual advantages compared with prevalent losses?”, best our knowledge. In this work, we aim at offering better understanding recommendation. Specifically, first theoretically reveal three model-agnostic advantages: (1) mitigating popularity bias, beneficial long-tail recommendation; (2) mining hard negative samples, offers informative gradients optimize parameters; (3) maximizing ranking metric, facilitates top- K performance. However, based on empirical studies, recognize that default choice cosine similarity function limits ability magnitudes representation vectors. As such, combinations models also fall short adjusting matrix factorization) may result poor representations. One step further, provide mathematical proof message passing schemes graph convolution networks can adjust magnitude according node degree, naturally compensates shortcoming SSM. Extensive experiments four benchmark justify analyses, demonstrating superiority Our implementations available both TensorFlow 1 PyTorch. 2

Language: Английский

---

Machine Unlearning: Solutions and Challenges

IEEE Transactions on Emerging Topics in Computational Intelligence, Journal Year: 2024, Volume and Issue: 8(3), P. 2150 - 2168

Published: April 9, 2024

Machine learning models may inadvertently memorize sensitive, unauthorized, or malicious data, posing risks of privacy breaches, security vulnerabilities, and performance degradation. To address these issues, machine unlearning has emerged as a critical technique to selectively remove specific training data points' influence on trained models. This paper provides comprehensive taxonomy analysis the solutions in unlearning. We categorize existing into exact approaches that thoroughly approximate efficiently minimize influence. By comprehensively reviewing solutions, we identify discuss their strengths limitations. Furthermore, propose future directions advance establish it an essential capability for trustworthy adaptive researchers with roadmap open problems, encouraging impactful contributions real-world needs selective removal.

Language: Английский

---

A Comprehensive Survey on Trustworthy Graph Neural Networks: Privacy, Robustness, Fairness, and Explainability

Deleted Journal, Journal Year: 2024, Volume and Issue: unknown

Published: Sept. 27, 2024

Abstract Graph neural networks (GNNs) have made rapid developments in the recent years. Due to their great ability modeling graph-structured data, GNNs are vastly used various applications, including high-stakes scenarios such as financial analysis, traffic predictions, and drug discovery. Despite potential benefiting humans real world, study shows that can leak private information, vulnerable adversarial attacks, inherit magnify societal bias from training data lack interpretability, which risk of causing unintentional harm users society. For example, existing works demonstrate attackers fool give outcome they desire with unnoticeable perturbation on graph. trained social may embed discrimination decision process, strengthening undesirable bias. Consequently, trust-worthy aspects emerging prevent GNN models increase users’ trust GNNs. In this paper, we a comprehensive survey computational privacy, robustness, fairness, explainability. each aspect, taxonomy related methods formulate general frameworks for multiple categories trustworthy We also discuss future research directions aspect connections between these help achieve trustworthiness.

Language: Английский

---

Unleashing the Power of Knowledge Graph for Recommendation via Invariant Learning

Proceedings of the ACM Web Conference 2022, Journal Year: 2024, Volume and Issue: unknown, P. 3745 - 3755

Published: May 8, 2024

Knowledge graph (KG) demonstrates substantial potential for enhancing the performance of recommender systems. Due to its rich semantic content and associations among interactive entities, it can effectively alleviate inherent limitations in collaborative filtering (CF), such as data sparsity or cold-start issues. However, most existing knowledge-aware recommendation models indiscriminately aggregate all information KG, without considering specifically relevant task. Such indiscriminate aggregation could introduce additional noisy knowledge into representation learning, which distort understanding users' genuine preferences, thereby sacrificing quality. In this paper, we principle invariance recommendation, culminating our Graph Invariant Learning (KGIL) framework. It aims discern harness task-relevant connections within KG enhance models. Specifically, employ multiple environment generators simulate diverse KG-environments. Then devise a novel attention learning mechanism user-item interaction graph, aiming learn environment-invariant subgraphs. Leveraging an adversarial optimization strategy, diversity environments, meanwhile, promote invariant across environments. We conduct extensive experiments on three datasets compare KGIL with state-of-the-art methods. The experimental results further demonstrate superiority approach.

Language: Английский

---

FUNU: Boosting Machine Unlearning Efficiency by Filtering Unnecessary Unlearning

Published: April 22, 2025

Language: Английский

---

TAPE: Tailored Posterior Difference for Auditing of Machine Unlearning

Published: April 22, 2025

Language: Английский

---

GraphGuard: Detecting and Counteracting Training Data Misuse in Graph Neural Networks

Published: Jan. 1, 2024

Language: Английский

---

Recommendation Unlearning via Influence Function

ACM Transactions on Recommender Systems, Journal Year: 2024, Volume and Issue: unknown

Published: Oct. 29, 2024

Recommendation unlearning is an emerging task to serve users for erasing unusable data ( e.g., some historical behaviors) from a well-trained recommender model. Existing methods process requests by fully or partially retraining the model after removing data. However, these are impractical due high computation cost of full and highly possible performance damage partial training. In this light, desired recommendation method should obtain similar as in more efficient manner, i.e., achieving complete, harmless unlearning. work, we propose new Influence Function-based Unlearning (IFRU) framework, which efficiently updates without estimating influence on via function . light that recent models use both constructions optimization loss computational graph neighborhood aggregation), IFRU jointly estimates direct spillover pursue complete Furthermore, importance-based pruning algorithm reduce function. applicable mainstream differentiable models. Extensive experiments demonstrate achieves than 250 times acceleration compared retraining-based with comparable retraining. Codes available at https://github.com/baiyimeng/IFRU.

Language: Английский

---

On the Effectiveness of Sampled Softmax Loss for Item Recommendation

arXiv (Cornell University), Journal Year: 2022, Volume and Issue: unknown

Published: Jan. 1, 2022

The learning objective plays a fundamental role to build recommender system. Most methods routinely adopt either pointwise or pairwise loss train the model parameters, while rarely pay attention softmax due its computational complexity when scaling up large datasets intractability for streaming data. sampled (SSM) emerges as an efficient substitute loss. Its special case, InfoNCE loss, has been widely used in self-supervised and exhibited remarkable performance contrastive learning. Nonetheless, limited recommendation work uses SSM objective. Worse still, none of them explores properties thoroughly answers ``Does suit item recommendation?'' ``What are conceptual advantages compared with prevalent losses?'', best our knowledge. In this work, we aim offer better understanding recommendation. Specifically, first theoretically reveal three model-agnostic advantages: (1) mitigating popularity bias; (2) mining hard negative samples; (3) maximizing ranking metric. However, based on empirical studies, recognize that default choice cosine similarity function limits ability magnitudes representation vectors. As such, combinations models also fall short adjusting may result poor representations. One step further, provide mathematical proof message passing schemes graph convolution networks can adjust magnitude according node degree, which naturally compensates shortcoming SSM. Extensive experiments four benchmark justify analyses, demonstrating superiority Our implementations available both TensorFlow PyTorch.

Language: Английский

---

Unlink to Unlearn: Simplifying Edge Unlearning in GNNs

Published: May 12, 2024

As concerns over data privacy intensify, unlearning in Graph Neural Networks (GNNs) has emerged as a prominent research frontier academia. This concept is pivotal enforcing the right to be forgotten, which entails selective removal of specific from trained GNNs upon user request. Our focuses on edge unlearning, process particular relevance real-world applications. Current state-of-the-art approaches like GNNDelete can eliminate influence edges yet suffer over-forgetting, means inadvertently removes excessive information beyond needed, leading significant performance decline for remaining edges. analysis identifies loss functions primary source over-forgetting and also suggests that may redundant effective unlearning. Building these insights, we simplify develop Unlink Unlearn (UtU), novel method facilitates exclusively through unlinking forget graph structure. extensive experiments demonstrate UtU delivers protection par with retrained model while preserving high accuracy downstream tasks, by upholding 97.3% model's capabilities 99.8% its link prediction accuracy. Meanwhile, requires only constant computational demands, underscoring advantage highly lightweight practical solution.

Language: Английский

---