Next POI Recommendation for Multi-gate Hybrid Experts Based on Sparse Sharing

Abstract

Abstract: Next POI recommendation refers to predicting the next POI that the user will visit within a certain period of time. User travel patterns are affected by heterogeneous contextual factors, including continuous values (for example, geographic distance, time interval) and discrete values (for example, social status, weekly status). This paper uses a sparse sharing structure to generate expert networks, and uses a multi-gate hybrid expert model to realize parameter sharing between multiple target tasks. This paper models multiple types of user behaviors based on multi-task learning, and proposes a multi-gate hybrid expert model based on sparse sharing. First, the sparse sharing structure is used to generate the corresponding expert network from the basic network for each target task, and the iterative amplitude pruning method is used to select the expert network. Secondly, the multi-gate hybrid expert model uses expert networks as the input layer. Each expert network has a different emphasis on the processing of input information, and uses multiple door networks to share and select task targets. Finally, the proposed algorithm is evaluated based on real data sets. The evaluation of the proposed algorithm based on real data sets verifies the effectiveness and practicability of the method in this chapter, as well as the prospect of applying multi-task learning and sparse structure sharing to the next point of interest recommendation.

Key words: location-based social networks, next POI recommendation, neural networks, multi-task learning, sparse sharing architectures

Kang Laisong, Liu Shifeng, Gong Daqing. Next POI Recommendation for Multi-gate Hybrid Experts Based on Sparse Sharing[J]. Management Review, 2023, 35(3): 148-159.

References

[1] 江若然,张玲玲.社交属性网下基于链路预测及节点度的推荐算法[J].管理评论, 2019,31(2):119-129
[2] 姜祎盼,张文,李健,等.基于异构网络元路径的App推荐算法[J].管理评论, 2020,32(4):160-170
[3] Abhadiomhen S. E., Nzeh R. C., Ganaa E. D., et al. Supervised Shallow Multi-Task Learning:Analysis of Methods[J]. Neural Processing Letters, 2022,54(3):2491-2508
[4] Caruana R. A. Multitask Learning:A Knowledge-Based Source of Inductive Bias[J]. Machine Learning Proceedings, 1993,10(1):41-48
[5] Duong L., Cohn T., Bird S., et al. Low Resource Dependency Parsing:Cross-Lingual Parameter Sharing in a Neural Network Parser[C]. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2:Short Papers), 2015
[6] Misra I., Shrivastava A., Gupta A., et al. Cross-Stitch Networks for Multi-task Learning[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016
[7] Chen L., Chen D., Yang F., et al. A Deep Multi-task Representation Learning Method for Time Series Classification and Retrieval[J]. Information Sciences, 2021,555:17-32
[8] Zhao S., Zhao T., Yang H., et al. STELLAR:Spatial-Temporal Latent Ranking for Successive Point-of-Interest Recommendation[C]. Thirtieth AAAI Conference on Artificial Intelligence, 2016
[9] Xie M., Yin H., Wang H., et al. Learning Graph-Based POI Embedding for Location-Based Recommendation[C]. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 2016
[10] 鲜学丰,陈晓杰,赵朋朋,等.基于上下文感知和个性化度量嵌入的下一个兴趣点推荐[J].计算机工程与科学, 2018,40(4):616-625
[11] 张奥雅,石美惠,申德荣,等.融合区域与朋友影响的下一兴趣点推荐[J/OL].计算机科学与探索:1-11[2021-02-06]. http://kns.cnki.net/kcms/detail/11.5602.tp.20201105.1051.012.html
[12] 柴瑞敏,殷臣,孟祥福,等.基于时空循环神经网络的下一个兴趣点推荐方法[J].智能系统学报, 2021,16(3):407-415
[13] 王立,张谧.基于LSTM的POI个性化推荐框架[J].计算机系统应用, 2018,27(12):56-61
[14] Bansal T., Belanger D., McCallum A. Ask the GRU:Multi-Task Learning for Deep Text Recommendations[C]. Proceedings of the 10th ACM Conference on Recommender Systems, 2016
[15] Caruana R. Multitask Learning[J]. Machine Learning, 1997,28(1):41-75
[16] Gehring J., Auli M., Grangier D., et al. Convolutional Sequence to Sequence Learning[C]. International Conference on Machine Learning, 2017
[17] Li G., Chen Q., Zheng B., et al. Group-Based Recurrent Neural Networks for POI Recommendation[J]. ACM Transactions on Data Science, 2020,1(1):1-18
[18] Burges C., Shaked T., Renshaw E., et al. Learning to Rank Using Gradient Descent[C]. Proceedings of the 22nd International Conference on Machine Learning, 2005
[19] Doon R., Rawat T. K., Gautam S. Cifar-10 Classification Using Deep Convolutional Neural Network[C]. 2018 IEEE Punecon, 2018
[20] Frankle J., Carbin M. The Lottery Ticket Hypothesis:Finding Sparse, Trainable Neural Networks[C]. The 7th International Conference on Learning Representations, 2018
[21] Jacobs R. A., Jordan M. I., Nowlan S. J., et al. Adaptive Mixtures of Local Experts[J]. Neural Computation, 1991,3(1):79-87
[22] Eigen D., Puhrsch C., Fergus R. Depth Map Prediction from a Single Image Using a Multi-Scale Deep Network[C]. Advances in Neural Information Processing Systems, 2014
[23] Shazeer N., Mirhoseini A., Maziarz K., et al. Outrageously Large Neural Networks:The Sparsely-Gated Mixture-of-Experts Layer[C]. International Conference on Learning Representations, 2017
[24] Bengio Y., Courville A., Vincent P. Representation Learning:A Review and New Perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013,35(8):1798-1828
[25] Cho E., Myers S. A., Leskovec J. Friendship and Mobility:User Movement in Location-Based Social Networks[C]. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2011
[26] Yin H., Cui B., Zhou X., et al. Joint Modeling of User Check-In Behaviors for Real-Time Point-of-Interest Recommendation[J]. ACM Transactions on Information Systems (TOIS), 2016,35(2):1-44
[27] Ma J., Zhao Z., Yi X., et al. Modeling Task Relationships in Multi-Task Learning with Multi-Gate Mixture-of-Experts[C]. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery&Data Mining, 2018
[28] Zhao P., Luo A., Liu Y., et al. Where to Go Next:A Spatio-Temporal Gated Network for Next POI Recommendation[J]. IEEE Transactions on Knowledge and Data Engineering, 2020,(1):1-1
[29] Liu Q., Wu S., Wang L., et al. Predicting the Next Location:A Recurrent Model with Spatial and Temporal Contexts[C]. Thirtieth AAAI Conference on Artificial Intelligence, 2016
[30] Li R., Shen Y., Zhu Y. Next Point-of-Interest Recommendation with Temporal and Multi-level Context Attention[C]. 2018 IEEE International Conference on Data Mining (ICDM), IEEE, 2018
[31] Manotumruksa J., Macdonald C., Ounis I. A Contextual Attention Recurrent Architecture for Context-Aware Venue Recommendation[C]. The 41st International ACM SIGIR Conference on Research&Development in Information Retrieval, 2018