基于稀疏共享的多门混合专家下一个兴趣点推荐算法

管理评论 ›› 2023, Vol. 35 ›› Issue (3): 148-159.

基于稀疏共享的多门混合专家下一个兴趣点推荐算法

康来松^1,2, 刘世峰¹, 宫大庆¹

1. 北京交通大学经济管理学院, 北京 100044;
2. 中共国家能源集团党校, 北京 102211

收稿日期:2021-03-01 出版日期:2023-03-28 发布日期:2023-04-28
通讯作者: 宫大庆(通讯作者)，北京交通大学经济管理学院副教授，硕士生导师，博士。
作者简介:康来松，中共国家能源集团党校主管，博士；刘世峰，北京交通大学经济管理学院教授，博士生导师，博士。
基金资助:
教育部人文社会科学研究青年基金项目（18YJC630068）。

Next POI Recommendation for Multi-gate Hybrid Experts Based on Sparse Sharing

Kang Laisong^1,2, Liu Shifeng¹, Gong Daqing¹

1. School of Economics and Management, Beijing Jiaotong University, Beijing 100044;
2. Party School of National Energy Investment Group Co., Ltd., Beijing 102211

Received:2021-03-01 Online:2023-03-28 Published:2023-04-28

摘要/Abstract

摘要： 下一个兴趣点(next POI)推荐指的是预测用户在特定时间段内将访问的下一个兴趣点。用户出行模式受到异构的上下文因素影响，包括连续值(例如地理距离、时间间隔)和离散值(例如社交状况、星期状况)。本文采用稀疏共享的结构来产生专家网络，并采用多门混合专家模型来实现多个目标任务之间的参数共享。本文基于多任务学习对用户的多种类型行为进行建模，提出了一种基于稀疏共享的多门混合专家模型。首先，使用稀疏共享结构针对每个目标任务从基础网络中生成对应的专家网络，并采用迭代幅度剪枝法选择专家网络。其次，多门混合专家模型采用专家网络作为输入层，每个专家网络对于输入信息的处理具有不同的侧重，并采用多个门网络对任务目标进行专家网络的共享选择。最后，基于真实数据集对所提出的算法进行评估，验证了本文方法的有效性和实用性，以及将多任务学习和稀疏结构共享应用于下一个兴趣点推荐的前景。文章最后也提出了管理启示。

关键词: 基于位置的社交网络, 下一个兴趣点推荐, 神经网络, 多任务学习, 稀疏共享

Abstract: Next POI recommendation refers to predicting the next POI that the user will visit within a certain period of time. User travel patterns are affected by heterogeneous contextual factors, including continuous values (for example, geographic distance, time interval) and discrete values (for example, social status, weekly status). This paper uses a sparse sharing structure to generate expert networks, and uses a multi-gate hybrid expert model to realize parameter sharing between multiple target tasks. This paper models multiple types of user behaviors based on multi-task learning, and proposes a multi-gate hybrid expert model based on sparse sharing. First, the sparse sharing structure is used to generate the corresponding expert network from the basic network for each target task, and the iterative amplitude pruning method is used to select the expert network. Secondly, the multi-gate hybrid expert model uses expert networks as the input layer. Each expert network has a different emphasis on the processing of input information, and uses multiple door networks to share and select task targets. Finally, the proposed algorithm is evaluated based on real data sets. The evaluation of the proposed algorithm based on real data sets verifies the effectiveness and practicability of the method in this chapter, as well as the prospect of applying multi-task learning and sparse structure sharing to the next point of interest recommendation.

Key words: location-based social networks, next POI recommendation, neural networks, multi-task learning, sparse sharing architectures

康来松, 刘世峰, 宫大庆. 基于稀疏共享的多门混合专家下一个兴趣点推荐算法[J]. 管理评论, 2023, 35(3): 148-159.

Kang Laisong, Liu Shifeng, Gong Daqing. Next POI Recommendation for Multi-gate Hybrid Experts Based on Sparse Sharing[J]. Management Review, 2023, 35(3): 148-159.

参考文献

[1] 江若然,张玲玲.社交属性网下基于链路预测及节点度的推荐算法[J].管理评论, 2019,31(2):119-129
[2] 姜祎盼,张文,李健,等.基于异构网络元路径的App推荐算法[J].管理评论, 2020,32(4):160-170
[3] Abhadiomhen S. E., Nzeh R. C., Ganaa E. D., et al. Supervised Shallow Multi-Task Learning:Analysis of Methods[J]. Neural Processing Letters, 2022,54(3):2491-2508
[4] Caruana R. A. Multitask Learning:A Knowledge-Based Source of Inductive Bias[J]. Machine Learning Proceedings, 1993,10(1):41-48
[5] Duong L., Cohn T., Bird S., et al. Low Resource Dependency Parsing:Cross-Lingual Parameter Sharing in a Neural Network Parser[C]. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2:Short Papers), 2015
[6] Misra I., Shrivastava A., Gupta A., et al. Cross-Stitch Networks for Multi-task Learning[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016
[7] Chen L., Chen D., Yang F., et al. A Deep Multi-task Representation Learning Method for Time Series Classification and Retrieval[J]. Information Sciences, 2021,555:17-32
[8] Zhao S., Zhao T., Yang H., et al. STELLAR:Spatial-Temporal Latent Ranking for Successive Point-of-Interest Recommendation[C]. Thirtieth AAAI Conference on Artificial Intelligence, 2016
[9] Xie M., Yin H., Wang H., et al. Learning Graph-Based POI Embedding for Location-Based Recommendation[C]. Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 2016
[10] 鲜学丰,陈晓杰,赵朋朋,等.基于上下文感知和个性化度量嵌入的下一个兴趣点推荐[J].计算机工程与科学, 2018,40(4):616-625
[11] 张奥雅,石美惠,申德荣,等.融合区域与朋友影响的下一兴趣点推荐[J/OL].计算机科学与探索:1-11[2021-02-06]. http://kns.cnki.net/kcms/detail/11.5602.tp.20201105.1051.012.html
[12] 柴瑞敏,殷臣,孟祥福,等.基于时空循环神经网络的下一个兴趣点推荐方法[J].智能系统学报, 2021,16(3):407-415
[13] 王立,张谧.基于LSTM的POI个性化推荐框架[J].计算机系统应用, 2018,27(12):56-61
[14] Bansal T., Belanger D., McCallum A. Ask the GRU:Multi-Task Learning for Deep Text Recommendations[C]. Proceedings of the 10th ACM Conference on Recommender Systems, 2016
[15] Caruana R. Multitask Learning[J]. Machine Learning, 1997,28(1):41-75
[16] Gehring J., Auli M., Grangier D., et al. Convolutional Sequence to Sequence Learning[C]. International Conference on Machine Learning, 2017
[17] Li G., Chen Q., Zheng B., et al. Group-Based Recurrent Neural Networks for POI Recommendation[J]. ACM Transactions on Data Science, 2020,1(1):1-18
[18] Burges C., Shaked T., Renshaw E., et al. Learning to Rank Using Gradient Descent[C]. Proceedings of the 22nd International Conference on Machine Learning, 2005
[19] Doon R., Rawat T. K., Gautam S. Cifar-10 Classification Using Deep Convolutional Neural Network[C]. 2018 IEEE Punecon, 2018
[20] Frankle J., Carbin M. The Lottery Ticket Hypothesis:Finding Sparse, Trainable Neural Networks[C]. The 7th International Conference on Learning Representations, 2018
[21] Jacobs R. A., Jordan M. I., Nowlan S. J., et al. Adaptive Mixtures of Local Experts[J]. Neural Computation, 1991,3(1):79-87
[22] Eigen D., Puhrsch C., Fergus R. Depth Map Prediction from a Single Image Using a Multi-Scale Deep Network[C]. Advances in Neural Information Processing Systems, 2014
[23] Shazeer N., Mirhoseini A., Maziarz K., et al. Outrageously Large Neural Networks:The Sparsely-Gated Mixture-of-Experts Layer[C]. International Conference on Learning Representations, 2017
[24] Bengio Y., Courville A., Vincent P. Representation Learning:A Review and New Perspectives[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013,35(8):1798-1828
[25] Cho E., Myers S. A., Leskovec J. Friendship and Mobility:User Movement in Location-Based Social Networks[C]. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2011
[26] Yin H., Cui B., Zhou X., et al. Joint Modeling of User Check-In Behaviors for Real-Time Point-of-Interest Recommendation[J]. ACM Transactions on Information Systems (TOIS), 2016,35(2):1-44
[27] Ma J., Zhao Z., Yi X., et al. Modeling Task Relationships in Multi-Task Learning with Multi-Gate Mixture-of-Experts[C]. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery&Data Mining, 2018
[28] Zhao P., Luo A., Liu Y., et al. Where to Go Next:A Spatio-Temporal Gated Network for Next POI Recommendation[J]. IEEE Transactions on Knowledge and Data Engineering, 2020,(1):1-1
[29] Liu Q., Wu S., Wang L., et al. Predicting the Next Location:A Recurrent Model with Spatial and Temporal Contexts[C]. Thirtieth AAAI Conference on Artificial Intelligence, 2016
[30] Li R., Shen Y., Zhu Y. Next Point-of-Interest Recommendation with Temporal and Multi-level Context Attention[C]. 2018 IEEE International Conference on Data Mining (ICDM), IEEE, 2018
[31] Manotumruksa J., Macdonald C., Ounis I. A Contextual Attention Recurrent Architecture for Context-Aware Venue Recommendation[C]. The 41st International ACM SIGIR Conference on Research&Development in Information Retrieval, 2018