Did the Government Service Rating System Improve the Effectiveness of Online Government Services?—Empirical Tests Based on a Natural Experiment Using Regression Discontinuity Design

Abstract

Abstract: Implementing the "Good/Poor Rating System" for government services is one of the reform measures that have received great attention and expectation since the 18th National Congress of the Communist Party of China. The policy effect of this reform and its optimization direction has gained extensive attention from academia. The Message Board for Leaders on People's Daily is an online political deliberation platform. As the epitome of online government services, the Message Board for Leaders added the satisfaction rating function in June 2019, which became the natural intervention of the "Good/Poor" rating system. Using the exogeneity of the satisfaction rating function, this paper applies deep learning methods to analyze the textual features of more than 27,000 netizens' posts and government responses and measures the effectiveness of online government service and related variables. This paper estimates the impact of netizens' satisfaction ratings on the effectiveness of online government service with regression discontinuity design. As expected, the netizens' satisfaction ratings play a decisive role in improving the effectiveness of government online services. Also, the positive impacts vary in degree and statistical significance among different local governments. The results remain robust with different estimation methods, model settings, and bandwidth selections. The conclusion of this study deepens the understanding of the "Good/Poor Rating System" and sheds light on the key issues that should be optimized.

Key words: the effectiveness of online government services, "good/poor rating system", public satisfaction

Wang Tianmei, Zhao Yuning, Yu Peng, Zhu Yanchun. Did the Government Service Rating System Improve the Effectiveness of Online Government Services?—Empirical Tests Based on a Natural Experiment Using Regression Discontinuity Design[J]. Management Review, 2024, 36(1): 251-263.

References

[1] 石亚军,吕勇. 政务服务"好差评"制度对强化监督深化改革的可预期性探析[J]. 中国行政管理, 2019,(12):10-15 Shi Y. J., Lv Y. To Establish a Government Service Rating System Is a Smart Move to Deepen and Supervise Reform[J]. Chinese Public Administration, 2019,(12):10-15
[2] 沈丽琴. 地方政府实施政务服务"好差评"制度的成效及其优化[J]. 中国行政管理, 2020,(12):144-146 Shen L. Q. The Effectiveness and Optimization of a Government Service Rating System Implemented by Local Governments[J]. Chinese Public Administration, 2020,(12):144-146
[3] Li H. An Empirical Research on the Construction of a Government Website Public Satisfaction Index Model in China[J]. Journal of Global Information Management, 2021,29(5):112-137
[4] 翟云. 政务服务"好差评"制度:政策意蕴、理论阐释与路径建构[J]. 行政管理改革, 2020,(3):66-72 Zhai Y. Evaluation System for the Government Service:Policy Implication, Theoretical Explanation and Implementation Path[J]. Administration Reform, 2020,(3):66-72
[5] 程波辉. 制度可实施性:政务服务"好差评"的元评估框架——基于19个城市的政策文本分析[J]. 行政论坛, 2022,29(4):48-55 Cheng B. H. Institutional Enforceability:A Meta Evaluation Framework of "Good and Bad Evaluation" of Government Services:An Analysis of Policy Texts Based on 19 Cities in China[J]. Administrative Tribune, 2022,29(4):48-55
[6] 吴建南,庄秋爽. 测量公众心中的绩效:顾客满意度指数在公共部门的分析应用[J]. 管理评论, 2005,17(5):53-57 Wu J. N., Zhuang Q. S. Measuring Performance in the Public Mind:Analytical Applications of the Customer Satisfaction Index in the Public Sector[J]. Management Review, 2005,17(5):53-57
[7] Ho A. T. K., Cho W. Government Communication Effectiveness and Satisfaction with Police Performance:A Large-scale Survey Study[J]. Public Administration Review, 2017,77(2):228-239
[8] 杨开峰,王璐璐,仇纳青. 营商环境建设、主观治理绩效评价与溢出效应——来自中国29省的经验证据[J]. 公共管理学报, 2023,20(1):106-120 Yang K. F., Wang L. L., Qiu N. Q. Business Environment, Subjective Performance Evaluation of Government Governance and Spillover Effect:Empirical Evidence from 29 Provinces in China[J]. Journal of Public Management, 2023,20(1):106-120
[9] 张建顺,匡浩宇. "放管服"改革与纳税人满意度:施策重点与优化路径——基于机器学习方法[J]. 公共管理学报, 2021,18(4):46-62 Zhang J. S., Kuang H. Y. Administration Reform and Taxpayer Satisfaction:Policy Focus and Optimization Paths:Based on Machine Learning Methods[J]. Journal of Public Management, 2021,18(4):46-62
[10] 保海旭. 信任对公共服务满意度的影响及其区域差异化研究——基于CGSS2015年中国28个省份的截面数据[J]. 管理评论, 2021,33(7):301-312 Bao H. X. The Influence of Trust on Public Service Satisfaction and Its Regional Differentiation:Based on CGSS 2015 Cross-Sectional Data of 28 Provinces in China[J]. Management Review, 2021,33(7):301-312
[11] 高学德,冯露露. 地方政府形象对公众满意度的影响研究——基于政府能力和意愿的分析[J]. 公共行政评论, 2022,15(6):116-135 Gao X. D., Feng L. L. Research on the Effect of Local Government Image on Public Satisfaction:Evidence from Government Competence and Willingness[J]. Journal of Public Administration, 2022,15(6):116-135
[12] Kelly J. M. Citizen Satisfaction and Administrative Performance Measures:Is There Really a Link?[J]. Urban Affairs Review, 2003,38(6):855-866
[13] Jilke S., Baekgaard M. The Political Psychology of Citizen Satisfaction:Does Functional Responsibility Matter?[J]. Journal of Public Administration Research and Theory, 2020,30(1):130-143
[14] Swindell D., Kelly J. Performance Measurement versus City Service Satisfaction:Intra-city Variations in Quality?[J]. Social Science Quarterly, 2005,86(3):704-723
[15] 王印红,朱玉洁. 基层政府"逆扁平化"组织扩张的多重逻辑——基于"管区制度"的案例研究[J]. 公共管理学报, 2020,17(4):21-31 Wang Y. H., Zhu Y. J. Multi-logic of "Anti-flattening" Organizational Expansion of Grassroots Government:Case Study Based on "District System"[J]. Journal of Public Management, 2020,17(4):21-31
[16] 易兰丽,范梓腾. 层级治理体系下的政策注意力识别偏好与政策采纳——以省级"互联网+政务服务"平台建设为例[J]. 公共管理学报, 2022,19(1):40-51 Yi L. L., Fan Z. T. Policy Attention Identification Preference and Policy Adoption under the Multilevel Governance System:Evidence from the Construction of Provincial "Internet + Government Service" Platform[J]. Journal of Public Management, 2022,19(1):40-51
[17] 孙宗锋,姜楠. 政府部门回应策略及其逻辑研究——以J市政务热线满意度考核为例[J]. 中国行政管理, 2021,(5):40-46 Sun Z. F., Jiang N. Study on the Responsive Strategies of Government Agencies and Its Logic:The Case of Satisfaction Assessment of Government Hotline in J City[J]. Chinese Public Administration, 2021,(5):40-46
[18] 宋向嵘. "线上"回应与"线下"治理:网络问政效果耦合影响因素研究[J]. 电子政务, 2020,(5):112-124 Song X. R. "Online" Response and "Offline" Governance:A Study of Coupled Influencing Factors on the Effectiveness of Online Questioning of Politics[J]. E-government, 2020,(5):112-124
[19] Wang Q., Niu M. Exploring the Relationship between Government Budget Information and Citizens' Perceptions of Public Service Performance in China[J]. Public Management Review, 2020,22(3):317-340
[20] Gao X., Wang Q., Shen X. Is Citizen Satisfaction Affected by the Equivalent Framing Effect? A Study Based on Situations Involving Poor Government Performance[J/OL]. International Review of Administrative Sciences, 2023. DOI:10.1177/00208523231173128
[21] 王玉君,韩冬临. 空气质量,环境污染感知与地方政府环境治理评价[J]. 中国软科学, 2019,(8):41-51 Wang Y. J., Han D. L. Air Quality, Perception of Environmental Pollution, and the Evaluation of Local Governments' Environmental Governance[J]. China Soft Science, 2019,(8):41-51
[22] 于大春,张华杰,宋万超. 绩效管理理论研究综述[J]. 情报杂志, 2010,29(S2):16-19 Yu D. C., Zhang H. J., Song W. C. A Review of Performance Management Theory Research[J]. Journal of Intelligence, 2010,29(S2):16-19
[23] 郑方辉,喻锋,覃事灿. 政府整体绩效评价:理论假说及其实证检验——以2008年度广东省为例[J]. 公共管理学报, 2011,8(3):13-23 Zheng F. H., Yu F., Qin S. C. Overall Government Performance Evaluation:Hypothesis, Theoretical Demonstration and Empirical Study on Guangdong Province 2008[J]. Journal of Public Management, 2011,8(3):13-23
[24] 沙勇忠,王峥嵘,詹建. 政民互动行为如何影响网络问政效果?——基于"问政泸州"的大数据探索与推论[J]. 公共管理学报, 2019,16(2):15-27 Sha Y. Z., Wang Z. R., Zhan J. How do the Interactions between Government and Public Affect the Effectiveness of Online Interaction Politics:The Exploration and Inference Based on Big Data of "Wenzheng Luzhou"[J]. Journal of Public Management, 2019,16(2):15-27
[25] 梁昌勇,朱龙,冷亚军. 基于结构方程模型的政府部门公众满意度测评[J]. 中国管理科学, 2012,20(S1):108-113 Liang C. Y., Zhu L., Leng Y. J. The Application of the Structual Equation Model in Public Satisfaction Evaluation of Government[J]. Chinese Journal of Management Science, 2012,20(S1):108-113
[26] 吴建南,阎波. 谁是"最佳"的价值判断者:区县政府绩效评价机制的利益相关主体分析[J]. 管理评论, 2006,18(4):46-53 Wu J. N., Yan B. Who Is the "Best" Value Judge:Analysis of the Stakeholder Subjects in the Performance Evaluation Mechanism of District and County Governments[J]. Management Review, 2006,18(4):46-53
[27] 谭海波. 技术管理能力、注意力分配与地方政府网站建设——一项基于TOE框架的组态分析[J]. 管理世界, 2019,35(9):81-94 Tan H. B. Technology Management Capabilities, Attention Allocation, and Local Government Website Development:An Analysis of a Configuration Based on the TOE Framework[J]. Journal of Management World, 2019,35(9):81-94
[28] 吕欣,裴瑞敏,刘凡. 电子政务信息资源共享的影响因素及安全风险分析[J]. 管理评论, 2013,25(6):161-169 Lv X., Pei R. M., Liu F. Influencing Factors and Security Analysis of E-government Information Sharing[J]. Management Review, 2013,25(6):161-169
[29] 杨慧,易兰丽,孟庆国. "互联网+政务服务"发展的测度与提升路径研究[J]. 中国行政管理, 2018,(11):39-44 Yang H., Yi L. L., Meng Q. G. Research on the Measurement and Improving Path of "Internet Plus Government Services"[J]. Chinese Public Administration, 2018,(11):39-44
[30] 吴建南,张攀,刘张立. "效能建设"十年扩散:面向中国省份的事件史分析[J]. 中国行政管理, 2014,(1):76-82 Wu J. N., Zhang P., Liu Z. L. Antecedents of "Xiaoneng" Building Diffusion among Chinese Provinces:An Event History Analysis[J]. Chinese Public Administration, 2014,(1):76-82
[31] 周志忍. 我国政府绩效评估需要思考的几个问题[J]. 行政管理改革, 2011,(4):37-41 Zhou Z. R. Several Issues to be Considered in China's Government Performance Evaluation[J]. Administration Reform, 2011,(4):37-41
[32] 王磊,易扬. 公共卫生危机中的数字政府回应如何纾解网络负面舆情——基于人民网"领导留言板"回复情况的调查[J]. 公共管理学报, 2022,19(4):65-78 Wang L., Yi Y. How does Government Responsiveness Alleviate Negative Public Opinion during COVID-19:Evidence from "Message Board for Leaders" in People.cn[J]. Journal of Public Management, 2022,19(4):65-78
[33] Feng X., Wang C., Wang J. Understanding How the Expression of Online Citizen Petitions Influences the Government Responses in China:An Empirical Study with Automatic Text Analytics[J]. Information Processing & Management, 2023,60(3):103330
[34] 徐凌. 论热线管理中公众导向型政府回应模式的有效建构[J]. 中国行政管理, 2020,(10):52-59 Xu L. On the Effective Construction of Public-oriented Government Response Mode in Hotline Management[J]. Chinese Public Administration, 2020,(10):52-59
[35] Li Y., Fang W., Sun H., et al. Pecidrl:Petition Expectation Correction and Identification Based on Deep Reinforcement Learning[J]. Information Processing & Management, 2023,60(3). DOI:10.1016/j.ipm.2023.103285
[36] 马翔,包国宪. 网络舆情事件中的公共价值偏好与政府回应绩效[J]. 公共管理学报, 2020,17(2):70-83 Ma X., Bao G. X. Public Value Preference in Internet Public Opinion Events and Government Response Performance[J]. Journal of Public Management, 2020,17(2):70-83
[37] 邓卫华,吕佩. 反转或缓解?突发事件政府回应有效性研究——基于在线文本情感分析[J]. 中国行政管理, 2021,(2):123-130 Deng W. H., Lv P. Reversion or Ease? Research on the Effectiveness of Government Response in Emergencies:Sentiment Analysis Based on Online Text[J]. Chinese Public Administration, 2021,(2):123-130
[38] 郑石明,兰雨潇,黎枫. 网络公共舆论与政府回应的互动逻辑——基于新冠肺炎疫情期间"领导留言板"的数据分析[J]. 公共管理学报, 2021,18(3):24-37 Zheng S. M., Lan Y. X., Li F. The Interactive Logic of Online Public Opinion and Government Response:Based on the Data Analysis of the "Leadership Message Board" during the COVID-19 Pandemic[J]. Journal of Public Management, 2021,18(3):24-37
[39] 孟天广,黄种滨,张小劲. 政务热线驱动的超大城市社会治理创新——以北京市"接诉即办"改革为例[J]. 公共管理学报, 2021,18(2):1-12 Meng T. G., Huang Z. B., Zhang X. J. Social Governance Innovation Driven by Government Hot-line in Megacity:A Case Study on Beijing's Reform of Responding to Whistles[J]. Journal of Public Management, 2021,18(2):1-12
[40] Jiang M., Xu H. Exploring Online Structures on Chinese Government Portals:Citizen Political Participation and Government Legitimation[J]. Social Science Computer Review, 2009,27(2):174-195
[41] 李莉,孟天广. 公众网络反腐败参与研究——以全国网络问政平台的大数据分析为例[J]. 中国行政管理, 2019,(1):45-52 Li L., Meng T. G. Public Participation in Anti-corruption:A Study of the Big Data from Message Board for Local Leaders[J]. Chinese Public Administration, 2019,(1):45-52
[42] Lee D. S., Lemieux T. Regression Discontinuity Designs in Economics[J]. Journal of Economic Literature, 2010,48(2):281-355
[43] Blander R. R. G. D. Mostly Harmless Econometrics:An Empiricist's Companion[J]. European Review of Agricultural Economics, 2011,38(2):281-283
[44] 中国社会科学院政治学研究所课题组. 坚持人民至上共创美好生活——北京党建引领接诉即办改革发展报告[J]. 管理世界, 2023,39(1):15-28 Chinese Academy of Social Sciences Research Group. Putting People First for a Better Life Together:Report on Beijing's Reform for Swift Responses to Public Complaints through Party Building[J]. Journal of Management World, 2023,39(1):15-28
[45] 孙柏瑛,周保民. 民意发包:驱动政府工作的新方式——以S县民生实事项目人大代表票决制为例[J]. 中国行政管理, 2023,39(4):30-37 Sun B. Y., Zhou B. M. "Subcontract of People's Will":A New Mode of Driving Government Work:An Example of LPC Deputies Vote-deciding System on People's Livelihood Projects in S County[J]. Chinese Public Administration, 2023,39(4):30-37
[46] 陈佳,高洁玉,赫郑飞. 公共政策执行中的"激励"研究——以W县退耕还林为例[J]. 中国行政管理, 2015,(6):113-118 Chen J., Gao J. Y., Hao Z. F. Study on Policy Implementation:Perspectives and Implications[J]. Chinese Public Administration, 2015,(6):113-118
[47] 董天天,徐璐璐,贺雯. 积极总是好的吗?积极元刻板印象对工作记忆的影响及其机制[J]. 心理学报, 2023,55(8):1344-1357 Dong T. T., Xu L. L., He W. Is Positivity Always Beneficial? The Effect of Positive Meta-stereotypes on Working Memory and Their Mechanism[J]. Acta Psychologica Sinica, 2023,55(8):1344-1357
[48] 谢谦,薛仙玲,付明卫. 断点回归设计方法应用的研究综述[J]. 经济与管理评论, 2019,35(2):69-79 Xie Q., Xue X. L., Fu M. W. A Literature Review of the Application of Regression Discontinuity Design[J]. Review of Economy and Management, 2019,35(2):69-79
[49] Tianguang M. Responsive Government:The Diversity and Institutional Performance of Online Political Deliberation Systems[J]. Social Sciences in China, 2019,40(4):148-172