Publications

Recent Preprints

[P1] Efficient Exploration via Reward-Biased Maximum Likelihood Estimation in Linear Contextual Bandits
Yu-Heng Hung and Ping-Chun Hsieh
Submitted to Machine Learning Journal, under review.

[P2] Value-Biased Maximum Likelihood Estimation for Model-based Reinforcement Learning in Discounted Linear MDPs
Yu-Heng Hung, Ping-Chun Hsieh, Akshay Mete, and P. R. Kumar
arXiv Technical Report (arXiv:2310.11515), Oct. 2023

[P3] AnesSim: A Learning-Based Simulation Platform for Closed-Loop Anesthesia Delivery
Yun-Ming Chan, Ping-Chun Hsieh, Yu-Ming Wang, Cheng-Chun Wu, Hao-Chun Lin, Cheng-Yi Xu, Yen-Yi Chen, Mu-Ting Lee, Yi-Ting Chang, Chun-Shu Wei, and Ching-Hui Shen
Submitted to Artificial Intelligence in Medicine, under review.

Journal Papers

[J2] Heavy-Traffic Analysis of QoE Optimality for On-Demand Video Streams Over Fading Channels
Ping-Chun Hsieh and I-Hong Hou
IEEE/ACM Transactions on Networking (TON), vol. 26, no. 4, Aug. 2018, pp. 1768 – 1781.

[J1] The Capacity of QoE for Wireless Networks with Unreliable Transmissions
I-Hong Hou and Ping-Chun Hsieh
Queueing Systems: Theory and Applications (QUESTA), vol. 87, no. 1-2, Oct. 2017, pp. 131-159.

Conference Papers

[C29] Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPs
Wei Hung, Shao-Hua Sun, and Ping-Chun Hsieh
International Conference on Learning Representations (ICLR), 2025 (Acceptance rate = 32%)

[C28] BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL
Yu-Heng Hung, Kai-Jie Lin, Yu-Heng Lin, Cheng Sun, Chien-Yi Wang*, and Ping-Chun Hsieh* (*: equal advising)
International Conference on Learning Representations (ICLR), 2025 (Acceptance rate = 32%)
(An extended version of our ICML 2024 AutoRL Workshop paper)

[C27] Diffusion-Reward Adversarial Imitation Learning
Chun-Mao Lai*, Hsiang-Chun Wang*, Ping-Chun Hsieh, Yu-Chiang Frank Wang, Min-Hung Chen, and Shao-Hua Sun (*: equal contribution)
Conference on Neural Information Processing Systems (NeurIPS), 2024 (Acceptance rate = 25.8%)

[C26] Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learning
Yen-Ju Chen, Nai-Chieh Huang, Ching-pei Lee, and Ping-Chun Hsieh
International Conference on Machine Learning (ICML), 2024 (Acceptance rate = 27.5%)
(An extended version of our ICML 2023 Frontiers4LCD Workshop paper)

[C25] Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning
Yun-Hsuan Lien, Ping-Chun Hsieh, Tzu-Mao Li, Yu-Shuen Wang
International Conference on Machine Learning (ICML), 2024 (Acceptance rate = 27.5%)

[C24] Offline Imitation of Badminton Player Behavior via Experiential Contexts and Brownian Motion
Kuang-Da Wang, Wei-Yao Wang, Ping-Chun Hsieh, and Wen-Chih Peng
European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2024 (Acceptance rate = 24.5%)

[C23] PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping
Nai-Chieh Huang, Ping-Chun Hsieh, Kuo-Hao Ho, and I-Chen Wu
AAAI Conference on Artificial Intelligence (AAAI), 2024 (Acceptance rate = 23.75%)

[C22] Towards Human-Like RL: Taming Non-Naturalistic Behavior in Deep RL via Adaptive Behavioral Costs in 3D Games
Kuo-Hao Ho, Ping-Chun Hsieh, Chiu-Chou Lin, You-Ren Luo, Feng-Jian Wang, and I-Chen Wu
Asian Conference on Machine Learning (ACML), 2023 (Long Oral; Acceptance rate = 35%)

[C21] Revisiting Domain Randomization via Relaxed State-Adversarial Policy Optimization
Yun-Hsuan Lien, Ping-Chun Hsieh, and Yu-Shuen Wang
International Conference on Machine Learning (ICML), 2023 (Acceptance rate = 28%)

[C20] Q-Pensieve: Boosting Sample Efficiency of Multi-Objective RL Through Memory Sharing of Q-Snapshots
Wei Hung*, Bo-Kai Huang*, Ping-Chun Hsieh, and Xi Liu (*: equal contribution)
International Conference on Learning Representations (ICLR), 2023 (Acceptance rate = 31.8%)

[C19] Coordinate Ascent for Off-Policy RL with Global Convergence Guarantees
Hsin-En Su*, Yen-Ju Chen*, Ping-Chun Hsieh, and Xi Liu (*: equal contribution)
International Conference on Artificial Intelligence and Statistics (AISTATS), 2023 (Acceptance rate = 29%)

[C18] Reward-Biased Maximum Likelihood Estimation for Neural Contextual Bandits: A Distributional Learning Perspective
Yu-Heng Hung and Ping-Chun Hsieh
AAAI Conference on Artificial Intelligence (AAAI), 2023 (Oral; Acceptance rate = 19.6%)

[C17] Neural Frank-Wolfe Policy Optimization for Region-of-Interest Intra-Frame Coding with HEVC/H.265
Yung-Han Ho, Chia-Hao Kao, Wen-Hsiao Peng, and Ping-Chun Hsieh
IEEE Visual Communications and Image Processing (VCIP), 2022.

[C16] Deterministic Bandwidth-Based Packet-Level Traffic Splitting for Datacenter Networks
Cheng-Yu Wu, Li-Hsing Yen, Ping-Chun Hsieh, and Chien-Chao Tseng
Asia-Pacific Network Operations and Management Symposium (APNOMS), 2022.
(Best Paper Award)

[C15] Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization
Bing-Jing Hsieh, Ping-Chun Hsieh, and Xi Liu
Conference on Neural Information Processing Systems (NeurIPS), 2021 (Acceptance rate = 26%)

[C14] NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RL
Khaled Nakhleh, Santosh Ganji, Ping-Chun Hsieh, I-Hong Hou, and Srinivas Shakkottai
Conference on Neural Information Processing Systems (NeurIPS), 2021 (Acceptance rate = 26%)

[C13] Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimization
Jyun-Li Lin*, Wei Hung*, Shang-Hsuan Yang*, Ping-Chun Hsieh, and Xi Liu (*: equal contribution)
Conference on Uncertainty in Artificial Intelligence (UAI), 2021 (Acceptance rate = 26.5%)

[C12] Reward-Biased Maximum Likelihood Estimation for Linear Stochastic Bandits
Yu-Heng Hung, Ping-Chun Hsieh, Xi Liu, and P. R. Kumar
AAAI Conference on Artificial Intelligence (AAAI), 2021 (Acceptance rate = 21%)

[C11] Optimal Wireless Scheduling for Remote Sensing Through Brownian Approximation
Daojing Guo, Ping-Chun Hsieh, and I-Hong Hou
IEEE International Conference on Computer Communications (INFOCOM), 2021 (Acceptance rate = 19.9%)

[C10] Exploration Through Reward Biasing: Reward-Biased Maximum Likelihood Estimation for Stochastic Multi-Armed Bandits
Xi Liu*, Ping-Chun Hsieh*, Yu-Heng Hung, Anirban Bhattacharya, and P. R. Kumar (*: equal contribution)
International Conference on Machine Learning (ICML), 2020 (Acceptance rate = 22%)

[C9] Fresher Content or Smoother Playback? A Brownian-Approximation Framework for Scheduling Real-Time Wireless Video Streams
Ping-Chun Hsieh, Xi Liu, and I-Hong Hou
ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), 2020 (Acceptance rate = 15%)
(Best Paper Award)

[C8] Stay With Me: Lifetime Maximization Through Heteroscedastic Linear Bandits With Reneging
Ping-Chun Hsieh*, Xi Liu*, Anirban Bhattacharya, and P. R. Kumar (*: equal contribution)
International Conference on Machine Learning (ICML), 2019 (Acceptance rate = 22.6%)

[C7] A Decentralized Medium Access Protocol for Real-Time Wireless Ad Hoc Networks With Unreliable Transmissions
Ping-Chun Hsieh and I-Hong Hou
IEEE International Conference on Distributed Computing Systems (ICDCS), 2018

[C6] PULS: Processor-Supported Ultra-Low Latency Scheduling
Simon Yau, Ping-Chun Hsieh, Rajarshi Bhattacharyya, Kartic Bhargav, Srinivas Shakkottai, I-Hong Hou, and P. R. Kumar
ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), 2018 (Acceptance rate = 17%)
(Best Paper Finalist)

[C5] An Experimental Study on Coverage Enhancement of LTE Cat-M1 for Machine-Type Communication
Ping-Chun Hsieh, Yupeng Jia, Darwin Parra, and Prabha Aithal
IEEE International Conference on Communications (ICC), 2018

[C4] Throughput-Optimal Scheduling For Multi-Hop Networked Transportation Systems With Switch-Over Delay
Ping-Chun Hsieh, Xi Liu, Jian Jiao, I-Hong Hou, Yunlong Zhang, and P.R. Kumar
ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), 2017 (Acceptance rate = 17%)
(Best Paper Award)

[C3] Heavy-Traffic Analysis of QoE Optimality for On-Demand Video Streams Over Fading Channels
Ping-Chun Hsieh and I-Hong Hou
IEEE International Conference on Computer Communications (INFOCOM), 2016 (Acceptance rate = 18.25%)

[C2] QoE-Optimal Scheduling for On-Demand Video Streams Over Unreliable Wireless Networks
I-Hong Hou and Ping-Chun Hsieh
ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), 2015 (Acceptance rate = 14.9%)

[C1] A Primary-Side-Control Quasi-Resonant Flyback Converter With Tight Output Voltage Regulation and Self-Calibrated Valley Switching
Ping-Chun Hsieh, Chia-Jung Chang, and Chern-Lin Chen
IEEE Energy Conversion Congress and Exposition (ECCE), 2013

Workshop and Demo Papers

[W5] Accelerated Policy Gradient: On the Nesterov Momentum for Reinforcement Learning
*Yen-Ju Chen, *Nai-Chieh Huang, and Ping-Chun Hsieh (*: equal contribution)
Frontiers4LCD Workshop at International Conference on Machine Learning (ICML), 2023

[W4] Generating Turn-Based Player Behavior via Experience from Demonstrations
Kuang-Da Wang, Wei-Yao Wang, Ping-Chun Hsieh, Wen-Chih Peng
SPIGM Workshop at International Conference on Machine Learning (ICML), 2023

[W3] Rethinking Deep Policy Gradient via State-Wise Policy Improvement
Kai-Chun Hu, Ping-Chun Hsieh, Ting-Han Wei, and I-Chen Wu
ICBINB Workshop @ NeurIPS 2020

[W2] Real-Time Streaming Graph Embedding Through Local Actions
Xi Liu, Ping-Chun Hsieh, Nick Duffield, Rui Chen, Muhe Xie, and Xidao Wen
DL4G-SDE Workshop at the World Wide Web Conference (WWW), 2019

[W1] WiMAC: Rapid Implementation Platform for User Deﬁnable MAC Protocols Through Separation
Simon Yau, Liang Ge, Ping-Chun Hsieh, I-Hong Hou, Shuguang Cui, P. R. Kumar, Amal Ekbal, and Nikhil Kundargi
ACM Special Interest Group on Data Communications (SIGCOMM) demo, 2015

Submissions and Archived Preprints

[S2] Developing Multiple-task Recommendations with Long-term Reward via Policy Distilled Reinforcement Learning
Xi Liu, Li Li, Ping-Chun Hsieh, Muhe Xie, Yong Ge, and Rui Chen
Submitted to ACM Transactions on Information Systems (TOIS), 2019

[S1] Delay-Optimal Scheduling for Queueing Systems with Switching Overhead
Ping-Chun Hsieh, I-Hong Hou, and Xi Liu
arXiv Technical Report (arXiv:1701.03831), Jan. 2017