Publications

RL Pre-Training

Reward-Free RL

• From Reward-Free Representations to Preferences: Rethinking Offline Preference-Based Reinforcement Learning
  Jun-Jie Yang, Chia-Heng Hsu, Kui-Yuan Chen, and Ping-Chun Hsieh
  International Conference on Machine Learning (ICML), 2026 (Acceptance rate = 26.6%)
  

• A Reward-Free Viewpoint on Multi-Objective Reinforcement Learning
  Ying-Tu Chen*, Wei Hung*, Bing-Shu Wu*, Zhang-Wei Hong, and Ping-Chun Hsieh (*: equal contribution)
  International Conference on Learning Representations (ICLR), 2026 (Acceptance rate = 28%)
  [Project Page]

Learning to Optimize via RL Pre-Training

• BOFormer: Learning to Solve Multi-Objective Bayesian Optimization via Non-Markovian RL
  Yu-Heng Hung, Kai-Jie Lin, Yu-Heng Lin, Chien-Yi Wang*, Cheng Sun, and Ping-Chun Hsieh* (*: equal advising)
  International Conference on Learning Representations (ICLR), 2025 (Acceptance rate = 32%)
  (An extended version of our ICML 2024 AutoRL Workshop paper)
  [Project Page] [Code] [Video]

• Reinforced Few-Shot Acquisition Function Learning for Bayesian Optimization
  Bing-Jing Hsieh, Ping-Chun Hsieh, and Xi Liu
  Conference on Neural Information Processing Systems (NeurIPS), 2021 (Acceptance rate = 26%)

Test-Time RL Methods

• Test-Time Alignment for Large Language Models via Textual Model Predictive Control
  Kuang-Da Wang*, Teng-Ruei Chen*, Yu Heng Hung, Guo-Xun Ko, Shuoyang Ding, Yueh-Hua Wu, Yu-Chiang Frank Wang, Chao-Han Huck Yang, Wen-Chih Peng, and Ping-Chun Hsieh (*: equal contribution)
  International Conference on Learning Representations (ICLR), 2026 (Acceptance rate = 28%)
  [Project Page]

• Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning
  Sze-Ann Chen, Zhi-Yi Chin, Kui-Yuan Chen, Chi-Yu Li, and Ping-Chun Hsieh
  Transactions on Machine Learning Research (TMLR), 2026

• Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantization
  Jian-Ting Guo, Yu-Cheng Chen, Ping-Chun Hsieh, Kuo-Hao Ho, Po-Wei Huang, Ti-Rong Wu, and I-Chen Wu
  Conference on Neural Information Processing Systems (NeurIPS), 2025 (Acceptance rate = 24.52%)
  [Project Page]

Single-Task RL

Cross-Domain RL

• Cross-Domain Policy Optimization via Bellman Consistency and Hybrid Critics
  Ming-Hong Chen*, Kuan-Chen Pan*, You-De Huang*, Xi Liu, and Ping-Chun Hsieh (*: equal contribution)
  International Conference on Learning Representations (ICLR), 2026 (Acceptance rate = 28%)
  [Project Page]

• Semi-Supervised Cross-Domain Imitation Learning
  Li-Min Chu, Kai-Siang Ma, Ming-Hong Chen, and Ping-Chun Hsieh
  Transactions on Machine Learning Research (TMLR), 2026
  [Code]

Action-Constrained RL

• Efficient Action-Constrained Reinforcement Learning via Acceptance-Rejection Method and Augmented MDPs
  Wei Hung, Shao-Hua Sun, and Ping-Chun Hsieh
  International Conference on Learning Representations (ICLR), 2025 (Acceptance rate = 32%)
  [Code] [Video]

• Action-Constrained Imitation Learning
  Chia-Han Yeh*, Tse-Sheng Nan*, Risto Vuorio, Wei Hung, Hung Yen Wu, Shao-Hua Sun, and Ping-Chun Hsieh (*: equal contribution)
  International Conference on Machine Learning (ICML), 2025 (Acceptance rate = 26.9%)
  [Code][Video]

• Escaping from Zero Gradient: Revisiting Action-Constrained Reinforcement Learning via Frank-Wolfe Policy Optimization
  Jyun-Li Lin*, Wei Hung*, Shang-Hsuan Yang*, Ping-Chun Hsieh, and Xi Liu (*: equal contribution)
  Conference on Uncertainty in Artificial Intelligence (UAI), 2021 (Acceptance rate = 26.5%)

Misc (MORL, Offline RL, and Robust RL)

• Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning
  Yun-Hsuan Lien, Ping-Chun Hsieh, Tzu-Mao Li, Yu-Shuen Wang
  International Conference on Machine Learning (ICML), 2024 (Acceptance rate = 27.5%)
  

• Q-Pensieve: Boosting Sample Efficiency of Multi-Objective RL Through Memory Sharing of Q-Snapshots
  Wei Hung*, Bo-Kai Huang*, Ping-Chun Hsieh, and Xi Liu (*: equal contribution)
  International Conference on Learning Representations (ICLR), 2023 (Acceptance rate = 31.8%)

• Relaxed Transition Kernels can Cure Underestimation in Adversarial Offline Reinforcement Learning
  Ziyu Wang, Ping-Chun Hsieh, Yu-Shuen Wang, and Yun-Hsuan Lien
  Asian Conference on Machine Learning (ACML), 2025

• Diffusion-Reward Adversarial Imitation Learning
  Chun-Mao Lai*, Hsiang-Chun Wang*, Ping-Chun Hsieh, Yu-Chiang Frank Wang, Min-Hung Chen, and Shao-Hua Sun (*: equal contribution)
  Conference on Neural Information Processing Systems (NeurIPS), 2024 (Acceptance rate = 25.8%)

• Revisiting Domain Randomization via Relaxed State-Adversarial Policy Optimization
  Yun-Hsuan Lien, Ping-Chun Hsieh, and Yu-Shuen Wang
  International Conference on Machine Learning (ICML), 2023 (Acceptance rate = 28%)

• NeurWIN: Neural Whittle Index Network For Restless Bandits Via Deep RL
  Khaled Nakhleh, Santosh Ganji, Ping-Chun Hsieh, I-Hong Hou, and Srinivas Shakkottai
  Conference on Neural Information Processing Systems (NeurIPS), 2021 (Acceptance rate = 26%)

RL and Bandits Theory

Global Convergence of RL

• Accelerated Policy Gradient: On the Convergence Rates of the Nesterov Momentum for Reinforcement Learning
  Yen-Ju Chen, Nai-Chieh Huang, Ching-pei Lee, and Ping-Chun Hsieh
  International Conference on Machine Learning (ICML), 2024 (Acceptance rate = 27.5%)
  (An extended version of our ICML 2023 Frontiers4LCD Workshop paper)

• PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping
  Nai-Chieh Huang, Ping-Chun Hsieh, Kuo-Hao Ho, and I-Chen Wu
  AAAI Conference on Artificial Intelligence (AAAI), 2024 (Acceptance rate = 23.75%)

• Coordinate Ascent for Off-Policy RL with Global Convergence Guarantees
  Hsin-En Su*, Yen-Ju Chen*, Ping-Chun Hsieh, and Xi Liu (*: equal contribution)
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2023 (Acceptance rate = 29%)

Bandits

• A Modularized Framework for Piecewise-Stationary Restless Bandits
  Kuan-Ta Li, Chia-Chun Lin, Ping-Chun Hsieh, and Yu-Chih Huang
  International Conference on Artificial Intelligence and Statistics (AISTATS), 2026 (Spotlight; Acceptance rate = 28%)

• Reward-Biased Maximum Likelihood Estimation for Neural Contextual Bandits: A Distributional Learning Perspective
  Yu-Heng Hung and Ping-Chun Hsieh
  AAAI Conference on Artificial Intelligence (AAAI), 2023 (Oral; Acceptance rate = 19.6%)

• Reward-Biased Maximum Likelihood Estimation for Linear Stochastic Bandits
  Yu-Heng Hung, Ping-Chun Hsieh, Xi Liu, and P. R. Kumar
  AAAI Conference on Artificial Intelligence (AAAI), 2021 (Acceptance rate = 21%)

• Exploration Through Reward Biasing: Reward-Biased Maximum Likelihood Estimation for Stochastic Multi-Armed Bandits
  Xi Liu*, Ping-Chun Hsieh*, Yu-Heng Hung, Anirban Bhattacharya, and P. R. Kumar (*: equal contribution)
  International Conference on Machine Learning (ICML), 2020 (Acceptance rate = 22%)

• Stay With Me: Lifetime Maximization Through Heteroscedastic Linear Bandits With Reneging
  Ping-Chun Hsieh*, Xi Liu*, Anirban Bhattacharya, and P. R. Kumar (*: equal contribution)
  International Conference on Machine Learning (ICML), 2019 (Acceptance rate = 22.6%)

Applications

• Extending Automatic Machine Translation Evaluation to Book-Length Documents
  Kuang-Da Wang, Shuoyang Ding, Chao-Han Huck Yang, Ping-Chun Hsieh, Wen-Chih Peng, Vitaly Lavrukhin, and Boris Ginsburg
  Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025 (Acceptance rate = 22.16%)
  

• DDOT: A Derivative-directed Dual-decoder Ordinary Differential Equation Transformer for Dynamic System Modeling
  Yang Chang, Kuang-Da Wang, Ping-Chun Hsieh, Cheng-Kuan Lin, and Wen-Chih Peng
  Pacific Asia Conference on Knowledge Discovery and Data Mining (PAKDD), 2025
  

• Offline Imitation of Badminton Player Behavior via Experiential Contexts and Brownian Motion
  Kuang-Da Wang, Wei-Yao Wang, Ping-Chun Hsieh, and Wen-Chih Peng
  European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2024 (Acceptance rate = 24.5%)
  

• Towards Human-Like RL: Taming Non-Naturalistic Behavior in Deep RL via Adaptive Behavioral Costs in 3D Games
  Kuo-Hao Ho, Ping-Chun Hsieh, Chiu-Chou Lin, You-Ren Luo, Feng-Jian Wang, and I-Chen Wu
  Asian Conference on Machine Learning (ACML), 2023 (Long Oral; Acceptance rate = 35%)

• Neural Frank-Wolfe Policy Optimization for Region-of-Interest Intra-Frame Coding with HEVC/H.265
  Yung-Han Ho, Chia-Hao Kao, Wen-Hsiao Peng, and Ping-Chun Hsieh
  IEEE Visual Communications and Image Processing (VCIP), 2022.

• Real-Time Streaming Graph Embedding Through Local Actions
  Xi Liu, Ping-Chun Hsieh, Nick Duffield, Rui Chen, Muhe Xie, and Xidao Wen
  DL4G-SDE Workshop at the World Wide Web Conference (WWW), 2019