Yuki Ichihara

Ph.D. student at MBZUAI

I am a Ph.D. student at MBZUAI, supervised by Junpei Komiyama.

I study how to make language-model generation more reliable and efficient, from decoding algorithms and statistical guarantees to reinforcement-learning methods for multi-objective alignment.

Email Google Scholar GitHub Paper Guides

Research

My work connects decoding, statistical reliability, and reinforcement learning to build language models that optimize what we actually care about.

Reliable Decoding

I study Best-of-N and Minimum Bayes Risk decoding, including when these methods work, how quickly they converge, and how to make them robust to imperfect rewards.

MBR guarantees Best-of-N

Multi-Objective Alignment

I develop reinforcement-learning methods that balance multiple rewards and constraints while reducing reward hacking and manual weight tuning.

MO-GRPO AW-GRPO Constrained RL

Efficient and Trustworthy Reasoning

I explore statistical signals and training objectives that make self-consistency and reasoning more reliable without unnecessary inference-time computation.

PDC OP²SD Prefix consistency CITE C-GRPO

Publications

Prefix-Denoising Consistency: Test-Time Verification for Diffusion Language Models Y. Ichihara, N. Iwase, M. A. Quamar, J. Komiyama. arXiv 2026 · Preprint Paper

Summary & figure

Prefix-Denoising Consistency (PDC) verifies diffusion-language-model outputs by keeping different prefixes, remasking the remaining suffix, and regenerating alternative reasoning trajectories. Across mathematical and commonsense reasoning benchmarks, voting over these regenerations improves the initial answer while using inference compute more effectively than independent full generations.

Figure 1. PDC regenerates masked suffixes from several fixed prefixes and votes over the resulting answers.
Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillation Y. Ichihara, N. Iwase, M. A. Quamar, J. Komiyama. arXiv 2026 · Preprint Paper Code

Summary & figure

OP²SD tests whether on-policy self-distillation truly requires the teacher to see the target problem's solution. Replacing it with a worked solution from another problem still improves the base model and remains competitive with OPSD, showing that context-induced teacher behavior is an important part of the distilled signal.

Figure 1. OPSD and OP²SD differ only in whether the teacher sees the target solution or another problem's solution.
Reliable Chain-of-Thought via Prefix Consistency N. Iwase, Y. Ichihara, M. A. Quamar, J. Komiyama. arXiv 2026 · Preprint Paper Code

Summary & figure

Prefix consistency estimates the reliability of a chain-of-thought by truncating it and checking whether regenerated continuations recover the same answer. Across reasoning models and benchmarks, it predicts correctness well and reaches standard majority-voting accuracy with substantially fewer tokens.

Figure 1. Prefix-consistency-weighted majority voting and the regeneration-based reliability signal.
CITE: Anytime-Valid Statistical Inference in LLM Self-Consistency H. Ota, N. Iwase, Y. Ichihara, J. Komiyama, M. Imaizumi. arXiv 2026 · Preprint Paper

Summary & figure

CITE provides anytime-valid statistical certification that a target response is the unique mode of an LLM's answer distribution. It controls false certification under adaptive stopping without assuming a fixed or known set of possible answers.

Figure 1. CITE's anytime-valid stopping rule for certifying a target response.
Consensus Group Relative Policy Optimization for Text Generation Y. Ichihara, Y. Jinnai, K. Ariu, E. Uchibe. EMNLP 2026 · Main Paper Code

Summary & figure

C-GRPO distills Minimum Bayes Risk decoding into GRPO training using only a utility function and policy-generated samples. It achieves performance comparable to MBR decoding in translation and summarization without its inference-time overhead.

Figure 1. Standard GRPO and the label-free consensus utility used by C-GRPO.
MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems Y. Ichihara, Y. Jinnai, T. Morimura, M. Sakamoto, R. Mitsuhashi, E. Uchibe. TACL 2026 · Journal Paper Code Guide

Summary & figure

MO-GRPO addresses reward hacking in multi-objective GRPO by automatically normalizing rewards according to their variance. It balances objectives without manual scale tuning and delivers more stable learning across control and text-generation tasks.

Figure 1. GRPO and MO-GRPO advantage functions when reward scales differ.
Auto-Weighted Group Relative Preference Optimization for Multi-Objective Text Generation Tasks Y. Ichihara, Y. Jinnai. EMNLP 2025 · Industry Paper Code

Summary & figure

AW-GRPO automatically adjusts multiple reward weights according to each objective's learning progress. On advertising text generation and public benchmarks, it improves the balance among objectives while reducing constraint violations.

Figure 1. Standard GRPO overfits one reward on WMT En-Ja, motivating automatic weighting.
Theoretical Guarantees for Minimum Bayes Risk Decoding Y. Ichihara, Y. Jinnai, K. Ariu, T. Morimura, E. Uchibe. ACL 2025 · Main Paper Guide

Summary & figure

This work provides finite-sample guarantees showing that MBR decoding approaches the optimal solution at a rate of O(n^−1/2) under stated assumptions. It also explains why MBR can converge faster than MAP decoding in several cases.

Figure 1. Regimes in which the regret bound of MBR or MAP decoding is smaller.
Evaluation of Best-of-N Sampling Strategies for Language Model Alignment Y. Ichihara, Y. Jinnai, T. Morimura, K. Abe, K. Ariu, M. Sakamoto, E. Uchibe. TMLR 2025 · Journal Paper

Summary & figure

This work studies regularized Best-of-N sampling as robust optimization against errors in proxy rewards. It introduces Stochastic RBoN with theoretical guarantees and shows how simple regularizers can reduce reward hacking on alignment benchmarks.

Figure 1. Sensitivity of regularized Best-of-N methods to beta across proxy reward models.
A Policy Gradient Primal-Dual Algorithm for Constrained MDPs with Uniform PAC Guarantees T. Kitamura, T. Kozuno, M. Kato, Y. Ichihara, S. Nishimori, A. Sannai, S. Sonoda, W. Kumagai, Y. Matsuo. RLSW 2024 · Workshop Paper

Summary & figure

This work proposes a policy-gradient primal-dual algorithm for online constrained MDPs with Uniform-PAC guarantees. It combines convergence to an optimal policy, sublinear regret, and polynomial sample complexity while controlling constraint violations.

Figure 1. Optimality gap and constraint violation of primal-dual CMDP algorithms.