Revisiting Value Iteration: Unified Analysis of Discounted and Average-Reward Cases
arXiv:2510.23914v2 Announce Type: replace Abstract: While Value Iteration (VI) is one of the most fundamental algorithms in Reinforcement Learning, its theoretical convergence guarantees still exhibit a persistent mismatch with empirical behavior. In the discounted-reward case, classical theory guarantees geometric convergence…
