reinforcement-learning optimal-control mathematical-foundations diffusion-models

Hamilton-Jacobi-Bellman Equation: Reinforcement Learning and Diffusion Models

dani2442.github.io

March 30, 2026

16 min read

🔥🔥🔥🔥🔥

56/100

Summary

Richard Bellman's 1952 paper established the foundation for optimal control and reinforcement learning. His later work in the 1950s connected continuous-time systems to a previously published physical result from the 1840s, formulating the optimal condition as a partial differential equation (PDE).

Key Takeaways

Richard Bellman published a foundational paper on dynamic programming in 1952, which laid the groundwork for optimal control and reinforcement learning.
The Hamilton-Jacobi-Bellman (HJB) equation, derived from Bellman's work, has the same mathematical structure as the Hamilton-Jacobi equation from classical mechanics.
Continuous-time reinforcement learning and diffusion models can be interpreted through the lens of stochastic optimal control, utilizing the principles established by Bellman.
The value function in reinforcement learning satisfies the Bellman equation, which maximizes immediate rewards plus continuation value.

Read original article

View story →