$ cat about.txt

Siddarth Venkatraman

RL at Mistral · PhD at Mila, Quebec AI Institute

I’m an RL intern at Mistral AI and a final-year PhD student at Mila, co-supervised by Glen Berseth and Esmeralda Whitammer. I work closely with Yoshua Bengio and was an academic collaborator at LawZero, helping develop safe and controllable AI systems.

I also work with LLNL on scaling off-policy RL for large reasoning models. Recently, I finished an internship at Valence Labs, where I trained flow bridges for molecular systems.

My research focuses on the science of RL for LLMs and inference scaling. Right now, I’m most excited (and concerned) about self-improvement and long-horizon RL. I’m also fascinated by the Machine Minds we’re spawning. I want to understand them better.

ls research/

12 papers · 2020–2026

LE CRITIQUE: Privileged Value Functions for LLM Reinforcement Learning

Siddarth Venkatraman, Matthieu Dinot, Laurence Aitchison

Preprint, 2026

Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models

Siddarth Venkatraman*, Vineet Jain*, Sarthak Mittal*, Vedant Shah, Johan Obando-Ceron, Yoshua Bengio, Brian R. Bartoldson, Bhavya Kailkhura, Guillaume Lajoie, Glen Berseth, Nikolay Malkin, Moksh Jain

Preprint

A Comedy of Estimators: On KL Regularization in RL Training of LLMs

Vedant Shah, Johan Obando-Ceron, Vineet Jain, Brian Bartoldson, Bhavya Kailkhura, Sarthak Mittal, Glen Berseth, Pablo Samuel Castro, Yoshua Bengio, Nikolay Malkin, Moksh Jain*, Siddarth Venkatraman*, Aaron Courville*

COLM 2026

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training

Brian R. Bartoldson, Siddarth Venkatraman, James Diffenderfer, Moksh Jain, Tal Ben-Nun, Seanie Lee, Minsu Kim, Johan Obando-Ceron, Yoshua Bengio, Bhavya Kailkhura

NeurIPS 2025

Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models

Siddarth Venkatraman*, Mohsin Hasan*, Minsu Kim, Luca Scimeca, Marcin Sendera, Yoshua Bengio, Glen Berseth, Nikolay Malkin

ICML 2025

Siddarth Venkatraman

ls research/

LE CRITIQUE: Privileged Value Functions for LLM Reinforcement Learning

Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models

A Comedy of Estimators: On KL Regularization in RL Training of LLMs

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training

Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models

Amortizing Intractable Inference in Diffusion Models for Vision, Language and Control

Reasoning with Latent Diffusion in Offline Reinforcement Learning

Learning Temporally Abstract World Models without Online Experimentation

Multi-Alpha Soft Actor-Critic: Overcoming Stochastic Biases in Maximum Entropy RL

MLNav: Learning to Safely Navigate on Martian Terrains

Machine Learning Based Path Planning for Improved Rover Navigation

Deep Residual Neural Networks for Image in Audio Steganography

Siddarth Venkatraman

$ls research/

LE CRITIQUE: Privileged Value Functions for LLM Reinforcement Learning

Recursive Self-Aggregation Unlocks Deep Thinking in Large Language Models

A Comedy of Estimators: On KL Regularization in RL Training of LLMs

Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training

Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models

Amortizing Intractable Inference in Diffusion Models for Vision, Language and Control

Reasoning with Latent Diffusion in Offline Reinforcement Learning

Learning Temporally Abstract World Models without Online Experimentation

Multi-Alpha Soft Actor-Critic: Overcoming Stochastic Biases in Maximum Entropy RL

MLNav: Learning to Safely Navigate on Martian Terrains

Machine Learning Based Path Planning for Improved Rover Navigation

Deep Residual Neural Networks for Image in Audio Steganography

ls research/