Clean RL is a great library if you're looking to get started doing some deep reinforcement learning! That plus gymnasium are pretty standard.

It's good for the world if we keep publishing improvements and optimizations to understandable primitives.

I am curious why not contribute back upstream, though.

This is awesome Vincent, Tensordict x CleanRL x torch.compile is the most ambitious crossover

This looks awesome. CleanRL has been incredibly useful for some of my students starting out in RL. Adding Pytorch's compilation capabilities is a fantastic addition.

Very cool! How does the optimized Pytoch code compare to the Jax implementation?

Wow this looks clean (no pun intended). Great speedups as well!