Training Hyperparameters for VMAS Reproduction

[Square596]

Hello,

Thank you for this helpful repository! I’m trying to reproduce the results of experiments from one of your papers, VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learning, using TorchRL. Specifically, I’m focusing on reproducing the results shown in Figure 4, but I haven’t been able to find certain training hyperparameters (e.g., number of epochs per iteration, batch size, optimizer, learning rate, GAE lambda) in the paper or code.

Could you provide any details on these?

Thank you for your help!

[matteobettini]

Hello!

Thanks for openeing this.

So, those experiment were run using RLLib with a setup similar to the one here (https://github.com/proroklab/HetGPPO/blob/main/train/train_give_way.py).

I think the hyperparameters should be those, the one i linked is give_way but other scenarios should be similar.

Since torchrl is different from rllib i would first try to reproduce in the original rllib setup and then try to match that with the colsest one in rllib

The H1 section of this paper https://matteobettini.com/publication/torchrl-a-data-driven-decision-making-library-for-pytorch/TorchRL-A-data-driven-decision-making-library-for-PyTorch.pdf might help with the mapping of hyperparameters from rllib to torch rl