how to train on multi-node

How to write sbatch script to run GRPO with vLLM on multiple nodes? What should be --num_processes?

such as 2 nodes, 16 gpus.

btw, how to train with resume? just set model_name_or_path to the path where trained ckpt saved?

@dhcode-cpp

How to write sbatch script to run GRPO with vLLM on multiple nodes? What should be --num_processes?

@dhcode-cpp hi, can you help me out?

Not supported in the short term.

Navigation Menu