Some questions about Webshop enviroment

[zijianh4]

Hi,
Thanks for your great work. I'm currently trying to run some experiments and everything is working well except for the Webshop enviroment. I run the setup_webshop.sh script with these 2 lines commented out:

sudo apt update
sudo apt install default-jdk

Since I'm running in a node in a SLURM system and I don't have the sudo permission. Then, I run the webshop experiment with the following command:
python train.py --config-name _6_webshop system.CUDA_VISIBLE_DEVICES=\'0,1,2,3\' model_path=Qwen/Qwen2.5-3B-Instruct trainer.project_name=ragen_webshop trainer.experiment_name=no_replay_3B trainer.total_training_steps=500 trainer.n_gpus_per_node=4 actor_rollout_ref.rollout.tensor_model_parallel_size=4
and the program just stucks at the step

CUDA_VISIBLE_DEVICES: 0,1,2,3
2025-08-24 15:16:13,248 INFO worker.py:1918 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265

Could you please help me with this issue? Thanks!

Update:
Before I run the setup_webshop.sh script, other environments are okay, but they are also failed after I run this setup script.

[ZihanWang314]

Hi @zijianh4 , we have found that this would be due to the parallel environment setup and have reverted that PR. could you please try again using the latest main branch?

[zijianh4]

Hi @ZihanWang314 ,
Thanks for your response. I run the command MKL_SERVICE_FORCE_INTEL=1 python train.py --config-name _6_webshop system.CUDA_VISIBLE_DEVICES=\'0,1,2,3\' model_path=Qwen/Qwen2.5-3B-Instruct trainer.project_name=ragen_webshop trainer.experiment_name=no_replay_3B trainer.total_training_steps=500 trainer.n_gpus_per_node=4 actor_rollout_ref.rollout.tensor_model_parallel_size=4I and get a new error like this:

Error executing job with overrides: ["system.CUDA_VISIBLE_DEVICES='0,1,2,3'", 'model_path=Qwen/Qwen2.5-3B-Instruct', 't
rainer.project_name=ragen_webshop', 'trainer.experiment_name=no_replay_3B', 'trainer.total_training_steps=500', 'train$
r.n_gpus_per_node=4', 'actor_rollout_ref.rollout.tensor_model_parallel_size=4']
Traceback (most recent call last):
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/train.py", line 158, in main
    run_ppo(config)
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/train.py", line 179, in run_ppo
    ray.get(runner.run.remote(config))
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/ray/_private/auto_init_hook.py", l$
ne 22, in auto_init_wrapper
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/ray/_private/client_mode_hook.py",
line 104, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/ray/_private/worker.py", line 2858$
 in get
    values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/ray/_private/worker.py", line 958,
in get_objects
    raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(AttributeError): ray::TaskRunner.run() (pid=1901096, ip=10.164.15.22, actor_id=c5655ddd1f8$
52348ffb2f4201000000, repr=<train.TaskRunner object at 0x1543ded4f380>)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/train.py", line 296, in run
    trainer.init_workers()
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/ragen/trainer/agent_trainer.py", line 369, in init_workers
    self.critic_wg.init_model()
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/verl/verl/single_controller/ray/base.py", line 49, in func
    output = ray.get(output)
             ^^^^^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^^^
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ray.exceptions.RayTaskError(AttributeError): ray::WorkerDict.critic_init_model() (pid=1904240, ip=10.164.15.22, actor_i
d=aac755c2b28a080d2b5bb6f101000000, repr=<verl.single_controller.ray.base.WorkerDict object at 0x143780c98530>)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1
928, in __getattr__
    raise AttributeError(
AttributeError: 'LoraModel' object has no attribute 'prepare_inputs_for_generation'

During handling of the above exception, another exception occurred:

ray::WorkerDict.critic_init_model() (pid=1904240, ip=10.164.15.22, actor_id=aac755c2b28a080d2b5bb6f101000000, repr=<ver
l.single_controller.ray.base.WorkerDict object at 0x143780c98530>)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/verl/verl/single_controller/ray/base.py", line 459, in func
    return getattr(self.worker_dict[key], name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/verl/verl/single_controller/base/decorator.py", line 465, in inn
er
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/ragen/workers/fsdp_workers.py", line 984, in init_model
    self.critic_module, self.critic_optimizer, self.critic_lr_scheduler = self._build_critic_model_optimizer(self.confi
g)
                                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^
  File "/gpfs/accounts/jiasi_root/jiasi0/zijianh/RAGEN/ragen/workers/fsdp_workers.py", line 885, in _build_critic_model
_optimizer
    critic_module = get_peft_model(critic_module, LoraConfig(**lora_config))
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/peft/mapping_func.py", line 125, in
 get_peft_model
    return MODEL_TYPE_TO_PEFT_MODEL_MAPPING[peft_config.task_type](
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/peft/peft_model.py", line 1816, in
__init__
    self.base_model_prepare_inputs_for_generation = self.base_model.prepare_inputs_for_generation
                                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/peft/tuners/lora/model.py", line 38
6, in __getattr__
    return getattr(self.model, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nfs/turbo/coe-mavens/zijianh/conda-envs/ragen/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1
928, in __getattr__
    raise AttributeError(
AttributeError: 'Qwen2ForTokenClassification' object has no attribute 'prepare_inputs_for_generation'. Did you mean: '_
return_value'?

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

This seems related to this issue #100. If I run without MKL_SERVICE_FORCE_INTEL=1, I will be stuck at the initialization step.