pickling errors in torch.load when checkpoints are downloaded via curl

[Mohinta2892]

Hi @theodore-zhao

Wondering if you get pickling errors as below when using downloaded checkpoints via option 2:

usr/local/lib/python3.10/dist-packages/torch/utils/_pytree.py:185: FutureWarning: optree is installed but the version is too old to support PyTorch Dynamo in C++ pytree. C++ pytree support is disabled. Please consider upgrading optree using `python3 -m pip install --upgrade 'optree>=0.13.0'`.
  warnings.warn(
Using device: cuda
/mnt/graid/biomedparse/BiomedParse/run_new_example_from_config.py:16: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  hydra.initialize(config_path="configs/model", job_name="example_prediction")
/usr/local/lib/python3.10/dist-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'biomedparse_3D': Defaults list is missing `_self_`. See https://hydra.cc/docs/1.2/upgrades/1.0_to_1.1/default_composition_order for more information
  warnings.warn(msg, UserWarning)
/usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:942: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
  warnings.warn(
Traceback (most recent call last):
  File "/mnt/graid/biomedparse/BiomedParse/run_new_example_from_config.py", line 19, in <module>
    model.load_pretrained("model_weights/biomedparse_v2.ckpt")
  File "/mnt/graid/biomedparse/BiomedParse/src/model/biomedparse_3D.py", line 393, in load_pretrained
    checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)
  File "/usr/local/lib/python3.10/dist-packages/torch/serialization.py", line 1494, in load
    raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
Please file an issue with the following so that we can make `weights_only=True` compatible with your use case: WeightsUnpickler error: Unsupported operand 65

I have simply used the curl command and then changed this line to point to my on disk ckpt

model.load_pretrained("model_weights/biomedparse_v2.ckpt")

Also, changed the line to include weights_only, however that did not help.

    checkpoint = torch.load(checkpoint_path, map_location="cpu", weights_only=True)

[theodore-zhao]

Hi, did you try weights_only=False? I think the new default is True, so manually switching to False make work.

[Mohinta2892]

Hi, @theodore-zhao

yes, I did, and it did not resolve the issue. I am using the HF version for now, which pulls the weights everytime the script is run.
It was not a problem before when the weights extraction worked via the docker you had shared.

[theodore-zhao]

Can you try downloading the weights manually and see if that work?