PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data

Codebase for PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data

This repository accompanies the paper PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data, which will appear at TMLR.

Project File Tree

├── README.md
├── halfcheetah
│   ├── VLBM.py
│   ├── d4rl.ipynb  # OPE
│   ├── d4rl_policies.json
│   ├── dg_b.py  # data generation
│   ├── learned_log_w_IS.pkl
│   ├── learned_log_w_PDIS.pkl
│   ├── log_w_CP.pkl
│   ├── log_w_IS.pkl
│   ├── log_w_PDIS.pkl
│   ├── saved_model # VAE checkpoint
│   └── utils.py
├── inventory
│   └── inventory.ipynb  # Simulator & OPE
├── mimic-iv
│   ├── VLBM.py
│   ├── dg_b.py  # data generation 
│   ├── mimic.ipynb  # OPE
│   ├── mimic_iv_behavior_trajectories.pkl
│   ├── mimic_iv_target_trajectories.pkl
│   ├── mimic_iv_trajectories_behavior.pkl
│   ├── mimic_iv_trajectories_target.pkl
│   ├── saved_model  # VAE checkpoint and behavior cloning policies
│   ├── skip_list.pkl
│   └── utils.py
└── sepsis
    ├── mdptoolboxSrc
    ├── sepsis.ipynb  # OPE
    └── sepsisSimDiabetes  # Simulator

Citation

If you use this code, please cite:

@misc{mandyam2025perrypolicyevaluationconfidence,
      title={PERRY: Policy Evaluation with Confidence Intervals using Auxiliary Data}, 
      author={Aishwarya Mandyam and Jason Meng and Ge Gao and Jiankai Sun and Mac Schwager and Barbara E. Engelhardt and Emma Brunskill},
      year={2025},
      eprint={2507.20068},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2507.20068}, 
}