[BUG] PPOLag numerically unstable with zero cost_limit in safe environments

[Mi-729]

Required prerequisites

• I have read the documentation https://omnisafe.readthedocs.io.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

What version of OmniSafe are you using?

0.5.0

System information

omnisafe was installed by "pip install -e ."
Python version: 3.9.25 (main, Nov 3 2025, 22:44:01) [MSC v.1929 64 bit (AMD64)] win32
omnisafe version: 0.5.0

Problem description

When training PPOLag Agent with cost_limit=0.0 or a very small values in relatively safe environments like "SafetyPointGoal1-v0", the algorithm crashes with an AssertionError.
The error occurs in omnisafe/algorithms/on_policy/naive_lagrange/ppo_lag.py, line 74:

Jc = self._logger.get_stats('Metrics/EpCost')[0]
assert not np.isnan(Jc), 'cost for updating lagrange multiplier is nan'

Root cause analysis:
When cost_limit is zero and the environment produces near-zero costs, numerical operations (likely division by zero or log(0) in Lagrange multiplier updates) produce NaN values in the cost computation.

Reproducible example code

reproduce_nan_issue.py

Command lines:

python reproduce_nan_issue.py

It's 100% reproducible with the provided code.

Traceback

Traceback (most recent call last):
  File "D:\Users\alanr\RL_Projects\omnisafe\reproduce_nan_issue.py", line 45, in <module>
    reproduce_nan_bug()
  File "D:\Users\alanr\RL_Projects\omnisafe\reproduce_nan_issue.py", line 42, in reproduce_nan_bug
    agent.learn()
  File "D:\Users\alanr\RL_Projects\omnisafe\omnisafe\algorithms\algo_wrapper.py", line 180, in learn
    ep_ret, ep_cost, ep_len = self.agent.learn()
  File "D:\Users\alanr\RL_Projects\omnisafe\omnisafe\algorithms\on_policy\base\policy_gradient.py", line 268, in learn
    self._update()
  File "D:\Users\alanr\RL_Projects\omnisafe\omnisafe\algorithms\on_policy\naive_lagrange\ppo_lag.py", line 74, in _update
    assert not np.isnan(Jc), 'cost for updating lagrange multiplier is nan'
AssertionError: cost for updating lagrange multiplier is nan

Expected behavior

No response

Additional context

No response

[Mi-729]

Hi @zmsn-2077 , thanks for your attention.

As the reporter who's familiar with the problem, I'd like to contribute the fix. I've already:

1. Identified the exact location: ppo_lag.py line 74
2. Prepared a fix approach (epsilon protection)
3. Can implement and add tests

Would you like me to proceed and submit a PR for your review?

[Mi-729]

Submitted a fix for this issue in PR #385.

Main fixes:

• Added NaN detection and handling in online_adapter.py
• Fixed assertion and gradient computation issues in ppo_lag.py
• Added debugging scripts for reproducing the issue

Training should now proceed normally without NaN errors or assertion failures.
@zmsn-2077 Please review when you have time. Thanks!