Using PTB workflow patcher for the first time

[ckunki]

When adding PTB workflow patcher to project BFSPY, I collected the following observations

Feature: step_customizations, INSERT_BEFORE

step_customizations currently only provides REPLACE and INSERT_AFTER.

For example step "Allow unprivileged user namespaces" would be suitable to be added as first step of a job, which is currently not possible.

Request: Enable step_customizations, INSERT_BEFORE to add a step before the first one.

Doc/Feature: step_customizations, Missing ID

When using step_customizations then each inserted or replaced step needs to
have an ID otherwise you will get errors from pydantic like shown below.

Proposal:

• Either add this to the documentation or
• (better, if not too much effort) explicitly check for this error and report it separately in a nice and concise way

nox > Session workflow:generate raised exception InvalidWorkflowPatcherYamlError("File '/home/chku/git/bucketfs-python/.workflow-patcher.yml' is malformed; it failed Pydantic validation.")
Traceback (most recent call last):
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/exasol/toolbox/util/workflows/patch_workflow.py", line 137, in content
    WorkflowPatcherConfig.model_validate(loaded_yaml)
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/pydantic/main.py", line 716, in model_validate
    return cls.__pydantic_validator__.validate_python(
pydantic_core._pydantic_core.ValidationError: 1 validation error for WorkflowPatcherConfig
workflows.0.step_customizations.0.content.0.id
  Field required [type=missing, input_value={'name': 'Allow unprivile..._unprivileged_userns=0'}, input_type=CommentedMap]
    For further information visit https://errors.pydantic.dev/2.12/v/missing

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/nox/sessions.py", line 1168, in execute
    self.func(session)
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/nox/_decorators.py", line 99, in __call__
    return self.func(*args, **kwargs)
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/exasol/toolbox/nox/_workflow.py", line 40, in generate_workflow
    update_workflow(workflow_choice=args.workflow_choice, config=PROJECT_CONFIG)
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/exasol/toolbox/util/workflows/workflow.py", line 101, in update_workflow
    patch_yaml = workflow_patcher.extract_by_workflow(
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/exasol/toolbox/util/workflows/patch_workflow.py", line 149, in extract_by_workflow
    inner_content = self.content["workflows"]
  File "/usr/lib64/python3.10/functools.py", line 981, in __get__
    val = self.func(instance)
  File "/home/chku/.cache/pypoetry/virtualenvs/exasol-bucketfs-BNlHXJOS-py3.10/lib/python3.10/site-packages/exasol/toolbox/util/workflows/patch_workflow.py", line 140, in content
    raise InvalidWorkflowPatcherYamlError(file_path=self.file_path) from ex
exasol.toolbox.util.workflows.exceptions.InvalidWorkflowPatcherYamlError: File '/home/chku/git/bucketfs-python/.workflow-patcher.yml' is malformed; it failed Pydantic validation.

Feature: Use only a Single Python version in Integration Tests

How to define integration tests, not iterating all python versions?

This is not a step-customization, but changes the job definition.

Feature: Add Environment Variables / Secrets to step

Support adding environment variables / secrets to a step.

Feature: Empty Line

PTB Workflow patcher should insert an empty line between workflow steps, even
when replacing two consecutive steps.

Frequent Use Cases

Here are some more use cases I could imagine occurring more often.
If not adding a dedicated feature, then maybe a few lines in the user guide could help here?

• Different name of Sonar token SONARQUBE_GITHUB_SECRET Instead of SONAR_TOKEN) enabling private repositories to use Exasol's onprem Sonar service

The following use cases can probably covered by supporting individual variants of GitHub workflow slow-checks.yml

• Allow unprivileged user namespaces for running Exasol UDFs on GitHub Runners with Apparmor restrictions, e.g. ubuntu-24.04.

  - name: Allow unprivileged user namespaces
    run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0

• Free diskspace
• Add additional pytest parameters e.g. --backend
• Add environment variables, e.g. SAAS_HOST