Skip to content
This repository was archived by the owner on Mar 9, 2026. It is now read-only.

Techdebt refatorar modulo de pre processamento codex neodv 1697#100

Draft
ceciliacsilva wants to merge 12 commits into
techdebt-refatorar-modulo-de-adminclient-do-codex-neodv-2150from
techdebt-refatorar-modulo-de-pre-processamento-codex-neodv-1697
Draft

Techdebt refatorar modulo de pre processamento codex neodv 1697#100
ceciliacsilva wants to merge 12 commits into
techdebt-refatorar-modulo-de-adminclient-do-codex-neodv-2150from
techdebt-refatorar-modulo-de-pre-processamento-codex-neodv-1697

Conversation

@ceciliacsilva

@ceciliacsilva ceciliacsilva commented Oct 22, 2025

Copy link
Copy Markdown

Description

This PR:

  • refactors sync preprocessing interface and implementation
  • refactors async preprocessing v1 interface and implementation

Related issues

How to test it (sync preprocessing)

  • install codex dependencies/virtualenv.
  • python -i src/mlops_codex/sync_preprocessing/client.py
>>> auth = {"bearer_token": <token>}
>>> >>> sp = {'name': 'do-codex', 'script_reference': 'build_df', 'python_version': 'Python310', 'source_file_path': '/path/neomaril/samples/BreastCancer/preprocessing/script.py', 'requirements_file_path': '/path/neomaril/samples/BreastCancer/preprocessing/requirements.txt', 'schema_file_path': '/path/neomaril/samples/BreastCancer/preprocessing/schema.json', 'group_name': 'datarisk'}
>>> spc = SyncPreprocessing(**sp)
>>> client = SyncPreprocessingClient(**auth)
>>> client.register(spc)
>>> client.host()
>>> client.get_status()
>>> client.describe()
>>> client.search()
>>> client.use_existing_preprocessing(client.search()[-1]) # or other position to set.

How to test it (async preprocessing)

  • install codex dependencies/virtualenv.
  • python -i src/mlops_codex/async_preprocessingV1/client.py
>>> auth = {"bearer_token": <token>}
>>> client = AsyncPreprocessingClientV1(**auth)
# na versão atual do server não é mais possível criar asyncpreproc V1, se não tiver nenhum para testar essa parte do fluxo, atualize a feature flag `ClassicPreprocessing.AsyncCreation` para verdadeiro e crie um preproc async v1 manualmente. Codex não irá oferecer a função de registro.
>>> client.use_existing_preprocessing(client.search()[-1]) # or other position to set.
>>> client.host()
>>> client.get_status()
>>> client.describe()
>>> client.search()

@ceciliacsilva ceciliacsilva self-assigned this Oct 22, 2025
@linear

linear Bot commented Oct 22, 2025

Copy link
Copy Markdown

@ceciliacsilva ceciliacsilva changed the base branch from master to techdebt-refatorar-modulo-de-adminclient-do-codex-neodv-2150 October 22, 2025 18:06
Comment thread src/mlops_codex/sync_preprocessing/client.py
Comment on lines +15 to +26
class InvalidPreprocessingError(Exception):
"""Invalid preprocessing error."""

def __init__(
self,
message: str = "Not a valid Neomaril preprocessing instance. `register` or `use_exisisting_preprocessing`.",
) -> None:
super().__init__(message)
self.message = message

def __str__(self):
return f"Could not send request, invalid Neomaril preprocessing instance."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think we create a python package for exception and centralize this?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A package with just exceptions? Is this the pattern in python?

Comment thread src/mlops_codex/preprocessing/client.py Outdated
Comment on lines +29 to +34
class SyncPreprocessingClient:
"""
Preprocessing base client.
"""

def __init__(self, token: str):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I think we should use pydantic even though we won't use all the feautures

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will change.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about this one, if I do this change I cannot make __bearer_token a private attribute.

Comment on lines +38 to +45
def register(self, sp: SyncPreprocessing) -> NeomarilSyncPreprocessing:
"""
List all MLOps groups the user has access to.

Returns:
list[str]: List of MLOps groups the user has access to.

"""

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my opinion, we should not shorten terms because it should be descriptive for the user.
Also, missing the parameter sp should be in the docstring because sphinx will generate the docs based on the docstrings

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc strings are wrong, you can ignore them for now.


Attributes:
name (str): human identifier.
python_version (str): a Neomaril supported python version.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should inform the available options

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should link neomaril doc here.

@ceciliacsilva ceciliacsilva force-pushed the techdebt-refatorar-modulo-de-pre-processamento-codex-neodv-1697 branch from a4ea774 to a420abb Compare October 29, 2025 12:07
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants