Skip to content

course and workshop material and cli enhancement - #42

Merged
lnauta merged 92 commits into
sara-nl:masterfrom
mherkazandjian:mher/spd-512/course-material
Sep 19, 2025
Merged

course and workshop material and cli enhancement#42
lnauta merged 92 commits into
sara-nl:masterfrom
mherkazandjian:mher/spd-512/course-material

Conversation

@mherkazandjian

@mherkazandjian mherkazandjian commented Aug 18, 2025

Copy link
Copy Markdown
Collaborator

This pull request introduces notebook based tutorials for the upcoming workshop. The notebooks follow the examples of the README and are appended with a walk through.
In addition to the tutorials a local development couchdb environment is added that is optional and can be used for local testing and development of the Picas client and CouchDB integration. It adds a Makefile for workflow automation, a Docker Compose configuration for CouchDB, a shell script to automate CouchDB initialization.

Tutorials and onboarding:

  • Added `examples/00,01,02,03,04....ipynb . The notebooks are rendered into .ipynb files from .py files using jupytext.
  • The examples are:
    - 00-environment-setup.ipynb
    - 01-database-setup.ipynb
    - 02-local-run.ipynb
    - 03-spider-run.ipynb
    - 04-slurm-snellius.ipynb

Development workflow automation:

  • Added a new Makefile with targets for building, installing, cleaning, testing, running tutorials, and managing Docker Compose services, enabling streamlined local development and reproducible builds.

CouchDB service and initialization:

  • Introduced a docker-compose.yml for running CouchDB and a setup container, including extensive documentation for common CouchDB operations and user/database setup.
  • Added init_couchdb_user.sh script to automate CouchDB initialization: waits for CouchDB readiness, creates database and user if missing, configures security, and sets up the Monitor view.

Add the picas-cli executable helper / wrapper script to initialize a configuration and change the couchdb password

Fix many style issues and add annotations and improve the documentation

Comment thread examples/notebooks/00-environment-setup.ipynb Outdated
Comment thread examples/notebooks/00-environment-setup.ipynb Outdated
Comment thread examples/notebooks/00-environment-setup.ipynb
Comment thread examples/notebooks/01-database-setup.ipynb
Comment thread examples/notebooks/01-database-setup.ipynb
Comment thread examples/notebooks/01-database-setup.ipynb
@mherkazandjian

Copy link
Copy Markdown
Collaborator Author

@lnauta i think i addressed all your comments. tnx alot :)

Comment thread examples/notebooks/01-database-setup.ipynb

@hailihu hailihu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed the changes related to poetry/packaging and tested the short local example. Some comments and questions.

Comment thread setup.py Outdated
Comment thread README.md Outdated
Comment thread README.md Outdated
Comment thread examples/create_views.py
Comment thread Makefile
Comment thread README.md
Comment thread README.md
Comment thread pyproject.toml
couchdb = "^1.2"
stopit = "^1.1.2"
setuptools = "^66"
setuptools = "^80"

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why, but i am getting this warning now when pushing tokens to the DB (or any command that connects to DB):

% python push_tokens.py quick
.../lib/python3.10/site-packages/couchdb/__init__.py:15: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  __version__ = __import__('pkg_resources').get_distribution('CouchDB').version

Could this be because of the version change in setuptools?

@xinan1911

Copy link
Copy Markdown
Contributor

Check the notebooks and it looks fine. Need to test them again after the merge.

@lnauta
lnauta merged commit 4c60fa6 into sara-nl:master Sep 19, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants