Most notebooks in this repo presume running in a Python environment where Apache Spark, Python modules and RAPIDS Spark jars are pre-installed
This issue proposes to assume the minimum, ideally an empty Python virtual environment. Ideally the instruction should read only:
Create a new venv
Open notebook in Jupyter, VSCode with Jupyter Extensions, Hosted Notebook environment
and start executing the notebook.
All additional packages are installed when needed with various kernel magics
For an example of a notebook striving to achieve this, see https://github.com/NVIDIA/spark-rapids-examples/blob/d10106573bd5ad18cb65d956d92033fd5d9bf05e/examples/SQL%2BDF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb
Most notebooks in this repo presume running in a Python environment where Apache Spark, Python modules and RAPIDS Spark jars are pre-installed
This issue proposes to assume the minimum, ideally an empty Python virtual environment. Ideally the instruction should read only:
Create a new venv
Open notebook in Jupyter, VSCode with Jupyter Extensions, Hosted Notebook environment
and start executing the notebook.
All additional packages are installed when needed with various kernel magics
For an example of a notebook striving to achieve this, see https://github.com/NVIDIA/spark-rapids-examples/blob/d10106573bd5ad18cb65d956d92033fd5d9bf05e/examples/SQL%2BDF-Examples/tpcds/notebooks/TPCDS-SF10.ipynb