per-user config

[gswifort]

I think it would be a good idea to introduce per-user configuration (in addition to per-environment):

• pyrx.ini file with settings such as PYRX_DISABLE_ONLOAD or PYRX_LOAD_REPL
• an additional onload.py file that will be loaded independently of the pyrx_onload.py file located in the local environment search path.

All of this could be handled by the _host_init.py module.

Issue: #289

[CEXT-Dan]

Okay, though I feel onload.py by itself is ambiguous. maybe something like pyrx_oninit or pyrx_on_user_init, pyrx_user_init?
I have references to the old pyrx.ini for C++ debug, but I don’t think I need it anymore, I’ll either remove it or rename it

[gswifort]

Ok, so maybe pyrx_init.py. I wouldn't like to include the word user in the name because in the future other locations may be added (usually such configurations can be on three levels: system, user, working directory, but I don't know if it's needed here)

[CEXT-Dan]

sounds great!

[gswifort]

I wonder if PYRX_DISABLE_ONLOAD should also disable loading pyrx_init.py

[CEXT-Dan]

Sounds like a good idea from a support perspective, people do weird stuff and have files all over the place. Having one switch to remove items that could be causing trouble seems like a good thing

[gswifort]

Settings contained in pyrx.ini should be identical to the settings at the level of environment variables (environment variables overwrite settings from the ini file).

I know about these variables:

• PYRX_LOG
• PYRX_DISABLE_ONLOAD
• PYRX_LOAD_REPL

Is there anything else?

[CEXT-Dan]

I also see the following in acedGetEnv (AutoCAD variables)
PYRX_ONLOAD_PATH
PYRX_LOG

I can remove PYRX_ONLOAD_PATH so its handled in _host_init.py
I the case of debugging the install of Python, PYRX_LOG would never be hit if there’s a problem loading python, right?

[CEXT-Dan]

edit
Not remove, but move PYRX_ONLOAD_PATH from
acedGetEnv
to
GetEnvironmentVariable

[gswifort]

I the case of debugging the install of Python, PYRX_LOG would never be hit if there’s a problem loading python, right?

Yes, I understand that you mean that there is no point in setting it in pyrx.ini? I guess you are right

[CEXT-Dan]

I moved these from acedGetEnv to GetEnvironmentVariable
PYRX_DISABLE_ONLOAD
PYRX_ONLOAD_PATH

I removed legacy PyRx.ini I used for debugging

would you like a build?

[gswifort]

would you like a build?

No, I'm not doing that yet, I'm just looking for ideas for now

[gswifort]

would you like a build?

Now is probably a good time :)

[CEXT-Dan]

I just made v2.2.7, I'll be out of town for the long weekend

[gswifort]

I think the pyrx_onload.py file loading should also be moved to _host_init.py

[CEXT-Dan]

Okay, just a note, it’s a real pain for me to work in branches. My debug builds are not committed, so it’s 30 minutes per switch. I would much rather work in main :)

[gswifort]

can't you stash them?

[CEXT-Dan]

Oh, I'll try. wasn't bad this time, only four minutes

[rdesparbes]

Okay, just a note, it’s a real pain for me to work in branches. My debug builds are not committed, so it’s 30 minutes per switch. I would much rather work in main :)

Do we need to add some folders to the .gitignore to ease this process?

[CEXT-Dan]

I have build folders that are currently ignored, my old 9900k is whining and moaning compiling 😁
I didn’t think about using stash, so I’ll try that.

[CEXT-Dan]

I noticed that HostApplicationServices.findFile throws on error
A similar function Ed.Core.findFile does not, I believe just returns an empty string on error

[gswifort]

I added settings (pyrx.config) and loading of the pyrx_onload.py file on the Python side: https://github.com/CEXT-Dan/PyRx/tree/289-per-user-config
I slightly changed the concept discussed earlier; it seems more understandable to me, but this is open for discussion. Instead of adding a new type of file (pyrx_init.py), I also used the pyrx_onload.py file in the user directory.

There are three levels of loading the pyrx_onload.py file:

• debug: located next to the ARX modules
• local: located in the host's search path (Ed.Core.findFile)
• user: located in the %appdata%\pyrx\pyrx_onload.py path

By default, all files are loaded in the order listed above (one from each level).
PYRX_DISABLE_ONLOAD (or disable_onload in pyrx.toml) disables loading from the local and user levels, while the debug level is always loaded (if it exists). I assume that the debug level will only be used by C++ developers and should not be used by end users.

Loading pyrx_onload.py files from subsequent levels can be disabled in the pyrx_onload.py file of the currently executed level. For example, if we want to load the debug level and skip the subsequent levels, we can include the following in the debug level's pyrx_onload.py file:

from pyrx.config import get_pyrx_settings, set_pyrx_settings

s = get_pyrx_settings()
s.disable_onload = True
set_pyrx_settings(s)

Settings are retrieved before loading each level, so changes to the settings introduced at a given level will affect the loading of files from subsequent levels.

To load pyrx_onload.py files, I used runpy.run_path instead of Ap.Application.loadPythonModule, as it does not have issues with loading multiple files with the same name. The same function could successfully be used to implement PYLOAD and PYRELOAD.

The current implementation is incomplete and is intended for testing purposes only. For example, there is no direct way to reload pyrx_onload files loaded in this manner.

To display settings do: python -m pyrx.config

[CEXT-Dan]

I’m not sure I fully understand pyrx_onload.py, the important thing is that the user’s version is loaded with the highest priority.
We have users that are currently using “AppData\Roaming\Bricsys\BricsCAD\V25x64\en_US\Support\pyrx_onload.py
As long as we don’t break that

[gswifort]

Yes, it is included in the local level (although in this case this name may not be the best) - it will be loaded anyway if it is in the ACAD search path or set in PYRX_ONLOAD_PATH (I forgot to write this, this variable is checked before searching with findFile).

Once again - by default files from all levels are loaded (debug, local, user), but only one for a given level (i.e. the first path found by findFile)

[gswifort]

But in my opinion files in such locations are from the user category and after merging this branch they should be moved by users to %appdata%\pyrx because the local level should be reserved mainly for the current working directory.

I see it like this:

• user: general tool, useful in every project or when drawing
• local: we start the host from the project directory we are working on and e.g. test commands are loaded

[CEXT-Dan]

I don’t agree, CAD users are very used to using their support paths, they have their on_start.lsp, on_docload.lsp, hatch patterns, fonts, etc,
The instructions for pyrx_onload.py should only be “put it in your support path”
The idea is PyRx integrate as much as possible into CAD.

[gswifort]

You are right - especially if we are thinking about users who do not create separate projects based on pyrx. Those who use pyrx in many contexts will probably want to use it as I shown above. Still, these are just recommendations, the pyrx_onload.py file will be loaded regardless of whether it is in the search path or in %appdata%\pyrx.

[CEXT-Dan]

An example is this thread
https://www.cadtutor.net/forum/topic/97450-export-table-with-blocks-to-excel-with-python/#findComment-666591

aridzv, just wanted to have the routine load at startup. He was trying to load python modules in on_start.lsp. Its so easy to just say
“Make a file pyrx_onload.py and put it in the same folder as on_start.lsp”

[CEXT-Dan]

I don’t see many things that are of interest to users in PyConfig
optimization_level is like the only one I see
What will this file contain? Also whats format, toml??

[gswifort]

yes, toml, example:

disable_onload = false
load_repl = true
onload_path = "path/to/your/pyrx_onload.py"

[gswifort]

mainly this file will contain settings used for pyrx initialization (in _host_init.py and in c++).

[gswifort]

@CEXT-Dan, so you don't like python -m pyrx.config?

[gswifort]

I would rather not call python.exe. we can read the files directly if they are toml

ok, but I think we should also read environment variables (which have higher priority).

The following configurations should be equivalent:

# pyrx.toml
disable_onload = false
load_repl = true
onload_path = 'path/to/your/pyrx_onload.py'
optimization_level = 2

# env
PYRX_DISABLE_ONLOAD=0
PYRX_LOAD_REPL=1
PYRX_ONLOAD_PATH=path/to/your/pyrx_onload.py
PYRX_OPTIMIZATION_LEVEL=2

[gswifort]

I would rather not call python.exe. we can read the files directly if they are toml

out of curiosity, why?

[CEXT-Dan]

I would rather not call python.exe. we can read the files directly if they are toml

out of curiosity, why?

Of course, No C/C++ programmer would, or should, it’s bad practice, bad program design.
I would have nightmares of Bjarne Stroustrup saying “have you learned nothing?”

I also feel like it’s an unnecessary security risk for a few variables we can easily do ourselves.

[gswifort]

I don't want this discussion to turn into a lecture on best practices, but I'm trying to understand how running a program that calls my own script negatively impacts security.

[CEXT-Dan]

I probably could have phrased that better.

It’s the act of calling any .EXE from a library, unattended, that I dislike, IMHO, its adding in an attack vector, could elevate the privileges of Python.exe in ways the user didn’t intend.
Calling EXEs, also causes little glitches I don’t like such as a screen flickers, possibly causing AutoCAD to lose focus

At the very least, it’s a high bar, and in my mind, a last resort. When I look our options for this task, the simplest solution is just to read the file

[CEXT-Dan]

In my mind, I would like to see something cool, for example, we can add a tab in the options dialog.
We save the settings to the user’s profile, then read the profile. the users can switch profiles launching different onload files

We wouldn’t need any INI or config files, and it’s all accessible from an interface the users are used to

[CEXT-Dan]

probably not the best option either, just food for thought

[gswifort]

cool option, but I'd still rather save settings (e.g. from the options window) to a pyrx.toml file (or some other text file). I work on PyRx almost every day, in different contexts - for each I need a different onload file and probably configuration (I assume there will be more configuration options in the future).

[CEXT-Dan]

Which folder are we going to use for the config,

On load
I can create the target folder, if it does not exist
copy pyrx.toml, from pyrx to the target folder, if it does not exist

[gswifort]

In my opinion:

1. findFile
2. Roaming Appdata

Both files should be read and findFile should have higher priority. What is not set in findFile should be retrieved from APPDATA.

[gswifort]

and of course all this is overwritten by environment variables

[gswifort]

You don't have to implement everything, in python it's easy (and already done). You can read only what you need (so for now optimization_level and log).

[CEXT-Dan]

I think the format should be categorized, example

[system]
optimization_level = 2

[user]
disable_onload = false
load_repl = true
onload_path = 'path/to/your/pyrx_onload.py'

[gswifort]

Sounds good.

[CEXT-Dan]

LOG is deferred; it’s in the loader module, I don’t want to add implementation to that project just yet.
I was actually thinking of removing the VAR and leaving it on

optimization_level can be overridden with text flags

[gswifort]

PyRx/PyRxCore/PyRxAppSettings.cpp

Lines 12 to 51 in e7055b7

	const std::tuple<bool, std::wstring> PyRxAppSettings::getOrCreateConfigPath()
	{
	static bool flag = false;
	static std::wstring foundPath;
	if (flag && foundPath.size() != 0)
	{
	return std::make_tuple(flag, foundPath);
	}
	std::error_code ec;
	wchar_t acstrPath[MAX_PATH];
	if (auto rt = acedFindFile(pyrx_config_name, acstrPath, MAX_PATH); rt == RTNORM)
	{
	flag = true;
	foundPath = std::wstring{ acstrPath };
	return std::make_tuple(flag, foundPath);
	}
	const auto roamingPath = PyRxApp::getAppDataPath() / pyrx_config_name;
	if (!std::filesystem::exists(roamingPath, ec))
	{
	#ifdef PYRXDEBUG
	std::filesystem::copy(SOLUTION_DIR / L"pyrx" / pyrx_config_name, roamingPath, ec);
	#else
	std::filesystem::copy(PyRxApp::modulePath() / pyrx_config_name, roamingPath, ec);
	#endif
	}
	if (!std::filesystem::exists(roamingPath, ec))
	{
	acutPrintf(_T("\nError to create config %ls %ls: "), __FUNCTIONW__, utf8_to_wstr(ec.message()).c_str());
	return std::make_tuple(false, std::wstring{ });
	}
	else
	{
	flag = true;
	foundPath = roamingPath.wstring();
	return std::make_tuple(flag, foundPath);
	}
	return std::make_tuple(false, std::wstring{ });
	}

If I understand correctly, after findFile finds a file (e.g. in the working directory), the file in APPDATA is no longer searched for? If the file in APPDATA contains the base configuration, and in the working directory only one variable that is to be overwritten, will this not work?

[CEXT-Dan]

Correct, it should be a single source, I have no intention of parsing multiple files searching for variables.
They can be already overridden with environment variables.

Acad already has a mechanism for switching states with profiles, we can add a search order.

• Profile
• Environment
• Config

optimization_level is not something that can be reset, it’s that way for the whole session
additionally, pyrx_onload.py is per session

[gswifort]

Ok, so I guess I'll have to change the behavior in python, I read all the config files and gradually fill in the config. I guess I'll have to give up on that to keep it consistent with c++.

[gswifort]

Although I still don't really like it 🫣 maybe @rdesparbes has some opinion on this..?

[rdesparbes]

Although I still don't really like it 🫣 maybe @rdesparbes has some opinion on this..?

I don't find multiple partial files particularly intuitive, it will probably cause some trouble. A single configuration at a well identified location + env variables + a .env file should give enough flexibility for most of use cases, no?
I totally support the choice of pydantic-settings on python's side, it's a great package.

[CEXT-Dan]

I also think we are looking at this from different perspectives, the way I see it

optimization_level
a system variable no end user is ever going to change

disable_onload
my view of this is mostly for support, if a user’s app is crashing, setting this would remove unknowns

load_repl
safty flag, the module introduced very negative behavior in AutoCAD, you can’t use the command line and it randomly crashes

onload_path
This one is interesting because of its utility.
My view is that users would never, or rarely use this, I think I added this so I could switch between loading my stuff for development. My intention was always to use find file. it’s also my intention this behave like acad.lsp Acaddoc.lsp
https://help.autodesk.com/view/ACDLT/2024/ENU/?guid=GUID-FDB4038D-1620-4A56-8824-D37729D42520
Ideally, there would only be one, located in their support path.

I think we’re not using pyrx_onload to its full potential, it in itself can load routines, per document with OnPyLoadDwg
It can have document reactors, command reactors, basically the full power of any arx app

[CEXT-Dan]

i did see a warning running the tests
you can set optimization_level=0, with bit 1 on , I.e.
./run_tests --pyinit 0x1 --no-slow --hosts acad

[gswifort]

good to know that pytest can handle it

[CEXT-Dan]

What steps are needed to get a basic pyrx_onload working? If its not ready yet, that’s fine, though I’ll need restore the C++ version so I can make releases

[gswifort]

I need to make some minor modifications due to your C++ implementation, I'll try to finish it today.

[CEXT-Dan]

I probably won’t make another release until next week. I've been goofing off 😁

[gswifort]

To load pyrx_onload.py files, I used runpy.run_path instead of Ap.Application.loadPythonModule, as it does not have issues with loading multiple files with the same name. The same function could successfully be used to implement PYLOAD and PYRELOAD.

@CEXT-Dan , what do you think about this? Regarding PYLOAD/PYRELOAD

[gswifort]

I don't know if we understand each other well. I don't want to change the current behavior. I just want to add loading of the %appdata%\pyrx\pyrx_onload_user.py file - this file should be loaded regardless of whether a pyrx_onload.py file is already loaded.

[CEXT-Dan]

I understand, I was thinking the names might be switched

IMHO, if we're going to have one with a reference to user, this would be the one that the user puts in their support path. developers ideally would avoid touching these files. however this is a break

It does not seem right to name the one in %appdata%\pyrx\pyrx_xxx_onload as user.

maybe
pyrx_base_onload
pyrx_onload_first

[gswifort]

Ok, I guess pyrx_base_onload.py
And the question is, which one should load first - it's possible that one will have some influence on the other (e.g. add or remove something from sys.path). It seems to me that pyrx_base_onload.py should load first.

[CEXT-Dan]

It seems to me that pyrx_base_onload.py should load first.

I agree

[gswifort]

Ok, I'll try to fix everything today.

[CEXT-Dan]

Missing pydantic-settings

Also I get

PyRx version <2.2.7.4216.20250430> loaded:
Python Interpreter Loaded successfully! :
GITPATH=M:\
PyErr No module named 'gen_pyi' ,Traceback - : 
PyErr Exception!(eInvalidInput), function loadPythonModule, Line 337, File PyApApplication.cpp:  ,Traceback - 
At <module> (M:\Dev\Projects\PyRxGit\pyrx\_host_init.py:94)
At main (M:\Dev\Projects\PyRxGit\pyrx\_host_init.py:85)
At pyrx_onload (M:\Dev\Projects\PyRxGit\pyrx\_host_init.py:58)
At load_pyrx_onload (M:\Dev\Projects\PyRxGit\pyrx\_host_init.py:50)
At run_path (<frozen runpy>:287)
At _run_module_code (<frozen runpy>:98)
At _run_code (<frozen runpy>:88)
At <module> (M:\Dev\Projects\PyRxGit\Bin\PyRxZ26.0\x64\Debug\pyrx_onload.py:34): 
internalLoad_host_init_py failed: 
M:\Dev\Projects\PyRxGit\Bin\PyRxZ26.0\x64\Debug\RxLoaderZ26.0.zrx load successfully!

[CEXT-Dan]

Ap.Application.loadPythonModule should already set the cwd btw. Might be necessary in case the code base spans multiple files

[gswifort]

Ap.Application.loadPythonModule should set the cwd btw. Might be necessary in case the code base spans multiple files

cwd or rather overwrite sys.path while loading?

[CEXT-Dan]

Ap.Application.loadPythonModule does both,

• sets the CWD,
• pushes to the front of sys.path
• loads the module
• pops CWD to previous
• pops the front of sys.path, then appends it

[gswifort]

I generally don't use cwd when I care about the path relative to the module, I always use file. On the contrary, changing cwd can in my opinion cause problems when someone wants to base on the actual cwd, e.g. previously set in pyrx_onload.

[CEXT-Dan]

I would prefer not to change this behavior

Some things to consider, we already have people using the package. We have no clue what people are doing, what if people a using relative paths in py_onload? AutoCAD will not see the module path as the working directory, then we are breaking people’s applications

[CEXT-Dan]

My objections to moving PYLOAD/PYRELOAD are.

• I don’t understand what benefit moving them adds to the project.
• they are ready commands regardless of the state of other python modules. If for some reason downstream modules fail to load, what happens?
• I can directly set my C++ debugger in PYLOAD and step through the loading process. I feel that this might limit my ability to give support. I can set my break point in Ap.Application.loadPythonModule, but if it’s never called, then what do I do?

[CEXT-Dan]

this is my onload file. it's copied and renamed with visual studio

[CEXT-Dan]

Skipping pyrx configuration file (C:\Users\Dan\AppData\Roaming\pyrx\pyrx.toml) as it is not the first one found (M:\Dev\Projects\PyRxGit\pyrx\pyrx.toml)

seems backwards

[gswifort]

ok, I didn't pay attention to the exact path, I thought it was your personal file. The problem is that it is searched by findFile (and shouldn't it be?). I understand that this is because the cwd is changed to the parent directory of PyRx.arx when it is loaded..?
The solution is probably to change pyrx.toml → pyrx.toml.template

[gswifort]

Besides, I don't think the pyrx.toml file should be written to APPDATA during PyRx initialization. In my opinion, it should be explicitly created by the user.

[CEXT-Dan]

it should be explicitly created by the user.

What’s your reasoning?

• I think it’s worth considering that we may want to write settings to this file someday. A command or a settings GUI.
• From a support perspective, I can say “go to this file and make this change”, instead of asking them to go to the package and make a copy of the file to this folder, then edit it

[gswifort]

An explicit command is perfectly fine. I understand your perspective, creating a command is easy so let's do it, you'll be able to say the same thing, but starting with "invoke command ...".

Why shouldn't we write this file? In version n the default settings will be some, you'll write the file in the APPDATA directory, in version n+100 the default settings will change (because for example we'll decide that optimization_level=2 is a bad idea - this is just an example), then you'll look into the user settings and conclude "the default optimization level is 1, but the user set it to 2 so apparently that's what they want", and that's not true.

[CEXT-Dan]

Ok, I will need to change this. I will remove Ap.Application.getFoundConfigPath() since it will return false by default

[CEXT-Dan]

temporarily I restored C++ pyrx_onload to give more time on this task

[gswifort]

c63a1ee
Why don't you search for pyrx.toml using findFile?

[CEXT-Dan]

I don’t think it’s a good idea to have multiple copies of configuration files, it adds complexity that I feel is not necessary.
Adds problems like this

[gswifort]

Ok, maybe not at this stage. But I think if there are more settings in the config file it will be necessary. It is standard procedure to have several levels of config files:

• vscode:
  • user
  • workspace
  • project
• pip:
  • Global: C:\ProgramData\pip\pip.ini
  • User: APPDATA%\pip\pip.ini
  • Site: VIRTUAL_ENV%\pip.ini

[gswifort]

But we can come back to it when it is actually needed or there are user reports.

[CEXT-Dan]

Not sure I understand this

Skipping pyrx onload path . as it is not a .py or .pyc file

[gswifort]

I assumed the path in the config file would point to a file, not a directory. But I can change that.

[gswifort]

I would like to add the ability to set the application window name in the user configuration. I often have several instances of ZWCAD open, each for a different pyrx-based project, and then I don't know which instance is for which project.

Ap.Application.wxApp().TopWindow.Title = "Project 1"

[gswifort]

I would like to add such an option, e.g.

[user]
top_window_title = "Project 1"

[gswifort]

I forgot that you don't want multiple config files. So PYRX_TOP_WINDOW_TITLE=Project 1

[CEXT-Dan]

PYRX_TOP_WINDOW_TITLE is fine

[CEXT-Dan]

Also, I can add something like

data = Ap.Application.customData()
Ap.Application.setCustomData(pyobject)

Store data for the session, could be any python object, maybe that would be helpful in some contexts

[gswifort]

Also, I can add something like

data = Ap.Application.customData()
Ap.Application.setCustomData(pyobject)

Store data for the session, could be any python object, maybe that would be helpful in some contexts

yes, it might be useful