yaml based configuration#36
Conversation
|
Hey @jbride, good job with the pull request mate! I wish GitHub would allow me to submit a proper review, so I'll do it with comments for now. Found an issue with the CPU miner config: I asked my LLM to provide a fix and then it provided this solution, maybe you want to port it to your branch. |
|
Another improvement would be replacing or removing references to the old environment variables. Found instances on the following files:
Some of the mentions are in the context of explaining this PR's changes but some are just residue, if left like that it might confuse and misinform future developers. |
|
nit: Maybe we should find ways to avoid committing real config files by accident and leaking secrets? For one we could add |
|
Hey @jbride, following up to see if you will continue with this implementation? I think this would be really useful. If you want to make me a contributor in your fork I can amend/add the fixes and improvements there as well. |
|
I realize it would be helpful if I took a look at this sooner rather than later. Thanks for your patience! |
|
Sorry about the delay. |
fix(config): complete migration from legacy env vars to unified config system The CPU miner board factory was still reading MUJINA_CPUMINER_THREADS via from_env() rather than using config passed through the transport event, causing the board to fail when only the new MUJINA__* style vars were set. This commit also fixes the following: - Add mujina.yaml to .gitignore - Thread CpuDeviceInfo through VirtualBoardFactoryFn so the board factory receives its config from the transport event instead of legacy env vars - Migrate ForcedRateConfig::from_env() to pool.forced_rate config key - Remove dead CpuMinerConfig::from_env() and its tests - Add bootstrap variable validation in Config::load_with() to detect MUJINA__CONFIG__* and MUJINA__DEFAULT__* misuse with a clear error - Document bootstrap variables (MUJINA_CONFIG_FILE_PATH, MUJINA_DEFAULT_CONFIG_PATH) and their distinction from MUJINA__* keys - Update all docs and source to use MUJINA__* style; purge legacy var references from README, api.md, container.md, cpu-mining.md, issue triage template, and stratum_v1 integration test - Consolidate config_priority_tests.rs and cpu_miner_tests.rs into daemon_integration_tests.rs
|
Hey @jayrmotta . |
There was a problem hiding this comment.
Thanks for the recent update @jbride and for working together on this!
I have tested once again and confirm the head eb6b6c9 works as the documentation suggests.
My comments are most about style and structure. When it comes to the overall design I'm not entirely convinced with merging properties from different config sources, I believe it might lead to unintended situations/configurations.
Last but not least, I would suggest cleaning up the commit history. Arguably you could have multiple commits, but right now I'd say they aren't following Mujina's guidelines to have atomic conventional commits only.
| # -- Bitaxe family (BM1370-based boards) -- | ||
| # NOTE: These fields are parsed but not yet wired to board behavior. | ||
| # They are reserved for near-term implementation. | ||
| bitaxe: | ||
| # Emergency shutdown temperature in Celsius. | ||
| # The board halts hashing if any sensor reads above this value. | ||
| temp_limit_c: 85.0 | ||
|
|
||
| # Fan speed bounds as a percentage of full speed. | ||
| # The thermal controller adjusts within this range. | ||
| fan_min_pct: 20 | ||
| fan_max_pct: 100 | ||
|
|
||
| # Maximum board power consumption in watts. | ||
| # Null disables the power cap (hardware default applies). | ||
| power_limit_w: ~ |
There was a problem hiding this comment.
I would rather just skip this and then each author will implement support for their upcoming features.
There was a problem hiding this comment.
Kept thinking about this and wanted to clarify, I'm coming from a place of implementing the thermal control for the bitaxes and not knowing for sure what should be exposed, if you check this config file it has many properties and each of them are arguably important for this implementation.
I believe the properties there might change a lot and so it's introduced already "out of sync" while both this and #33 are open at the same time. If anything I'd be more inclined exposing them all and giving our users the power of customizing everything, but having safe defaults.
| # ------------------------------------------------------------ | ||
| # hash_thread — ASIC hash thread tuning | ||
| # NOTE: These fields are parsed but not yet wired to board behavior. | ||
| # They are reserved for near-term implementation. | ||
| # ------------------------------------------------------------ | ||
| hash_thread: | ||
| # Difficulty target configured on-chip for share reporting. | ||
| # Lower values produce more frequent shares (useful for health | ||
| # monitoring); higher values reduce message volume on large | ||
| # installations. The scheduler still applies pool difficulty | ||
| # filtering before forwarding shares to the pool. | ||
| chip_target_difficulty: 256 |
There was a problem hiding this comment.
nit: May be worth pointing out which PR, branch, discussion, etc, led you to this near-term implementation.
| ### 2.2. User-specified location | ||
|
|
||
| Set `MUJINA_CONFIG_FILE_PATH` to an absolute path to load a second config file | ||
| that supplements (and overrides) the default location: | ||
|
|
||
| ```sh | ||
| MUJINA_CONFIG_FILE_PATH=/home/operator/mujina.yaml mujina-minerd | ||
| ``` | ||
|
|
||
| Keys present in the user-specified file take precedence over the same keys in | ||
| `/etc/mujina/mujina.yaml`. Keys absent from the user file fall back to the | ||
| default file, then to hard-coded defaults. | ||
|
|
||
| An example config file is provided at `configs/mujina.example.yaml`. | ||
|
|
||
| ### 2.3. Bootstrap variables | ||
|
|
||
| `MUJINA_CONFIG_FILE_PATH` and `MUJINA_DEFAULT_CONFIG_PATH` use single | ||
| underscores and are **not** part of the `MUJINA__*` config-key system. They are | ||
| bootstrap variables read by the loader before the config system is constructed, | ||
| so the double-underscore naming convention does not apply to them. | ||
|
|
||
| `MUJINA_DEFAULT_CONFIG_PATH` overrides the default system config path | ||
| (`/etc/mujina/mujina.yaml`). It is primarily intended for testing, where | ||
| writing to `/etc` requires root access: | ||
|
|
||
| ```sh | ||
| MUJINA_DEFAULT_CONFIG_PATH=/tmp/mujina-test.yaml mujina-minerd | ||
| ``` |
There was a problem hiding this comment.
nit: I find it a bit awkward that we are making the default separator a double underscore but this property remains single underscore. Perhaps we can find a way for it to also follow the same pattern or to drop it and support it as a CLI flag instead?
| ## 4. CLI Flags | ||
|
|
||
| CLI flags override all other sources, including environment variables. They are | ||
| intended for one-off overrides and testing, not permanent configuration. | ||
|
|
||
| `mujina-minerd` accepts the following flags: | ||
|
|
||
| ``` | ||
| USAGE: | ||
| mujina-minerd [OPTIONS] | ||
|
|
||
| OPTIONS: | ||
| -c, --config <PATH> Config file path (overrides MUJINA_CONFIG_FILE_PATH) | ||
| --log-level <LEVEL> Log level: error, warn, info, debug, trace [default: info] | ||
| --api-listen <ADDR> API listen address [default: 127.0.0.1:7785] | ||
| --pool-url <URL> Pool URL, e.g. stratum+tcp://pool.example.com:3333 | ||
| --pool-user <USER> Pool worker username | ||
| --pool-pass <PASS> Pool worker password | ||
| -h, --help Print help | ||
| -V, --version Print version | ||
| ``` |
There was a problem hiding this comment.
nit: So the CLI flags are like an opinionated API since it doesn't support the same set of parameters nor does it reflect the same namespace as this pull request enables for the alternative config sources (env variables or YAML files.)
Why can't we do the same here? Should the other sources go for abitrary keys instead of enforcing/coupling external APIs to internal structures?
Probably this is more of a style preference, worth mentioning and see how it resonates, but wouldn't block the overall solution.
| const DEFAULT_CONFIG_PATH_ENV_VAR: &str = "MUJINA_DEFAULT_CONFIG_PATH"; | ||
| const CONFIG_FILE_ENV_VAR: &str = "MUJINA_CONFIG_FILE_PATH"; |
There was a problem hiding this comment.
Maybe these should be derived using the prefix and the separator, so it's not awkard having different patterns or prefixes if the users decide to customize.
| #[derive(Debug, Clone, Deserialize, Serialize)] | ||
| #[serde(default, deny_unknown_fields)] | ||
| pub struct BitaxeConfig { | ||
| pub temp_limit_c: f32, | ||
| pub fan_min_pct: u8, | ||
| pub fan_max_pct: u8, | ||
| pub power_limit_w: Option<f32>, | ||
| } | ||
|
|
||
| /// Power limits | ||
| pub power_limit: Option<f32>, | ||
| impl Default for BitaxeConfig { | ||
| fn default() -> Self { | ||
| Self { | ||
| temp_limit_c: 85.0, | ||
| fan_min_pct: 20, | ||
| fan_max_pct: 100, | ||
| power_limit_w: None, | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This will probably be affected by #33, you could omit it here and if approved before/after the other one I'll make amends.
- Remove unimplemented BitaxeConfig and HashThreadConfig from config struct and example YAML; accepted keys should have effect - Drop MUJINA_CONFIG_FILE_PATH and MUJINA_DEFAULT_CONFIG_PATH bootstrap env vars; config file path is now CLI-only via --config - Replace named CLI flags (--pool-url, --log-level, etc.) with a generic --set key=value flag using the same dot-path namespace as the YAML config, giving full coverage without coupling CLI names to internal struct fields - Update integration tests, docs, and example YAML to match
|
@jayrmotta Sounds good. The following is a summary of changes with the latest commit:
|
|
Hello @jayrmotta . Any other suggestions you may have before @rkuester reviews this PR ? |
|
Hey @jbride! Sorry it took me long to answer, been a bit hectic over here. At this point I would also like a third person to run and review it. Looking forward to having this in the main branch. |
|
Thanks @jayrmotta . |
|
From dev call: @rkuester to write up feedback so there are specifics on config design to discuss |
|
Thanks for all the work here, @jbride, and for your patience through the back-and-forth. Thanks @jayrmotta too for the careful testing and review along the way. Digging into this PR on dev call #2 is what made me realize we should pin down what we actually want from configuration before reviewing an implementation line by line. That's the "write up config design specifics" item from the call. I've now done that, written up as MIP-0001, the first Mujina Improvement Proposal: (Also introduced in an Ideas discussion, #57, for anyone following along there.) A couple of things worth saying. The requirements are a draft for discussion, not a spec to go implement, and I'd much rather you and @jayrmotta poke holes in them first, so comments and line suggestions on that PR are very welcome, especially where they conflict with what you ran into building this. And since this PR is the reason the requirements exist and remains a valuable reference for them, I'd like to keep it open while we converge on them rather than do a line-by-line review right now. Once we're agreed on the requirements, let's plan the implementation together. They reach beyond what this PR set out to do (runtime changes made through the API and persisted, live subscriptions to changes, etc.), so the realistic shape is an incremental path rather than one big change, and this PR is a strong starting point for that. Please don't read any of this as the work being tossed. It's the opposite. My next step is walking the dev-call group through the requirements for review, and I'd like to have you both in that. |
Details of implementation can be found here.