config,server: add runtime-switchable alias profiles#774
Open
sousekd wants to merge 1 commit into
Open
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds a top-level ChangesRuntime-Switchable Profile Aliases
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@config-schema.json`:
- Around line 374-375: The schema description for the "aliases" map is
inaccurate; update the "description" for the aliases property to state that each
value must reference a valid target which can be either a key in the top-level
"models" map, an existing alias name, or a key used by "setParamsByID" in
runtime profiles, that an empty string or YAML null (~) disables the local alias
while the profile is active, and that unknown or non-existent targets are
rejected by runtime validation (not treated as disabled) so editor/schema
validation should surface those errors earlier.
In `@docs/configuration.md`:
- Line 80: Update the table row for the `hooks` entry in docs/configuration.md:
change the phrase "event driven functionality" to use a hyphen as "event-driven
functionality" so the compound adjective is correctly formed for the `hooks`
description.
In `@proxy/proxymanager.go`:
- Around line 100-111: ActiveProfile currently returns the internal map
profile.Aliases directly, exposing shared mutable state; modify ActiveProfile
(the method on ProxyManager) to make and return a defensive copy of
profile.Aliases while still holding pm.activeProfileMu.RLock() (and before
deferring the RUnlock), so callers get a shallow-copied map (or nil if aliases
is nil) instead of the original map, preserving encapsulation and avoiding
concurrent map mutation issues.
In `@ui-svelte/src/components/Header.svelte`:
- Around line 108-113: The profile selector in Header.svelte currently only uses
a title attribute which is not sufficient for screen readers; update the
<select> used with $activeProfile and handleProfileChange to include an explicit
accessible label by either adding aria-label="Active profile" (or
aria-labelledby with a visible <label> tied to an id on the <select>) so
assistive tech can reliably identify the control; ensure the select has an id if
you choose the <label for="..."> approach and keep existing event handler
handleProfileChange intact.
In `@ui-svelte/src/lib/types.ts`:
- Around line 15-19: The Profile interface's aliases field currently disallows
null but needs to accept disabled aliases; update the type of Profile.aliases
from Record<string, string> to allow null values (e.g., Record<string, string |
null>) and adjust any related usages (parsing/consumption of aliases) to handle
null as a disabled alias; target the Profile interface declaration to make this
change.
In `@ui-svelte/src/stores/api.ts`:
- Around line 178-186: The setActiveProfile function currently posts to
/api/profiles/activate but doesn't update the local activeProfile store on
success; update the store immediately after a successful response (i.e., inside
setActiveProfile after confirming response.ok and before exiting the try block)
by calling the activeProfile store's setter (e.g., activeProfile.set(name) or
the appropriate store update function) so the UI reflects the new profile even
if SSE is delayed or disconnected.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 08823bd6-88a2-484b-9b96-61b68374cf3e
📒 Files selected for processing (16)
config-schema.jsonconfig.example.yamldocs/configuration.mdllama-swap.goproxy/config/config.goproxy/config/config_posix_test.goproxy/config/config_test.goproxy/config/config_windows_test.goproxy/events.goproxy/proxymanager.goproxy/proxymanager_api.goproxy/proxymanager_test.goui-svelte/src/App.svelteui-svelte/src/components/Header.svelteui-svelte/src/lib/types.tsui-svelte/src/stores/api.ts
💤 Files with no reviewable changes (1)
- llama-swap.go
sousekd
force-pushed
the
open-pr/runtime-profiles
branch
3 times, most recently
from
8d08d23 to
13d6b37
Compare
There was a problem hiding this comment.
♻️ Duplicate comments (1)
docs/configuration.md (1)
80-80:⚠️ Potential issue | 🟡 Minor | ⚡ Quick winHyphenate the compound adjective in the feature description.
Line 80 should use “event-driven functionality” for correct grammar.
Proposed fix
-| `hooks` | event driven functionality | +| `hooks` | event-driven functionality |🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/configuration.md` at line 80, Update the feature description for the `hooks` entry to use the hyphenated compound adjective "event-driven functionality" instead of "event driven functionality"; locate the table row containing the `hooks` column in the documentation and replace the phrase accordingly so the `hooks` description reads "event-driven functionality".
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@docs/configuration.md`:
- Line 80: Update the feature description for the `hooks` entry to use the
hyphenated compound adjective "event-driven functionality" instead of "event
driven functionality"; locate the table row containing the `hooks` column in the
documentation and replace the phrase accordingly so the `hooks` description
reads "event-driven functionality".
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 70953996-d2d1-45ce-ad1a-cfd9f655688c
📒 Files selected for processing (16)
config-schema.jsonconfig.example.yamldocs/configuration.mdllama-swap.goproxy/config/config.goproxy/config/config_posix_test.goproxy/config/config_test.goproxy/config/config_windows_test.goproxy/events.goproxy/proxymanager.goproxy/proxymanager_api.goproxy/proxymanager_test.goui-svelte/src/App.svelteui-svelte/src/components/Header.svelteui-svelte/src/lib/types.tsui-svelte/src/stores/api.ts
💤 Files with no reviewable changes (1)
- llama-swap.go
✅ Files skipped from review due to trivial changes (1)
- config.example.yaml
🚧 Files skipped from review as they are similar to previous changes (12)
- proxy/config/config_windows_test.go
- ui-svelte/src/App.svelte
- proxy/config/config_posix_test.go
- config-schema.json
- ui-svelte/src/stores/api.ts
- proxy/config/config_test.go
- ui-svelte/src/lib/types.ts
- proxy/events.go
- proxy/proxymanager_api.go
- proxy/proxymanager_test.go
- proxy/proxymanager.go
- proxy/config/config.go
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/configuration.md`:
- Around line 443-444: Clarify that while the documentation line "target may be
a model ID, an alias, or a setParamsByID variant key" allows an alias, profile
aliases must not point to other profile aliases (no profile→profile chains);
explicitly state that profile alias resolution permits at most one static-alias
indirection (alias→model or alias→setParamsByID) and that profile aliases cannot
shadow a model ID, to prevent users from creating multi-step alias chains.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 9550f7c3-5a09-47a8-bc7c-b2f357df19dc
📒 Files selected for processing (16)
config-schema.jsonconfig.example.yamldocs/configuration.mdllama-swap.goproxy/config/config.goproxy/config/config_posix_test.goproxy/config/config_test.goproxy/config/config_windows_test.goproxy/events.goproxy/proxymanager.goproxy/proxymanager_api.goproxy/proxymanager_test.goui-svelte/src/App.svelteui-svelte/src/components/Header.svelteui-svelte/src/lib/types.tsui-svelte/src/stores/api.ts
💤 Files with no reviewable changes (1)
- llama-swap.go
✅ Files skipped from review due to trivial changes (2)
- config.example.yaml
- ui-svelte/src/App.svelte
🚧 Files skipped from review as they are similar to previous changes (12)
- proxy/config/config_windows_test.go
- ui-svelte/src/components/Header.svelte
- proxy/config/config_posix_test.go
- proxy/events.go
- config-schema.json
- ui-svelte/src/lib/types.ts
- ui-svelte/src/stores/api.ts
- proxy/proxymanager.go
- proxy/proxymanager_api.go
- proxy/config/config_test.go
- proxy/config/config.go
- proxy/proxymanager_test.go
sousekd
force-pushed
the
open-pr/runtime-profiles
branch
from
13d6b37 to
aaf3a96
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
proxy/config/config_test.go (1)
297-314: ⚡ Quick winAdd a same-profile chain rejection case.
This currently verifies profile→profile chaining across two profiles. Please also add a case where an alias in the same profile points to another alias in that same profile to lock in the one-step rule.
As per coding guidelines, `**/*_test.go`: Follow test naming conventions like `TestProxyManager_`, `TestProcessGroup_`, etc.Proposed test addition
func TestConfig_Profiles_TargetIsAnotherProfileAlias_Rejected(t *testing.T) { @@ } + +func TestConfig_Profiles_TargetIsAliasInSameProfile_Rejected(t *testing.T) { + content := ` +models: + model1: + cmd: path/to/cmd --arg1 one + proxy: "http://localhost:8080" +profiles: + a: + aliases: + from-a: model1 + from-b: from-a +` + _, err := LoadConfigFromReader(strings.NewReader(content)) + require.Error(t, err) + assert.Contains(t, err.Error(), `profile a: alias "from-b" target "from-a" is not a known model or alias`) +}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@proxy/config/config_test.go` around lines 297 - 314, Add a new assertion that covers the same-profile alias chaining case and rename the test to match naming conventions: update the test named TestConfig_Profiles_TargetIsAnotherProfileAlias_Rejected to a TestProxyManager_* name (e.g. TestProxyManager_Config_Profiles_TargetIsAnotherProfileAlias_Rejected), and modify the YAML content passed to LoadConfigFromReader so one profile contains two aliases where the second alias in the same profile references the first (e.g. profiles: a: aliases: from-a: model1; from-b: from-a), then call LoadConfigFromReader and require an error asserting the message mentions the profile "a" and that alias "from-b" target "from-a" is not a known model or alias; keep function name and call sites (LoadConfigFromReader) as the reference points to locate the test.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@proxy/config/config_test.go`:
- Around line 297-314: Add a new assertion that covers the same-profile alias
chaining case and rename the test to match naming conventions: update the test
named TestConfig_Profiles_TargetIsAnotherProfileAlias_Rejected to a
TestProxyManager_* name (e.g.
TestProxyManager_Config_Profiles_TargetIsAnotherProfileAlias_Rejected), and
modify the YAML content passed to LoadConfigFromReader so one profile contains
two aliases where the second alias in the same profile references the first
(e.g. profiles: a: aliases: from-a: model1; from-b: from-a), then call
LoadConfigFromReader and require an error asserting the message mentions the
profile "a" and that alias "from-b" target "from-a" is not a known model or
alias; keep function name and call sites (LoadConfigFromReader) as the reference
points to locate the test.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: 0c2807d9-7a9b-445b-9fe1-41aa97ef5fa2
📒 Files selected for processing (16)
config-schema.jsonconfig.example.yamldocs/configuration.mdllama-swap.goproxy/config/config.goproxy/config/config_posix_test.goproxy/config/config_test.goproxy/config/config_windows_test.goproxy/events.goproxy/proxymanager.goproxy/proxymanager_api.goproxy/proxymanager_test.goui-svelte/src/App.svelteui-svelte/src/components/Header.svelteui-svelte/src/lib/types.tsui-svelte/src/stores/api.ts
💤 Files with no reviewable changes (1)
- llama-swap.go
✅ Files skipped from review due to trivial changes (1)
- config.example.yaml
🚧 Files skipped from review as they are similar to previous changes (11)
- ui-svelte/src/App.svelte
- proxy/events.go
- proxy/config/config_posix_test.go
- ui-svelte/src/components/Header.svelte
- proxy/config/config_windows_test.go
- ui-svelte/src/lib/types.ts
- ui-svelte/src/stores/api.ts
- proxy/proxymanager_test.go
- proxy/proxymanager_api.go
- proxy/proxymanager.go
- proxy/config/config.go
sousekd
force-pushed
the
open-pr/runtime-profiles
branch
from
aaf3a96 to
395792a
Compare
Contributor
Author
|
Owner
|
@sousekd this is an interesting idea. I'm doing a massive refactor of the routing back end right now that would make implementing more dynamic swapping of the configuration easier (hopefully) to implement and maintain. I'll keep an eye on how this feature develops and if or how to implement it into llama-swap core. |
Contributor
Author
|
Thank you. I'd be perfectly happy if the issue is addressed in any other way, or at least the new system would allow similar patch possible. |
sousekd
force-pushed
the
open-pr/runtime-profiles
branch
from
395792a to
aa1095d
Compare
Introduce named profiles that remap aliases at runtime without a config reload, applied as a routing-agnostic layer in front of the group and matrix backends. A profile is a map of alias name to target (model ID, static alias, or setParamsByID variant key); an empty string or YAML ~ disables the alias while that profile is active. Profile aliases resolve in a single step and cannot shadow a model ID; unknown targets are rejected at load time. The active profile is selected via POST /api/profiles/activate/<name> and surfaced through GET /api/profiles plus an SSE profileChanged event so the UI dropdown stays in sync. - config: ProfileConfig type, profile-aware resolution helpers, two-pass load-time validation of alias names and targets - server: active-profile state guarded by an RWMutex, a front-layer middleware that rewrites the requested model in context before filters and dispatch, list/activate endpoints, EffectiveAliasesFor applied to /v1/models and model status, and an SSE broadcast - ui: header dropdown, store, types, SSE handler - docs, config.example.yaml, config-schema.json: profiles section
sousekd
force-pushed
the
open-pr/runtime-profiles
branch
from
aa1095d to
b6164ce
Compare
sousekd
changed the title
Contributor
Author
|
Rebased onto current |
Owner
|
OK! I'm quite swamped at the moment but I'll take a look. I like this idea though it is a rather niche feature. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
llama-swap already supports aliases, which makes it possible to give models semantic, intent-based names -
llm-plan,llm-code,llm-image,llm-assistant-smart- and let clients address those instead of concrete model IDs likeqwen3.6-27borglm-5.1. Agents and harnesses keep these names in client configuration, which is inconvenient to change and not meant to be edited mid-session.profiles:makes it possible to reroute aliases to different models using the UI profile-selector dropdown (or via API), without restarting llama-swap or reconfiguring any client.Example
With
plan-smarteractive,llm-planresolves toglm-5.1instead of the defaultqwen3.6-27b, andimage-genis disabled to keep the focused session free of stray load cycles. No restart, no client changes.Compatibility
profiles:is absent, behaviour is unchanged and the UI dropdown does not appear.groups,matrix, orsetParamsByIDsemantics for non-profile requests.profiles:block.~- all scoped to the active profile.What it touches
Config.RealModelNameWithProfileoverlays the active profile's aliases on top of static aliases.setParamsByIDlookup uses the profile-rewritten request name, so a profile alias can target a variant key (e.g.llm-task → some-model:nothink) and trigger that variant's params./v1/modelslisting reflects the active profile's alias reassignments automatically.Notes for review
@mostlygeek, happy to change anything you'd prefer different - naming, UI, scope.
We use llama-swap in a small team with a growing number of client apps running both on the server and on users' machines. Profiles let us experiment and switch between focused configurations without restarts and worrying that one stray request triggers a long unload/load cycle. We keep profiles like
coding-smarter,coding-faster,general-chat, each routing aliases to different set of models. All clients continue to work without fighting for a specific model.A couple of design choices worth mentioning:
aliasProfiles?) might fit better thanprofiles. I choseprofilesbecause the same config might carry other runtime-switchable overrides later. Not sure about this.setParamsByIDvariant keys), but profile-to-profile chains are rejected at load time. To keep it simple.