diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..c50ec64 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,123 @@ +# Changelog + +All notable changes to this project are documented here. + +The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [0.4.0] - 2026-05-22 + +> Breaking release. Target manifests and `agpack.yml` were overhauled to make every resource generic. The old MCP-specific syntax has no in-place upgrade — see the migration block below. + +### Added + +- **YAML target manifests + `target_definitions:` in `agpack.yml`.** Built-in targets (Claude, Cursor, Gemini, Codex, OpenCode, Copilot) now ship as YAML manifests; users can define or override targets directly in `agpack.yml`. New CLI: `agpack targets list` and `agpack targets show `. +- **Three resource kinds: `copy-directory`, `copy-file`, `edit-file`.** Declared per resource as `kind:` + `path:`. Per-kind logic lives in `agpack.kinds`; the deployer is a thin orchestrator. +- **Generic `edit-file` for any JSON/TOML config**, not just MCP. Patches are `{key, value, strategy: replace | append}` under `dependencies:`. The engine walks the dotted `key`, auto-creates intermediates, and on cleanup reverses replaces by key and appends by deep-equality match (no markers in user files). The built-in `claude` target ships an `edit-file` resource for `.claude/settings.json` (hooks/permissions) alongside `mcp`. +- **Per-target `vars:` with `${name}` substitution in patches.** Each target manifest declares a `vars:` map (e.g. `bucket: mcpServers` for Claude, `mcp` for OpenCode, `mcp_servers` for Codex). One patch `key: ${bucket}.filesystem` deploys correctly everywhere. Target vars take precedence over env vars on collision. +- **Open resource taxonomy.** `skills`, `commands`, `agents`, `mcp` are conventions used by the built-ins — any string is a valid resource-type name in both `dependencies:` and `target_definitions`. + +### Changed + +- Configuration errors now raise structured exceptions instead of failing late during deploy. +- Logging no longer echoes secrets pulled from `env:` substitutions. + +### Removed + +- The MCP-specific encoder (`McpServer`, transport-aware encoding, `merge: {servers_key, defaults, transports}`). Per-target quirks are now explicit patches. +- `format:` on edit-file resources (inferred from path extension). +- `layout: directory | file` on copy resources (replaced by `kind: copy-directory | copy-file`). + +### Migration from 0.3.x + +Old `agpack.yml`: + +```yaml +dependencies: + mcp: + - name: filesystem + command: npx + args: ["-y", "@modelcontextprotocol/server-filesystem", "."] + env: + API_KEY: ${API_KEY} +``` + +New `agpack.yml`: + +```yaml +dependencies: + mcp: + - key: ${bucket}.filesystem # resolves per-target via vars + value: + command: npx + args: ["-y", "@modelcontextprotocol/server-filesystem", "."] + env: + API_KEY: ${API_KEY} +``` + +`${bucket}` resolves per target from each built-in's `vars:`: `mcpServers` for Claude/Cursor/Gemini, `mcp_servers` for Codex, `mcp` for OpenCode, `servers` for Copilot. + +For custom `target_definitions:`, drop the `merge:` block and `format:` from edit-file resources, replace `layout: directory|file` with `kind: copy-directory|copy-file`, and add `vars: { bucket: }` on edit-file resources whose patches use `${bucket}.`. + +**Lockfile:** the format changed and agpack does not read pre-release lockfiles — delete `.agpack.lock.yml` before the first sync. Legacy singular resource-type keys (`skill`/`command`/`agent`) are remapped to plural on read, so cleanup of older lockfiles still works. + +## [0.3.1] - 2026-04-05 + +### Added + +- Multiple URLs per dependency: `url:` accepts a list, tried in order (e.g. HTTPS primary with SSH fallback). The first URL is canonical for identity and display. + +## [0.3.0] - 2026-04-05 + +### Added + +- Global config at `~/.config/agpack/agpack.yml` for dependencies and MCP servers shared across projects. +- Per-project opt-out via `global: false` in `agpack.yml`, or `agpack sync --no-global` from the CLI. +- Project dependencies and MCP servers win on conflict when merged with the global config. + +## [0.2.1] - 2026-03-26 + +### Changed + +- Improved progress / status styling. + +## [0.2.0] - 2026-03-26 + +### Added + +- Folder-of-skills support: a dependency `path` pointing at a directory of subfolders expands to one skill per subfolder. + +## [0.1.7] - 2026-03-26 + +### Fixed + +- Added a proper timeout for `git fetch` operations and disabled interactive git prompts so a stalled clone can no longer hang sync indefinitely (#2). + +## [0.1.6] - 2026-03-25 + +### Added + +- `${VAR}` env-variable substitution in MCP server `env:` and other config values, resolved from `.env` or the shell environment. + +## [0.1.5] - 2026-03-23 + +### Added + +- MCP server config merging into each tool's project-level config file. + +### Changed + +- User-feedback improvements during sync (status / progress output). + +### Fixed + +- Type-check failures and CI configuration. + +[Unreleased]: https://github.com/PhilippTh/agpack/compare/v0.4.0...HEAD +[0.4.0]: https://github.com/PhilippTh/agpack/compare/v0.3.1...v0.4.0 +[0.3.1]: https://github.com/PhilippTh/agpack/compare/v0.3.0...v0.3.1 +[0.3.0]: https://github.com/PhilippTh/agpack/compare/v0.2.1...v0.3.0 +[0.2.1]: https://github.com/PhilippTh/agpack/compare/v0.2.0...v0.2.1 +[0.2.0]: https://github.com/PhilippTh/agpack/compare/v0.1.7...v0.2.0 +[0.1.7]: https://github.com/PhilippTh/agpack/compare/v0.1.6...v0.1.7 +[0.1.6]: https://github.com/PhilippTh/agpack/compare/v0.1.5...v0.1.6 +[0.1.5]: https://github.com/PhilippTh/agpack/releases/tag/v0.1.5 diff --git a/README.md b/README.md index 7216b05..c8ce065 100644 --- a/README.md +++ b/README.md @@ -9,20 +9,18 @@ [![License: GPL-3.0](https://img.shields.io/badge/license-GPL--3.0-green)](LICENSE) [![Sponsor](https://img.shields.io/badge/Sponsor-GitHub%20Sponsors-ea4aaa)](https://github.com/sponsors/PhilippTh) -Declare your AI agent resources in a YAML file, run `agpack sync`, and they get deployed to every coding tool you use. - -agpack fetches skills, commands, agents, and MCP server configs from git repos and copies them to the right places for Claude Code, OpenCode, Codex, Cursor, GitHub Copilot, Gemini CLI, Windsurf, and Google Antigravity. +A package manager for AI coding tools. Declare skills, commands, agents, MCP servers, hooks, and any other resource your editor or CLI consumes in one `agpack.yml`. Run `agpack sync` and agpack fetches them from git and deploys them where each tool expects — Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode, Windsurf, Google Antigravity, and any custom tool you describe in a YAML manifest. ## Why -Every AI coding tool has its own directory structure for skills, its own config format for MCP servers, its own spot for custom commands. If you use more than one tool -- or share resources across projects -- you end up manually copying files and keeping multiple configs in sync. +Every AI coding tool has its own conventions: `.claude/skills/`, `.cursor/commands/`, `.codex/agents/`, `.github/prompts/`. MCP server configs live in subtly different shapes across tools (`mcpServers` vs `mcp_servers` vs `mcp` vs `servers`). Sharing the same resources across tools — and updating them across projects — turns into manual copy-paste with quiet drift. -agpack replaces that with a single `agpack.yml` that describes what you want and where it comes from. +agpack replaces that with a single declarative file. It owns the keys it writes, leaves everything else alone, and remembers exactly what it did so removing a dependency cleanly rolls back. ## Install ```bash -pipx install agpack # or: uv tool install agpack +pipx install agpack # or: uv tool install agpack ``` Requires Python 3.11+ and `git` on PATH. @@ -30,10 +28,10 @@ Requires Python 3.11+ and `git` on PATH. ## Quick start ```bash -agpack init # creates agpack.yml with commented-out examples +agpack init # creates agpack.yml with commented examples ``` -Edit `agpack.yml`: +A minimal `agpack.yml`: ```yaml targets: @@ -45,90 +43,136 @@ dependencies: - url: https://github.com/owner/repo path: skills/my-skill - commands: - - url: https://github.com/owner/repo - path: commands/review.md - - agents: - - url: https://github.com/owner/repo - path: agents/backend-expert.md - + # MCP server entries deploy as patches into each target's config file. + # ${bucket} is supplied by the target manifest, so the same patch + # writes mcpServers.filesystem on Claude and mcp.filesystem on OpenCode. mcp: - - name: filesystem - command: npx - args: ["-y", "@modelcontextprotocol/server-filesystem", "."] + - key: ${bucket}.filesystem + value: + command: npx + args: ["-y", "@modelcontextprotocol/server-filesystem", "."] ``` ```bash agpack sync ``` -Skills get copied to `.claude/skills/`, `.opencode/skills/`, etc. Commands and agents go to their respective directories. MCP server definitions get merged into each tool's config file. Run `agpack sync` again after editing the config -- removed dependencies get cleaned up automatically. +Skills land in `.claude/skills/my-skill/` and `.opencode/skills/my-skill/`. The MCP server is added to `.mcp.json` and `opencode.json`. Run `agpack sync` again after editing — removed entries are cleaned up automatically and the lockfile remembers what to restore. -## Dependencies +## The model + +Every resource agpack deploys falls into one of three **kinds**. Once you know the kinds, the rest of the tool is just "declare resources of these kinds; declare which tools (targets) get them." + +| Kind | What it does | What you write in `agpack.yml` | +|------------------|-----------------------------------------------------------------------------------------------------------------------|----------------------------------------| +| `copy-directory` | Copy a directory tree from a fetched git repo into `//`. A folder-of-folders expands to one bundle each. | `{ url, path?, ref? }` | +| `copy-file` | Copy individual files from a fetched git repo into `/`. A folder-of-files expands to one item per file. | `{ url, path?, ref? }` | +| `edit-file` | Read a JSON or TOML config, apply patches, write it back. Only touches keys agpack owns; everything else is preserved. | `{ key, value, strategy? }` | -### URLs and pinning +A **target** is a YAML manifest that maps resource type names (`skills`, `commands`, `mcp`, anything you like) to a kind + destination path. agpack ships built-in manifests for the eight common tools and lets you override them or add your own. -The `url` field takes any valid `git clone` URL -- HTTPS, SSH, local paths, whatever git understands. Authentication is handled by your system git config (SSH keys, credential helpers, etc.). +## Dependencies + +`dependencies:` is keyed by resource type name. The value is a list of entries. The shape of each entry depends on the kind the target uses for that resource type: -Use `ref` to pin a dependency to a specific tag or commit: +- **copy-directory / copy-file** entries are fetched from git: `{ url, path?, ref? }`. +- **edit-file** entries are inline patches: `{ key, value, strategy? }`. ```yaml -- url: https://github.com/owner/repo - path: skills/my-skill - ref: v1.2.0 +dependencies: -- url: git@gitlab.com:myorg/myrepo.git - path: skills/my-skill - ref: abc1234 + skills: # copy-directory on every built-in + - url: https://github.com/owner/skills-repo + path: skills/code-review + ref: v1.2.0 # tag, branch, or commit SHA + + commands: # copy-file + - url: https://github.com/owner/cmds + path: commands/review.md + + agents: # copy-file + - url: https://github.com/owner/agents + path: agents/backend-expert.md + + mcp: # edit-file + - key: ${bucket}.filesystem + value: + command: npx + args: ["-y", "@modelcontextprotocol/server-filesystem", "."] ``` -### Fallback URLs +### URLs, pinning, and fallbacks -`url` can be a list. When it is, agpack tries each URL in order until one succeeds. This is useful when team members use different auth methods (SSH vs HTTPS), or when you want to fall back to a mirror: +`url` takes any string `git clone` accepts — HTTPS, SSH, local paths. Auth goes through your system git config (SSH keys, credential helpers, etc.). -```yaml -# Tried in order -- works for both SSH and HTTPS users -- url: - - https://github.com/owner/repo - - git@github.com:owner/repo.git - path: skills/my-skill +`url` can also be a list of fallback URLs, tried in order: -# Internal mirror with public fallback +```yaml - url: - - https://git.internal.company.com/team/repo - - https://github.com/company/repo + - git@github.com:owner/repo.git # SSH for team members with keys + - https://github.com/owner/repo # HTTPS fallback path: skills/my-skill + ref: v1.2.0 ``` ### Directory expansion -The `path` field can point to a single file, a single folder, or a parent directory containing multiple items. When it points at a directory, agpack figures out what's inside: +`path` can point at a single file, a single folder, or a parent directory containing multiple items. agpack figures out what's inside: -- **Skills** -- a directory with top-level files is deployed as one skill. A directory containing only subdirectories deploys each subfolder as a separate skill. -- **Commands & Agents** -- every non-hidden file is deployed individually. If the directory only contains subdirectories, files inside those are collected instead. +| `path:` points at… | What deploys | +|-----------------------------------|---------------------------------------------| +| One skill folder (with files) | One skill bundle named after the folder | +| A folder of skill subfolders | One bundle per subfolder | +| One command/agent file | One file | +| A folder of command/agent files | Every non-hidden file | +| A folder of subfolders | Files collected from each subfolder | -```yaml -skills: - - url: https://github.com/owner/repo - path: skills/my-skill # deploys one skill +Sync fails with an explicit error if a folder contains nothing deployable. + +## Patches (`edit-file`) - - url: https://github.com/owner/repo - path: skills # deploys each subfolder as a separate skill +A patch is a `{ key, value, strategy }` triple. `key` is a dotted path into the destination config file. `value` is whatever Python value the consuming tool expects to find there (a dict for an MCP server entry, a string for a permission, a dict for a hook entry — agpack is schema-agnostic). `strategy` is `replace` (the default — overwrites whatever's at the path) or `append` (treats the path as a list and adds one element). -commands: - - url: https://github.com/owner/repo - path: commands/review.md # deploys one file +```yaml +dependencies: - - url: https://github.com/owner/repo - path: commands # deploys every file inside + mcp: # strategy defaults to replace + - key: ${bucket}.filesystem + value: + command: npx + args: ["-y", "@modelcontextprotocol/server-filesystem", "."] + env: + API_KEY: ${API_KEY} + + hooks: # an append patch — hooks live in a list + - key: ${bucket}.PreToolUse + strategy: append + value: + matcher: "Write|Edit" + hooks: + - type: command + command: "$${CLAUDE_PROJECT_DIR}/lint.sh" + + permissions: + - key: ${bucket}.allow + strategy: append + value: "Read(/etc/**)" ``` -If the directory contains no deployable files, sync fails with an error. +A few things to know: + +- **`${bucket}`** is a per-target variable supplied by each target manifest. Claude's `mcp` resource ships `bucket: mcpServers`, Codex's ships `bucket: mcp_servers`, OpenCode's ships `bucket: mcp`. One patch deploys correctly to all three. +- **`$${X}`** writes a literal `${X}` to the destination file. Useful when the consuming tool resolves variables at runtime — Claude Code itself expands `${CLAUDE_PROJECT_DIR}` when a hook fires. +- **Dotted keys with literal dots**: use `\.` inside a segment to embed a literal dot. `mcpServers.example\.com` writes to `mcpServers["example.com"]`. +- **Intermediate dicts auto-create** for replace patches. `append` requires the path to resolve to a list (created empty if absent). +- **Cleanup is surgical**: `replace` restores the value that was there *before* agpack overwrote it (or deletes the key if agpack created it); `append` removes the exact item agpack added by deep-equality. agpack never deletes content it didn't write. -### Environment variables +## Variables and substitution -Use `${VAR_NAME}` in any string value to reference environment variables. This works in URLs, paths, refs, MCP commands, args, env values, and server URLs. +Use `${name}` in any string. The lookup table merges two sources, target wins on collision: + +1. **Target vars** — declared by an `edit-file` resource's `vars:` block. Per-target, only visible inside patches targeting that resource. +2. **Environment vars** — project `.env`, then global `.env`, then shell environment. Available in dependency URLs/paths/refs and recursively in patch keys and values. ```yaml dependencies: @@ -137,86 +181,260 @@ dependencies: path: skills/my-skill mcp: - - name: context7 - command: npx - args: ["-y", "@context7/mcp-server"] - env: - CONTEXT7_API_KEY: ${CONTEXT7_API_KEY} + - key: ${bucket}.context7 # ${bucket} from the target manifest + value: + command: npx + args: ["-y", "@context7/mcp-server"] + env: + CONTEXT7_API_KEY: ${CONTEXT7_API_KEY} # from env ``` -Variables are resolved from up to three sources (highest priority first): +`$$` writes a literal `$` (so `$${X}` becomes `${X}` in the file — pass runtime variables through to the consuming tool). Missing `${name}` references error at apply time, naming the variable and the patch context. -1. `.env` in the project root (same directory as `agpack.yml`) -2. `.env` in the global config directory (`~/.config/agpack/`) -3. Shell environment +## Targets -If a referenced variable is not found in any source, sync fails with an error. The `.env` parser supports `KEY=VALUE`, quoted values, `# comments`, blank lines, and `export` prefixes. +```bash +agpack targets list # show every available target +agpack targets show claude # print Claude's manifest as YAML +``` -## Global config +### Bundled targets -A global config defines dependencies shared across all your projects -- skills, agents, or MCP servers you want everywhere without repeating them in each `agpack.yml`. +agpack ships manifests for eight tools: `claude`, `codex`, `copilot`, `cursor`, `gemini`, `opencode`, `windsurf`, `antigravity`. The manifests themselves are the source of truth — browse them at [`agpack/builtin_targets/`](agpack/builtin_targets/) or introspect locally: ```bash -agpack init --global # creates ~/.config/agpack/agpack.yml +agpack targets list # every target + its resource types +agpack targets show claude # the full manifest as YAML ``` -The global config uses the same `dependencies` block but has no `targets` (those are always per-project): +One thing worth knowing that isn't obvious from the file names: + +- **Windsurf and Antigravity have no per-project MCP config.** Their MCP configs live in user-global locations (`~/.codeium/windsurf/mcp_config.json` and `~/.gemini/antigravity/mcp_config.json`), which agpack does not manage. + +### Custom and overridden targets + +Add a `target_definitions:` block to override a built-in or add a new tool: ```yaml -# ~/.config/agpack/agpack.yml +targets: + - claude + - my-internal-tool # custom target, defined below + +target_definitions: + + # Override the built-in claude target — full replacement, no deep merge. + claude: + skills: + kind: copy-directory + path: .my-claude/skills + commands: + kind: copy-file + path: .my-claude/commands + + # Brand-new target. Declare any resource type names you want; the same + # names must appear under `dependencies:` to be deployed. + my-internal-tool: + skills: + kind: copy-directory + path: .myaitool/skills + rules: + kind: copy-file + path: .myaitool/rules + settings: + kind: edit-file + path: .myaitool/settings.json # format inferred from .json/.toml + vars: + bucket: mcpServers +``` + +Precedence (highest first): project `target_definitions:` → global `target_definitions:` → bundled built-in. When a name appears in `target_definitions:`, that entry **fully replaces** the built-in; agpack does not deep-merge. + +Tip: `agpack targets show ` prints the resolved manifest as YAML — copy-paste it into `target_definitions:` as a starting point. + +### Resource type names are open + +`skills`, `commands`, `agents`, `mcp`, `hooks` are not reserved — they're just the names the built-in target manifests use. Your custom targets can declare any name (`rules`, `prompts`, `personas`, `lints`, `examples`). agpack only matches names between `dependencies:` and target resource blocks; if the same name appears in multiple targets they must agree on `kind:`. + +## Global config + +A global config shares dependencies across every project on your machine — skills, agents, or MCP servers you want everywhere. + +```bash +agpack init --global # creates ~/.config/agpack/agpack.yml +``` + +```yaml +# ~/.config/agpack/agpack.yml — same shape, no `targets:` (those stay per-project) dependencies: skills: - url: https://github.com/owner/shared-skills - path: skills/my-standard-skill + path: skills/code-review + + mcp: + - key: ${bucket}.context7 + value: + command: npx + args: ["-y", "@upstash/context7-mcp@latest"] + env: + CONTEXT7_API_KEY: ${CONTEXT7_API_KEY} +``` + +Global entries are merged into each project sync. Fetch entries are deduplicated by URL+path; patch entries by key (for `replace`) or full content (for `append`). Project entries win on conflict. + +Skip the global config with `--no-global` on the command line or `global: false` in `agpack.yml`. Override the default path with `AGPACK_GLOBAL_CONFIG`. + +## Safety + +agpack writes files the user often hand-edits. Three guarantees keep that safe: + +- **TOML format preservation.** TOML files (e.g. `.codex/config.toml`) round-trip through `tomlkit`. Comments, key ordering, and whitespace on sections agpack didn't touch survive every sync. +- **Idempotent writes.** Files are only written when the serialised text actually differs from disk. Running `agpack sync` twice in a row never modifies a file the second time. No mtime churn, no spurious git diffs. +- **Surgical cleanup.** Every `replace` patch snapshots the value that was at its key *before* agpack first ran. If you remove that patch from `agpack.yml`, the next sync restores the snapshot — your hand-written `mcpServers.foo` survives even if agpack temporarily owned it. Patches agpack created from nothing get deleted; patches that overwrote existing data get reverted. `append` patches are removed by deep-equality from their target list. +The lockfile (`.agpack.lock.yml`) is the source of truth — commit it alongside `agpack.yml` so the whole team's syncs converge. Every file write is atomic (write-to-temp-then-rename); agpack never partially writes a file. + +## Recipes + +### One MCP server across multiple tools + +```yaml +targets: [claude, codex, opencode] +dependencies: mcp: - - name: context7 - command: npx - args: ["-y", "@upstash/context7-mcp@latest"] - env: - CONTEXT7_API_KEY: ${CONTEXT7_API_KEY} + - key: ${bucket}.context7 # bucket differs per target + value: + command: npx + args: ["-y", "@upstash/context7-mcp@latest"] +``` + +### Private skills with a token from `.env` + +```yaml +# .env +GITHUB_TOKEN=ghp_xxx +``` + +```yaml +dependencies: + skills: + - url: https://x-access-token:${GITHUB_TOKEN}@github.com/company/private-skills + path: skills/internal +``` + +(SSH keys via `git@github.com:...` are usually simpler — `${GITHUB_TOKEN}` in URLs is for CI where SSH isn't available.) + +### Pin to a tag / commit + +```yaml +- url: https://github.com/owner/skills-repo + path: skills/my-skill + ref: v1.2.0 # tag, branch, or commit SHA +``` + +### Add a Claude Code hook + +```yaml +dependencies: + hooks: + - key: ${bucket}.PreToolUse + strategy: append + value: + matcher: "Write|Edit" + hooks: + - type: command + command: "$${CLAUDE_PROJECT_DIR}/.claude/hooks/lint.sh" ``` -Global dependencies are merged with the project config during sync. If the same dependency or MCP server appears in both, the project version wins. +`$${CLAUDE_PROJECT_DIR}` is written verbatim — Claude Code expands it when the hook fires. + +### Support a tool agpack doesn't ship a target for + +```yaml +targets: [my-cli] +dependencies: + rules: + - url: https://github.com/owner/rules-repo + path: rules + +target_definitions: + my-cli: + rules: + kind: copy-file + path: .mycli/rules + settings: + kind: edit-file + path: .mycli/settings.json +``` -To skip the global config, either pass `--no-global` on the command line or add `global: false` to your project's `agpack.yml`. The default path (`~/.config/agpack/agpack.yml`) can be overridden with the `AGPACK_GLOBAL_CONFIG` environment variable. +### Address a config key that contains a dot -## Target mapping +```yaml +dependencies: + mcp: + - key: ${bucket}.example\.com # writes mcpServers["example.com"] + value: { command: ... } +``` -| Target | Skills | Commands | Agents | MCP Config | -|--------|--------|----------|--------|------------| -| Claude | `.claude/skills//` | `.claude/commands/` | `.claude/agents/` | `.mcp.json` | -| OpenCode | `.opencode/skills//` | `.opencode/commands/` | `.opencode/agents/` | `opencode.json` | -| Codex | `.agents/skills//` | -- | -- | `.codex/config.toml` | -| Cursor | `.cursor/skills//` | -- | `.cursor/agents/` | `.cursor/mcp.json` | -| Copilot | `.github/skills//` | `.github/prompts/` | `.github/agents/` | `.vscode/mcp.json` | -| Gemini CLI | `.gemini/skills//` | `.gemini/commands/` | -- | `.gemini/settings.json` | -| Windsurf | `.windsurf/skills//` | -- | -- | -- *(global only)* | -| Antigravity | `.gemini/skills//` | `.gemini/commands/` | -- | `.gemini/settings.json` | +### Roll back -Unsupported resource types are skipped silently. MCP definitions are merged into each tool's config file without touching servers agpack didn't create. Windsurf's MCP config is global (`~/.codeium/windsurf/mcp_config.json`) rather than per-project, so agpack does not manage it. +Delete the entry from `agpack.yml`, run `agpack sync`. The lockfile diff drives cleanup: copy-kind files get removed, edit-file `replace` patches restore the prior value, `append` patches get the exact item removed. ## Commands ``` -agpack init [--config PATH] [--global] Scaffold a new config file -agpack sync [--config PATH] [--no-global] Fetch and deploy all dependencies +agpack init [--config PATH] [--global] Scaffold a new config file +agpack sync [--config PATH] [--no-global] Fetch and deploy all dependencies [--dry-run] [--verbose] -agpack status [--config PATH] [--no-global] Show installed vs configured state +agpack status [--config PATH] [--no-global] Show installed vs configured state +agpack targets list [--config PATH] [--no-global] Show all available targets and source +agpack targets show [--config PATH] [--no-global] Print the resolved manifest for one ``` -## How it works +## Limitations + +- **JSON formatting is not preserved.** Python's stdlib `json` has no format-preserving parser. agpack canonicalises (`indent=2`) on the *first* write to a JSON file; subsequent syncs are idempotent and won't rewrite it. Hand-edit JSON files in the same shape agpack emits to avoid churn. +- **Edit-file currently supports only JSON and TOML.** Format is inferred from the path extension; other config formats (YAML, INI, custom DSLs) are not patchable. +- **`target_definitions:` is full replacement, not extension.** Naming a built-in under `target_definitions:` replaces it wholesale — there is no deep-merge. To add one resource type to Claude (e.g. `personas:`) you have to restate every resource type Claude already declares. Tip: `agpack targets show claude` prints the resolved manifest, copy it under `target_definitions.claude:` as a starting point. The trade-off is intentional — deep-merge across built-in upgrades was judged too magical to ship behind a single config knob. +- **Cross-target resource types must share a kind.** If two targets both declare a `commands` resource, they have to agree on whether `commands` is `copy-file` or `copy-directory`. When you genuinely need different shapes, declare them under different names and list each entry under the corresponding name. The repetition is the price of clarity: + + ```yaml + targets: [tool-a, tool-b] + + dependencies: + commands-files: # for targets that want flat files + - url: https://github.com/owner/cmds + path: commands/review.md + commands-patches: # for targets that want config patches + - key: ${bucket}.review + value: { command: "..." } + + target_definitions: + tool-a: + commands-files: + kind: copy-file + path: .tool-a/commands + tool-b: + commands-patches: + kind: edit-file + path: .tool-b/config.json + vars: { bucket: commands } + ``` +- **Windsurf and Antigravity MCP configs are global**, not per-project. agpack doesn't write user-global config files. + +## Changelog + +See [CHANGELOG.md](CHANGELOG.md). On first `agpack sync` after a major version upgrade, files in the old (pre-upgrade) locations are cleaned up automatically — the lockfile remembers exactly where the previous sync wrote them. -1. Loads `agpack.yml` and the global config (if present), merges them -2. Resolves `${VAR}` references from `.env` files and the shell -3. Reads `.agpack.lock.yml` to diff against the previous state -4. Cleans up files from removed dependencies -5. Shallow-clones each repo (sparse checkout when `path` is set), copies files to all target directories -6. Merges MCP configs into each tool's config file -7. Writes an updated lockfile +## How it works -Every file write is atomic (write-to-temp-then-rename). agpack never partially writes a file and never deletes anything it didn't create. +1. Loads `agpack.yml` and (optionally) the global config; merges them. +2. Resolves `${VAR}` references from `.env` files and the shell env (for fetch entries). Patches resolve `${name}` per-target at apply time. +3. Reads `.agpack.lock.yml` to diff against the previous state. +4. Cleans up files from removed copy-kind dependencies. +5. Shallow-clones each repo (sparse checkout when `path` is set) and copies files to every target that declares the matching resource type. +6. Reconciles edit-file resources: per file, diff old applied patches against the desired set, undo what's gone, apply what's new, leave matched patches alone. Files are only written when their text actually changed. +7. Writes the updated lockfile. ## License -GPL-3.0 -- see [LICENSE](LICENSE). +GPL-3.0 — see [LICENSE](LICENSE). diff --git a/agpack/__init__.py b/agpack/__init__.py index be93a6f..33802f7 100644 --- a/agpack/__init__.py +++ b/agpack/__init__.py @@ -1,3 +1,3 @@ """agpack - Fetch and deploy AI agent resources from git repos.""" -__version__ = "0.3.1" +__version__ = "0.4.0" diff --git a/agpack/builtin_targets/antigravity.yml b/agpack/builtin_targets/antigravity.yml new file mode 100644 index 0000000..b30fb88 --- /dev/null +++ b/agpack/builtin_targets/antigravity.yml @@ -0,0 +1,12 @@ +# Google Antigravity + +skills: + kind: copy-directory + path: .agent/skills + +commands: + kind: copy-file + path: .agent/workflows + +# MCP omitted: Antigravity MCP is global only +# (~/.gemini/antigravity/mcp_config.json). diff --git a/agpack/builtin_targets/claude.yml b/agpack/builtin_targets/claude.yml new file mode 100644 index 0000000..dc0d2aa --- /dev/null +++ b/agpack/builtin_targets/claude.yml @@ -0,0 +1,40 @@ +# Claude Code (Anthropic CLI) + +skills: + kind: copy-directory + path: .claude/skills + +commands: + kind: copy-file + path: .claude/commands + +agents: + kind: copy-file + path: .claude/agents + +mcp: + kind: edit-file + path: .mcp.json + vars: + bucket: mcpServers + +# Catch-all for anything in .claude/settings.json — apiKeyHelper, +# cleanupPeriodDays, statusline, etc. Patches write full keys. +settings: + kind: edit-file + path: .claude/settings.json + +# Semantic shortcuts for the two most common settings.json sections. +# Same file as `settings:` above — patches just have a friendlier +# bucket name. https://code.claude.com/docs/en/hooks +hooks: + kind: edit-file + path: .claude/settings.json + vars: + bucket: hooks + +permissions: + kind: edit-file + path: .claude/settings.json + vars: + bucket: permissions diff --git a/agpack/builtin_targets/codex.yml b/agpack/builtin_targets/codex.yml new file mode 100644 index 0000000..8d56b1c --- /dev/null +++ b/agpack/builtin_targets/codex.yml @@ -0,0 +1,15 @@ +# OpenAI Codex CLI + +skills: + kind: copy-directory + path: .codex/skills + +agents: + kind: copy-file + path: .codex/agents + +mcp: + kind: edit-file + path: .codex/config.toml + vars: + bucket: mcp_servers diff --git a/agpack/builtin_targets/copilot.yml b/agpack/builtin_targets/copilot.yml new file mode 100644 index 0000000..4abc169 --- /dev/null +++ b/agpack/builtin_targets/copilot.yml @@ -0,0 +1,19 @@ +# GitHub Copilot in VS Code + +skills: + kind: copy-directory + path: .github/skills + +commands: + kind: copy-file + path: .github/prompts + +agents: + kind: copy-file + path: .github/agents + +mcp: + kind: edit-file + path: .vscode/mcp.json + vars: + bucket: servers diff --git a/agpack/builtin_targets/cursor.yml b/agpack/builtin_targets/cursor.yml new file mode 100644 index 0000000..29d1287 --- /dev/null +++ b/agpack/builtin_targets/cursor.yml @@ -0,0 +1,15 @@ +# Cursor (cursor.com) + +skills: + kind: copy-directory + path: .cursor/skills + +commands: + kind: copy-file + path: .cursor/commands + +mcp: + kind: edit-file + path: .cursor/mcp.json + vars: + bucket: mcpServers diff --git a/agpack/builtin_targets/gemini.yml b/agpack/builtin_targets/gemini.yml new file mode 100644 index 0000000..21fcba5 --- /dev/null +++ b/agpack/builtin_targets/gemini.yml @@ -0,0 +1,15 @@ +# Gemini CLI (Google) + +skills: + kind: copy-directory + path: .gemini/skills + +commands: + kind: copy-file + path: .gemini/commands + +mcp: + kind: edit-file + path: .gemini/settings.json + vars: + bucket: mcpServers diff --git a/agpack/builtin_targets/opencode.yml b/agpack/builtin_targets/opencode.yml new file mode 100644 index 0000000..666b1f7 --- /dev/null +++ b/agpack/builtin_targets/opencode.yml @@ -0,0 +1,19 @@ +# OpenCode (opencode.ai) + +skills: + kind: copy-directory + path: .opencode/skills + +commands: + kind: copy-file + path: .opencode/commands + +agents: + kind: copy-file + path: .opencode/agents + +mcp: + kind: edit-file + path: opencode.json + vars: + bucket: mcp diff --git a/agpack/builtin_targets/windsurf.yml b/agpack/builtin_targets/windsurf.yml new file mode 100644 index 0000000..9c0d58c --- /dev/null +++ b/agpack/builtin_targets/windsurf.yml @@ -0,0 +1,12 @@ +# Windsurf (Codeium) + +skills: + kind: copy-directory + path: .windsurf/skills + +commands: + kind: copy-file + path: .windsurf/workflows + +# MCP omitted: Windsurf MCP is global only +# (~/.codeium/windsurf/mcp_config.json). diff --git a/agpack/cli.py b/agpack/cli.py index 7789b56..aefcb72 100644 --- a/agpack/cli.py +++ b/agpack/cli.py @@ -3,14 +3,16 @@ from __future__ import annotations import time -from collections.abc import Callable from concurrent.futures import ThreadPoolExecutor from concurrent.futures import as_completed from dataclasses import dataclass from dataclasses import field +from importlib.resources import files as importlib_files from pathlib import Path +from typing import Any import click +import yaml from rich.progress import Progress from rich.progress import TaskID from rich.rule import Rule @@ -18,51 +20,58 @@ from agpack import __version__ from agpack.config import AgpackConfig -from agpack.config import ConfigError from agpack.config import DependencySource from agpack.config import GlobalConfig from agpack.config import load_config from agpack.config import load_global_config from agpack.config import merge_configs +from agpack.config import resolve_config +from agpack.config import resolve_global_config_path from agpack.deployer import cleanup_deployed_files -from agpack.deployer import deploy_single_agent -from agpack.deployer import deploy_single_command -from agpack.deployer import deploy_single_skill -from agpack.deployer import detect_agent_items -from agpack.deployer import detect_command_items -from agpack.deployer import detect_skill_items +from agpack.deployer import cleanup_orphaned_edits +from agpack.deployer import deploy_item +from agpack.deployer import detect_items +from agpack.deployer import sync_edit_resource from agpack.display import console from agpack.display import create_sync_progress -from agpack.envsubst import resolve_config -from agpack.fetcher import FetchError +from agpack.envsubst import build_env +from agpack.errors import ConfigError +from agpack.errors import EditFileError +from agpack.errors import FetchError +from agpack.errors import TargetSchemaError from agpack.fetcher import FetchResult from agpack.fetcher import cleanup_fetch from agpack.fetcher import fetch_dependency +from agpack.kinds import EditFileResource +from agpack.lockfile import AppliedPatch +from agpack.lockfile import EditLockEntry from agpack.lockfile import InstalledEntry from agpack.lockfile import Lockfile -from agpack.lockfile import McpLockEntry from agpack.lockfile import find_removed_dependencies -from agpack.lockfile import find_removed_mcp_servers from agpack.lockfile import read_lockfile from agpack.lockfile import write_lockfile -from agpack.mcp import McpError -from agpack.mcp import cleanup_mcp_server -from agpack.mcp import deploy_mcp_servers +from agpack.patch import Patch +from agpack.patch import match_key +from agpack.registry import list_builtins +from agpack.registry import load_builtin +from agpack.target_schema import TargetDef _MAX_FETCH_WORKERS = 8 -def _source_file_count(deployed: list[str]) -> int: - """Count unique source files from a deployed-paths list. +def _source_file_count(item_path: Path) -> int: + """Count source files in a deploy item. - Deployed paths look like ``//`` where - ```` is always two components (e.g. ``.claude/skills``). - Stripping those two components and deduplicating gives the number of - unique source files, regardless of how many targets received copies. + A file item is 1; a directory item is the number of non-``.git`` files in its tree. Counting on the source side + avoids any assumption about target path depth — user-defined targets can put deployments at any nesting level. """ - if not deployed: - return 0 - return len({Path(f).parts[2:] for f in deployed}) + if item_path.is_file(): + return 1 + return sum( + 1 + for f in item_path.rglob("*") + if f.is_file() and not any(p.startswith(".git") for p in f.relative_to(item_path).parts) + ) @dataclass @@ -73,27 +82,33 @@ class SyncResult: verbose_lines: list[str] = field(default_factory=list) -def _sync_resource_type( +def _sync_resource_type( # noqa: C901 # orchestrator: fetch + deploy + lockfile + progress fan-out deps: list[DependencySource], - detect_fn: Callable[[FetchResult], list[tuple[str, Path]]], - deploy_item_fn: Callable[[str, Path, list[str], Path, bool, bool], list[str]], resource_type: str, - config: AgpackConfig, + target_defs: list[TargetDef], project_root: Path, new_lockfile: Lockfile, progress: Progress, + env_vars: dict[str, str], *, dry_run: bool, verbose: bool, ) -> SyncResult: - """Fetch and deploy a list of dependencies of a single type. + """Fetch and deploy a list of copy-kind dependencies. - On error, writes a partial lockfile (with what has been synced so far) - before raising ClickException. + The resource type's kind (``copy-directory`` / ``copy-file``) is looked up from any target that declares it — all + targets sharing the resource type share the kind (validated by :func:`_resource_kinds`). On error, writes a partial + lockfile (with what has been synced so far) before raising ClickException. """ if not deps: return SyncResult() + # Pick a representative resource for the detect phase — any target supporting this resource type works, since + # _resource_kinds validated they all use the same kind. + detect_resource = next( + target.resources[resource_type] for target in target_defs if resource_type in target.resources + ) + # Phase 1: parallel fetch — collect all results and errors results: list[tuple[DependencySource, FetchResult]] = [] errors: list[str] = [] @@ -109,7 +124,7 @@ def _sync_resource_type( ) with ThreadPoolExecutor(max_workers=min(_MAX_FETCH_WORKERS, len(deps))) as executor: - futures = {executor.submit(fetch_dependency, dep): dep for dep in deps} + futures = {executor.submit(fetch_dependency, dep, env_vars): dep for dep in deps} for future in as_completed(futures): dep = futures[future] tid = task_ids[dep.identity] @@ -126,9 +141,7 @@ def _sync_resource_type( cleanup_fetch(result) if not dry_run: write_lockfile(project_root, new_lockfile) - raise click.ClickException( - "\n".join([f"Failed to fetch {len(errors)} {resource_type}(s):"] + errors) - ) + raise click.ClickException("\n".join([f"Failed to fetch {len(errors)} {resource_type} dep(s):"] + errors)) # Phase 3: sequential deploy — update progress rows, collect verbose output sync = SyncResult() @@ -136,15 +149,14 @@ def _sync_resource_type( tid = task_ids[dep.identity] try: - items = detect_fn(result) + items = detect_items(result, detect_resource, resource_type) except Exception as exc: progress.update(tid, completed=2, icon="[red]✗[/red]", detail=str(exc)) cleanup_fetch(result) if not dry_run: write_lockfile(project_root, new_lockfile) - raise click.ClickException( - f"Error deploying {resource_type} '{dep.name}': {exc}" - ) from exc + msg = f"Error deploying {resource_type} '{dep.name}': {exc}" + raise click.ClickException(msg) from exc is_expanded = len(items) > 1 @@ -166,13 +178,14 @@ def _sync_resource_type( all_deployed: list[str] = [] for idx, (item_name, item_path) in enumerate(items): try: - files = deploy_item_fn( + files = deploy_item( item_name, item_path, - config.targets, + resource_type, + target_defs, project_root, - dry_run, - False, + dry_run=dry_run, + verbose=False, ) except Exception as exc: if is_expanded: @@ -186,14 +199,13 @@ def _sync_resource_type( cleanup_fetch(result) if not dry_run: write_lockfile(project_root, new_lockfile) - raise click.ClickException( - f"Error deploying {resource_type} '{dep.name}': {exc}" - ) from exc + msg = f"Error deploying {resource_type} '{dep.name}': {exc}" + raise click.ClickException(msg) from exc all_deployed.extend(files) if is_expanded: - n = _source_file_count(files) + n = _source_file_count(item_path) progress.update( sub_task_ids[idx], completed=1, @@ -201,13 +213,12 @@ def _sync_resource_type( detail=f"Copied {n} {'file' if n == 1 else 'files'}", ) - # Update parent row — show source file count, not total across targets - src_files = _source_file_count(all_deployed) + # Update parent row — count source files (not multiplied across targets) if is_expanded: - item_label = f"{resource_type}s" if len(items) != 1 else resource_type - detail = f"Copied {len(items)} {item_label}" + detail = f"Copied {len(items)} {resource_type}" sync.count += len(items) else: + src_files = _source_file_count(items[0][1]) file_label = "file" if src_files == 1 else "files" detail = f"Copied {src_files} {file_label}" sync.count += 1 @@ -231,6 +242,114 @@ def _sync_resource_type( return sync +def _sync_edit_resource( + resource_type: str, + patches: list[Patch], + applied_old: list[AppliedPatch], + target_defs: list[TargetDef], + project_root: Path, + new_lockfile: Lockfile, + env_vars: dict[str, str], + *, + dry_run: bool, + verbose: bool, +) -> int: + """Reconcile edit-file patches and record the new applied set. + + Diffs ``patches`` against ``applied_old`` per file and only touches each file once. Returns the count for the + summary line (number of currently-desired patches, regardless of whether the file was rewritten). + """ + if not patches and not applied_old: + return 0 + try: + applied_new = sync_edit_resource( + resource_type, + desired=patches, + applied_old=applied_old, + targets=target_defs, + project_root=project_root, + env_vars=env_vars, + dry_run=dry_run, + verbose=verbose, + ) + except (ConfigError, EditFileError) as exc: + # ConfigError surfaces from ${VAR} resolution inside _resolve_patch; EditFileError covers every other + # apply-time failure (collision detection, JSON/TOML I/O, dotted-key navigation). + if not dry_run: + write_lockfile(project_root, new_lockfile) + raise click.ClickException(str(exc)) from exc + if applied_new: + new_lockfile.edits.append(EditLockEntry(resource_type=resource_type, applied=applied_new)) + return len(patches) + + +def _resource_kinds(targets: list[TargetDef]) -> dict[str, str]: + """Build the resource-type → kind map across all configured targets. + + Raises: + click.ClickException: If two targets declare the same resource + type name with different kinds. Resource type names are + cross-target identifiers; agpack will not treat + ``commands`` as copy-directory for one target and + copy-file for another. + """ + kinds: dict[str, str] = {} + sources: dict[str, str] = {} + for idx, target in enumerate(targets): + target_label = f"target #{idx}" + for rt, resource in target.resources.items(): + seen = kinds.get(rt) + if seen is None: + kinds[rt] = resource.kind + sources[rt] = target_label + elif seen != resource.kind: + msg = ( + f"Resource '{rt}' has conflicting kinds across " + f"configured targets: '{seen}' (from {sources[rt]}) vs " + f"'{resource.kind}' (from {target_label}). All targets " + f"declaring the same resource type must agree on its kind." + ) + raise click.ClickException(msg) + return kinds + + +def _resolve_targets(config: AgpackConfig) -> list[TargetDef]: + """Resolve ``config.targets`` to TargetDef objects. + + Precedence: ``config.target_definitions`` (already merged from project + global) → bundled built-in manifests. + Unknown names raise a ``ClickException`` listing both pools. + + Duplicate names in ``config.targets`` are deduplicated with a warning — listing the same target twice would + otherwise double-apply every ``append`` patch in :func:`sync_edit_resource`. + """ + resolved: list[TargetDef] = [] + seen: set[str] = set() + for name in config.targets: + if name in seen: + console.print( + f"[yellow]warning[/yellow]: target '{name}' is listed multiple times in 'targets:'; using it once." + ) + continue + seen.add(name) + if name in config.target_definitions: + resolved.append(config.target_definitions[name]) + continue + try: + resolved.append(load_builtin(name)) + except TargetSchemaError as exc: + builtins = ", ".join(list_builtins()) + user_defs = ", ".join(sorted(config.target_definitions)) or "(none)" + msg = ( + f"Unknown target '{name}'.\n" + f" Built-in targets: {builtins}\n" + f" Your target_definitions: {user_defs}\n" + "Add an entry under 'target_definitions' in agpack.yml to " + "define a custom target." + ) + raise click.ClickException(msg) from exc + return resolved + + def _load_and_merge_global( config: AgpackConfig, *, @@ -238,15 +357,14 @@ def _load_and_merge_global( ) -> tuple[AgpackConfig, GlobalConfig | None]: """Load the global config and merge it into *config*. - Returns the (possibly merged) config and the raw global config - (needed later for ``.env`` resolution). If the global config - doesn't exist or is disabled, the original config is returned - unchanged together with ``None``. + Returns the (possibly merged) config and the raw global config (needed later for ``.env`` resolution). If the + global config doesn't exist or is disabled, the original config is returned unchanged together with ``None``. """ try: global_cfg = load_global_config() except ConfigError as exc: - raise click.ClickException(f"Global config error: {exc}") from exc + msg = f"Global config error: {exc}" + raise click.ClickException(msg) from exc if global_cfg is not None: if verbose: @@ -277,7 +395,7 @@ def main() -> None: is_flag=True, help="Ignore the global config file.", ) -def sync(dry_run: bool, config_path: str, verbose: bool, no_global: bool) -> None: +def sync(dry_run: bool, config_path: str, verbose: bool, no_global: bool) -> None: # noqa: C901 """Fetch all dependencies and deploy to all target directories.""" cfg_path = Path(config_path).resolve() project_root = cfg_path.parent @@ -294,115 +412,114 @@ def sync(dry_run: bool, config_path: str, verbose: bool, no_global: bool) -> Non if not no_global and config.use_global: config, global_cfg = _load_and_merge_global(config, verbose=verbose) - # 3. Resolve ${VAR} references in config values + # 3. Resolve ${VAR} references in fetch dependencies; patches are resolved per-target at apply time using + # `env_vars` plus the target's own `vars`. try: - resolve_config(config, project_root, global_config=global_cfg, verbose=verbose) + env_vars = resolve_config(config, project_root, global_config=global_cfg, verbose=verbose) except ConfigError as exc: raise click.ClickException(str(exc)) from exc - # 4. Read existing lockfile - old_lockfile = read_lockfile(project_root) + # 4. Resolve target names to manifests + build the kind map + target_defs = _resolve_targets(config) + resource_kinds = _resource_kinds(target_defs) - # 5. Build set of current dependency identities + # 5. Read existing lockfile and build the set of current fetch-dependency identities (copy kinds only — patches + # are tracked separately under lockfile.edits). + old_lockfile = read_lockfile(project_root) current_identities: set[str] = set() - for dep in [*config.skills, *config.commands, *config.agents]: - current_identities.add(dep.identity) + for deps_list in config.dependencies.values(): + for dep in deps_list: + if isinstance(dep, DependencySource): + current_identities.add(dep.identity) - current_mcp_names = {m.name for m in config.mcp} - - # 6. Clean up removed dependencies + # 6. Clean up removed copy-kind dependencies. removed_deps = find_removed_dependencies(old_lockfile, current_identities) for entry in removed_deps: if verbose or dry_run: console.print(f"Removing {entry.type} '{entry.identity}'...") - cleanup_deployed_files( - entry.deployed_files, project_root, dry_run=dry_run, verbose=verbose - ) - - # 7. Clean up removed MCP servers - removed_mcp = find_removed_mcp_servers(old_lockfile, current_mcp_names) - for mcp_entry in removed_mcp: - if verbose or dry_run: - console.print(f"Removing MCP server '{mcp_entry.name}'...") - cleanup_mcp_server( - mcp_entry.name, - mcp_entry.targets, - project_root, - config.targets, - dry_run=dry_run, - verbose=verbose, - ) - - # 8. Fetch and deploy dependencies + cleanup_deployed_files(entry.deployed_files, project_root, dry_run=dry_run, verbose=verbose) + + # 7. Index old applied edits by resource type. We don't unapply them here — sync_edit_resource diffs against this + # and touches each file at most once. Files only get written when their text actually changes (combined with + # tomlkit for TOML, comments and key ordering on untouched sections survive across syncs). + old_applied_by_rt: dict[str, list[AppliedPatch]] = {} + if old_lockfile is not None: + for edit in old_lockfile.edits: + old_applied_by_rt[edit.resource_type] = list(edit.applied) + + # 8. Fetch and deploy dependencies in YAML order. Copy kinds go through fetch+detect+deploy; edit-file kinds run + # the diff-based sync against the lockfile's prior applied state. new_lockfile = Lockfile() counts: dict[str, int] = {} - - resource_types = [ - (config.skills, detect_skill_items, deploy_single_skill, "skill"), - (config.commands, detect_command_items, deploy_single_command, "command"), - (config.agents, detect_agent_items, deploy_single_agent, "agent"), - ] - all_verbose_lines: list[str] = [] with create_sync_progress() as progress: - for deps, detect_fn, deploy_item_fn, resource_type in resource_types: + for resource_type in config.dependencies: + kind = resource_kinds.get(resource_type) + if kind is None: + # Configured but no target supports it. With open-ended resource type names this is almost always a + # typo (``mpc`` instead of ``mcp``), so warn loudly rather than silently dropping the entries. + if config.dependencies[resource_type]: + known = ", ".join(sorted(resource_kinds)) or "(none)" + console.print( + f"[yellow]warning[/yellow]: dependency " + f"'{resource_type}' is configured but no target " + f"declares it — entries will be skipped. " + f"Resource types declared by current targets: " + f"{known}." + ) + continue + entries = config.dependencies[resource_type] + if kind == "edit-file": + counts[resource_type] = _sync_edit_resource( + resource_type, + [e for e in entries if isinstance(e, Patch)], + old_applied_by_rt.pop(resource_type, []), + target_defs, + project_root, + new_lockfile, + env_vars, + dry_run=dry_run, + verbose=verbose, + ) + continue + # Copy kind. + fetch_entries = [e for e in entries if isinstance(e, DependencySource)] sync = _sync_resource_type( - deps, - detect_fn, - deploy_item_fn, + fetch_entries, resource_type, - config, + target_defs, project_root, new_lockfile, progress, + env_vars, dry_run=dry_run, verbose=verbose, ) counts[resource_type] = sync.count all_verbose_lines.extend(sync.verbose_lines) + # 8b. Resource types that existed in the old lockfile but are gone from the current config — unapply every patch + # they recorded so the user's structured config files don't keep stale agpack content. + for resource_type, leftovers in old_applied_by_rt.items(): + if not leftovers: + continue + if verbose or dry_run: + console.print(f"Removing all '{resource_type}' patches (no longer in config)...") + cleanup_orphaned_edits(leftovers, project_root, dry_run=dry_run, verbose=verbose) + for line in all_verbose_lines: console.print(line) - # MCP servers - mcp_count = 0 - if config.mcp: - with console.status("Deploying MCP servers..."): - try: - mcp_result = deploy_mcp_servers( - config.mcp, - config.targets, - project_root, - dry_run=dry_run, - verbose=verbose, - ) - except McpError as exc: - if not dry_run: - write_lockfile(project_root, new_lockfile) - raise click.ClickException(str(exc)) from exc - - for server_name, target_paths in mcp_result.items(): - new_lockfile.mcp.append( - McpLockEntry(name=server_name, targets=target_paths) - ) - mcp_count += 1 - - # 9. Write lockfile + # 9. Write lockfile. if not dry_run: write_lockfile(project_root, new_lockfile) - # 10. Summary + # 10. Summary — one chip per resource type that had configured deps. elapsed = time.monotonic() - start_time target_count = len(config.targets) - items = [ - ("skills", "skill"), - ("commands", "command"), - ("agents", "agent"), - ] - parts = [f"[bold]{counts.get(k, 0)}[/bold] {name}" for name, k in items] - parts.append(f"[bold]{mcp_count}[/bold] MCP servers") - summary = ", ".join(parts) + parts = [f"[bold]{counts.get(rt, 0)}[/bold] {rt}" for rt in config.dependencies] + summary = ", ".join(parts) if parts else "[dim]nothing to deploy[/dim]" targets = f"[bold]{target_count}[/bold] targets" console.print() console.print(Rule(f"{summary} → {targets} [dim]({elapsed:.2f}s)[/dim]")) @@ -421,7 +538,7 @@ def sync(dry_run: bool, config_path: str, verbose: bool, no_global: bool) -> Non is_flag=True, help="Ignore the global config file.", ) -def status(config_path: str, no_global: bool) -> None: +def status(config_path: str, no_global: bool) -> None: # noqa: C901 """Show the current state of installed resources vs the config.""" cfg_path = Path(config_path).resolve() project_root = cfg_path.parent @@ -431,31 +548,26 @@ def status(config_path: str, no_global: bool) -> None: except ConfigError as exc: raise click.ClickException(str(exc)) from exc - # Load and merge global config + global_cfg: GlobalConfig | None = None if not no_global and config.use_global: - config, _ = _load_and_merge_global(config) + config, global_cfg = _load_and_merge_global(config) + + target_defs = _resolve_targets(config) + env_vars = build_env(project_root, global_cfg.config_dir if global_cfg else None) lockfile = read_lockfile(project_root) - # Build lookup from lockfile installed_map: dict[str, InstalledEntry] = {} + edits_by_type: dict[str, EditLockEntry] = {} if lockfile: for entry in lockfile.installed: installed_map[entry.identity] = entry + for edit in lockfile.edits: + edits_by_type[edit.resource_type] = edit - mcp_map: dict[str, McpLockEntry] = {} - if lockfile: - for mcp_entry in lockfile.mcp: - mcp_map[mcp_entry.name] = mcp_entry - - resource_sections = [ - ("Skills", config.skills), - ("Commands", config.commands), - ("Agents", config.agents), - ] - for label, deps in resource_sections: + for resource_type, deps in config.dependencies.items(): table = Table( - title=label, + title=resource_type.capitalize(), title_style="bold", show_header=False, box=None, @@ -468,49 +580,64 @@ def status(config_path: str, no_global: bool) -> None: table.add_row("[dim]no dependencies configured[/dim]") else: for dep in deps: - installed = installed_map.get(dep.identity) - if installed: - short_ref = installed.resolved_ref[:7] - table.add_row( - f"[green]✓[/green] {dep.name}", - f"{dep.url} @ {short_ref}", - ) + if isinstance(dep, DependencySource): + installed = installed_map.get(dep.identity) + if installed: + short_ref = installed.resolved_ref[:7] + table.add_row( + f"[green]✓[/green] {dep.name}", + f"{dep.url} @ {short_ref}", + ) + else: + table.add_row( + f"[red]✗[/red] {dep.name}", + "not yet synced", + ) else: - table.add_row( - f"[red]✗[/red] {dep.name}", - "not yet synced", - ) + synced = _is_patch_synced(dep, resource_type, target_defs, edits_by_type, env_vars) + mark = "[green]✓[/green]" if synced else "[red]✗[/red]" + detail = dep.strategy if synced else "not yet synced" + table.add_row(f"{mark} {dep.key}", detail) console.print(table) console.print() - # MCP - table = Table( - title="MCP Servers", - title_style="bold", - show_header=False, - box=None, - padding=(0, 1), - title_justify="left", - ) - table.add_column(style="bold", no_wrap=True) - table.add_column(style="dim") - if not config.mcp: - table.add_row("[dim]no servers configured[/dim]") - else: - for server in config.mcp: - mcp_installed = mcp_map.get(server.name) - if mcp_installed and mcp_installed.targets: - targets_str = ", ".join(mcp_installed.targets) - table.add_row( - f"[green]✓[/green] {server.name}", - f"→ {targets_str}", - ) - else: - table.add_row( - f"[red]✗[/red] {server.name}", - "not yet synced", - ) - console.print(table) + +def _is_patch_synced( + dep: Patch, + resource_type: str, + target_defs: list[TargetDef], + edits_by_type: dict[str, EditLockEntry], + env_vars: dict[str, str], +) -> bool: + """True iff every target that owns this resource type has the patch recorded in the lockfile. + + The dep is resolved per-owner-target via :meth:`EditFileResource.patch_identity` so any target-specific value + substitution participates in the identity hash. + """ + recorded = edits_by_type.get(resource_type) + if recorded is None: + return False + applied_by_id: dict[tuple[str, tuple[Any, ...]], AppliedPatch] = { + (ap.file_path, match_key(ap.strategy, ap.key, ap.value_hash)): ap for ap in recorded.applied + } + owners = [t for t in target_defs if isinstance(t.resources.get(resource_type), EditFileResource)] + if not owners: + return False + for target in owners: + resource = target.resources[resource_type] + assert isinstance(resource, EditFileResource) # narrowed above + try: + identity = resource.patch_identity(dep, env_vars) + except (ConfigError, EditFileError): + return False + if (resource.path, identity) not in applied_by_id: + return False + return True + + +def _read_init_template(name: str) -> str: + """Read an ``init`` scaffold from ``agpack/templates/``.""" + return importlib_files("agpack.templates").joinpath(name).read_text(encoding="utf-8") @main.command() @@ -529,47 +656,14 @@ def status(config_path: str, no_global: bool) -> None: ) def init(config_path: str, is_global: bool) -> None: """Scaffold a new agpack.yml.""" - from agpack.config import _resolve_global_config_path - if is_global: - path = _resolve_global_config_path() + path = resolve_global_config_path() if path.exists(): console.print(f"{path} already exists — doing nothing.") return path.parent.mkdir(parents=True, exist_ok=True) - - template = """\ -# Global agpack config — dependencies here are included in every project. -# Override per-project with 'global: false' in your project agpack.yml, -# or run agpack sync --no-global. - -dependencies: - skills: - # Shared skills available in all projects: - # - url: https://github.com/owner/repo - # path: skills/my-skill - # ref: v1.0.0 - - commands: - # Shared commands available in all projects: - # - url: https://github.com/owner/repo - # path: commands/my-command.md - - agents: - # Shared agents available in all projects: - # - url: https://github.com/owner/repo - # path: agents/my-agent.md - - mcp: - # Shared MCP servers available in all projects: - # - name: my-server - # command: npx - # args: ["-y", "@example/mcp-server"] - # env: - # API_KEY: ${API_KEY} # resolved from .env or shell environment -""" - path.write_text(template, encoding="utf-8") + path.write_text(_read_init_template("init_global.yml"), encoding="utf-8") console.print(f"[green]✓[/green] Created [bold]{path}[/bold]") return @@ -578,48 +672,168 @@ def init(config_path: str, is_global: bool) -> None: console.print(f"{path.name} already exists — doing nothing.") return - template = """\ -# Set to false to ignore the global config (~/.config/agpack/agpack.yml): -# global: false - -targets: - # - claude - # - opencode - # - codex - # - cursor - # - copilot - -dependencies: - skills: - # Point to a single skill folder: - # - url: https://github.com/owner/repo - # path: skills/my-skill - # ref: v1.0.0 - # Multiple URLs (tried in order, e.g. HTTPS + SSH fallback): - # - url: - # - https://github.com/owner/repo - # - git@github.com:owner/repo.git - # path: skills/my-skill - # Or to a directory of skill folders (each subfolder is deployed separately): - # - url: https://github.com/owner/repo - # path: skills - - commands: - # Point to a single file or a directory of command files: - # - url: https://github.com/owner/repo - # path: commands/my-command.md - - agents: - # Point to a single file or a directory of agent files: - # - url: https://github.com/owner/repo - # path: agents/my-agent.md - - mcp: - # - name: my-server - # command: npx - # args: ["-y", "@example/mcp-server"] - # env: - # API_KEY: ${API_KEY} # resolved from .env or shell environment -""" - path.write_text(template, encoding="utf-8") + path.write_text(_read_init_template("init_project.yml"), encoding="utf-8") console.print(f"[green]✓[/green] Created [bold]{path.name}[/bold]") + + +# --------------------------------------------------------------------------- +# `agpack targets` — inspect available target manifests +# --------------------------------------------------------------------------- + + +def _load_user_target_definitions(config_path: str, no_global: bool) -> dict[str, TargetDef]: + """Load target_definitions from project + global config. + + A missing project config is fine (the user may not have run ``init`` yet) but a broken one is not — config errors + are propagated so the user sees the parse failure instead of silently losing their target_definitions. Project + entries win by name over global. + """ + cfg_path = Path(config_path).resolve() + project_defs: dict[str, TargetDef] = {} + include_global = not no_global + + if cfg_path.exists(): + try: + config = load_config(cfg_path) + except ConfigError as exc: + raise click.ClickException(str(exc)) from exc + project_defs = dict(config.target_definitions) + include_global = include_global and config.use_global + + if not include_global: + return project_defs + + try: + global_cfg = load_global_config() + except ConfigError as exc: + msg = f"Global config error: {exc}" + raise click.ClickException(msg) from exc + + if global_cfg is None: + return project_defs + + merged = dict(project_defs) + for name, td in global_cfg.target_definitions.items(): + merged.setdefault(name, td) + return merged + + +def _load_raw_target_definition(name: str, config_path: str, no_global: bool) -> dict[str, Any] | None: + """Re-read the raw YAML for a user-defined target, or None if absent. + + Used by ``targets show`` so we can print exactly what the user wrote rather than round-tripping through TargetDef + and reconstructing. + """ + cfg_path = Path(config_path).resolve() + include_global = not no_global + if cfg_path.exists(): + data = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {} + td = (data.get("target_definitions") or {}).get(name) + if isinstance(td, dict): + return td + include_global = include_global and data.get("global", True) + + if not include_global: + return None + + global_path = resolve_global_config_path() + if not global_path.exists(): + return None + data = yaml.safe_load(global_path.read_text(encoding="utf-8")) or {} + td = (data.get("target_definitions") or {}).get(name) + return td if isinstance(td, dict) else None + + +def _read_builtin_yaml(name: str) -> str: + """Return the on-disk YAML text for a built-in target.""" + return importlib_files("agpack.builtin_targets").joinpath(f"{name}.yml").read_text(encoding="utf-8") + + +def _resource_summary(target: TargetDef) -> str: + if not target.resources: + return "[dim]none[/dim]" + return ", ".join(target.resources) + + +@main.group() +def targets() -> None: + """Inspect available target manifests (built-in and user-defined).""" + + +@targets.command("list") +@click.option( + "--config", + "config_path", + default="./agpack.yml", + type=click.Path(), + help="Path to config file (used to discover user target_definitions).", +) +@click.option( + "--no-global", + is_flag=True, + help="Ignore the global config file.", +) +def targets_list(config_path: str, no_global: bool) -> None: + """List all available targets — built-ins and user-defined.""" + user_defs = _load_user_target_definitions(config_path, no_global) + builtin_names = set(list_builtins()) + + table = Table( + title="Available targets", + title_style="bold", + show_header=True, + header_style="bold", + box=None, + padding=(0, 2), + title_justify="left", + ) + table.add_column("Name") + table.add_column("Source") + table.add_column("Resources") + + all_names = sorted(builtin_names | set(user_defs)) + for name in all_names: + if name in user_defs: + target = user_defs[name] + source = "[yellow]user (overrides built-in)[/yellow]" if name in builtin_names else "[cyan]user[/cyan]" + else: + target = load_builtin(name) + source = "[dim]built-in[/dim]" + table.add_row(name, source, _resource_summary(target)) + + console.print(table) + + +@targets.command("show") +@click.argument("name") +@click.option( + "--config", + "config_path", + default="./agpack.yml", + type=click.Path(), + help="Path to config file (used to discover user target_definitions).", +) +@click.option( + "--no-global", + is_flag=True, + help="Ignore the global config file.", +) +def targets_show(name: str, config_path: str, no_global: bool) -> None: + """Print the resolved manifest for *name* as YAML. + + Useful as a starting point for copying into ``target_definitions:`` to customise a built-in. + """ + raw = _load_raw_target_definition(name, config_path, no_global) + if raw is not None: + click.echo(yaml.safe_dump(raw, default_flow_style=False, sort_keys=False), nl=False) + return + + if name in list_builtins(): + click.echo(_read_builtin_yaml(name), nl=False) + return + + user_defs = _load_user_target_definitions(config_path, no_global) + builtins = ", ".join(list_builtins()) + user_names = ", ".join(sorted(user_defs)) or "(none)" + msg = f"Unknown target '{name}'.\n Built-in targets: {builtins}\n Your target_definitions: {user_names}" + raise click.ClickException(msg) diff --git a/agpack/config.py b/agpack/config.py index 48f2367..63c1e90 100644 --- a/agpack/config.py +++ b/agpack/config.py @@ -1,7 +1,14 @@ -"""agpack.yml parsing and validation.""" +"""``agpack.yml`` parsing, validation, and fetch-dependency ``${VAR}`` validation. + +Pulls together the leaf modules into the load-time pipeline: parsed YAML → typed :class:`AgpackConfig` / +:class:`GlobalConfig` → eager fetch-dep ``${VAR}`` validation that returns the merged env table for downstream +apply-time substitution. The substitution primitives themselves live in :mod:`agpack.envsubst`; this module +sequences them. +""" from __future__ import annotations +import json import os from dataclasses import dataclass from dataclasses import field @@ -10,7 +17,13 @@ import yaml -from agpack.targets import VALID_TARGETS +from agpack.envsubst import build_env +from agpack.envsubst import resolve_env_vars +from agpack.errors import ConfigError +from agpack.errors import TargetSchemaError +from agpack.patch import Patch +from agpack.target_schema import TargetDef +from agpack.target_schema import parse_target_def # --------------------------------------------------------------------------- # Constants @@ -18,6 +31,7 @@ DEFAULT_GLOBAL_CONFIG_DIR = Path.home() / ".config" / "agpack" + # --------------------------------------------------------------------------- # Data classes # --------------------------------------------------------------------------- @@ -25,12 +39,10 @@ @dataclass class DependencySource: - """A parsed skill, command, or agent dependency. + """A parsed skill, command, or agent dependency (copy-kind input). - The ``urls`` list contains one or more git clone URLs. The first - entry is the canonical (primary) URL used for identity and display. - Remaining entries are fallback URLs tried in order when earlier ones - fail. + The ``urls`` list contains one or more git clone URLs. The first entry is the canonical (primary) URL used for + identity and display. Remaining entries are fallback URLs tried in order when earlier ones fail. """ urls: list[str] @@ -47,7 +59,6 @@ def name(self) -> str: """Derive the resource name (last path segment, or url basename).""" if self.path: return self.path.rstrip("/").rsplit("/", 1)[-1] - # Strip trailing .git and take the last segment of the URL cleaned = self.url.rstrip("/") if cleaned.endswith(".git"): cleaned = cleaned[:-4] @@ -62,28 +73,25 @@ def identity(self) -> str: return key -@dataclass -class McpServer: - """An MCP server definition.""" - - name: str - type: str = "stdio" # stdio | sse | http - command: str | None = None - args: list[str] = field(default_factory=list) - env: dict[str, str] = field(default_factory=dict) - url: str | None = None +# A dependency entry under ``dependencies.`` is either a fetched resource (``DependencySource``) for copy kinds, +# or a structured patch (``Patch``) for edit-file kinds. +DependencyEntry = DependencySource | Patch @dataclass class AgpackConfig: - """Parsed and validated agpack.yml.""" + """Parsed and validated agpack.yml. + + ``dependencies`` is an open dict keyed by resource type name. Each entry is either a :class:`DependencySource` + (copy kinds — fetched from git) or a :class:`Patch` (edit-file kinds — applied to a structured config file). All + entries under a given key must be of the same type; the actual kind is enforced at sync time against the target + manifest. + """ targets: list[str] - skills: list[DependencySource] = field(default_factory=list) - commands: list[DependencySource] = field(default_factory=list) - agents: list[DependencySource] = field(default_factory=list) - mcp: list[McpServer] = field(default_factory=list) + dependencies: dict[str, list[DependencyEntry]] = field(default_factory=dict) use_global: bool = True + target_definitions: dict[str, TargetDef] = field(default_factory=dict) @dataclass @@ -93,206 +101,208 @@ class GlobalConfig: Contains only dependencies — no targets. """ - skills: list[DependencySource] = field(default_factory=list) - commands: list[DependencySource] = field(default_factory=list) - agents: list[DependencySource] = field(default_factory=list) - mcp: list[McpServer] = field(default_factory=list) + dependencies: dict[str, list[DependencyEntry]] = field(default_factory=dict) + target_definitions: dict[str, TargetDef] = field(default_factory=dict) config_dir: Path = field(default_factory=lambda: DEFAULT_GLOBAL_CONFIG_DIR) """Directory containing the global config (used to locate .env).""" # --------------------------------------------------------------------------- -# Validation errors +# YAML parsing # --------------------------------------------------------------------------- -class ConfigError(Exception): - """Raised when agpack.yml is invalid.""" - - -# --------------------------------------------------------------------------- -# Parsing -# --------------------------------------------------------------------------- - - -def _parse_dependency(raw: dict[str, Any], context: str) -> DependencySource: - """Parse a single dependency entry (object form). - - Args: - raw: The raw YAML dict for this dependency. - context: Human-readable location for error messages (e.g. "skills[0]"). - """ - if not isinstance(raw, dict): - raise ConfigError( - f"{context}: expected an object with 'url' key, got {type(raw).__name__}" - ) - +def _parse_fetch_entry(raw: dict[str, Any], context: str) -> DependencySource: + """Parse a fetch (copy-kind) dependency entry: ``url`` (+ optional path/ref).""" raw_url = raw.get("url") - if raw_url is None: - raise ConfigError(f"{context}: missing required field 'url'") - if isinstance(raw_url, str): if not raw_url: - raise ConfigError(f"{context}: 'url' must not be empty") + msg = f"{context}: 'url' must not be empty" + raise ConfigError(msg) urls = [raw_url] elif isinstance(raw_url, list): if not raw_url: - raise ConfigError(f"{context}: 'url' must not be empty") + msg = f"{context}: 'url' must not be empty" + raise ConfigError(msg) urls = [str(u) for u in raw_url] else: - raise ConfigError(f"{context}: 'url' must be a string or list of strings") + msg = f"{context}: 'url' must be a string or list of strings" + raise ConfigError(msg) path = raw.get("path") if path is not None and not isinstance(path, str): - raise ConfigError(f"{context}: 'path' must be a string") + msg = f"{context}: 'path' must be a string" + raise ConfigError(msg) ref = raw.get("ref") if ref is not None: ref = str(ref) + known = {"url", "path", "ref"} + extra = set(raw) - known + if extra: + msg = f"{context}: unknown fields {sorted(extra)}" + raise ConfigError(msg) + return DependencySource(urls=urls, path=path, ref=ref) -def _parse_mcp(raw: dict[str, Any], context: str) -> McpServer: - """Parse a single MCP server entry.""" - if not isinstance(raw, dict): - raise ConfigError(f"{context}: expected an object, got {type(raw).__name__}") - - name = raw.get("name") - if not name: - raise ConfigError(f"{context}: missing required field 'name'") - - server_type = str(raw.get("type", "stdio")) - if server_type not in ("stdio", "sse", "http"): - raise ConfigError( - f"{context}: 'type' must be 'stdio', 'sse', or 'http', got '{server_type}'" - ) - - if server_type == "stdio": - command = raw.get("command") - if not command: - raise ConfigError( - f"{context}: stdio MCP server '{name}'" - " is missing required field 'command'" - ) - return McpServer( - name=name, - type=server_type, - command=str(command), - args=[str(a) for a in raw.get("args", [])], - env={str(k): str(v) for k, v in raw.get("env", {}).items()}, - ) - else: - url = raw.get("url") - if not url: - raise ConfigError( - f"{context}: {server_type} MCP server '{name}'" - " is missing required field 'url'" - ) - return McpServer( - name=name, - type=server_type, - url=str(url), - ) - - -def _parse_dependencies( - deps: dict[str, Any], prefix: str = "" -) -> tuple[ - list[DependencySource], - list[DependencySource], - list[DependencySource], - list[McpServer], -]: - """Parse all dependency lists from a ``dependencies`` mapping. - - Args: - deps: The raw ``dependencies`` dict from YAML. - prefix: Optional prefix for error context (e.g. ``"global "``). - - Returns: - A tuple of (skills, commands, agents, mcp). +_VALID_STRATEGIES = ("replace", "append") + + +def _parse_patch_entry(raw: dict[str, Any], context: str) -> Patch: + """Parse an edit-file entry: ``key``, ``value``, optional ``strategy``.""" + key = raw.get("key") + if not isinstance(key, str) or not key: + msg = f"{context}: 'key' must be a non-empty string" + raise ConfigError(msg) + + if "value" not in raw: + msg = f"{context}: missing required field 'value'" + raise ConfigError(msg) + + strategy = raw.get("strategy", "replace") + if strategy not in _VALID_STRATEGIES: + msg = f"{context}: 'strategy' must be one of {_VALID_STRATEGIES}, got {strategy!r}" + raise ConfigError(msg) + + known = {"key", "value", "strategy"} + extra = set(raw) - known + if extra: + msg = f"{context}: unknown fields {sorted(extra)}" + raise ConfigError(msg) + + return Patch(key=key, value=raw["value"], strategy=strategy) + + +def _parse_dependency_entry(raw: Any, context: str) -> DependencyEntry: + """Parse one entry; the shape decides fetch vs patch. + + An entry with ``url`` is a fetch entry (copy kind). An entry with ``key`` is a patch entry (edit-file kind). + Anything else is an error. """ - skills = [ - _parse_dependency(s, f"{prefix}dependencies.skills[{i}]") - for i, s in enumerate(deps.get("skills") or []) - ] - commands = [ - _parse_dependency(c, f"{prefix}dependencies.commands[{i}]") - for i, c in enumerate(deps.get("commands") or []) - ] - agents = [ - _parse_dependency(a, f"{prefix}dependencies.agents[{i}]") - for i, a in enumerate(deps.get("agents") or []) - ] - mcp = [ - _parse_mcp(m, f"{prefix}dependencies.mcp[{i}]") - for i, m in enumerate(deps.get("mcp") or []) - ] - return skills, commands, agents, mcp + if not isinstance(raw, dict): + msg = f"{context}: expected an object, got {type(raw).__name__}" + raise ConfigError(msg) + has_url = "url" in raw + has_key = "key" in raw + if has_url and has_key: + msg = f"{context}: entry has both 'url' (fetch) and 'key' (patch) — these are mutually exclusive" + raise ConfigError(msg) + if has_url: + return _parse_fetch_entry(raw, context) + if has_key: + return _parse_patch_entry(raw, context) + msg = f"{context}: entry must have either 'url' (fetch) or 'key' (patch)" + raise ConfigError(msg) + + +def _parse_target_definitions(raw: Any, prefix: str = "") -> dict[str, TargetDef]: + """Parse a ``target_definitions`` mapping into TargetDef objects.""" + if raw is None: + return {} + if not isinstance(raw, dict): + msg = f"{prefix}target_definitions: must be a mapping, got {type(raw).__name__}" + raise ConfigError(msg) + result: dict[str, TargetDef] = {} + for key, value in raw.items(): + context = f"{prefix}target_definitions.{key}" + if not isinstance(key, str) or not key: + msg = f"{context}: target name must be a non-empty string" + raise ConfigError(msg) + try: + result[key] = parse_target_def(value, name=key, context=context) + except TargetSchemaError as exc: + raise ConfigError(str(exc)) from exc -def load_config(path: Path) -> AgpackConfig: - """Load and validate agpack.yml. + return result - Args: - path: Path to the agpack.yml file. - Returns: - A validated AgpackConfig. +def _parse_dependencies(deps: dict[str, Any], prefix: str = "") -> dict[str, list[DependencyEntry]]: + """Parse the ``dependencies`` mapping into a {resource_type: [entries]} dict. - Raises: - ConfigError: If the config is invalid. + Each entry is either a :class:`DependencySource` (fetch) or a :class:`Patch` (patch) depending on whether it has + ``url:`` or ``key:``. Mixed lists are rejected — a resource type is either fetch-only or patch-only. + + Patch duplicates are *not* rejected here; they're caught at apply time in + :meth:`agpack.kinds.edit_file.EditFileResource.sync_patches`, where the resolved keys (post ``${var}`` + substitution) are known. """ + out: dict[str, list[DependencyEntry]] = {} + for rt, raw_list in deps.items(): + if not isinstance(rt, str) or not rt: + msg = f"{prefix}dependencies: keys must be non-empty strings, got {rt!r}" + raise ConfigError(msg) + items = raw_list or [] + entries: list[DependencyEntry] = [ + _parse_dependency_entry(item, f"{prefix}dependencies.{rt}[{i}]") for i, item in enumerate(items) + ] + # Reject mixed fetch + patch within one resource type. + if entries: + first_kind = type(entries[0]) + for i, entry in enumerate(entries[1:], 1): + if type(entry) is not first_kind: + msg = ( + f"{prefix}dependencies.{rt}[{i}]: cannot mix fetch and " + f"patch entries under the same resource type" + ) + raise ConfigError(msg) + out[rt] = entries + return out + + +def load_config(path: Path) -> AgpackConfig: + """Load and validate agpack.yml.""" if not path.exists(): - raise ConfigError(f"Config file not found: {path}") + msg = f"Config file not found: {path}" + raise ConfigError(msg) try: data = yaml.safe_load(path.read_text(encoding="utf-8")) except yaml.YAMLError as exc: - raise ConfigError(f"Failed to parse YAML: {exc}") from exc + msg = f"Failed to parse YAML: {exc}" + raise ConfigError(msg) from exc if not isinstance(data, dict): - raise ConfigError("Config file must be a YAML mapping") + msg = "Config file must be a YAML mapping" + raise ConfigError(msg) - # Targets targets = data.get("targets") if not targets or not isinstance(targets, list): - raise ConfigError("Missing or invalid 'targets' (must be a list)") + msg = "Missing or invalid 'targets' (must be a list)" + raise ConfigError(msg) for t in targets: - if t not in VALID_TARGETS: - raise ConfigError( - f"Unrecognised target '{t}'. Valid targets: {sorted(VALID_TARGETS)}" - ) + if not isinstance(t, str) or not t: + msg = f"'targets' entries must be non-empty strings, got {t!r}" + raise ConfigError(msg) - # Global config opt-out use_global = data.get("global", True) if not isinstance(use_global, bool): - raise ConfigError("'global' must be true or false") + msg = "'global' must be true or false" + raise ConfigError(msg) - # Dependencies deps = data.get("dependencies", {}) if not isinstance(deps, dict): - raise ConfigError("'dependencies' must be a mapping") + msg = "'dependencies' must be a mapping" + raise ConfigError(msg) - skills, commands, agents, mcp = _parse_dependencies(deps) + dependencies = _parse_dependencies(deps) + target_definitions = _parse_target_definitions(data.get("target_definitions")) return AgpackConfig( targets=targets, - skills=skills, - commands=commands, - agents=agents, - mcp=mcp, + dependencies=dependencies, use_global=use_global, + target_definitions=target_definitions, ) -def _resolve_global_config_path() -> Path: +def resolve_global_config_path() -> Path: """Return the global config file path. Respects the ``AGPACK_GLOBAL_CONFIG`` environment variable. - Falls back to ``~/.config/agpack/agpack.yml``. """ override = os.environ.get("AGPACK_GLOBAL_CONFIG") if override: @@ -301,22 +311,9 @@ def _resolve_global_config_path() -> Path: def load_global_config(path: Path | None = None) -> GlobalConfig | None: - """Load the global agpack config. - - Args: - path: Explicit path to the global config file. - If *None*, the path is resolved via ``AGPACK_GLOBAL_CONFIG`` - or the default ``~/.config/agpack/agpack.yml``. - - Returns: - A :class:`GlobalConfig` if the file exists and is valid, - or *None* if the file does not exist. - - Raises: - ConfigError: If the file exists but is malformed. - """ + """Load the global agpack config (or None if missing).""" if path is None: - path = _resolve_global_config_path() + path = resolve_global_config_path() if not path.exists(): return None @@ -324,26 +321,27 @@ def load_global_config(path: Path | None = None) -> GlobalConfig | None: try: data = yaml.safe_load(path.read_text(encoding="utf-8")) except yaml.YAMLError as exc: - raise ConfigError(f"Failed to parse global config YAML: {exc}") from exc + msg = f"Failed to parse global config YAML: {exc}" + raise ConfigError(msg) from exc - # An empty file yields None from safe_load if data is None: return GlobalConfig(config_dir=path.parent) if not isinstance(data, dict): - raise ConfigError("Global config file must be a YAML mapping") + msg = "Global config file must be a YAML mapping" + raise ConfigError(msg) deps = data.get("dependencies", {}) if not isinstance(deps, dict): - raise ConfigError("Global config 'dependencies' must be a mapping") + msg = "Global config 'dependencies' must be a mapping" + raise ConfigError(msg) - skills, commands, agents, mcp = _parse_dependencies(deps, prefix="global ") + dependencies = _parse_dependencies(deps, prefix="global ") + target_definitions = _parse_target_definitions(data.get("target_definitions"), prefix="global ") return GlobalConfig( - skills=skills, - commands=commands, - agents=agents, - mcp=mcp, + dependencies=dependencies, + target_definitions=target_definitions, config_dir=path.parent, ) @@ -351,43 +349,104 @@ def load_global_config(path: Path | None = None) -> GlobalConfig | None: def merge_configs(project: AgpackConfig, global_cfg: GlobalConfig) -> AgpackConfig: """Merge a global config into a project config. - Global dependencies are appended after project dependencies. - Duplicates are resolved in favour of the project config: - - - Dependencies are deduplicated by :attr:`DependencySource.identity`. - - MCP servers are deduplicated by :attr:`McpServer.name`. + Global dependencies are appended after project dependencies. Fetch entries are deduplicated by + :attr:`DependencySource.identity`; patch entries are deduplicated by ``(key, value, strategy)`` content. Project + entries always win on conflict. Returns a **new** :class:`AgpackConfig`; the inputs are not mutated. """ - project_mcp_names = {m.name for m in project.mcp} - - def _merge_deps( - project_list: list[DependencySource], - global_list: list[DependencySource], - ) -> list[DependencySource]: - seen = {d.identity for d in project_list} - merged = list(project_list) - for dep in global_list: - if dep.identity not in seen: - merged.append(dep) - seen.add(dep.identity) - return merged - - skills = _merge_deps(project.skills, global_cfg.skills) - commands = _merge_deps(project.commands, global_cfg.commands) - agents = _merge_deps(project.agents, global_cfg.agents) - - # Merge MCP — project names take precedence - mcp = list(project.mcp) - for server in global_cfg.mcp: - if server.name not in project_mcp_names: - mcp.append(server) + dependencies: dict[str, list[DependencyEntry]] = {rt: list(deps) for rt, deps in project.dependencies.items()} + for rt, global_deps in global_cfg.dependencies.items(): + bucket = dependencies.setdefault(rt, []) + seen: set[Any] = {_dedup_key(e) for e in bucket} + for dep in global_deps: + k = _dedup_key(dep) + if k not in seen: + bucket.append(dep) + seen.add(k) + + target_definitions = dict(project.target_definitions) + for name, target in global_cfg.target_definitions.items(): + if name not in target_definitions: + target_definitions[name] = target return AgpackConfig( targets=project.targets, - skills=skills, - commands=commands, - agents=agents, - mcp=mcp, + dependencies=dependencies, use_global=project.use_global, + target_definitions=target_definitions, + ) + + +def _dedup_key(entry: DependencyEntry) -> Any: + """Return a hashable identity for an entry, for dedup during merge. + + Fetch entries are deduped by url+path. Patch entries depend on + strategy: + + - ``replace``: deduped by ``key`` — two patches that both set the + same key are duplicates (project wins). + - ``append``: deduped by ``(key, value)`` — two appends at the + same key with different values are *distinct* entries. + """ + if isinstance(entry, DependencySource): + return ("fetch", entry.identity) + if entry.strategy == "replace": + return ("patch", "replace", entry.key) + # ``default=str`` covers any oddballs that aren't natively JSON-encodable (datetimes, custom scalars from YAML). + # The output is a deterministic string identity for dict-key use, not something we round-trip. + value_repr = json.dumps(entry.value, sort_keys=True, default=str) + return ("patch", "append", entry.key, value_repr) + + +# --------------------------------------------------------------------------- +# Fetch-dependency ${VAR} validation +# --------------------------------------------------------------------------- + + +def resolve_config( + config: AgpackConfig, + project_root: Path, + *, + global_config: GlobalConfig | None = None, + verbose: bool = False, +) -> dict[str, str]: + """Validate that every ``${VAR}`` in fetch dependencies is resolvable. + + Walks every URL / path / ref template in copy-kind dependencies and resolves it eagerly so missing variables fail + fast (before any slow git clones), but **does not mutate** the entries — the template strings stay verbatim on + :class:`DependencySource` so they can be written to the lockfile, progress lines, and error messages without + leaking secrets like ``${GITHUB_TOKEN}``. Substitution actually happens inside + :func:`agpack.fetcher.fetch_dependency`, where the resolved URL is used for the one ``git clone`` call and then + discarded. + + Patch entries (edit-file kind) are *not* substituted at load time: their ``${name}`` references are resolved + per-target at apply time so that target ``vars`` can win over environment variables (see + :meth:`agpack.kinds.edit_file.EditFileResource.apply_patches`). + + Returns the merged environment table for downstream apply-time substitution. + + Resolution order (highest priority first): + + 1. Project ``.env`` (from *project_root*) + 2. Global ``.env`` (from the global config directory, if provided) + 3. Shell environment (``os.environ``) + """ + merged = build_env( + project_root, + global_config.config_dir if global_config is not None else None, + verbose=verbose, ) + + for entries in config.dependencies.values(): + for entry in entries: + if isinstance(entry, DependencySource): + ctx = f"dependency '{entry.name}'" + for u in entry.urls: + resolve_env_vars(u, merged, context=ctx) + if entry.path is not None: + resolve_env_vars(entry.path, merged, context=ctx) + if entry.ref is not None: + resolve_env_vars(entry.ref, merged, context=ctx) + + return merged diff --git a/agpack/deployer.py b/agpack/deployer.py index 28f3a2a..b3485bc 100644 --- a/agpack/deployer.py +++ b/agpack/deployer.py @@ -1,363 +1,171 @@ -"""File copy logic and directory mapping for deploying resources.""" +"""Orchestration over :mod:`agpack.kinds`. + +Per-kind behavior lives on the resource dataclasses themselves (in :mod:`agpack.kinds`); this module only loops over +targets and forwards to the right kind. The two public entrypoints handle the two fundamentally different deployment +shapes: + +* Copy kinds (``copy-directory`` / ``copy-file``): a tree of items is fetched from a git repo, detected, and copied + to each target that declares the matching resource type. :func:`detect_items` and :func:`deploy_item` cover this + path. +* Edit kind (``edit-file``): a list of :class:`~agpack.kinds.Patch` operations declared inline in ``agpack.yml`` is + reconciled against the lockfile's record of previously-applied patches. :func:`sync_edit_resource` walks every + target with a matching edit-file resource and does a diff-based per-file sync: removed ``replace`` patches delete + the leaf; removed ``append`` patches drop the previously-appended list entry; unchanged patches are left strictly + alone so the file isn't even written when nothing semantically changed. +""" from __future__ import annotations -import os -import shutil -import tempfile -from dataclasses import dataclass -from dataclasses import field +from collections import defaultdict from pathlib import Path from agpack.display import console +from agpack.errors import DeployError from agpack.fetcher import FetchResult -from agpack.targets import AGENT_DIRS -from agpack.targets import COMMAND_DIRS -from agpack.targets import SKILL_DIRS - - -class DeployError(Exception): - """Raised when a file deployment fails.""" - - -@dataclass -class DeployResult: - """Result of deploying one or more assets from a single dependency.""" - - files: list[str] = field(default_factory=list) - expanded_items: list[str] = field(default_factory=list) - - -def _atomic_copy_file(src: Path, dst: Path) -> None: - """Copy a file atomically using write-to-temp-then-rename. - - Creates parent directories as needed. - """ - dst.parent.mkdir(parents=True, exist_ok=True) - - # Write to a temp file in the same directory, then rename - fd, tmp_path = tempfile.mkstemp(dir=dst.parent, prefix=".agpack-tmp-") - try: - os.close(fd) - shutil.copy2(str(src), tmp_path) - os.replace(tmp_path, str(dst)) - except Exception: - # Clean up temp file on failure - try: - os.unlink(tmp_path) - except OSError: - pass - raise - - -def _copy_tree(src_dir: Path, dst_dir: Path) -> list[str]: - """Recursively copy a directory, returning list of relative paths written. - - Uses atomic file copies for each file. - """ - deployed: list[str] = [] - for src_file in sorted(src_dir.rglob("*")): - if src_file.is_file(): - # Skip git metadata - rel = src_file.relative_to(src_dir) - if any(part.startswith(".git") for part in rel.parts): - continue - dst_file = dst_dir / rel - _atomic_copy_file(src_file, dst_file) - deployed.append(str(dst_file)) - return deployed - - -def _find_asset_subfolders(path: Path) -> list[Path]: - """Return immediate subdirectories that contain at least one file. - - Only non-.git subdirectories are considered. Files may be nested - arbitrarily deep inside the subfolder. - """ - subfolders: list[Path] = [] - for item in sorted(path.iterdir()): - if item.is_dir() and not item.name.startswith(".git"): - has_files = any( - f.is_file() - and not any(p.startswith(".git") for p in f.relative_to(item).parts) - for f in item.rglob("*") - ) - if has_files: - subfolders.append(item) - return subfolders - - -def _find_top_level_files(path: Path) -> list[Path]: - """Return non-hidden files at the top level of a directory.""" - return sorted( - item - for item in path.iterdir() - if item.is_file() and not item.name.startswith(".") - ) - - -# --------------------------------------------------------------------------- -# Item detection — figure out what a dependency expands to -# --------------------------------------------------------------------------- - - -def detect_skill_items(fetch_result: FetchResult) -> list[tuple[str, Path]]: - """Return ``(name, path)`` pairs for the skill items in a fetch result. - - A directory with top-level files is treated as a single skill. - A directory with only subdirectories expands to one skill per subfolder. - """ - local_path = fetch_result.local_path - - if local_path.is_dir() and not _find_top_level_files(local_path): - subfolders = _find_asset_subfolders(local_path) - if not subfolders: - raise DeployError( - f"'{fetch_result.source.name}' is a directory but does not contain " - f"any skill folders. Provide a path to a skill folder or a " - f"directory containing skill folders." - ) - return [(sf.name, sf) for sf in subfolders] - - return [(fetch_result.source.name, local_path)] - - -def detect_file_items( - fetch_result: FetchResult, asset_type: str -) -> list[tuple[str, Path]]: - """Return ``(name, path)`` pairs for file assets (commands / agents).""" - local_path = fetch_result.local_path - - if local_path.is_dir(): - files = _find_top_level_files(local_path) - if not files: - for sf in _find_asset_subfolders(local_path): - files.extend(_find_top_level_files(sf)) - if not files: - article = "an" if asset_type[0] in "aeiou" else "a" - raise DeployError( - f"'{fetch_result.source.name}' is a directory but does not contain " - f"any {asset_type} files. Provide a path to {article} {asset_type} " - f"file or a directory containing {asset_type} files." - ) - return [(f.name, f) for f in files] - - return [(fetch_result.source.name, local_path)] - - -def detect_command_items(fetch_result: FetchResult) -> list[tuple[str, Path]]: - """Return ``(name, path)`` pairs for command items.""" - return detect_file_items(fetch_result, "command") - - -def detect_agent_items(fetch_result: FetchResult) -> list[tuple[str, Path]]: - """Return ``(name, path)`` pairs for agent items.""" - return detect_file_items(fetch_result, "agent") - - -# --------------------------------------------------------------------------- -# Single-item deployment -# --------------------------------------------------------------------------- - - -def deploy_single_skill( - skill_name: str, - skill_path: Path, - targets: list[str], +from agpack.kinds import CopyResource +from agpack.kinds import EditFileResource +from agpack.kinds import ResourceDef +from agpack.lockfile import AppliedPatch +from agpack.patch import Patch +from agpack.target_schema import TargetDef + +# =========================================================================== +# Copy kinds — fetch + detect + deploy +# =========================================================================== + + +def detect_items(fetch_result: FetchResult, resource: ResourceDef, label: str) -> list[tuple[str, Path]]: + """Return ``(name, source-path)`` pairs for the items in a fetch result.""" + if isinstance(resource, CopyResource): + return resource.detect(fetch_result, label) + msg = f"detect_items called with a {resource.kind} resource; only copy kinds support detection" + raise DeployError(msg) + + +def deploy_item( + name: str, + src_path: Path, + resource_type: str, + targets: list[TargetDef], project_root: Path, - dry_run: bool, - verbose: bool, + dry_run: bool = False, + verbose: bool = False, ) -> list[str]: - """Deploy a single skill folder/file to all applicable target directories.""" - all_deployed: list[str] = [] - + """Deploy one item to every target that supports ``resource_type``.""" + deployed: list[str] = [] for target in targets: - target_dir = SKILL_DIRS.get(target) - if target_dir is None: - continue - - dst = project_root / target_dir / skill_name - - if dry_run: - if verbose: - console.print(f"[dry-run] copy {skill_path} → {dst}") - if skill_path.is_dir(): - for f in sorted(skill_path.rglob("*")): - if f.is_file() and not any( - p.startswith(".git") for p in f.relative_to(skill_path).parts - ): - rel = dst / f.relative_to(skill_path) - all_deployed.append(str(rel.relative_to(project_root))) - else: - all_deployed.append( - str((dst / skill_path.name).relative_to(project_root)) - ) + resource = target.resources.get(resource_type) + if not isinstance(resource, CopyResource): continue + deployed.extend(resource.deploy_item(name, src_path, project_root, dry_run=dry_run, verbose=verbose)) - newly_deployed: list[str] = [] - if skill_path.is_dir(): - deployed = _copy_tree(skill_path, dst) - newly_deployed = [str(Path(d).relative_to(project_root)) for d in deployed] - else: - dst_file = dst / skill_path.name - _atomic_copy_file(skill_path, dst_file) - newly_deployed = [str(dst_file.relative_to(project_root))] + if verbose and not dry_run: + for entry in deployed: + console.print(f" {entry}") - all_deployed.extend(newly_deployed) + return deployed - if verbose: - for deployed_path in newly_deployed: - console.print(f" {deployed_path}") - return all_deployed +# =========================================================================== +# Edit-file kind — patches +# =========================================================================== -def deploy_skill( - fetch_result: FetchResult, - targets: list[str], +def sync_edit_resource( + resource_type: str, + desired: list[Patch], + applied_old: list[AppliedPatch], + targets: list[TargetDef], project_root: Path, + env_vars: dict[str, str] | None = None, + *, dry_run: bool = False, verbose: bool = False, -) -> DeployResult: - """Deploy a skill to all applicable target directories. +) -> list[AppliedPatch]: + """Reconcile every target's edit-file resource of ``resource_type``. - If the fetched path is a directory that contains files, it is deployed as - a single skill. If it contains only subdirectories (no top-level files), - each subdirectory that itself contains files is deployed as a separate - skill. An error is raised when neither condition is met. - """ - items = detect_skill_items(fetch_result) - expanded_items = [name for name, _ in items] if len(items) > 1 else [] + ``desired`` is the current list of patches from ``agpack.yml``. ``applied_old`` is the lockfile's record of what + was applied for this resource type on the previous sync (pass the entries from one ``EditLockEntry``). - all_deployed: list[str] = [] - for skill_name, skill_path in items: - deployed = deploy_single_skill( - skill_name, skill_path, targets, project_root, dry_run, verbose - ) - all_deployed.extend(deployed) - - return DeployResult(files=all_deployed, expanded_items=expanded_items) - - -# --------------------------------------------------------------------------- -# Commands & Agents (single-file assets) -# --------------------------------------------------------------------------- - - -def deploy_single_file( - filename: str, - file_path: Path, - targets: list[str], - target_dirs: dict[str, str], - project_root: Path, - dry_run: bool, - verbose: bool, -) -> list[str]: - """Deploy a single file to all applicable target directories.""" - all_deployed: list[str] = [] + Each target with a matching edit-file resource gets its own per-file diff: matching patches are left alone, removed + patches are reversed (``replace`` deletes the leaf; ``append`` drops the previously-appended entry), added patches + are applied fresh. Targets that don't declare this resource type are silently skipped; targets with a non-edit-file + kind for this name are also skipped (the cross-target kind consistency check in ``cli._resource_kinds`` should have + caught any actual conflict). + The returned :class:`AppliedPatch` list is the new authoritative state for the lockfile. + """ + # Group old applied entries by file_path so each target picks up only what was previously written to *its* file. + old_by_file: dict[str, list[AppliedPatch]] = defaultdict(list) + for entry in applied_old: + old_by_file[entry.file_path].append(entry) + + new_applied: list[AppliedPatch] = [] + matched_any = False + touched_files: set[str] = set() + + # Dedup by `resource.path`: when two targets resolve to the same edit-file (duplicate `targets:` entries, or two + # distinct targets both pointing at e.g. `.mcp.json`), reconciling once per target double-applies every `append` + # patch on every sync, leaving orphaned entries the lockfile can't track. for target in targets: - target_dir = target_dirs.get(target) - if target_dir is None: + resource = target.resources.get(resource_type) + if not isinstance(resource, EditFileResource): continue - - dst = project_root / target_dir / filename - - if dry_run: - if verbose: - console.print(f"[dry-run] copy → {dst}") - all_deployed.append(str(dst.relative_to(project_root))) + matched_any = True + if resource.path in touched_files: continue + touched_files.add(resource.path) + new_applied.extend( + resource.sync_patches( + applied_old=old_by_file.get(resource.path, []), + desired_new=desired, + project_root=project_root, + env_vars=env_vars, + dry_run=dry_run, + verbose=verbose, + ) + ) - _atomic_copy_file(file_path, dst) - all_deployed.append(str(dst.relative_to(project_root))) - - if verbose: - console.print(f" {dst.relative_to(project_root)}") - - return all_deployed - - -def deploy_single_command( - filename: str, - file_path: Path, - targets: list[str], - project_root: Path, - dry_run: bool, - verbose: bool, -) -> list[str]: - """Deploy a single command file to all applicable target directories.""" - return deploy_single_file( - filename, file_path, targets, COMMAND_DIRS, project_root, dry_run, verbose - ) - - -def deploy_single_agent( - filename: str, - file_path: Path, - targets: list[str], - project_root: Path, - dry_run: bool, - verbose: bool, -) -> list[str]: - """Deploy a single agent file to all applicable target directories.""" - return deploy_single_file( - filename, file_path, targets, AGENT_DIRS, project_root, dry_run, verbose - ) - - -def _deploy_file_asset( - fetch_result: FetchResult, - targets: list[str], - target_dirs: dict[str, str], - asset_type: str, - project_root: Path, - dry_run: bool, - verbose: bool, -) -> DeployResult: - """Deploy single-file asset(s) to all applicable target directories. - - If the fetched path is a directory, non-hidden files are collected from - the top level (or from subfolders if the top level has none). An error - is raised when no deployable files are found. - """ - items = detect_file_items(fetch_result, asset_type) - expanded_items = [name for name, _ in items] if len(items) > 1 else [] + # Files that used to be touched by this resource type but aren't any more (e.g. a target was removed from + # ``targets:``) need their old patches reversed too. Resolved keys live on the entries, so cleanup needs no + # target lookup. + for file_path, leftovers in old_by_file.items(): + if file_path in touched_files or not leftovers: + continue + EditFileResource(path=file_path).cleanup_patches(leftovers, project_root, dry_run=dry_run, verbose=verbose) - all_deployed: list[str] = [] - for filename, file_path in items: - deployed = deploy_single_file( - filename, file_path, targets, target_dirs, project_root, dry_run, verbose + if not matched_any and desired: + console.print( + f"[yellow]warning[/yellow]: {len(desired)} '{resource_type}' " + f"patch(es) configured but no target declares an edit-file " + f"resource for '{resource_type}'." ) - all_deployed.extend(deployed) - return DeployResult(files=all_deployed, expanded_items=expanded_items) + return new_applied -def deploy_command( - fetch_result: FetchResult, - targets: list[str], +def cleanup_orphaned_edits( + applied_old: list[AppliedPatch], project_root: Path, + *, dry_run: bool = False, verbose: bool = False, -) -> DeployResult: - """Deploy command file(s) to all applicable target directories.""" - return _deploy_file_asset( - fetch_result, targets, COMMAND_DIRS, "command", project_root, dry_run, verbose - ) +) -> None: + """Reverse every recorded patch for a resource type that no longer exists in ``dependencies:``. + Each :class:`AppliedPatch` carries its already-resolved key, so cleanup needs neither the originating target nor + the env table — it dispatches directly to the file for each path it has records for. + """ + by_file: dict[str, list[AppliedPatch]] = defaultdict(list) + for ap in applied_old: + by_file[ap.file_path].append(ap) + for file_path, entries in by_file.items(): + EditFileResource(path=file_path).cleanup_patches(entries, project_root, dry_run=dry_run, verbose=verbose) -def deploy_agent( - fetch_result: FetchResult, - targets: list[str], - project_root: Path, - dry_run: bool = False, - verbose: bool = False, -) -> DeployResult: - """Deploy agent file(s) to all applicable target directories.""" - return _deploy_file_asset( - fetch_result, targets, AGENT_DIRS, "agent", project_root, dry_run, verbose - ) + +# =========================================================================== +# Cleanup (copy-kind files) +# =========================================================================== def cleanup_deployed_files( @@ -383,11 +191,7 @@ def cleanup_deployed_files( def _cleanup_empty_dirs(deployed_files: list[str], project_root: Path) -> None: - """Remove empty parent directories left behind after file deletion. - - Only removes directories that are within known agpack-managed prefixes. - """ - # Collect unique parent directories, sorted deepest-first + """Remove empty parent directories left behind after file deletion.""" dirs_to_check: set[Path] = set() for rel_path in deployed_files: path = project_root / rel_path diff --git a/agpack/display.py b/agpack/display.py index d983a24..a1c65af 100644 --- a/agpack/display.py +++ b/agpack/display.py @@ -17,8 +17,8 @@ class StatusColumn(ProgressColumn): """Spinner while a task is running, status icon when finished. - The icon is read from the task's ``icon`` field so each task can - show a different result (e.g. green checkmark vs red cross). + The icon is read from the task's ``icon`` field so each task can show a different result (e.g. green checkmark vs + red cross). """ def __init__(self) -> None: diff --git a/agpack/envsubst.py b/agpack/envsubst.py index ea41877..16747c6 100644 --- a/agpack/envsubst.py +++ b/agpack/envsubst.py @@ -1,24 +1,69 @@ -"""Environment variable substitution for agpack config values.""" +"""``${VAR}`` substitution and ``.env`` loading. + +Leaf module: depends only on :mod:`agpack.errors`. Used by :mod:`agpack.config` (eager fetch-dep validation), +:mod:`agpack.fetcher` (URL/path/ref substitution at clone time), and :mod:`agpack.kinds.edit_file` (per-target patch +resolution at apply time). +""" from __future__ import annotations import os import re from pathlib import Path +from typing import Any + +from agpack.errors import ConfigError -from agpack.config import AgpackConfig -from agpack.config import ConfigError -from agpack.config import GlobalConfig +# Matches either ``$$`` (escape — emit literal ``$``) or ``${name}`` (substitute). ``$${name}`` therefore writes a +# literal ``${name}`` to the destination, letting users pass through variables resolved by the consuming tool at +# runtime (e.g. Claude Code's ``${CLAUDE_PROJECT_DIR}`` inside hook commands). +_VAR_PATTERN = re.compile(r"\$\$|\$\{([^}]+)}") -_VAR_PATTERN = re.compile(r"\$\{([^}]+)}") + +def resolve_env_vars(value: str, env: dict[str, str], *, context: str = "") -> str: + """Replace all ``${name}`` references in *value* from *env*. + + ``$$`` writes a literal ``$`` (so ``$${X}`` produces ``${X}``). + Raises :class:`ConfigError` if a ``${name}`` reference is not defined. + """ + + def _replace(match: re.Match[str]) -> str: + if match.group(0) == "$$": + return "$" + name = match.group(1) + if name in env: + return env[name] + hint = context + ": " if context else "" + msg = ( + f"{hint}variable '{name}' is not defined. " + f"Define it in .env, your shell environment, or as a target " + f"var; or use $${{{name}}} to write a literal ${{{name}}}." + ) + raise ConfigError(msg) from None + + return _VAR_PATTERN.sub(_replace, value) + + +def resolve_env_vars_recursive(value: Any, env: dict[str, str], *, context: str = "") -> Any: + """Walk a JSON-ish value substituting ``${name}`` in every string leaf. + + Used for patch values, where the same ``${name}`` semantics apply to nested dicts and lists. Non-string scalars + pass through unchanged. + """ + if isinstance(value, str): + return resolve_env_vars(value, env, context=context) + if isinstance(value, dict): + return {k: resolve_env_vars_recursive(v, env, context=context) for k, v in value.items()} + if isinstance(value, list): + return [resolve_env_vars_recursive(v, env, context=context) for v in value] + return value def load_dotenv(project_root: Path) -> dict[str, str]: """Load variables from a ``.env`` file in *project_root*. - Returns an empty dict when the file does not exist. - Supports ``KEY=VALUE``, optional quoting, ``# comments``, - blank lines, and an optional ``export`` prefix. + Returns an empty dict when the file does not exist. Supports ``KEY=VALUE``, optional quoting, ``# comments``, blank + lines, and an optional ``export`` prefix. """ env_path = project_root / ".env" if not env_path.exists(): @@ -50,29 +95,9 @@ def load_dotenv(project_root: Path) -> dict[str, str]: return result -def resolve_env_vars(value: str, env: dict[str, str], *, context: str = "") -> str: - """Replace all ``${VAR}`` references in *value* from *env*. - - Raises :class:`ConfigError` if a referenced variable is not defined. - """ - - def _replace(match: re.Match[str]) -> str: - var_name = match.group(1) - try: - return env[var_name] - except KeyError: - hint = context + ": " if context else "" - raise ConfigError( - f"{hint}environment variable '{var_name}' is not set. " - f"Define it in .env or your shell environment." - ) from None - - return _VAR_PATTERN.sub(_replace, value) - - -def _build_env( +def build_env( project_root: Path, - global_config: GlobalConfig | None = None, + global_config_dir: Path | None = None, *, verbose: bool = False, ) -> dict[str, str]: @@ -80,13 +105,13 @@ def _build_env( Resolution order (highest priority first): 1. Project ``.env`` (from *project_root*) - 2. Global ``.env`` (from the global config directory) + 2. Global ``.env`` (from *global_config_dir*) 3. Shell environment (``os.environ``) - """ - global_dotenv: dict[str, str] = {} - if global_config is not None: - global_dotenv = load_dotenv(global_config.config_dir) + Takes ``global_config_dir`` as a :class:`Path` (not a ``GlobalConfig`` object) so this module stays a leaf — the + caller in :mod:`agpack.config` unwraps the directory before calling. + """ + global_dotenv = load_dotenv(global_config_dir) if global_config_dir is not None else {} project_dotenv = load_dotenv(project_root) merged = {**os.environ, **global_dotenv, **project_dotenv} @@ -96,50 +121,6 @@ def _build_env( if global_dotenv: console.print(f" Loaded {len(global_dotenv)} variable(s) from global .env") if project_dotenv: - console.print( - f" Loaded {len(project_dotenv)} variable(s) from project .env" - ) + console.print(f" Loaded {len(project_dotenv)} variable(s) from project .env") return merged - - -def resolve_config( - config: AgpackConfig, - project_root: Path, - *, - global_config: GlobalConfig | None = None, - verbose: bool = False, -) -> None: - """Resolve ``${VAR}`` references in config values in-place. - - Substitutes ``${VAR}`` references in **all** string fields across - the config: dependency URLs, paths, refs, MCP commands, args, env - values, and MCP URLs. - - Resolution order for variables (highest priority first): - - 1. Project ``.env`` (from *project_root*) - 2. Global ``.env`` (from the global config directory, if provided) - 3. Shell environment (``os.environ``) - """ - merged = _build_env(project_root, global_config, verbose=verbose) - - # Dependency fields: urls, path, ref - for dep in [*config.skills, *config.commands, *config.agents]: - ctx = f"dependency '{dep.name}'" - dep.urls = [resolve_env_vars(u, merged, context=ctx) for u in dep.urls] - if dep.path is not None: - dep.path = resolve_env_vars(dep.path, merged, context=ctx) - if dep.ref is not None: - dep.ref = resolve_env_vars(dep.ref, merged, context=ctx) - - # MCP server fields: command, args, env values, url - for server in config.mcp: - ctx = f"mcp server '{server.name}'" - if server.command is not None: - server.command = resolve_env_vars(server.command, merged, context=ctx) - server.args = [resolve_env_vars(a, merged, context=ctx) for a in server.args] - for key, value in server.env.items(): - server.env[key] = resolve_env_vars(value, merged, context=ctx) - if server.url is not None: - server.url = resolve_env_vars(server.url, merged, context=ctx) diff --git a/agpack/errors.py b/agpack/errors.py new file mode 100644 index 0000000..9e89ba6 --- /dev/null +++ b/agpack/errors.py @@ -0,0 +1,27 @@ +"""Exception types raised across the agpack pipeline. + +Centralised here so every module can ``from agpack.errors import X`` without dragging in the modules that *raise* the +exception. Keeps every other module a clean leaf with respect to the error surface. +""" + +from __future__ import annotations + + +class ConfigError(Exception): + """Raised when an agpack config (``agpack.yml``, ``.env``, or a ``${VAR}`` reference) is invalid.""" + + +class EditFileError(Exception): + """Raised when an edit-file deployment or cleanup fails.""" + + +class DeployError(Exception): + """Raised when a copy-kind deployment fails.""" + + +class FetchError(Exception): + """Raised when a git fetch operation fails.""" + + +class TargetSchemaError(Exception): + """Raised when a target manifest fails to parse or validate.""" diff --git a/agpack/fetcher.py b/agpack/fetcher.py index 9f0d92d..b533270 100644 --- a/agpack/fetcher.py +++ b/agpack/fetcher.py @@ -11,18 +11,28 @@ from pathlib import Path from agpack.config import DependencySource +from agpack.envsubst import resolve_env_vars +from agpack.errors import FetchError -# Timeout (in seconds) for any single git subprocess call. Prevents -# indefinite hangs when the remote is unreachable or git is waiting for -# credentials that will never arrive. +# Timeout (in seconds) for any single git subprocess call. Prevents indefinite hangs when the remote is unreachable +# or git is waiting for credentials that will never arrive. _GIT_TIMEOUT_SECONDS = 120 # Match 7-40 hex chars (commit SHA) _SHA_RE = re.compile(r"^[0-9a-f]{7,40}$", re.IGNORECASE) -class FetchError(Exception): - """Raised when a git fetch operation fails.""" +def _redact(text: str) -> str: + """Strip ``user:pass@`` userinfo from any ``scheme://...`` URL embedded in *text*. + + Git echoes the resolved URL it was asked to clone, which is the one place a resolved ``${GITHUB_TOKEN}`` still + reaches user-visible output. :func:`_run_git` applies this unconditionally to every returned CompletedProcess + (args, stdout, stderr); call sites use it directly on URL inputs they construct error messages from. + + SSH-style ``git@github.com:owner/repo`` URLs have no ``://`` and are left untouched — their ``user@host`` form is + the scheme, not a secret. + """ + return re.compile(r"([a-z][a-z0-9+\-.]*://)[^@/\s]+@").sub(r"\1", text) @dataclass @@ -40,20 +50,19 @@ def _is_sha(ref: str) -> bool: return bool(_SHA_RE.match(ref)) -def _run_git( - args: list[str], cwd: str | Path | None = None -) -> subprocess.CompletedProcess[str]: - """Run a git command and return the result. +def _run_git(args: list[str], cwd: str | Path | None = None) -> subprocess.CompletedProcess[str]: + """Run a git command and return the result with userinfo scrubbed from stdout/stderr/args. + + The subprocess inherits the current environment with ``GIT_TERMINAL_PROMPT=0`` injected so that git never blocks + waiting for interactive credentials input (e.g. when an HTTPS URL is used but only SSH authentication is + configured). A timeout is also enforced to guard against network hangs. - The subprocess inherits the current environment with - ``GIT_TERMINAL_PROMPT=0`` injected so that git never blocks waiting - for interactive credentials input (e.g. when an HTTPS URL is used but - only SSH authentication is configured). A timeout is also enforced to - guard against network hangs. + Every returned :class:`CompletedProcess` has its ``args``, ``stdout``, and ``stderr`` passed through + :func:`_redact` so a resolved ``${GITHUB_TOKEN}`` in the URL never reaches a caller that might log the result. """ env = {**os.environ, "GIT_TERMINAL_PROMPT": "0"} try: - return subprocess.run( + result = subprocess.run( ["git", *args], cwd=cwd, capture_output=True, @@ -63,11 +72,17 @@ def _run_git( ) except subprocess.TimeoutExpired: return subprocess.CompletedProcess( - args=["git", *args], + args=["git", *(_redact(a) for a in args)], returncode=1, stdout="", stderr=f"Git operation timed out after {_GIT_TIMEOUT_SECONDS} seconds", ) + return subprocess.CompletedProcess( + args=["git", *(_redact(a) for a in args)], + returncode=result.returncode, + stdout=_redact(result.stdout), + stderr=_redact(result.stderr), + ) def _clone( @@ -98,7 +113,10 @@ def _clone( result = _run_git(cmd) if result.returncode != 0: - raise FetchError(f"Failed to clone {url}:\n{result.stderr}") + # ``url`` is the caller-supplied value (may carry a resolved ``${GITHUB_TOKEN}``) so it still needs scrubbing + # here; ``result.stderr`` was already redacted inside _run_git. + msg = f"Failed to clone {_redact(url)}:\n{result.stderr}" + raise FetchError(msg) # If ref is a SHA, we need to fetch and checkout that specific commit if is_sha and ref is not None: @@ -122,11 +140,13 @@ def _checkout_sha(clone_dir: Path, sha: str) -> None: # If unshallow fails (e.g. already full), try a regular fetch result = _run_git(["fetch", "origin"], cwd=clone_dir) if result.returncode != 0: - raise FetchError(f"Failed to fetch commit {sha}:\n{result.stderr}") + msg = f"Failed to fetch commit {sha}:\n{result.stderr}" + raise FetchError(msg) checkout = _run_git(["checkout", sha], cwd=clone_dir) if checkout.returncode != 0: - raise FetchError(f"Failed to checkout commit {sha}:\n{checkout.stderr}") + msg = f"Failed to checkout commit {sha}:\n{checkout.stderr}" + raise FetchError(msg) def _setup_sparse_checkout(clone_dir: Path, path: str) -> bool: @@ -179,21 +199,27 @@ def _try_clone( return clone_dir -def fetch_dependency(source: DependencySource) -> FetchResult: +def fetch_dependency(source: DependencySource, env: dict[str, str] | None = None) -> FetchResult: """Fetch a dependency from a remote git repo. - Clones the repo (with sparse checkout when possible), extracts the - relevant path, and returns a FetchResult with the local path to - the extracted content. + Clones the repo (with sparse checkout when possible), extracts the relevant path, and returns a FetchResult with + the local path to the extracted content. - When the primary URL fails and additional URLs are configured, - each is tried in order until one succeeds. + When the primary URL fails and additional URLs are configured, each is tried in order until one succeeds. - The caller is responsible for cleaning up the returned local_path's - parent temp directory when done. + ``${VAR}`` references in :attr:`DependencySource.urls`, :attr:`~DependencySource.path`, and + :attr:`~DependencySource.ref` are resolved here against *env* and used only for the ``git`` invocation — the + resolved strings are never written back to ``source`` and never returned. Pre-cloning eager validation in + :func:`agpack.config.resolve_config` guarantees missing variables fail before we get here, so a ``ConfigError`` + from this resolve path is a programmer error. + + The caller is responsible for cleaning up the returned local_path's parent temp directory when done. Args: - source: The dependency to fetch. + source: The dependency to fetch (URL/path/ref are templates). + env: Variable table for ``${VAR}`` substitution at clone time. + Defaults to an empty dict (templates with no ``${VAR}`` + will work without one). Returns: A FetchResult with the local path and resolved ref. @@ -201,13 +227,22 @@ def fetch_dependency(source: DependencySource) -> FetchResult: Raises: FetchError: If all URLs fail. """ - urls = source.urls + env = env or {} + ctx = f"dependency '{source.name}'" + + # Resolve path/ref once (URL is resolved per-attempt below so each fallback URL gets its own context label in any + # error message). + resolved_path = resolve_env_vars(source.path, env, context=ctx) if source.path else None + resolved_ref_template = resolve_env_vars(source.ref, env, context=ctx) if source.ref else None + tmpdir = Path(tempfile.mkdtemp(prefix="agpack-")) try: last_error: FetchError | None = None - for url in urls: + for url_template in source.urls: + resolved_url = resolve_env_vars(url_template, env, context=ctx) + # Each attempt needs a clean clone directory clone_target = tmpdir / "repo" if clone_target.exists(): @@ -215,29 +250,32 @@ def fetch_dependency(source: DependencySource) -> FetchResult: try: clone_dir = _try_clone( - url=url, - ref=source.ref, - path=source.path, + url=resolved_url, + ref=resolved_ref_template, + path=resolved_path, tmpdir=tmpdir, ) except FetchError as exc: last_error = exc continue - resolved_ref = _get_resolved_ref(clone_dir) + head_sha = _get_resolved_ref(clone_dir) # Determine the local path to the content - if source.path: - content_path = clone_dir / source.path + if resolved_path: + content_path = clone_dir / resolved_path if not content_path.exists(): - raise FetchError(f"Path '{source.path}' not found in {url}") + # ``url_template`` (not ``resolved_url``) so we never leak a substituted token even if the template + # itself appears in the surfaced error. + msg = f"Path '{resolved_path}' not found in {url_template}" + raise FetchError(msg) else: content_path = clone_dir return FetchResult( source=source, local_path=content_path, - resolved_ref=resolved_ref, + resolved_ref=head_sha, _tmpdir=tmpdir, ) diff --git a/agpack/kinds/__init__.py b/agpack/kinds/__init__.py new file mode 100644 index 0000000..e9f31e9 --- /dev/null +++ b/agpack/kinds/__init__.py @@ -0,0 +1,24 @@ +"""The three asset kinds agpack knows how to deploy. + +Each kind owns its own dataclass + behaviour in its own submodule (``copy_directory``, ``copy_file``, ``edit_file``). +This package's ``__init__`` re-exports the resource classes and the two union aliases for callers that want them +together. Domain types that are *not* kind behaviour (the patch model, the exception hierarchy) live elsewhere — +:mod:`agpack.patch` and :mod:`agpack.errors` — so importing one kind module doesn't drag those in. +""" + +from agpack.kinds.copy_directory import CopyDirectoryResource +from agpack.kinds.copy_file import CopyFileResource +from agpack.kinds.edit_file import EditFileResource +from agpack.kinds.edit_file import infer_config_format + +ResourceDef = CopyDirectoryResource | CopyFileResource | EditFileResource +CopyResource = CopyDirectoryResource | CopyFileResource + +__all__ = [ + "CopyDirectoryResource", + "CopyFileResource", + "CopyResource", + "EditFileResource", + "ResourceDef", + "infer_config_format", +] diff --git a/agpack/kinds/_shared.py b/agpack/kinds/_shared.py new file mode 100644 index 0000000..c48cb1e --- /dev/null +++ b/agpack/kinds/_shared.py @@ -0,0 +1,93 @@ +"""Shared filesystem helpers for the three deploy kinds. + +Atomic file I/O and directory-traversal helpers used by both copy kinds plus :mod:`agpack.kinds.edit_file`. Nothing +kind-specific belongs here — if you find yourself about to add behaviour that knows about a single kind, put it in +that kind's module instead. Exception types live in :mod:`agpack.errors`. +""" + +from __future__ import annotations + +import contextlib +import os +import shutil +import tempfile +from pathlib import Path + + +def atomic_copy_file(src: Path, dst: Path) -> None: + """Copy a file atomically using write-to-temp-then-rename.""" + dst.parent.mkdir(parents=True, exist_ok=True) + fd, tmp_path = tempfile.mkstemp(dir=dst.parent, prefix=".agpack-tmp-") + tmp = Path(tmp_path) + try: + os.close(fd) + shutil.copy2(src, tmp) + tmp.replace(dst) + except Exception: + with contextlib.suppress(OSError): + tmp.unlink() + raise + + +def _atomic_write(path: Path, content: str) -> None: + """Write text content to a file atomically.""" + path.parent.mkdir(parents=True, exist_ok=True) + fd, tmp_path = tempfile.mkstemp(dir=path.parent, prefix=".agpack-edit-") + tmp = Path(tmp_path) + try: + os.close(fd) + tmp.write_text(content, encoding="utf-8") + tmp.replace(path) + except Exception: + with contextlib.suppress(OSError): + tmp.unlink() + raise + + +def write_if_changed(path: Path, new_text: str) -> bool: + """Atomic write, but skip when the file already has this exact content. + + Returns ``True`` if a write happened, ``False`` if we noticed the file was already byte-identical and skipped. This + is what keeps every-sync churn off the user's structured config files when nothing semantically changed. + """ + if path.exists(): + try: + if path.read_text(encoding="utf-8") == new_text: + return False + except OSError: + pass # Fall through to the write — file may be unreadable. + _atomic_write(path, new_text) + return True + + +def copy_tree(src_dir: Path, dst_dir: Path) -> list[str]: + """Recursively copy a directory; return absolute destination paths.""" + deployed: list[str] = [] + for src_file in sorted(src_dir.rglob("*")): + if src_file.is_file(): + rel = src_file.relative_to(src_dir) + if any(part.startswith(".git") for part in rel.parts): + continue + dst_file = dst_dir / rel + atomic_copy_file(src_file, dst_file) + deployed.append(str(dst_file)) + return deployed + + +def find_asset_subfolders(path: Path) -> list[Path]: + """Return immediate subdirectories that contain at least one file.""" + subfolders: list[Path] = [] + for item in sorted(path.iterdir()): + if item.is_dir() and not item.name.startswith(".git"): + has_files = any( + f.is_file() and not any(p.startswith(".git") for p in f.relative_to(item).parts) + for f in item.rglob("*") + ) + if has_files: + subfolders.append(item) + return subfolders + + +def find_top_level_files(path: Path) -> list[Path]: + """Return non-hidden files at the top level of a directory.""" + return sorted(item for item in path.iterdir() if item.is_file() and not item.name.startswith(".")) diff --git a/agpack/kinds/copy_directory.py b/agpack/kinds/copy_directory.py new file mode 100644 index 0000000..e7151ff --- /dev/null +++ b/agpack/kinds/copy_directory.py @@ -0,0 +1,85 @@ +"""``kind: copy-directory`` — items deploy as ``//``. + +Used for resources whose consumer expects a *folder per item* on disk (Claude skills are the canonical example: each +skill is a directory containing ``SKILL.md`` plus assets). A source directory of only-subdirectories expands to one +bundle per subfolder; a source directory with top-level files becomes a single bundle. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import TYPE_CHECKING +from typing import ClassVar + +from agpack.display import console +from agpack.errors import DeployError +from agpack.kinds._shared import atomic_copy_file +from agpack.kinds._shared import copy_tree +from agpack.kinds._shared import find_asset_subfolders +from agpack.kinds._shared import find_top_level_files + +if TYPE_CHECKING: + from agpack.fetcher import FetchResult + + +@dataclass(frozen=True) +class CopyDirectoryResource: + """Deploys items as directory bundles under ``//``. + + A directory dependency with top-level files is treated as a single bundle; a directory containing only + subdirectories expands to one bundle per subfolder. + """ + + path: str + kind: ClassVar[str] = "copy-directory" + + def detect(self, fetch_result: FetchResult, label: str) -> list[tuple[str, Path]]: + local_path = fetch_result.local_path + + if local_path.is_dir() and not find_top_level_files(local_path): + subfolders = find_asset_subfolders(local_path) + if not subfolders: + msg = ( + f"'{fetch_result.source.name}' is a directory but does not " + f"contain any {label} folders. Provide a path to a {label} " + f"folder or a directory containing {label} folders." + ) + raise DeployError(msg) + return [(sf.name, sf) for sf in subfolders] + + return [(fetch_result.source.name, local_path)] + + def deploy_item( + self, + item_name: str, + src_path: Path, + project_root: Path, + *, + dry_run: bool = False, + verbose: bool = False, + ) -> list[str]: + dst = project_root / self.path / item_name + deployed: list[str] = [] + + if dry_run: + if src_path.is_dir(): + for f in sorted(src_path.rglob("*")): + if f.is_file() and not any(p.startswith(".git") for p in f.relative_to(src_path).parts): + rel = dst / f.relative_to(src_path) + deployed.append(str(rel.relative_to(project_root))) + else: + deployed.append(str((dst / src_path.name).relative_to(project_root))) + if verbose: + console.print(f"[dry-run] copy {src_path} → {dst}") + return deployed + + if src_path.is_dir(): + for copied in copy_tree(src_path, dst): + deployed.append(str(Path(copied).relative_to(project_root))) + else: + dst_file = dst / src_path.name + atomic_copy_file(src_path, dst_file) + deployed.append(str(dst_file.relative_to(project_root))) + + return deployed diff --git a/agpack/kinds/copy_file.py b/agpack/kinds/copy_file.py new file mode 100644 index 0000000..2eecb45 --- /dev/null +++ b/agpack/kinds/copy_file.py @@ -0,0 +1,69 @@ +"""``kind: copy-file`` — items deploy as ``/`` (flat files). + +Used for resources whose consumer expects a *file per item* on disk (Claude commands, agents, Codex prompts). When the +source is a directory, top-level files become items; if the top level has no files, agpack recurses into +subdirectories to find them. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import TYPE_CHECKING +from typing import ClassVar + +from agpack.display import console +from agpack.errors import DeployError +from agpack.kinds._shared import atomic_copy_file +from agpack.kinds._shared import find_asset_subfolders +from agpack.kinds._shared import find_top_level_files + +if TYPE_CHECKING: + from agpack.fetcher import FetchResult + + +@dataclass(frozen=True) +class CopyFileResource: + """Deploys items as individual files at ``/``.""" + + path: str + kind: ClassVar[str] = "copy-file" + + def detect(self, fetch_result: FetchResult, label: str) -> list[tuple[str, Path]]: + local_path = fetch_result.local_path + + if local_path.is_dir(): + files = find_top_level_files(local_path) + if not files: + for sf in find_asset_subfolders(local_path): + files.extend(find_top_level_files(sf)) + if not files: + article = "an" if label[0] in "aeiou" else "a" + msg = ( + f"'{fetch_result.source.name}' is a directory but does not " + f"contain any {label} files. Provide a path to {article} " + f"{label} file or a directory containing {label} files." + ) + raise DeployError(msg) + return [(f.name, f) for f in files] + + return [(fetch_result.source.name, local_path)] + + def deploy_item( + self, + item_name: str, + src_path: Path, + project_root: Path, + *, + dry_run: bool = False, + verbose: bool = False, + ) -> list[str]: + dst = project_root / self.path / item_name + + if dry_run: + if verbose: + console.print(f"[dry-run] copy → {dst}") + return [str(dst.relative_to(project_root))] + + atomic_copy_file(src_path, dst) + return [str(dst.relative_to(project_root))] diff --git a/agpack/kinds/edit_file.py b/agpack/kinds/edit_file.py new file mode 100644 index 0000000..77d37db --- /dev/null +++ b/agpack/kinds/edit_file.py @@ -0,0 +1,490 @@ +"""``kind: edit-file`` — surgically patch a structured config file. + +Reads a JSON or TOML file, applies a list of :class:`Patch` operations (``replace`` or ``append``), and writes it +back. The diff-based :meth:`EditFileResource.sync_patches` is what makes this safe across syncs: + +* TOML files round-trip through :mod:`tomlkit` so comments, key ordering, and whitespace on untouched sections + survive unchanged. +* JSON files use :mod:`json` (no format-preserving alternative in stdlib); ``write_if_changed`` skips the write + entirely when the serialised text is byte-identical to disk, so unchanged files don't churn either. +* Cleanup is symmetric with copy-kind cleanup: a removed ``replace`` patch deletes the key (just like a removed + copy file is unlinked), and a removed ``append`` patch removes the previously-appended list entry. agpack does + not try to remember what existed at a key before it first applied a ``replace`` — that value was overwritten when + the patch was first applied, the same way ``cp`` overwrites an existing file. +""" + +from __future__ import annotations + +import json +from dataclasses import dataclass +from dataclasses import field +from pathlib import Path +from typing import Any +from typing import ClassVar +from typing import Literal + +import tomlkit +from tomlkit.exceptions import TOMLKitError + +from agpack.display import console +from agpack.envsubst import resolve_env_vars +from agpack.envsubst import resolve_env_vars_recursive +from agpack.errors import EditFileError +from agpack.kinds._shared import write_if_changed +from agpack.lockfile import AppliedPatch +from agpack.patch import Patch +from agpack.patch import match_key +from agpack.patch import value_hash + +# --------------------------------------------------------------------------- +# Format inference +# --------------------------------------------------------------------------- + + +_FORMAT_BY_SUFFIX = {".json": "json", ".toml": "toml"} + + +def infer_config_format(path: str) -> Literal["json", "toml"]: + """Return the format for an edit-file config path. + + The extension is the single source of truth — there is no override. + """ + lower = path.lower() + if lower.endswith(".toml"): + return "toml" + if lower.endswith(".json"): + return "json" + valid = ", ".join(sorted(_FORMAT_BY_SUFFIX)) + msg = f"cannot infer config format from '{path}' — path must end in one of: {valid}" + raise EditFileError(msg) + + +# --------------------------------------------------------------------------- +# Dotted-path navigation +# --------------------------------------------------------------------------- + + +def _split_key(key: str) -> list[str]: + """Split a dotted patch key into segments, honouring backslash escapes. + + ``.`` separates segments; ``\\.`` produces a literal dot inside a segment; ``\\\\`` produces a literal backslash. + This lets users address keys that contain dots (e.g. MCP server names with dotted identifiers, Java package keys, + hostnames): + + * ``mcpServers.foo`` → ``["mcpServers", "foo"]`` + * ``mcpServers.example\\.com/srv`` → ``["mcpServers", "example.com/srv"]`` + * ``a\\\\b.c`` → ``["a\\b", "c"]`` + + A trailing unescaped backslash is taken literally. Empty segments (``a..b``, ``.x``, ``x.``) are rejected so that + ambiguous keys fail loudly instead of silently navigating into ``""``. + """ + if not key: + msg = "patch key must be non-empty" + raise EditFileError(msg) + + segments: list[str] = [] + current: list[str] = [] + i = 0 + while i < len(key): + c = key[i] + if c == "\\" and i + 1 < len(key): + # Escape: emit the next character literally. Covers both ``\.`` (literal dot, no segment break) and ``\\`` + # (literal backslash). + current.append(key[i + 1]) + i += 2 + continue + if c == ".": + segments.append("".join(current)) + current = [] + i += 1 + continue + current.append(c) + i += 1 + segments.append("".join(current)) + + if any(not s for s in segments): + msg = f"patch key {key!r} has an empty segment — use '\\.' to embed a literal dot inside a segment" + raise EditFileError(msg) + return segments + + +def _walk_to_parent(root: dict[str, Any], segments: list[str]) -> tuple[dict[str, Any], str]: + """Walk ``root`` along ``segments[:-1]``, returning (parent, last_segment). + + Missing intermediate dicts are auto-created. If an existing intermediate value is *not* a dict, raises + :class:`EditFileError` rather than silently overwriting user data. + """ + parent: dict[str, Any] = root + for seg in segments[:-1]: + if seg in parent and not isinstance(parent[seg], dict): + msg = f"patch path traverses non-dict at '{seg}': existing value is {type(parent[seg]).__name__}" + raise EditFileError(msg) + parent = parent.setdefault(seg, {}) + return parent, segments[-1] + + +def _walk_readonly(root: dict[str, Any], segments: list[str]) -> tuple[dict[str, Any] | None, str]: + """Walk ``root`` along ``segments[:-1]`` without auto-creating anything. + + Returns ``(parent_dict, leaf_segment)`` if every intermediate segment resolved to a dict; ``(None, "")`` if any + segment was missing or pointed at a non-dict. Used by every read/undo path where missing keys must silently no-op + (cleanup, previous-value capture, undo). + """ + parent: Any = root + for seg in segments[:-1]: + if not isinstance(parent, dict) or seg not in parent: + return None, "" + parent = parent[seg] + if not isinstance(parent, dict): + return None, "" + return parent, segments[-1] + + +def _apply_patch(root: dict[str, Any], patch: Patch) -> None: + """Apply ``patch`` to ``root`` in-place — idempotent for both strategies. + + ``replace`` assigns to the leaf (the same assignment is a no-op on re-apply). ``append`` only adds the value when + no existing list element already hashes to the same value, so re-applying a patch that's already present is a + no-op. This idempotence is what makes committing the lockfile + cloning fresh work: when ``sync_patches`` re-walks + its diff and finds an "unchanged" entry, calling ``_apply_patch`` ensures the destination file actually has the + entry without churning files that already do. + """ + segments = _split_key(patch.key) + parent, leaf = _walk_to_parent(root, segments) + + if patch.strategy == "replace": + parent[leaf] = patch.value + return + + # append + bucket = parent.setdefault(leaf, []) + if not isinstance(bucket, list): + msg = f"patch with strategy='append' targets non-list at '{patch.key}': got {type(bucket).__name__}" + raise EditFileError(msg) + new_hash = value_hash(patch.value) + for item in bucket: + if value_hash(item) == new_hash: + return # already present — keep idempotent + bucket.append(patch.value) + + +def _undo_resolved(root: dict[str, Any], strategy: str, resolved_key: str, expected_hash: str) -> bool: + """Reverse one applied patch on ``root`` in-place. + + The caller pre-resolves ``${var}`` substitutions in the key; the lockfile stores resolved keys and a SHA256 hash + of the resolved value (:func:`value_hash`). + + ``replace`` deletes the leaf. ``append`` walks the list at the path, hashes each element, and removes the first + hash-match. Silent no-op if the target is missing or already gone. + """ + parent, leaf = _walk_readonly(root, _split_key(resolved_key)) + if parent is None or leaf not in parent: + return False + + if strategy == "replace": + del parent[leaf] + return True + + bucket = parent[leaf] + if not isinstance(bucket, list): + return False + for i, item in enumerate(bucket): + if value_hash(item) == expected_hash: + del bucket[i] + return True + return False + + +# --------------------------------------------------------------------------- +# JSON / TOML I/O +# --------------------------------------------------------------------------- + + +def _read_existing(config_path: Path, format_: str) -> dict[str, Any]: + """Read an existing JSON/TOML config file, or return an empty dict. + + TOML files are parsed with :mod:`tomlkit` so comments, key ordering, and whitespace survive the round-trip. The + returned :class:`tomlkit.TOMLDocument` behaves as a dict for navigation and mutation; ``tomlkit.dumps`` later + re-emits the document preserving everything we didn't touch. + + JSON has no equivalent format-preserving parser in stdlib; canonicalization on write is unavoidable, but the + :func:`_write_if_changed` guard at write time prevents no-op churn on unchanged content. + """ + if not config_path.exists(): + return tomlkit.document() if format_ == "toml" else {} + text = config_path.read_text(encoding="utf-8") + try: + data: Any + data = json.loads(text) if format_ == "json" else tomlkit.parse(text) + except (json.JSONDecodeError, TOMLKitError, OSError) as exc: + msg = f"Failed to read {config_path}: {exc}" + raise EditFileError(msg) from exc + + if not isinstance(data, dict): + msg = f"{config_path}: top-level must be a mapping" + raise EditFileError(msg) + return data + + +def _dump(data: dict[str, Any], format_: str) -> str: + """Serialise a dict back to JSON or TOML text. + + For TOML, ``data`` is always a :class:`TOMLDocument` because :func:`_read_existing` returns one (a fresh + :func:`tomlkit.document` if the file didn't exist, otherwise the parsed document). ``tomlkit.dumps`` accepts any + mapping, but passing the document we already have keeps comments, ordering, and whitespace intact on untouched + sections. + """ + if format_ == "json": + return json.dumps(data, indent=2) + "\n" + return tomlkit.dumps(data) + + +# --------------------------------------------------------------------------- +# EditFileResource +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class EditFileResource: + """Applies :class:`Patch` operations to a JSON or TOML config file. + + The file format is inferred from :attr:`path`'s extension. Patches are fully generic — agpack reads the file, + applies each patch (replace or append), and writes it back atomically. + + :attr:`vars` is a mapping of substitution variables made available to every patch when this resource is the apply + target. They are referenced as ``${name}`` in patch ``key`` strings and recursively in patch ``value`` strings. + Target ``vars`` win over environment variables on name collision — the target manifest is the canonical source for + per-target structure like bucket names. + """ + + path: str + vars: dict[str, str] = field(default_factory=dict) + kind: ClassVar[str] = "edit-file" + + @property + def format(self) -> Literal["json", "toml"]: + return infer_config_format(self.path) + + def apply_patches( + self, + patches: list[Patch], + project_root: Path, + env_vars: dict[str, str] | None = None, + *, + dry_run: bool = False, + verbose: bool = False, + ) -> list[AppliedPatch]: + """Apply each patch to the config file at :attr:`path`. + + Thin wrapper around :meth:`sync_patches` with no prior state — every patch is treated as freshly added. + """ + return self.sync_patches( + applied_old=[], + desired_new=patches, + project_root=project_root, + env_vars=env_vars, + dry_run=dry_run, + verbose=verbose, + ) + + def _resolve_patch(self, patch: Patch, env_vars: dict[str, str]) -> Patch: + """Return a new Patch with all ${name} references substituted. + + Target ``vars`` override env vars on collision. + """ + table = {**env_vars, **self.vars} + ctx = f"patch {self.path}:{patch.key}" + return Patch( + key=resolve_env_vars(patch.key, table, context=ctx), + value=resolve_env_vars_recursive(patch.value, table, context=ctx), + strategy=patch.strategy, + ) + + def patch_identity(self, patch: Patch, env_vars: dict[str, str]) -> tuple[Any, ...]: + """Return the :func:`match_key` identity *patch* would have on disk when applied to this resource. + + Resolves the patch's ``${name}`` references against env + target vars, hashes the resolved value, and combines + them with strategy + the **resolved** key into the same tuple :meth:`sync_patches` builds internally for + diffing — and the same identity the lockfile records. Used by :func:`agpack.cli.status` to test whether the + lockfile already has this patch. + + Raises: + ConfigError: If a ``${name}`` reference cannot be resolved. + """ + resolved = self._resolve_patch(patch, env_vars) + return match_key(resolved.strategy, resolved.key, value_hash(resolved.value)) + + def cleanup_patches( + self, + patches: list[AppliedPatch], + project_root: Path, + env_vars: dict[str, str] | None = None, + *, + dry_run: bool = False, + verbose: bool = False, + ) -> None: + """Undo a list of previously-applied patches on the config file at :attr:`path`. + + Thin wrapper over :meth:`sync_patches` with empty ``desired_new``: every entry falls into the removes + branch of the diff, which resolves the key via ``self.vars`` + *env_vars* and undoes the patch + (``replace`` deletes the leaf; ``append`` removes the previously-appended list entry by hash match). + + Silent no-op if the file is missing — nothing to clean up. + """ + if not patches: + return + if not (project_root / self.path).exists(): + return + self.sync_patches( + applied_old=patches, + desired_new=[], + project_root=project_root, + env_vars=env_vars, + dry_run=dry_run, + verbose=verbose, + ) + + def sync_patches( # noqa: C901 # three-way diff: matches/removes/adds + dry-run + format inference + self, + applied_old: list[AppliedPatch], + desired_new: list[Patch], + project_root: Path, + env_vars: dict[str, str] | None = None, + *, + dry_run: bool = False, + verbose: bool = False, + ) -> list[AppliedPatch]: + """Reconcile the file at :attr:`path` to match *desired_new*. + + The diff-based path that backs both ``apply`` and ``cleanup``: + + * Patches in *applied_old* but not in *desired_new* are undone (``replace`` deletes the leaf; ``append`` + removes the previously-appended list entry, located by value hash). + * Patches in *desired_new* but not in *applied_old* are applied fresh. + * Patches that appear in both with identical value hashes are left untouched — no file mutation, no churn. + * For ``replace`` patches whose key matches but value hash differs, the file is updated to the new value. + + Diff identity uses the **resolved** key (post-``${var}`` substitution) plus the value hash. That's also what + the lockfile records — see :class:`AppliedPatch` for the assumption that keys are structural, not secret. + + Returns the list of :class:`AppliedPatch` records to write to the lockfile. + """ + env_vars = env_vars or {} + + # Resolve current desired patches. + resolved_new = [self._resolve_patch(p, env_vars) for p in desired_new] + + # Catch two patches that resolve to the same identity (e.g. literal ``mcpServers.foo`` and ``${bucket}.foo`` + # with ``bucket=mcpServers``). Parse-time can't see this because target vars aren't known yet, so this is + # the earliest moment the collision is detectable. Failing here also prevents the diff dict from silently + # last-write-wins on collisions. + seen: dict[tuple[Any, ...], int] = {} + for i, p in enumerate(resolved_new): + mk = match_key(p.strategy, p.key, value_hash(p.value)) + if mk in seen: + first = seen[mk] + first_src = desired_new[first].key + second_src = desired_new[i].key + src_detail = "" if first_src == second_src else f" (unresolved keys: {first_src!r} and {second_src!r})" + msg = ( + f"{self.path}: patches at indices {first} and {i} resolve to the same identity " + f"(strategy={p.strategy}, key={p.key!r}){src_detail}." + ) + raise EditFileError(msg) + seen[mk] = i + + config_path = project_root / self.path + + if dry_run: + if verbose: + for p in resolved_new: + console.print(f"[dry-run] {p.strategy} {self.path}:{p.key}") + return [ + AppliedPatch( + file_path=self.path, + key=rp.key, + strategy=rp.strategy, + value_hash=value_hash(rp.value), + ) + for rp in resolved_new + ] + + if not applied_old and not resolved_new and not config_path.exists(): + return [] + + try: + format_ = self.format + except EditFileError: + # Stale lockfile pointing at an unknown extension — drop the old records. + return [] + + data = _read_existing(config_path, format_) + + # Diff identities use RESOLVED keys + value hashes on both sides (lockfile entries already store resolved + # keys; desired patches were resolved above). + old_by_match: dict[tuple[Any, ...], AppliedPatch] = { + match_key(ap.strategy, ap.key, ap.value_hash): ap for ap in applied_old + } + new_by_match: dict[tuple[Any, ...], Patch] = { + match_key(rp.strategy, rp.key, value_hash(rp.value)): rp for rp in resolved_new + } + + result: list[AppliedPatch] = [] + verbose_lines: list[str] = [] + + for mk in old_by_match.keys() & new_by_match.keys(): + ap = old_by_match[mk] + rp = new_by_match[mk] + new_hash = value_hash(rp.value) + if ap.value_hash == new_hash: + # Lockfile and config agree this patch is current. We still call ``_apply_patch`` so the destination + # file actually contains the entry — covers the committed-lockfile + fresh-clone case where the + # lockfile records a patch but the destination file was never created on this machine. + # ``_apply_patch`` is idempotent (replace = assign; append checks for an existing hash-match), so this + # is a no-op when the file already has the entry — ``write_if_changed`` will skip the disk write. + _apply_patch(data, rp) + result.append(ap) + continue + # ``replace`` with same key, different value: apply the new value. + _apply_patch(data, rp) + result.append( + AppliedPatch( + file_path=self.path, + key=rp.key, + strategy=rp.strategy, + value_hash=new_hash, + ) + ) + verbose_lines.append(f" update {self.path}:{rp.key}") + + for mk in old_by_match.keys() - new_by_match.keys(): + ap = old_by_match[mk] + # ``ap.key`` is already resolved — no re-resolution needed; navigate straight to the leaf. + if _undo_resolved(data, ap.strategy, ap.key, ap.value_hash): + verbose_lines.append(f" remove {self.path}:{ap.key}") + + for mk in new_by_match.keys() - old_by_match.keys(): + rp = new_by_match[mk] + _apply_patch(data, rp) + result.append( + AppliedPatch( + file_path=self.path, + key=rp.key, + strategy=rp.strategy, + value_hash=value_hash(rp.value), + ) + ) + verbose_lines.append(f" {rp.strategy} {self.path}:{rp.key}") + + new_text = _dump(data, format_) + try: + wrote = write_if_changed(config_path, new_text) + except (OSError, TypeError, ValueError) as exc: + msg = f"Failed to write {config_path}: {exc}" + raise EditFileError(msg) from exc + + if verbose and wrote: + for line in verbose_lines: + console.print(line) + + return result diff --git a/agpack/lockfile.py b/agpack/lockfile.py index cb0f1a3..36f78f3 100644 --- a/agpack/lockfile.py +++ b/agpack/lockfile.py @@ -2,6 +2,7 @@ from __future__ import annotations +import contextlib import os import tempfile from dataclasses import dataclass @@ -14,18 +15,21 @@ import yaml from agpack import __version__ +from agpack.display import console LOCKFILE_NAME = ".agpack.lock.yml" @dataclass class InstalledEntry: - """A single installed dependency in the lockfile.""" + """A single installed copy-kind dependency in the lockfile.""" url: str path: str | None resolved_ref: str - type: str # "skill", "command", "agent" + type: str + """Resource type name as it appears in agpack.yml (``skills`` / ``commands`` / ``agents`` / any user-defined + name).""" deployed_files: list[str] = field(default_factory=list) @property @@ -38,11 +42,35 @@ def identity(self) -> str: @dataclass -class McpLockEntry: - """A single MCP server entry in the lockfile.""" +class AppliedPatch: + """An edit-file patch recorded for future cleanup. - name: str - targets: list[str] = field(default_factory=list) + :attr:`key` is the **resolved** dotted path as it lives on disk (e.g. ``mcpServers.filesystem``) — what cleanup + needs to navigate to the leaf. :attr:`value_hash` is a SHA256 fingerprint of the resolved value (not the value + itself), so a value interpolated from ``${API_KEY}`` never ends up in the lockfile. + + **Assumption: patch keys are structural, not secret.** Keys describe where in the JSON/TOML a value lives — + ``mcpServers.filesystem``, ``hooks.PreToolUse``. If a user ever wrote ``${SECRET}.foo`` as a key, the resolved + value would land here in plaintext. Don't put secrets in patch keys. + + * ``replace`` cleanup deletes the leaf — symmetric with how copy kinds clean up the files they wrote. If the + user had a value at that key before agpack first ran, ``replace`` overwrote it; cleanup does not try to + magically restore it. + * ``append`` cleanup walks the list at the path, hashes each element, and removes the first hash-match. + """ + + file_path: str + key: str + strategy: str + value_hash: str + + +@dataclass +class EditLockEntry: + """All patches applied for one resource type across all targets.""" + + resource_type: str + applied: list[AppliedPatch] = field(default_factory=list) @dataclass @@ -52,24 +80,56 @@ class Lockfile: generated_at: str = "" agpack_version: str = __version__ installed: list[InstalledEntry] = field(default_factory=list) - mcp: list[McpLockEntry] = field(default_factory=list) + edits: list[EditLockEntry] = field(default_factory=list) + + +# --------------------------------------------------------------------------- +# Read / write +# --------------------------------------------------------------------------- + +_CORRUPT_LOCKFILE_WARNING = ( + "Treating lockfile as missing — agpack has no record of which patches " + "it previously applied, so cleanup of patches you have since removed " + "from agpack.yml will not happen automatically. Restore the lockfile " + "from version control if you have it, or remove leftover agpack-written " + "entries from your config files by hand." +) -def read_lockfile(project_root: Path) -> Lockfile | None: + +def read_lockfile(project_root: Path) -> Lockfile | None: # noqa: C901 """Read the lockfile from disk. - Returns None if the lockfile doesn't exist. + Returns ``None`` if the file is absent. If the file exists but is unreadable, malformed, or has the wrong top-level + shape, emits a loud warning explaining what guarantee is now broken (without the lockfile, agpack cannot clean up + patches the user has since removed from ``agpack.yml``) and returns ``None``. """ path = project_root / LOCKFILE_NAME if not path.exists(): return None try: - data = yaml.safe_load(path.read_text(encoding="utf-8")) - except (yaml.YAMLError, OSError): + text = path.read_text(encoding="utf-8") + except OSError as exc: + console.print( + f"[bold yellow]warning[/bold yellow]: cannot read lockfile {path}: {exc}.\n {_CORRUPT_LOCKFILE_WARNING}" + ) + return None + + try: + data = yaml.safe_load(text) + except yaml.YAMLError as exc: + console.print( + f"[bold yellow]warning[/bold yellow]: lockfile {path} is corrupt: {exc}.\n {_CORRUPT_LOCKFILE_WARNING}" + ) return None if not isinstance(data, dict): + console.print( + f"[bold yellow]warning[/bold yellow]: lockfile {path} has " + f"unexpected shape (expected a YAML mapping, got " + f"{type(data).__name__}).\n {_CORRUPT_LOCKFILE_WARNING}" + ) return None lockfile = Lockfile( @@ -90,13 +150,25 @@ def read_lockfile(project_root: Path) -> Lockfile | None: ) ) - for mcp_data in data.get("mcp", []): - if not isinstance(mcp_data, dict): + for edit_data in data.get("edits", []): + if not isinstance(edit_data, dict): continue - lockfile.mcp.append( - McpLockEntry( - name=mcp_data.get("name", ""), - targets=mcp_data.get("targets", []), + applied: list[AppliedPatch] = [] + for raw in edit_data.get("applied", []): + if not isinstance(raw, dict): + continue + applied.append( + AppliedPatch( + file_path=raw.get("file_path", ""), + key=raw.get("key", ""), + strategy=raw.get("strategy", "replace"), + value_hash=raw.get("value_hash", ""), + ) + ) + lockfile.edits.append( + EditLockEntry( + resource_type=edit_data.get("resource_type", ""), + applied=applied, ) ) @@ -112,7 +184,7 @@ def write_lockfile(project_root: Path, lockfile: Lockfile) -> None: "generated_at": lockfile.generated_at, "agpack_version": lockfile.agpack_version, "installed": [], - "mcp": [], + "edits": [], } for entry in lockfile.installed: @@ -126,52 +198,47 @@ def write_lockfile(project_root: Path, lockfile: Lockfile) -> None: entry_data["path"] = entry.path data["installed"].append(entry_data) - for mcp_entry in lockfile.mcp: - data["mcp"].append( + for edit in lockfile.edits: + data["edits"].append( { - "name": mcp_entry.name, - "targets": mcp_entry.targets, + "resource_type": edit.resource_type, + "applied": [ + { + "file_path": p.file_path, + "key": p.key, + "strategy": p.strategy, + "value_hash": p.value_hash, + } + for p in edit.applied + ], } ) path = project_root / LOCKFILE_NAME content = yaml.dump(data, default_flow_style=False, sort_keys=False) - # Atomic write fd, tmp_path = tempfile.mkstemp(dir=project_root, prefix=".agpack-lock-tmp-") + tmp = Path(tmp_path) try: os.close(fd) - Path(tmp_path).write_text(content, encoding="utf-8") - os.replace(tmp_path, str(path)) + tmp.write_text(content, encoding="utf-8") + tmp.replace(path) except Exception: - try: - os.unlink(tmp_path) - except OSError: - pass + with contextlib.suppress(OSError): + tmp.unlink() raise +# --------------------------------------------------------------------------- +# Diff helpers +# --------------------------------------------------------------------------- + + def find_removed_dependencies( old_lockfile: Lockfile | None, current_identities: set[str], ) -> list[InstalledEntry]: - """Find dependencies that were in the old lockfile but are no longer configured.""" + """Find copy-kind dependencies present in the old lockfile but absent now.""" if old_lockfile is None: return [] - - return [ - entry - for entry in old_lockfile.installed - if entry.identity not in current_identities - ] - - -def find_removed_mcp_servers( - old_lockfile: Lockfile | None, - current_mcp_names: set[str], -) -> list[McpLockEntry]: - """Find MCP servers that were in the old lockfile but are no longer configured.""" - if old_lockfile is None: - return [] - - return [entry for entry in old_lockfile.mcp if entry.name not in current_mcp_names] + return [entry for entry in old_lockfile.installed if entry.identity not in current_identities] diff --git a/agpack/mcp.py b/agpack/mcp.py deleted file mode 100644 index 7324063..0000000 --- a/agpack/mcp.py +++ /dev/null @@ -1,294 +0,0 @@ -"""MCP config merge logic (JSON + TOML).""" - -from __future__ import annotations - -import json -import os -import tempfile -import tomllib -from pathlib import Path -from typing import Any - -import tomli_w - -from agpack.config import McpServer -from agpack.display import console -from agpack.targets import MCP_TARGETS -from agpack.targets import McpTargetConfig - - -class McpError(Exception): - """Raised when an MCP config merge fails.""" - - -def _build_server_object(server: McpServer, target: str = "") -> dict[str, Any]: - """Build the config object for a single MCP server. - - Different targets expect different schemas, so the output varies by target. - """ - if target == "opencode": - return _build_opencode_server_object(server) - - if server.type == "stdio": - obj: dict[str, Any] = {} - # VS Code / Copilot requires an explicit "type" field for stdio servers - if target == "copilot": - obj["type"] = "stdio" - obj["command"] = server.command - if server.args: - obj["args"] = server.args - if server.env: - obj["env"] = server.env - return obj - else: - obj = {"url": server.url, "type": server.type} - return obj - - -def _build_opencode_server_object(server: McpServer) -> dict[str, Any]: - """Build an MCP server object in opencode's expected schema. - - opencode uses: type "local"/"remote", command as array, "environment" key. - """ - if server.type == "stdio": - cmd = [server.command] + server.args if server.args else [server.command] - obj: dict[str, Any] = {"type": "local", "command": cmd} - if server.env: - obj["environment"] = server.env - return obj - else: - return {"type": "remote", "url": server.url} - - -def _atomic_write(path: Path, content: str) -> None: - """Write content to a file atomically.""" - path.parent.mkdir(parents=True, exist_ok=True) - fd, tmp_path = tempfile.mkstemp(dir=path.parent, prefix=".agpack-mcp-") - try: - os.close(fd) - Path(tmp_path).write_text(content, encoding="utf-8") - os.replace(tmp_path, str(path)) - except Exception: - try: - os.unlink(tmp_path) - except OSError: - pass - raise - - -def _merge_json( - config_path: Path, - servers_key: str, - servers: dict[str, dict[str, Any]], -) -> None: - """Merge MCP servers into a JSON config file.""" - existing: dict[str, Any] = {} - if config_path.exists(): - try: - existing = json.loads(config_path.read_text(encoding="utf-8")) - except (json.JSONDecodeError, OSError) as exc: - raise McpError(f"Failed to read {config_path}: {exc}") from exc - - if servers_key not in existing: - existing[servers_key] = {} - - existing[servers_key].update(servers) - - _atomic_write(config_path, json.dumps(existing, indent=2) + "\n") - - -def _merge_toml( - config_path: Path, - servers_key: str, - servers: dict[str, dict[str, Any]], -) -> None: - """Merge MCP servers into a TOML config file.""" - existing: dict[str, Any] = {} - if config_path.exists(): - try: - existing = tomllib.loads(config_path.read_text(encoding="utf-8")) - except (tomllib.TOMLDecodeError, OSError) as exc: - raise McpError(f"Failed to read {config_path}: {exc}") from exc - - if servers_key not in existing: - existing[servers_key] = {} - - existing[servers_key].update(servers) - - _atomic_write(config_path, tomli_w.dumps(existing)) - - -def deploy_mcp_servers( - mcp_servers: list[McpServer], - targets: list[str], - project_root: Path, - dry_run: bool = False, - verbose: bool = False, -) -> dict[str, list[str]]: - """Deploy MCP server definitions to all applicable target config files. - - Returns a dict mapping server name to list of config file paths - (relative to project_root) that were written. - """ - result: dict[str, list[str]] = {} - - for server in mcp_servers: - written_to: list[str] = [] - - for target in targets: - target_cfg = MCP_TARGETS.get(target) - if target_cfg is None: - continue - - server_obj = _build_server_object(server, target=target) - servers_dict = {server.name: server_obj} - - config_path = project_root / target_cfg.config_path - rel_path = target_cfg.config_path - - if dry_run: - if verbose: - console.print(f"[dry-run] merge MCP '{server.name}' → {rel_path}") - written_to.append(rel_path) - continue - - try: - if target_cfg.format == "json": - _merge_json(config_path, target_cfg.servers_key, servers_dict) - elif target_cfg.format == "toml": - _merge_toml(config_path, target_cfg.servers_key, servers_dict) - except McpError: - raise - except Exception as exc: - raise McpError( - f"Failed to write MCP config to {config_path}: {exc}" - ) from exc - - if verbose: - console.print(f" MCP '{server.name}' → {rel_path}") - - written_to.append(rel_path) - - result[server.name] = written_to - - return result - - -def cleanup_mcp_server( - server_name: str, - target_paths: list[str], - project_root: Path, - targets: list[str], - dry_run: bool = False, - verbose: bool = False, -) -> None: - """Remove an MCP server from all listed config files.""" - for rel_path in target_paths: - config_path = project_root / rel_path - if not config_path.exists(): - continue - - # Find the target config for this path - target_cfg: McpTargetConfig | None = None - for target in targets: - cfg = MCP_TARGETS.get(target) - if cfg and cfg.config_path == rel_path: - target_cfg = cfg - break - - if target_cfg is None: - # Try to infer format from the file extension - if rel_path.endswith(".json"): - _remove_from_json(config_path, server_name, dry_run) - elif rel_path.endswith(".toml"): - _remove_from_toml(config_path, server_name, dry_run) - continue - - if dry_run: - if verbose: - console.print(f"[dry-run] remove MCP '{server_name}' from {rel_path}") - continue - - if target_cfg.format == "json": - _remove_server_from_json(config_path, target_cfg.servers_key, server_name) - elif target_cfg.format == "toml": - _remove_server_from_toml(config_path, target_cfg.servers_key, server_name) - - if verbose: - console.print(f" removed MCP '{server_name}' from {rel_path}") - - -def _remove_server_from_json( - config_path: Path, - servers_key: str, - server_name: str, -) -> None: - """Remove a server from a JSON config file.""" - try: - data = json.loads(config_path.read_text(encoding="utf-8")) - except (json.JSONDecodeError, OSError): - return - - servers = data.get(servers_key, {}) - if server_name in servers: - del servers[server_name] - _atomic_write(config_path, json.dumps(data, indent=2) + "\n") - - -def _remove_server_from_toml( - config_path: Path, - servers_key: str, - server_name: str, -) -> None: - """Remove a server from a TOML config file.""" - try: - data = tomllib.loads(config_path.read_text(encoding="utf-8")) - except (tomllib.TOMLDecodeError, OSError): - return - - servers = data.get(servers_key, {}) - if server_name in servers: - del servers[server_name] - _atomic_write(config_path, tomli_w.dumps(data)) - - -def _remove_from_json( - config_path: Path, - server_name: str, - dry_run: bool, -) -> None: - """Remove a server from a JSON file, trying common server keys.""" - if dry_run: - return - try: - data = json.loads(config_path.read_text(encoding="utf-8")) - except (json.JSONDecodeError, OSError): - return - - for key in ("mcpServers", "mcp", "servers"): - servers = data.get(key, {}) - if isinstance(servers, dict) and server_name in servers: - del servers[server_name] - _atomic_write(config_path, json.dumps(data, indent=2) + "\n") - return - - -def _remove_from_toml( - config_path: Path, - server_name: str, - dry_run: bool, -) -> None: - """Remove a server from a TOML file, trying common server keys.""" - if dry_run: - return - try: - data = tomllib.loads(config_path.read_text(encoding="utf-8")) - except (tomllib.TOMLDecodeError, OSError): - return - - for key in ("mcp_servers",): - servers = data.get(key, {}) - if isinstance(servers, dict) and server_name in servers: - del servers[server_name] - _atomic_write(config_path, tomli_w.dumps(data)) - return diff --git a/agpack/patch.py b/agpack/patch.py new file mode 100644 index 0000000..365f241 --- /dev/null +++ b/agpack/patch.py @@ -0,0 +1,75 @@ +"""The patch data model — shared between config parsing and edit-file deployment. + +A :class:`Patch` is a top-level domain type: it appears in ``agpack.yml`` (parsed by :mod:`agpack.config`), it gets +applied to a structured config file (by :mod:`agpack.kinds.edit_file`), and a hash of its resolved value is recorded +in the lockfile. Keeping it in its own leaf module means neither parsing nor application has to import the other. +""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from typing import Any +from typing import Literal + +Strategy = Literal["replace", "append"] + + +@dataclass(frozen=True) +class Patch: + """A single change to apply to a structured config file. + + Attributes: + key: Dotted path into the config file (``mcpServers.filesystem`` + or ``hooks.PreToolUse``). Intermediate dicts are auto-created. + value: What to put at the path. For ``append``, this is a single + element appended to the list at ``key``. + strategy: ``"replace"`` overwrites whatever's at the path; + ``"append"`` requires the path to resolve to a list (created + empty if absent) and appends ``value``. + """ + + key: str + value: Any + strategy: Strategy = "replace" + + +def match_key(strategy: str, key: str, value_hash: str) -> tuple[Any, ...]: + """Identity tuple for diffing patches across syncs. + + ``replace`` patches identify by ``(key,)`` — same key with a different value is still the same *slot* (an update). + ``append`` patches identify by ``(key, value_hash)`` — different appended values are distinct list elements. + + Callers normalise to primitives before calling: pull ``strategy`` and ``key`` from the patch directly, and use + :func:`_value_hash` on the resolved value for a :class:`Patch` or the stored ``value_hash`` for an applied entry. + """ + if strategy == "replace": + return ("replace", key) + return ("append", key, value_hash) + + +def value_hash(value: Any) -> str: + """SHA256 of the canonical JSON form of *value* (post-:func:`_unwrap` of tomlkit wrappers). + + Used both as in-memory identity for ``append`` patches and as the lockfile's record of what was applied. Storing + the hash rather than the value keeps secrets interpolated via ``${VAR}`` out of the lockfile. + """ + canonical = json.dumps(_unwrap(value), sort_keys=True, default=str) + return "sha256:" + hashlib.sha256(canonical.encode("utf-8")).hexdigest() + + +def _unwrap(value: Any) -> Any: + """Convert a tomlkit ``Item`` to its plain-Python equivalent. + + :meth:`tomlkit.items.Item.unwrap` already recurses through nested Tables and Arrays, so a single call returns a + fully-plain dict / list / scalar. For values that are already plain (JSON-loaded data, lockfile values, primitives) + this is a no-op. Used at the boundary between tomlkit-managed data and the lockfile / equality checks, which both + want plain Python. + """ + if hasattr(value, "unwrap"): + try: + return value.unwrap() + except Exception: # noqa: BLE001, S110 # defensive: any tomlkit wrapper failure falls through to the plain value + pass + return value diff --git a/agpack/registry.py b/agpack/registry.py new file mode 100644 index 0000000..bad5900 --- /dev/null +++ b/agpack/registry.py @@ -0,0 +1,60 @@ +"""Target manifest discovery and loading. + +Built-in target manifests ship as YAML files inside the package (``agpack/builtin_targets/``). This module loads them +via ``importlib.resources`` so they continue to work when agpack is installed from a wheel. + +The full resolution chain (project ``target_definitions`` → global ``target_definitions`` → built-in) lives in a later +commit; this module currently exposes only the built-in surface. +""" + +from __future__ import annotations + +from importlib.resources import files + +import yaml + +from agpack.errors import TargetSchemaError +from agpack.target_schema import TargetDef +from agpack.target_schema import parse_target_def + +_BUILTIN_PACKAGE = "agpack.builtin_targets" +_BUILTIN_SUFFIX = ".yml" + + +def list_builtins() -> list[str]: + """Return the sorted list of built-in target names.""" + names: list[str] = [] + for entry in files(_BUILTIN_PACKAGE).iterdir(): + if entry.is_file() and entry.name.endswith(_BUILTIN_SUFFIX): + names.append(entry.name[: -len(_BUILTIN_SUFFIX)]) + return sorted(names) + + +def load_builtin(name: str) -> TargetDef: + """Load a single built-in target manifest by name. + + Args: + name: The target name (matches the YAML filename without + extension, e.g. ``"claude"``). + + Raises: + TargetSchemaError: If the manifest does not exist or is invalid. + """ + resource = files(_BUILTIN_PACKAGE).joinpath(f"{name}{_BUILTIN_SUFFIX}") + if not resource.is_file(): + available = ", ".join(list_builtins()) + msg = f"No built-in target named '{name}'. Available: {available}" + raise TargetSchemaError(msg) + + try: + data = yaml.safe_load(resource.read_text(encoding="utf-8")) + except yaml.YAMLError as exc: + msg = f"Failed to parse built-in target '{name}': {exc}" + raise TargetSchemaError(msg) from exc + + return parse_target_def(data, name=name, context=f"builtin_targets/{name}{_BUILTIN_SUFFIX}") + + +def load_all_builtins() -> dict[str, TargetDef]: + """Load every shipped built-in manifest, keyed by name.""" + return {name: load_builtin(name) for name in list_builtins()} diff --git a/agpack/target_schema.py b/agpack/target_schema.py new file mode 100644 index 0000000..44cae82 --- /dev/null +++ b/agpack/target_schema.py @@ -0,0 +1,179 @@ +"""Target manifest schema — parser, validator, and the resource-type unions. + +A *target* describes the filesystem location of each resource type a single AI tool consumes. Each top-level key is +the resource type name (``skills``, ``commands``, ``mcp``, ``settings``, anything user-defined); each value declares a +resource definition via the ``kind:`` field. + +The actual deploy/cleanup behavior lives on the kind classes in :mod:`agpack.kinds.copy_directory`, +:mod:`agpack.kinds.copy_file`, and :mod:`agpack.kinds.edit_file`. This module is only responsible for turning YAML +into well-typed resource definitions and exposing the union types used in signatures elsewhere. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from dataclasses import field +from typing import Any + +from agpack.errors import TargetSchemaError +from agpack.kinds import CopyDirectoryResource +from agpack.kinds import CopyFileResource +from agpack.kinds import EditFileResource +from agpack.kinds import ResourceDef +from agpack.kinds import infer_config_format + +# --------------------------------------------------------------------------- +# Constants +# --------------------------------------------------------------------------- + +_VALID_KINDS = ("copy-directory", "copy-file", "edit-file") + + +# --------------------------------------------------------------------------- +# TargetDef +# --------------------------------------------------------------------------- + + +@dataclass(frozen=True) +class TargetDef: + """A fully-resolved target manifest. + + :attr:`name` is the YAML filename (built-ins) or the mapping key under ``target_definitions:`` in ``agpack.yml``. + It's recorded in the lockfile so cleanup can re-resolve ``${var}`` references against this target's ``vars`` long + after the resource type has been removed from ``dependencies:``. + """ + + name: str = "" + resources: dict[str, ResourceDef] = field(default_factory=dict) + + +# --------------------------------------------------------------------------- +# Helpers +# --------------------------------------------------------------------------- + + +def _require_mapping(value: Any, context: str) -> dict[str, Any]: + if not isinstance(value, dict): + msg = f"{context}: expected a mapping, got {type(value).__name__}" + raise TargetSchemaError(msg) + return value + + +def _require_string(value: Any, context: str) -> str: + if not isinstance(value, str) or not value: + msg = f"{context}: expected a non-empty string" + raise TargetSchemaError(msg) + return value + + +def _reject_extra(known: set[str], data: dict[str, Any], context: str) -> None: + extra = set(data) - known + if extra: + msg = f"{context}: unknown keys {sorted(extra)}; valid: {sorted(known)}" + raise TargetSchemaError(msg) + + +# --------------------------------------------------------------------------- +# Per-kind parsers +# --------------------------------------------------------------------------- + + +def _parse_copy_directory(data: dict[str, Any], context: str) -> CopyDirectoryResource: + _reject_extra({"kind", "path"}, data, context) + path = _require_string(data.get("path"), f"{context}.path") + return CopyDirectoryResource(path=path) + + +def _parse_copy_file(data: dict[str, Any], context: str) -> CopyFileResource: + _reject_extra({"kind", "path"}, data, context) + path = _require_string(data.get("path"), f"{context}.path") + return CopyFileResource(path=path) + + +def _parse_edit_file(data: dict[str, Any], context: str) -> EditFileResource: + _reject_extra({"kind", "path", "vars"}, data, context) + path = _require_string(data.get("path"), f"{context}.path") + # Validate extension at parse time so malformed manifests fail loudly. + try: + infer_config_format(path) + except Exception as exc: + msg = f"{context}.path: {exc}" + raise TargetSchemaError(msg) from exc + + raw_vars = data.get("vars", {}) + if not isinstance(raw_vars, dict): + msg = f"{context}.vars: must be a mapping, got {type(raw_vars).__name__}" + raise TargetSchemaError(msg) + target_vars: dict[str, str] = {} + for key, value in raw_vars.items(): + if not isinstance(key, str) or not key: + msg = f"{context}.vars: keys must be non-empty strings, got {key!r}" + raise TargetSchemaError(msg) + if not isinstance(value, str): + msg = f"{context}.vars.{key}: value must be a string, got {type(value).__name__}" + raise TargetSchemaError(msg) + target_vars[key] = value + + return EditFileResource(path=path, vars=target_vars) + + +# --------------------------------------------------------------------------- +# Resource block parser (kind dispatch) +# --------------------------------------------------------------------------- + + +def _parse_resource(raw: Any, context: str) -> ResourceDef: + data = _require_mapping(raw, context) + + if "layout" in data: + msg = ( + f"{context}.layout: deprecated — use 'kind: copy-directory' or " + f"'kind: copy-file' instead. (Was: layout: {data['layout']!r}.)" + ) + raise TargetSchemaError(msg) + if "merge" in data: + msg = ( + f"{context}.merge: removed — edit-file resources now take only " + f"'kind' and 'path'. Patches live under dependencies in agpack.yml." + ) + raise TargetSchemaError(msg) + + kind = data.get("kind") + if kind not in _VALID_KINDS: + msg = f"{context}.kind: must be one of {_VALID_KINDS}, got {kind!r}" + raise TargetSchemaError(msg) + + if kind == "copy-directory": + return _parse_copy_directory(data, context) + if kind == "copy-file": + return _parse_copy_file(data, context) + return _parse_edit_file(data, context) + + +# --------------------------------------------------------------------------- +# Public entrypoint +# --------------------------------------------------------------------------- + + +def parse_target_def(raw: Any, *, name: str = "", context: str = "target") -> TargetDef: + """Parse a target manifest from a raw dict (loaded YAML). + + Each top-level key is a resource type name (``skills`` / ``commands`` / ``mcp`` / any user-defined name). Each + block must declare a ``kind:`` (``copy-directory`` / ``copy-file`` / ``edit-file``) and a ``path:``. + + *name* is stored verbatim on the returned :class:`TargetDef` — callers know what the target is called (built-in + filename or the mapping key under ``target_definitions:``) and pass it in. + + Raises: + TargetSchemaError: If the manifest is malformed. + """ + data = _require_mapping(raw, context) + + resources: dict[str, ResourceDef] = {} + for key, value in data.items(): + if not isinstance(key, str) or not key: + msg = f"{context}: keys must be non-empty strings, got {key!r}" + raise TargetSchemaError(msg) + resources[key] = _parse_resource(value, f"{context}.{key}") + + return TargetDef(name=name, resources=resources) diff --git a/agpack/targets.py b/agpack/targets.py deleted file mode 100644 index 7d73d2a..0000000 --- a/agpack/targets.py +++ /dev/null @@ -1,112 +0,0 @@ -"""Target directory mapping constants for each supported AI coding tool.""" - -from __future__ import annotations - -from dataclasses import dataclass - -# Recognised target names -VALID_TARGETS = frozenset( - { - "claude", - "opencode", - "codex", - "cursor", - "copilot", - "gemini", - "windsurf", - "antigravity", - } -) - -# --------------------------------------------------------------------------- -# Skills -# --------------------------------------------------------------------------- - -SKILL_DIRS: dict[str, str] = { - "claude": ".claude/skills", - "opencode": ".opencode/skills", - "codex": ".agents/skills", - "cursor": ".cursor/skills", - "copilot": ".github/skills", - "gemini": ".gemini/skills", - "windsurf": ".windsurf/skills", - "antigravity": ".gemini/skills", # Antigravity shares the .gemini/ namespace -} - -# --------------------------------------------------------------------------- -# Commands -# --------------------------------------------------------------------------- - -# codex, cursor, and windsurf do not support commands -COMMAND_DIRS: dict[str, str] = { - "claude": ".claude/commands", - "opencode": ".opencode/commands", - "copilot": ".github/prompts", - "gemini": ".gemini/commands", - "antigravity": ".gemini/commands", # shares .gemini/ namespace -} - -# --------------------------------------------------------------------------- -# Agents -# --------------------------------------------------------------------------- - -# codex, gemini, antigravity, and windsurf do not support agents -AGENT_DIRS: dict[str, str] = { - "claude": ".claude/agents", - "opencode": ".opencode/agents", - "cursor": ".cursor/agents", - "copilot": ".github/agents", -} - -# --------------------------------------------------------------------------- -# MCP config files -# --------------------------------------------------------------------------- - - -@dataclass(frozen=True) -class McpTargetConfig: - """Describes how a target stores MCP server definitions.""" - - config_path: str - format: str # "json" or "toml" - servers_key: str # top-level key in the config that holds the servers dict - - -# windsurf: MCP config is global (~/.codeium/windsurf/mcp_config.json), not per-project -MCP_TARGETS: dict[str, McpTargetConfig] = { - "claude": McpTargetConfig( - config_path=".mcp.json", - format="json", - servers_key="mcpServers", - ), - "opencode": McpTargetConfig( - config_path="opencode.json", - format="json", - servers_key="mcp", - ), - "codex": McpTargetConfig( - config_path=".codex/config.toml", - format="toml", - servers_key="mcp_servers", - ), - "cursor": McpTargetConfig( - config_path=".cursor/mcp.json", - format="json", - servers_key="mcpServers", - ), - "copilot": McpTargetConfig( - config_path=".vscode/mcp.json", - format="json", - servers_key="servers", - ), - "gemini": McpTargetConfig( - config_path=".gemini/settings.json", - format="json", - servers_key="mcpServers", - ), - "antigravity": McpTargetConfig( # shares .gemini/ namespace - config_path=".gemini/settings.json", - format="json", - servers_key="mcpServers", - ), -} diff --git a/agpack/templates/init_global.yml b/agpack/templates/init_global.yml new file mode 100644 index 0000000..bca7510 --- /dev/null +++ b/agpack/templates/init_global.yml @@ -0,0 +1,31 @@ +# Global agpack config — dependencies here are included in every project. +# Override per-project with 'global: false' in your project agpack.yml, +# or run agpack sync --no-global. + +dependencies: + skills: + # Shared skills available in all projects: + # - url: https://github.com/owner/repo + # path: skills/my-skill + # ref: v1.0.0 + + commands: + # Shared commands available in all projects: + # - url: https://github.com/owner/repo + # path: commands/my-command.md + + agents: + # Shared agents available in all projects: + # - url: https://github.com/owner/repo + # path: agents/my-agent.md + + mcp: + # Shared MCP servers as patches. ${bucket} is supplied by each + # built-in target (mcpServers / mcp_servers / mcp / servers), so + # one patch deploys correctly to every target. + # - key: ${bucket}.my-server + # value: + # command: npx + # args: ["-y", "@example/mcp-server"] + # env: + # API_KEY: ${API_KEY} # from .env or shell env diff --git a/agpack/templates/init_project.yml b/agpack/templates/init_project.yml new file mode 100644 index 0000000..602ad96 --- /dev/null +++ b/agpack/templates/init_project.yml @@ -0,0 +1,94 @@ +# Set to false to ignore the global config (~/.config/agpack/agpack.yml): +# global: false + +targets: + # - claude + # - opencode + # - codex + # - cursor + # - copilot + +dependencies: + skills: + # Point to a single skill folder: + # - url: https://github.com/owner/repo + # path: skills/my-skill + # ref: v1.0.0 + # Multiple URLs (tried in order, e.g. HTTPS + SSH fallback): + # - url: + # - https://github.com/owner/repo + # - git@github.com:owner/repo.git + # path: skills/my-skill + # Or to a directory of skill folders (each subfolder is deployed separately): + # - url: https://github.com/owner/repo + # path: skills + + commands: + # Point to a single file or a directory of command files: + # - url: https://github.com/owner/repo + # path: commands/my-command.md + + agents: + # Point to a single file or a directory of agent files: + # - url: https://github.com/owner/repo + # path: agents/my-agent.md + + # edit-file resources take patches instead of git URLs. ${bucket} + # is supplied by the target manifest (e.g. mcpServers for Claude, + # mcp for OpenCode, mcp_servers for Codex), so a single patch + # deploys to every configured target. + mcp: + # - key: ${bucket}.my-server + # value: + # command: npx + # args: ["-y", "@example/mcp-server"] + # env: + # API_KEY: ${API_KEY} # from .env or shell env + + # Claude Code hooks (.claude/settings.json → hooks.): + # hooks: + # - key: ${bucket}.PreToolUse # bucket="hooks" on Claude + # strategy: append # default is 'replace' + # value: + # matcher: "Write|Edit" + # hooks: + # - type: command + # # $${} escapes — Claude Code resolves at hook fire time: + # command: "$${CLAUDE_PROJECT_DIR}/.claude/hooks/lint.sh" + # + # permissions: + # - key: ${bucket}.allow # bucket="permissions" + # strategy: append + # value: "Read(/etc/**)" + +# Override a built-in target or define a brand-new one. +# Use 'agpack targets list' to see all available targets and +# 'agpack targets show ' to print a starting manifest you can copy here. +# target_definitions: +# claude: +# # Override the built-in claude target — replace semantics (no merge). +# skills: +# kind: copy-directory # one of: copy-directory, copy-file, edit-file +# path: .my-claude/skills +# commands: +# kind: copy-file +# path: .my-claude/commands +# mcp: +# kind: edit-file # patches written from dependencies.mcp +# path: .mcp.json # format (json|toml) inferred from extension +# vars: # exposed to patches as ${name} +# bucket: mcpServers # so patch ${bucket}.x → mcpServers.x +# +# my-internal-tool: +# # Brand-new target — also listed under 'targets:' above to be used. +# skills: +# kind: copy-directory +# path: .myaitool/skills +# # Resource type names are open — declare any name (rules, prompts, +# # personas, …) and use the same name in 'dependencies:' above. +# rules: +# kind: copy-file +# path: .myaitool/rules +# settings: +# kind: edit-file +# path: .myaitool/settings.json diff --git a/pyproject.toml b/pyproject.toml index d3269e2..767487a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -12,7 +12,7 @@ dependencies = [ "click>=8.0", "PyYAML>=6.0", "rich>=13.0", - "tomli-w>=1.0", + "tomlkit>=0.13", ] classifiers = [ "Development Status :: 3 - Alpha", @@ -34,33 +34,65 @@ Funding = "https://github.com/sponsors/PhilippTh" agpack = "agpack.cli:main" [tool.ruff] -# Enable pycodestyle (`E`), Pyflakes (`F`), and import sorting (`I`) -lint.select = ["E", "F", "I", "B", "C4", "ARG", "N", "COM", "UP", "W"] -lint.ignore = ["B008", "COM812"] # Ignore explicit function call in function decorator -lint.fixable = ["ALL"] -lint.unfixable = [] +line-length = 120 -# Same as Black -line-length = 88 -indent-width = 4 +[tool.ruff.lint] +fixable = ["ALL"] +select = [ -# Assume Python 3.12 -target-version = "py312" + # Level 1 + "E", # pycodestyle + "F", # pyflakes + "W", # warnings + "C901", # McCabe, + "I", # isort + "N", # pep8-naming + "UP", # pyupgrade -[tool.ruff.lint.isort] -force-single-line = true + # Level 2 + "B", # flake8-bugbear (common bugs) + "S", # bandit (security) + "C4", # flake8-comprehensions + "SIM", # flake8-simplify + "ARG", # flake8-unused-arguments + "PTH", # flake8-use-pathlib + "RET", # flake8-return + "FBT", # flake8-boolean-trap + + # Level 3 + + "PT", # flake8-pytest-style (if you use pytest) + "EM", # flake8-errmsg (better error messages) + "G", # flake8-logging-format + "ISC", # flake8-implicit-str-concat + "Q", # flake8-quotes (consistent quote style) + +] +ignore = [ + "S101", # Use of assert (common in tests) + "FBT001", # Boolean positional arg in function definition + "FBT002", # Boolean default arg in function definition +] [tool.ruff.format] quote-style = "double" indent-style = "space" -skip-magic-trailing-comma = false -line-ending = "auto" +docstring-code-format = true -[tool.ruff.lint.pydocstyle] -convention = "google" +[tool.ruff.lint.isort] +force-single-line = true [tool.ruff.lint.per-file-ignores] -"__init__.py" = ["F401"] # Ignore unused imports in __init__.py files +# Hardcoded-password, /tmp paths, and subprocess checks are noise in tests. +"tests/*" = ["S105", "S106", "S108", "S603", "S607"] +# Cloning git repos is this module's whole purpose — running the +# system ``git`` binary on URL arguments is intentional, not a smell. +# S607 also false-fires on ``r"\\1"`` (regex back-reference replacement +# strings) used inside FetchError messages here. +"agpack/fetcher.py" = ["S603", "S607"] + +[tool.ruff.lint.mccabe] +max-complexity = 10 [tool.pytest.ini_options] testpaths = ["tests"] @@ -72,6 +104,9 @@ build-backend = "hatchling.build" [tool.hatch.version] path = "agpack/__init__.py" +[tool.hatch.build.targets.wheel] +packages = ["agpack"] + [dependency-groups] dev = [ "pytest>=9.0.2", diff --git a/tests/test_builtin_targets.py b/tests/test_builtin_targets.py new file mode 100644 index 0000000..64c9761 --- /dev/null +++ b/tests/test_builtin_targets.py @@ -0,0 +1,107 @@ +"""Regression tests for the shipped built-in target manifests. + +Built-in manifests are intentionally tiny in the patch-based world: each entry is just ``kind`` + ``path``. These +assertions lock in the file locations so a typo can't silently break a target. +""" + +from __future__ import annotations + +from agpack.kinds import CopyDirectoryResource +from agpack.kinds import CopyFileResource +from agpack.kinds import EditFileResource +from agpack.registry import load_builtin + + +def test_claude_paths() -> None: + target = load_builtin("claude") + assert isinstance(target.resources["skills"], CopyDirectoryResource) + assert target.resources["skills"].path == ".claude/skills" + assert isinstance(target.resources["commands"], CopyFileResource) + assert target.resources["commands"].path == ".claude/commands" + assert target.resources["agents"].path == ".claude/agents" + + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.path == ".mcp.json" + assert mcp.vars == {"bucket": "mcpServers"} + + settings = target.resources["settings"] + assert isinstance(settings, EditFileResource) + assert settings.path == ".claude/settings.json" + + hooks = target.resources["hooks"] + assert isinstance(hooks, EditFileResource) + assert hooks.path == ".claude/settings.json" + assert hooks.vars == {"bucket": "hooks"} + + permissions = target.resources["permissions"] + assert isinstance(permissions, EditFileResource) + assert permissions.path == ".claude/settings.json" + assert permissions.vars == {"bucket": "permissions"} + + +def test_opencode_paths() -> None: + target = load_builtin("opencode") + assert target.resources["skills"].path == ".opencode/skills" + assert target.resources["commands"].path == ".opencode/commands" + assert target.resources["agents"].path == ".opencode/agents" + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.path == "opencode.json" + + +def test_codex_paths() -> None: + target = load_builtin("codex") + assert target.resources["skills"].path == ".codex/skills" + assert isinstance(target.resources["agents"], CopyFileResource) + assert target.resources["agents"].path == ".codex/agents" + assert "commands" not in target.resources + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.format == "toml" + + +def test_cursor_paths() -> None: + target = load_builtin("cursor") + assert target.resources["skills"].path == ".cursor/skills" + assert target.resources["commands"].path == ".cursor/commands" + assert "agents" not in target.resources + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.path == ".cursor/mcp.json" + + +def test_copilot_paths() -> None: + target = load_builtin("copilot") + assert target.resources["skills"].path == ".github/skills" + assert target.resources["commands"].path == ".github/prompts" + assert target.resources["agents"].path == ".github/agents" + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.path == ".vscode/mcp.json" + + +def test_gemini_paths() -> None: + target = load_builtin("gemini") + assert target.resources["skills"].path == ".gemini/skills" + assert target.resources["commands"].path == ".gemini/commands" + assert "agents" not in target.resources + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.path == ".gemini/settings.json" + + +def test_windsurf_paths() -> None: + target = load_builtin("windsurf") + assert target.resources["skills"].path == ".windsurf/skills" + assert target.resources["commands"].path == ".windsurf/workflows" + assert "agents" not in target.resources + assert "mcp" not in target.resources + + +def test_antigravity_paths() -> None: + target = load_builtin("antigravity") + assert target.resources["skills"].path == ".agent/skills" + assert target.resources["commands"].path == ".agent/workflows" + assert "agents" not in target.resources + assert "mcp" not in target.resources diff --git a/tests/test_cli_parallel.py b/tests/test_cli_parallel.py index e1678fd..299a503 100644 --- a/tests/test_cli_parallel.py +++ b/tests/test_cli_parallel.py @@ -11,13 +11,15 @@ import pytest from agpack.cli import _MAX_FETCH_WORKERS +from agpack.cli import _resource_kinds from agpack.cli import _sync_resource_type -from agpack.config import AgpackConfig from agpack.config import DependencySource from agpack.display import create_sync_progress from agpack.fetcher import FetchError from agpack.fetcher import FetchResult from agpack.lockfile import Lockfile +from agpack.registry import load_builtin +from agpack.target_schema import TargetDef def _make_dep(name: str) -> DependencySource: @@ -30,26 +32,25 @@ def _make_result(dep: DependencySource, tmp_path: Path) -> FetchResult: return FetchResult(source=dep, local_path=d, resolved_ref="abc1234", _tmpdir=d) -def _make_config(targets: list[str] | None = None) -> AgpackConfig: - return AgpackConfig( - targets=targets or ["claude"], - ) +def _make_targets(names: list[str] | None = None) -> list[TargetDef]: + return [load_builtin(n) for n in (names or ["claude"])] -def _fake_detect(result: FetchResult) -> list[tuple[str, Path]]: - """Detect function that returns a single item per fetch result.""" +def _fake_detect_items(result: FetchResult, _resource: object, _resource_type: str) -> list[tuple[str, Path]]: + """Stand-in for ``agpack.deployer.detect_items``.""" return [(result.source.name, result.local_path)] def _fake_deploy_item( name: str, _path: Path, - _targets: list[str], + _resource_type: str, + _targets: list[TargetDef], _project_root: Path, - _dry_run: bool, - _verbose: bool, + dry_run: bool = False, # noqa: ARG001 # mock must mirror deploy_item's kwargs + verbose: bool = False, # noqa: ARG001 ) -> list[str]: - """Deploy function that pretends to deploy a single file.""" + """Stand-in for ``agpack.deployer.deploy_item``.""" return [f"{name}.md"] @@ -57,56 +58,58 @@ class TestParallelFetchAllSucceed: def test_all_fetched_and_deployed(self, tmp_path: Path) -> None: deps = [_make_dep("a"), _make_dep("b"), _make_dep("c")] fake_results = {dep.name: _make_result(dep, tmp_path) for dep in deps} - config = _make_config() + target_defs = _make_targets() new_lockfile = Lockfile() - def fake_fetch(dep: DependencySource) -> FetchResult: + def fake_fetch(dep: DependencySource, _env=None) -> FetchResult: return fake_results[dep.name] - deploy_item_fn = MagicMock(side_effect=_fake_deploy_item) + deploy_item_mock = MagicMock(side_effect=_fake_deploy_item) with ( patch("agpack.cli.fetch_dependency", side_effect=fake_fetch), patch("agpack.cli.cleanup_fetch"), + patch("agpack.cli.detect_items", side_effect=_fake_detect_items), + patch("agpack.cli.deploy_item", deploy_item_mock), create_sync_progress() as progress, ): sync = _sync_resource_type( deps, - _fake_detect, - deploy_item_fn, - "skill", - config, + "skills", + target_defs, tmp_path, new_lockfile, progress, + {}, dry_run=False, verbose=False, ) assert sync.count == 3 - assert deploy_item_fn.call_count == 3 + assert deploy_item_mock.call_count == 3 assert len(new_lockfile.installed) == 3 def test_lockfile_entries_added(self, tmp_path: Path) -> None: deps = [_make_dep("x")] fake_result = _make_result(deps[0], tmp_path) - config = _make_config() + target_defs = _make_targets() new_lockfile = Lockfile() with ( patch("agpack.cli.fetch_dependency", return_value=fake_result), patch("agpack.cli.cleanup_fetch"), + patch("agpack.cli.detect_items", side_effect=_fake_detect_items), + patch("agpack.cli.deploy_item", side_effect=_fake_deploy_item), create_sync_progress() as progress, ): _sync_resource_type( deps, - _fake_detect, - _fake_deploy_item, - "skill", - config, + "skills", + target_defs, tmp_path, new_lockfile, progress, + {}, dry_run=False, verbose=False, ) @@ -118,30 +121,32 @@ def test_lockfile_entries_added(self, tmp_path: Path) -> None: class TestParallelFetchCollectAllErrors: def test_all_errors_reported(self, tmp_path: Path) -> None: deps = [_make_dep("a"), _make_dep("b"), _make_dep("c")] - config = _make_config() + target_defs = _make_targets() new_lockfile = Lockfile() - def fake_fetch(dep: DependencySource) -> FetchResult: - raise FetchError(f"failed {dep.name}") + def fake_fetch(dep: DependencySource, _env=None) -> FetchResult: + msg = f"failed {dep.name}" + raise FetchError(msg) - detect_fn = MagicMock() - deploy_item_fn = MagicMock() + detect_items_mock = MagicMock() + deploy_item_mock = MagicMock() with ( patch("agpack.cli.fetch_dependency", side_effect=fake_fetch), patch("agpack.cli.write_lockfile") as mock_write, + patch("agpack.cli.detect_items", detect_items_mock), + patch("agpack.cli.deploy_item", deploy_item_mock), create_sync_progress() as progress, pytest.raises(click.ClickException) as exc_info, ): _sync_resource_type( deps, - detect_fn, - deploy_item_fn, - "skill", - config, + "skills", + target_defs, tmp_path, new_lockfile, progress, + {}, dry_run=False, verbose=False, ) @@ -151,36 +156,38 @@ def fake_fetch(dep: DependencySource) -> FetchResult: assert "failed b" in msg assert "failed c" in msg assert "3" in msg - detect_fn.assert_not_called() - deploy_item_fn.assert_not_called() + detect_items_mock.assert_not_called() + deploy_item_mock.assert_not_called() mock_write.assert_called_once() def test_partial_failure_cleans_up_successes(self, tmp_path: Path) -> None: deps = [_make_dep("ok"), _make_dep("bad")] fake_result = _make_result(deps[0], tmp_path) - config = _make_config() + target_defs = _make_targets() - def fake_fetch(dep: DependencySource) -> FetchResult: + def fake_fetch(dep: DependencySource, _env=None) -> FetchResult: if dep.name == "bad": - raise FetchError("boom") + msg = "boom" + raise FetchError(msg) return fake_result with ( patch("agpack.cli.fetch_dependency", side_effect=fake_fetch), patch("agpack.cli.cleanup_fetch") as mock_cleanup, patch("agpack.cli.write_lockfile"), + patch("agpack.cli.detect_items", MagicMock()), + patch("agpack.cli.deploy_item", MagicMock()), create_sync_progress() as progress, pytest.raises(click.ClickException), ): _sync_resource_type( deps, - MagicMock(), - MagicMock(), - "skill", - config, + "skills", + target_defs, tmp_path, Lockfile(), progress, + {}, dry_run=False, verbose=False, ) @@ -189,7 +196,7 @@ def fake_fetch(dep: DependencySource) -> FetchResult: def test_dry_run_skips_lockfile_write(self, tmp_path: Path) -> None: deps = [_make_dep("bad")] - config = _make_config() + target_defs = _make_targets() with ( patch("agpack.cli.fetch_dependency", side_effect=FetchError("boom")), @@ -200,13 +207,12 @@ def test_dry_run_skips_lockfile_write(self, tmp_path: Path) -> None: ): _sync_resource_type( deps, - MagicMock(), - MagicMock(), - "skill", - config, + "skills", + target_defs, tmp_path, Lockfile(), progress, + {}, dry_run=True, verbose=False, ) @@ -216,37 +222,39 @@ def test_dry_run_skips_lockfile_write(self, tmp_path: Path) -> None: def test_deploy_not_called_when_any_fetch_fails(self, tmp_path: Path) -> None: deps = [_make_dep("a"), _make_dep("b")] fake_result = _make_result(deps[0], tmp_path) - config = _make_config() - detect_fn = MagicMock() - deploy_item_fn = MagicMock() + target_defs = _make_targets() + detect_items_mock = MagicMock() + deploy_item_mock = MagicMock() - def fake_fetch(dep: DependencySource) -> FetchResult: + def fake_fetch(dep: DependencySource, _env=None) -> FetchResult: if dep.name == "b": - raise FetchError("nope") + msg = "nope" + raise FetchError(msg) return fake_result with ( patch("agpack.cli.fetch_dependency", side_effect=fake_fetch), patch("agpack.cli.cleanup_fetch"), patch("agpack.cli.write_lockfile"), + patch("agpack.cli.detect_items", detect_items_mock), + patch("agpack.cli.deploy_item", deploy_item_mock), create_sync_progress() as progress, pytest.raises(click.ClickException), ): _sync_resource_type( deps, - detect_fn, - deploy_item_fn, - "skill", - config, + "skills", + target_defs, tmp_path, Lockfile(), progress, + {}, dry_run=False, verbose=False, ) - detect_fn.assert_not_called() - deploy_item_fn.assert_not_called() + detect_items_mock.assert_not_called() + deploy_item_mock.assert_not_called() class TestParallelFetchEdgeCases: @@ -257,13 +265,12 @@ def test_empty_deps_returns_zero(self, tmp_path: Path) -> None: ): sync = _sync_resource_type( [], - MagicMock(), - MagicMock(), - "skill", - _make_config(), + "skills", + _make_targets(), tmp_path, Lockfile(), progress, + {}, dry_run=False, verbose=False, ) @@ -272,7 +279,7 @@ def test_empty_deps_returns_zero(self, tmp_path: Path) -> None: def test_concurrency_capped_at_max_workers(self, tmp_path: Path) -> None: deps = [_make_dep(str(i)) for i in range(20)] - config = _make_config() + target_defs = _make_targets() captured: list[int] = [] real_init = ThreadPoolExecutor.__init__ @@ -282,24 +289,25 @@ def capturing_init(self, *args, max_workers=None, **kwargs): captured.append(max_workers) real_init(self, *args, max_workers=max_workers, **kwargs) - def fake_fetch(dep: DependencySource) -> FetchResult: + def fake_fetch(dep: DependencySource, _env=None) -> FetchResult: return _make_result(dep, tmp_path) with ( patch("agpack.cli.fetch_dependency", side_effect=fake_fetch), patch("agpack.cli.cleanup_fetch"), + patch("agpack.cli.detect_items", side_effect=_fake_detect_items), + patch("agpack.cli.deploy_item", side_effect=_fake_deploy_item), patch.object(ThreadPoolExecutor, "__init__", capturing_init), create_sync_progress() as progress, ): _sync_resource_type( deps, - _fake_detect, - _fake_deploy_item, - "skill", - config, + "skills", + target_defs, tmp_path, Lockfile(), progress, + {}, dry_run=False, verbose=False, ) @@ -310,26 +318,55 @@ def fake_fetch(dep: DependencySource) -> FetchResult: def test_deploy_error_writes_lockfile(self, tmp_path: Path) -> None: deps = [_make_dep("a")] fake_result = _make_result(deps[0], tmp_path) - config = _make_config() + target_defs = _make_targets() with ( patch("agpack.cli.fetch_dependency", return_value=fake_result), patch("agpack.cli.cleanup_fetch"), patch("agpack.cli.write_lockfile") as mock_write, + patch("agpack.cli.detect_items", side_effect=_fake_detect_items), + patch( + "agpack.cli.deploy_item", + MagicMock(side_effect=RuntimeError("disk full")), + ), create_sync_progress() as progress, pytest.raises(Exception, match="Error deploying"), ): _sync_resource_type( deps, - _fake_detect, - MagicMock(side_effect=RuntimeError("disk full")), - "skill", - config, + "skills", + target_defs, tmp_path, Lockfile(), progress, + {}, dry_run=False, verbose=False, ) mock_write.assert_called_once() + + +class TestResourceKinds: + def test_returns_union_of_target_resources(self) -> None: + kinds = _resource_kinds(_make_targets(["claude", "codex"])) + # claude has skills/commands/agents/mcp; codex has skills/agents/mcp. + assert kinds["skills"] == "copy-directory" + assert kinds["commands"] == "copy-file" + assert kinds["agents"] == "copy-file" + assert kinds["mcp"] == "edit-file" + + def test_raises_on_conflicting_kinds(self) -> None: + from agpack.target_schema import parse_target_def + + target_a = parse_target_def({"rules": {"kind": "copy-directory", "path": ".a/rules"}}) + target_b = parse_target_def({"rules": {"kind": "copy-file", "path": ".b/rules"}}) + with pytest.raises(click.ClickException, match="conflicting kinds"): + _resource_kinds([target_a, target_b]) + + def test_arbitrary_resource_name_supported(self) -> None: + from agpack.target_schema import parse_target_def + + target = parse_target_def({"rules": {"kind": "copy-file", "path": ".my-tool/rules"}}) + kinds = _resource_kinds([target]) + assert kinds == {"rules": "copy-file"} diff --git a/tests/test_config.py b/tests/test_config.py index 5c7a235..d3ec7f8 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -8,12 +8,13 @@ from agpack.config import AgpackConfig from agpack.config import ConfigError +from agpack.config import DependencyEntry from agpack.config import DependencySource from agpack.config import GlobalConfig -from agpack.config import McpServer from agpack.config import load_config from agpack.config import load_global_config from agpack.config import merge_configs +from agpack.patch import Patch # --------------------------------------------------------------------------- # Helpers @@ -21,14 +22,61 @@ def _write_config(tmp_path: Path, text: str) -> Path: - """Write *text* to ``agpack.yml`` inside *tmp_path* and return the path.""" p = tmp_path / "agpack.yml" p.write_text(text, encoding="utf-8") return p +def _write_global_config(tmp_path: Path, text: str) -> Path: + p = tmp_path / "agpack.yml" + p.write_text(text, encoding="utf-8") + return p + + +def _make_project_config( + *, + skills: list[DependencyEntry] | None = None, + commands: list[DependencyEntry] | None = None, + agents: list[DependencyEntry] | None = None, + mcp: list[DependencyEntry] | None = None, + **kwargs: object, +) -> AgpackConfig: + deps: dict[str, list[DependencyEntry]] = {} + for name, lst in ( + ("skills", skills), + ("commands", commands), + ("agents", agents), + ("mcp", mcp), + ): + if lst: + deps[name] = lst + defaults: dict[str, object] = {"targets": ["claude"], "dependencies": deps} + defaults.update(kwargs) + return AgpackConfig(**defaults) # type: ignore[arg-type] + + +def _make_global_config( + *, + skills: list[DependencyEntry] | None = None, + commands: list[DependencyEntry] | None = None, + agents: list[DependencyEntry] | None = None, + mcp: list[DependencyEntry] | None = None, + **kwargs: object, +) -> GlobalConfig: + deps: dict[str, list[DependencyEntry]] = {} + for name, lst in ( + ("skills", skills), + ("commands", commands), + ("agents", agents), + ("mcp", mcp), + ): + if lst: + deps[name] = lst + return GlobalConfig(dependencies=deps, **kwargs) # type: ignore[arg-type] + + # --------------------------------------------------------------------------- -# 1. Valid config with all fields +# 1. Valid config: fetch + patch dependencies # --------------------------------------------------------------------------- @@ -49,37 +97,52 @@ def test_load_valid_full_config(tmp_path: Path) -> None: agents: - url: https://github.com/owner/agent-repo mcp: - - name: my-server - type: stdio - command: node - args: ["server.js"] - env: - TOKEN: abc + - key: mcpServers.my-server + value: + command: node + args: ["server.js"] + env: + TOKEN: abc """, ) cfg = load_config(cfg_path) assert isinstance(cfg, AgpackConfig) assert cfg.targets == ["claude", "opencode"] + skills = cfg.dependencies["skills"] + assert len(skills) == 1 + assert isinstance(skills[0], DependencySource) + assert skills[0].url == "https://gitlab.com/owner/skill-repo" + assert skills[0].ref == "v2.0" - assert len(cfg.skills) == 1 - assert cfg.skills[0].url == "https://gitlab.com/owner/skill-repo" - assert cfg.skills[0].path == "skills/foo" - assert cfg.skills[0].ref == "v2.0" - - assert len(cfg.commands) == 1 - assert cfg.commands[0].url == "https://github.com/owner/cmd-repo" + mcp = cfg.dependencies["mcp"] + assert len(mcp) == 1 + assert isinstance(mcp[0], Patch) + assert mcp[0].key == "mcpServers.my-server" + assert mcp[0].strategy == "replace" + assert mcp[0].value["command"] == "node" - assert len(cfg.agents) == 1 - assert cfg.agents[0].url == "https://github.com/owner/agent-repo" - assert len(cfg.mcp) == 1 - mcp = cfg.mcp[0] - assert mcp.name == "my-server" - assert mcp.type == "stdio" - assert mcp.command == "node" - assert mcp.args == ["server.js"] - assert mcp.env == {"TOKEN": "abc"} +def test_patch_with_append_strategy(tmp_path: Path) -> None: + cfg_path = _write_config( + tmp_path, + """\ +targets: + - claude +dependencies: + settings: + - key: hooks.PreToolUse + strategy: append + value: + matcher: "Write|Edit" + hooks: [{type: command, command: "x"}] +""", + ) + cfg = load_config(cfg_path) + entry = cfg.dependencies["settings"][0] + assert isinstance(entry, Patch) + assert entry.strategy == "append" + assert entry.value["matcher"] == "Write|Edit" # --------------------------------------------------------------------------- @@ -88,107 +151,109 @@ def test_load_valid_full_config(tmp_path: Path) -> None: def test_missing_targets(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -name: pack -version: "1" -""", - ) + cfg_path = _write_config(tmp_path, "name: pack\nversion: '1'\n") with pytest.raises(ConfigError, match="Missing or invalid 'targets'"): load_config(cfg_path) def test_targets_not_a_list(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: claude -""", - ) + cfg_path = _write_config(tmp_path, "targets: claude\n") with pytest.raises(ConfigError, match="Missing or invalid 'targets'"): load_config(cfg_path) def test_empty_targets(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: [] -""", - ) + cfg_path = _write_config(tmp_path, "targets: []\n") with pytest.raises(ConfigError, match="Missing or invalid 'targets'"): load_config(cfg_path) -# --------------------------------------------------------------------------- -# 5. Unrecognised target name -# --------------------------------------------------------------------------- +def test_unknown_target_name_accepted_at_parse_time(tmp_path: Path) -> None: + cfg_path = _write_config(tmp_path, "targets:\n - not-a-builtin\n") + config = load_config(cfg_path) + assert config.targets == ["not-a-builtin"] -def test_unrecognised_target(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - not-a-target -""", - ) - with pytest.raises(ConfigError, match="Unrecognised target 'not-a-target'"): +def test_target_must_be_non_empty_string(tmp_path: Path) -> None: + cfg_path = _write_config(tmp_path, "targets:\n - ''\n") + with pytest.raises(ConfigError, match="non-empty strings"): load_config(cfg_path) # --------------------------------------------------------------------------- -# 6. Dependency with missing url +# 3. Fetch entry validation # --------------------------------------------------------------------------- -def test_dependency_missing_url(tmp_path: Path) -> None: +def test_fetch_entry_missing_url(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: skills: - path: some/path """, ) - with pytest.raises(ConfigError, match="missing required field 'url'"): + with pytest.raises(ConfigError, match="either 'url' .* or 'key'"): load_config(cfg_path) -# --------------------------------------------------------------------------- -# 7. Dependency with non-string url -# --------------------------------------------------------------------------- +def test_fetch_entry_url_must_be_string_or_list(tmp_path: Path) -> None: + cfg_path = _write_config( + tmp_path, + """\ +targets: [claude] +dependencies: + skills: + - url: 42 +""", + ) + with pytest.raises(ConfigError, match="'url' must be a string or list"): + load_config(cfg_path) -def test_dependency_url_must_be_string(tmp_path: Path) -> None: +def test_fetch_entry_url_empty(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: skills: - - url: 123 + - url: "" """, ) - with pytest.raises(ConfigError, match="'url' must be a string"): + with pytest.raises(ConfigError, match="'url' must not be empty"): load_config(cfg_path) -# --------------------------------------------------------------------------- -# 8. Dependency with all optional fields (path, ref) -# --------------------------------------------------------------------------- +def test_fetch_entry_url_as_list(tmp_path: Path) -> None: + cfg_path = _write_config( + tmp_path, + """\ +targets: [claude] +dependencies: + skills: + - url: + - https://github.com/owner/repo + - git@github.com:owner/repo.git + path: skills/foo +""", + ) + cfg = load_config(cfg_path) + skill = cfg.dependencies["skills"][0] + assert isinstance(skill, DependencySource) + assert skill.urls == [ + "https://github.com/owner/repo", + "git@github.com:owner/repo.git", + ] -def test_dependency_all_optional_fields(tmp_path: Path) -> None: +def test_fetch_entry_with_all_optional_fields(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: commands: - url: https://gitlab.com/org/repo @@ -197,159 +262,117 @@ def test_dependency_all_optional_fields(tmp_path: Path) -> None: """, ) cfg = load_config(cfg_path) - dep = cfg.commands[0] - assert dep.url == "https://gitlab.com/org/repo" + dep = cfg.dependencies["commands"][0] + assert isinstance(dep, DependencySource) assert dep.path == "sub/dir" assert dep.ref == "main" # --------------------------------------------------------------------------- -# 9. MCP stdio server – valid +# 4. Patch entry validation # --------------------------------------------------------------------------- -def test_mcp_stdio_valid(tmp_path: Path) -> None: +def test_patch_missing_value(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: mcp: - - name: stdio-srv - type: stdio - command: python - args: ["-m", "server"] - env: - PORT: "8080" + - key: mcpServers.fs """, ) - cfg = load_config(cfg_path) - srv = cfg.mcp[0] - assert srv.name == "stdio-srv" - assert srv.type == "stdio" - assert srv.command == "python" - assert srv.args == ["-m", "server"] - assert srv.env == {"PORT": "8080"} - assert srv.url is None - - -# --------------------------------------------------------------------------- -# 10. MCP stdio server – missing command -# --------------------------------------------------------------------------- + with pytest.raises(ConfigError, match="missing required field 'value'"): + load_config(cfg_path) -def test_mcp_stdio_missing_command(tmp_path: Path) -> None: +def test_patch_invalid_strategy(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: mcp: - - name: bad-stdio - type: stdio + - key: mcpServers.fs + value: {command: x} + strategy: merge """, ) - with pytest.raises(ConfigError, match="missing required field 'command'"): + with pytest.raises(ConfigError, match="'strategy' must be one of"): load_config(cfg_path) -# --------------------------------------------------------------------------- -# 11. MCP sse server – valid -# --------------------------------------------------------------------------- - - -def test_mcp_sse_valid(tmp_path: Path) -> None: +def test_patch_unknown_field(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: mcp: - - name: sse-srv - type: sse - url: http://localhost:3000/sse + - key: mcpServers.fs + value: {} + bogus: 1 """, ) - cfg = load_config(cfg_path) - srv = cfg.mcp[0] - assert srv.name == "sse-srv" - assert srv.type == "sse" - assert srv.url == "http://localhost:3000/sse" - assert srv.command is None - - -# --------------------------------------------------------------------------- -# 12. MCP sse server – missing url -# --------------------------------------------------------------------------- + with pytest.raises(ConfigError, match="unknown fields"): + load_config(cfg_path) -def test_mcp_sse_missing_url(tmp_path: Path) -> None: +def test_append_patches_with_distinct_values_allowed(tmp_path: Path) -> None: + """Two append patches at the same key with different values are distinct list elements — both must survive.""" cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: - mcp: - - name: bad-sse - type: sse + permissions: + - key: permissions.allow + strategy: append + value: "Read(/etc/**)" + - key: permissions.allow + strategy: append + value: "Read(/var/**)" """, ) - with pytest.raises(ConfigError, match="missing required field 'url'"): - load_config(cfg_path) - - -# --------------------------------------------------------------------------- -# 13. MCP http server – valid -# --------------------------------------------------------------------------- + cfg = load_config(cfg_path) + assert len(cfg.dependencies["permissions"]) == 2 -def test_mcp_http_valid(tmp_path: Path) -> None: +def test_mixed_fetch_and_patch_rejected(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: - mcp: - - name: http-srv - type: http - url: http://localhost:9000/api + mixed: + - url: https://github.com/owner/repo + - key: foo + value: 1 """, ) - cfg = load_config(cfg_path) - srv = cfg.mcp[0] - assert srv.name == "http-srv" - assert srv.type == "http" - assert srv.url == "http://localhost:9000/api" - - -# --------------------------------------------------------------------------- -# 14. MCP entry missing name -# --------------------------------------------------------------------------- + with pytest.raises(ConfigError, match="cannot mix fetch and patch"): + load_config(cfg_path) -def test_mcp_missing_name(tmp_path: Path) -> None: +def test_entry_with_both_url_and_key_rejected(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - claude +targets: [claude] dependencies: mcp: - - type: stdio - command: node + - url: https://github.com/owner/repo + key: mcpServers.x + value: {} """, ) - with pytest.raises(ConfigError, match="missing required field 'name'"): + with pytest.raises(ConfigError, match="mutually exclusive"): load_config(cfg_path) # --------------------------------------------------------------------------- -# 15. DependencySource.name property +# 5. DependencySource properties # --------------------------------------------------------------------------- @@ -358,36 +381,9 @@ def test_dependency_source_name_from_path() -> None: assert dep.name == "my-skill" -def test_dependency_source_name_from_path_trailing_slash() -> None: - dep = DependencySource( - urls=["https://github.com/org/repo"], path="skills/my-skill/" - ) - assert dep.name == "my-skill" - - -def test_dependency_source_name_from_url() -> None: - dep = DependencySource(urls=["https://github.com/org/repo-name"]) - assert dep.name == "repo-name" - - def test_dependency_source_name_from_url_with_dotgit() -> None: - dep = DependencySource(urls=["https://github.com/org/repo-name.git"]) - assert dep.name == "repo-name" - - -def test_dependency_source_name_from_url_trailing_slash() -> None: - dep = DependencySource(urls=["https://github.com/org/repo-name/"]) - assert dep.name == "repo-name" - - -# --------------------------------------------------------------------------- -# 16. DependencySource.identity property -# --------------------------------------------------------------------------- - - -def test_dependency_source_identity_without_path() -> None: - dep = DependencySource(urls=["https://github.com/org/repo"]) - assert dep.identity == "https://github.com/org/repo" + dep = DependencySource(urls=["https://github.com/org/repo.git"]) + assert dep.name == "repo" def test_dependency_source_identity_with_path() -> None: @@ -395,171 +391,67 @@ def test_dependency_source_identity_with_path() -> None: assert dep.identity == "https://github.com/org/repo::sub/dir" -def test_dependency_source_identity_different_url() -> None: - dep = DependencySource(urls=["https://gitlab.com/org/repo"]) - assert dep.identity == "https://gitlab.com/org/repo" - - # --------------------------------------------------------------------------- -# 17. Empty dependencies section +# 6. Empty / minimal configs # --------------------------------------------------------------------------- def test_empty_dependencies(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -dependencies: {} -""", - ) + cfg_path = _write_config(tmp_path, "targets: [claude]\ndependencies: {}\n") cfg = load_config(cfg_path) - assert cfg.skills == [] - assert cfg.commands == [] - assert cfg.agents == [] - assert cfg.mcp == [] + assert cfg.dependencies == {} def test_no_dependencies_key(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -""", - ) + cfg_path = _write_config(tmp_path, "targets: [claude]\n") cfg = load_config(cfg_path) - assert cfg.skills == [] - assert cfg.commands == [] - assert cfg.agents == [] - assert cfg.mcp == [] + assert cfg.dependencies == {} -# --------------------------------------------------------------------------- -# 18. Mixed dependency formats -# --------------------------------------------------------------------------- - - -def test_mixed_dependencies(tmp_path: Path) -> None: +def test_dependencies_preserve_yaml_order(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ -targets: - - cursor - - copilot +targets: [claude] dependencies: + agents: + - url: https://github.com/a/agents skills: - - url: https://github.com/a/b - - url: https://github.com/c/d - path: deep/nested - ref: v1 + - url: https://github.com/a/skills commands: - - url: https://github.com/e/f - agents: - - url: https://gitlab.com/g/h - mcp: - - name: s1 - type: stdio - command: node - - name: s2 - type: sse - url: http://example.com + - url: https://github.com/a/commands """, ) cfg = load_config(cfg_path) - - assert len(cfg.skills) == 2 - assert cfg.skills[0].url == "https://github.com/a/b" - assert cfg.skills[0].path is None - assert cfg.skills[1].url == "https://github.com/c/d" - assert cfg.skills[1].path == "deep/nested" - assert cfg.skills[1].ref == "v1" - - assert len(cfg.commands) == 1 - - assert len(cfg.agents) == 1 - assert cfg.agents[0].url == "https://gitlab.com/g/h" - - assert len(cfg.mcp) == 2 - assert cfg.mcp[0].type == "stdio" - assert cfg.mcp[1].type == "sse" + assert list(cfg.dependencies) == ["agents", "skills", "commands"] # --------------------------------------------------------------------------- -# 19. use_global field +# 7. use_global field # --------------------------------------------------------------------------- def test_use_global_defaults_to_true(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -""", - ) - cfg = load_config(cfg_path) - assert cfg.use_global is True + cfg_path = _write_config(tmp_path, "targets: [claude]\n") + assert load_config(cfg_path).use_global is True def test_use_global_false(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -name: pack -version: "1" -global: false -targets: - - claude -""", - ) - cfg = load_config(cfg_path) - assert cfg.use_global is False - - -def test_use_global_true_explicit(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -name: pack -version: "1" -global: true -targets: - - claude -""", - ) - cfg = load_config(cfg_path) - assert cfg.use_global is True + cfg_path = _write_config(tmp_path, "global: false\ntargets: [claude]\n") + assert load_config(cfg_path).use_global is False def test_use_global_non_bool_raises(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -name: pack -version: "1" -global: "yes" -targets: - - claude -""", - ) + cfg_path = _write_config(tmp_path, "global: 'yes'\ntargets: [claude]\n") with pytest.raises(ConfigError, match="'global' must be true or false"): load_config(cfg_path) # --------------------------------------------------------------------------- -# 20. load_global_config +# 8. Global config # --------------------------------------------------------------------------- -def _write_global_config(tmp_path: Path, text: str) -> Path: - """Write *text* to a global config file inside *tmp_path*.""" - p = tmp_path / "agpack.yml" - p.write_text(text, encoding="utf-8") - return p - - def test_load_global_config_full(tmp_path: Path) -> None: path = _write_global_config( tmp_path, @@ -568,51 +460,41 @@ def test_load_global_config_full(tmp_path: Path) -> None: skills: - url: https://github.com/org/skills path: skills/shared - commands: - - url: https://github.com/org/commands - agents: - - url: https://github.com/org/agents - path: agents/shared.md mcp: - - name: global-server - command: npx - args: ["-y", "@example/server"] - env: - KEY: value + - key: mcpServers.global-srv + value: + command: npx + args: ["-y", "@example/server"] """, ) cfg = load_global_config(path) assert cfg is not None - assert len(cfg.skills) == 1 - assert cfg.skills[0].url == "https://github.com/org/skills" - assert cfg.skills[0].path == "skills/shared" - assert len(cfg.commands) == 1 - assert len(cfg.agents) == 1 - assert len(cfg.mcp) == 1 - assert cfg.mcp[0].name == "global-server" + assert len(cfg.dependencies["skills"]) == 1 + mcp = cfg.dependencies["mcp"] + assert isinstance(mcp[0], Patch) + assert mcp[0].key == "mcpServers.global-srv" assert cfg.config_dir == tmp_path def test_load_global_config_missing_file(tmp_path: Path) -> None: - path = tmp_path / "nonexistent.yml" - assert load_global_config(path) is None + assert load_global_config(tmp_path / "nonexistent.yml") is None def test_load_global_config_empty_file(tmp_path: Path) -> None: - path = _write_global_config(tmp_path, "") - cfg = load_global_config(path) + cfg = load_global_config(_write_global_config(tmp_path, "")) assert cfg is not None - assert cfg.skills == [] - assert cfg.commands == [] - assert cfg.agents == [] - assert cfg.mcp == [] + assert cfg.dependencies == {} -def test_load_global_config_empty_dependencies(tmp_path: Path) -> None: - path = _write_global_config(tmp_path, "dependencies: {}\n") - cfg = load_global_config(path) +def test_load_global_config_env_var_override(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + custom_dir = tmp_path / "custom" + custom_dir.mkdir() + custom_path = custom_dir / "my-global.yml" + custom_path.write_text("dependencies:\n skills:\n - url: https://example.com/repo\n") + monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(custom_path)) + cfg = load_global_config() assert cfg is not None - assert cfg.skills == [] + assert cfg.config_dir == custom_dir def test_load_global_config_malformed_yaml(tmp_path: Path) -> None: @@ -629,382 +511,187 @@ def test_load_global_config_not_a_mapping(tmp_path: Path) -> None: def test_load_global_config_dependencies_not_a_mapping(tmp_path: Path) -> None: path = _write_global_config(tmp_path, "dependencies: [bad]\n") - with pytest.raises( - ConfigError, match="Global config 'dependencies' must be a mapping" - ): + with pytest.raises(ConfigError, match="Global config 'dependencies' must be a mapping"): load_global_config(path) -def test_load_global_config_env_var_override( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - custom_dir = tmp_path / "custom" - custom_dir.mkdir() - custom_path = custom_dir / "my-global.yml" - custom_path.write_text( - "dependencies:\n skills:\n - url: https://example.com/repo\n" - ) - monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(custom_path)) - - cfg = load_global_config() - assert cfg is not None - assert len(cfg.skills) == 1 - assert cfg.config_dir == custom_dir - - -def test_load_global_config_env_var_nonexistent( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(tmp_path / "nope.yml")) - assert load_global_config() is None - - -def test_load_global_config_skills_only(tmp_path: Path) -> None: - path = _write_global_config( - tmp_path, - """\ -dependencies: - skills: - - url: https://github.com/org/repo - path: skills/only -""", - ) - cfg = load_global_config(path) - assert cfg is not None - assert len(cfg.skills) == 1 - assert cfg.commands == [] - assert cfg.agents == [] - assert cfg.mcp == [] - - # --------------------------------------------------------------------------- -# 21. merge_configs +# 9. merge_configs # --------------------------------------------------------------------------- -def _make_project_config(**kwargs: object) -> AgpackConfig: - defaults: dict[str, object] = { - "targets": ["claude"], - } - defaults.update(kwargs) - return AgpackConfig(**defaults) # type: ignore[arg-type] - - def test_merge_basic() -> None: project = _make_project_config( skills=[DependencySource(urls=["https://github.com/a/b"], path="skills/proj")], ) - global_cfg = GlobalConfig( - skills=[ - DependencySource(urls=["https://github.com/c/d"], path="skills/global") - ], + global_skill = DependencySource(urls=["https://github.com/c/d"], path="skills/global") + global_cfg = _make_global_config( + skills=[global_skill], commands=[DependencySource(urls=["https://github.com/e/f"])], ) merged = merge_configs(project, global_cfg) - - assert len(merged.skills) == 2 - assert merged.skills[0].url == "https://github.com/a/b" # project first - assert merged.skills[1].url == "https://github.com/c/d" # global appended - assert len(merged.commands) == 1 - assert merged.commands[0].url == "https://github.com/e/f" + assert len(merged.dependencies["skills"]) == 2 + assert merged.dependencies["skills"][0].url == "https://github.com/a/b" # type: ignore[union-attr] -def test_merge_project_wins_on_duplicate_dep() -> None: +def test_merge_dedupes_fetch_by_identity() -> None: dep = DependencySource(urls=["https://github.com/a/b"], path="skills/shared") project = _make_project_config(skills=[dep]) - global_cfg = GlobalConfig( - skills=[ - DependencySource(urls=["https://github.com/a/b"], path="skills/shared") - ], - ) - merged = merge_configs(project, global_cfg) - - # Duplicate should be deduped — only the project entry survives - assert len(merged.skills) == 1 - assert merged.skills[0] is dep - - -def test_merge_project_wins_on_duplicate_mcp() -> None: - project_server = McpServer(name="ctx7", command="npx", args=["project-version"]) - global_server = McpServer(name="ctx7", command="npx", args=["global-version"]) - - project = _make_project_config(mcp=[project_server]) - global_cfg = GlobalConfig(mcp=[global_server]) + dup = DependencySource(urls=["https://github.com/a/b"], path="skills/shared") + global_cfg = _make_global_config(skills=[dup]) merged = merge_configs(project, global_cfg) + assert len(merged.dependencies["skills"]) == 1 - assert len(merged.mcp) == 1 - assert merged.mcp[0].args == ["project-version"] - -def test_merge_empty_global() -> None: - project = _make_project_config( - skills=[DependencySource(urls=["https://github.com/a/b"])], - ) - global_cfg = GlobalConfig() +def test_merge_dedupes_patches_by_content() -> None: + p = Patch(key="mcpServers.fs", value={"command": "npx"}) + project = _make_project_config(mcp=[p]) + dup = Patch(key="mcpServers.fs", value={"command": "npx"}) + global_cfg = _make_global_config(mcp=[dup]) merged = merge_configs(project, global_cfg) + assert len(merged.dependencies["mcp"]) == 1 - assert len(merged.skills) == 1 - assert merged.commands == [] - -def test_merge_empty_project_deps() -> None: - project = _make_project_config() - global_cfg = GlobalConfig( - skills=[DependencySource(urls=["https://github.com/c/d"])], - mcp=[McpServer(name="s1", command="cmd")], - ) - merged = merge_configs(project, global_cfg) - - assert len(merged.skills) == 1 - assert merged.skills[0].url == "https://github.com/c/d" - assert len(merged.mcp) == 1 - - -def test_merge_preserves_project_metadata() -> None: - project = _make_project_config(targets=["opencode"], use_global=False) - global_cfg = GlobalConfig( - skills=[DependencySource(urls=["https://github.com/a/b"])], - ) +def test_merge_keeps_distinct_patches() -> None: + project = _make_project_config(mcp=[Patch(key="mcpServers.fs", value={"command": "npx"})]) + global_cfg = _make_global_config(mcp=[Patch(key="mcpServers.other", value={"command": "x"})]) merged = merge_configs(project, global_cfg) - - assert merged.targets == ["opencode"] - assert merged.use_global is False + assert len(merged.dependencies["mcp"]) == 2 def test_merge_does_not_mutate_inputs() -> None: project = _make_project_config( skills=[DependencySource(urls=["https://github.com/a/b"])], ) - global_cfg = GlobalConfig( + global_cfg = _make_global_config( skills=[DependencySource(urls=["https://github.com/c/d"])], ) - orig_project_skills = list(project.skills) - orig_global_skills = list(global_cfg.skills) - + orig_project = list(project.dependencies["skills"]) + orig_global = list(global_cfg.dependencies["skills"]) merge_configs(project, global_cfg) - - assert project.skills == orig_project_skills - assert global_cfg.skills == orig_global_skills - - -def test_merge_cross_type_identity_not_deduped() -> None: - """A skill and a command with the same identity are NOT deduped.""" - dep = DependencySource(urls=["https://github.com/a/b"], path="shared") - project = _make_project_config(skills=[dep]) - global_cfg = GlobalConfig( - commands=[DependencySource(urls=["https://github.com/a/b"], path="shared")], - ) - merged = merge_configs(project, global_cfg) - - assert len(merged.skills) == 1 - assert len(merged.commands) == 1 + assert project.dependencies["skills"] == orig_project + assert global_cfg.dependencies["skills"] == orig_global # --------------------------------------------------------------------------- -# 22. url as list (multiple URLs / fallbacks) +# 10. load_config error paths # --------------------------------------------------------------------------- -def test_url_as_list(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -dependencies: - skills: - - url: - - https://github.com/owner/repo - - git@github.com:owner/repo.git - path: skills/foo -""", - ) - cfg = load_config(cfg_path) - assert cfg.skills[0].urls == [ - "https://github.com/owner/repo", - "git@github.com:owner/repo.git", - ] - assert cfg.skills[0].url == "https://github.com/owner/repo" - - -def test_url_as_string(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -dependencies: - skills: - - url: https://github.com/owner/repo -""", - ) - cfg = load_config(cfg_path) - assert cfg.skills[0].urls == ["https://github.com/owner/repo"] +def test_config_file_not_found(tmp_path: Path) -> None: + with pytest.raises(ConfigError, match="Config file not found"): + load_config(tmp_path / "nonexistent.yml") -def test_url_empty_list_raises(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -dependencies: - skills: - - url: [] -""", - ) - with pytest.raises(ConfigError, match="'url' must not be empty"): +def test_config_malformed_yaml(tmp_path: Path) -> None: + cfg_path = _write_config(tmp_path, ":\n - [invalid yaml") + with pytest.raises(ConfigError, match="Failed to parse YAML"): load_config(cfg_path) -def test_url_invalid_type_raises(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -dependencies: - skills: - - url: 42 -""", - ) - with pytest.raises(ConfigError, match="'url' must be a string or list"): +def test_config_not_a_mapping(tmp_path: Path) -> None: + cfg_path = _write_config(tmp_path, "- a list\n- not a mapping\n") + with pytest.raises(ConfigError, match="Config file must be a YAML mapping"): load_config(cfg_path) -def test_url_as_list_in_global_config(tmp_path: Path) -> None: - path = tmp_path / "agpack.yml" - path.write_text( - """\ -dependencies: - skills: - - url: - - https://github.com/org/repo - - git@github.com:org/repo.git - path: skills/shared -""" - ) - cfg = load_global_config(path) - assert cfg is not None - assert cfg.skills[0].urls == [ - "https://github.com/org/repo", - "git@github.com:org/repo.git", - ] +def test_config_dependencies_not_a_mapping(tmp_path: Path) -> None: + cfg_path = _write_config(tmp_path, 'targets: [claude]\ndependencies: "oops"\n') + with pytest.raises(ConfigError, match="'dependencies' must be a mapping"): + load_config(cfg_path) # --------------------------------------------------------------------------- -# 23. _parse_dependency validation errors +# 11. target_definitions parsing # --------------------------------------------------------------------------- -def test_dependency_entry_not_a_dict(tmp_path: Path) -> None: +def test_target_definitions_parses_new_target(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ targets: - - claude -dependencies: - skills: - - just-a-string -""", - ) - with pytest.raises(ConfigError, match="expected an object"): - load_config(cfg_path) - + - my-tool -def test_dependency_url_empty_string(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, - """\ -targets: - - claude -dependencies: - skills: - - url: "" +target_definitions: + my-tool: + skills: + kind: copy-directory + path: .my-tool/skills + mcp: + kind: edit-file + path: .my-tool/config.json """, ) - with pytest.raises(ConfigError, match="'url' must not be empty"): - load_config(cfg_path) + config = load_config(cfg_path) + td = config.target_definitions["my-tool"] + assert td.resources["skills"].path == ".my-tool/skills" + assert td.resources["mcp"].path == ".my-tool/config.json" -def test_dependency_path_not_a_string(tmp_path: Path) -> None: +def test_target_definitions_overrides_builtin(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ targets: - claude -dependencies: - skills: - - url: https://github.com/org/repo - path: 123 + +target_definitions: + claude: + skills: + kind: copy-directory + path: .my-claude/skills """, ) - with pytest.raises(ConfigError, match="'path' must be a string"): - load_config(cfg_path) + config = load_config(cfg_path) + td = config.target_definitions["claude"] + assert td.resources["skills"].path == ".my-claude/skills" + # Replace semantics: nothing else inherited. + assert "mcp" not in td.resources + assert "commands" not in td.resources -def test_mcp_entry_not_a_dict(tmp_path: Path) -> None: +def test_target_definitions_not_a_mapping(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, - """\ -targets: - - claude -dependencies: - mcp: - - just-a-string -""", + 'targets: [claude]\ntarget_definitions: "oops"\n', ) - with pytest.raises(ConfigError, match="expected an object"): + with pytest.raises(ConfigError, match="target_definitions: must be a mapping"): load_config(cfg_path) -def test_mcp_invalid_type(tmp_path: Path) -> None: +def test_target_definitions_invalid_inner_schema_raises(tmp_path: Path) -> None: cfg_path = _write_config( tmp_path, """\ targets: - - claude -dependencies: - mcp: - - name: bad - type: grpc - command: something + - my-tool +target_definitions: + my-tool: + skills: + kind: not-a-real-kind + path: .x/skills """, ) - with pytest.raises(ConfigError, match="'type' must be 'stdio', 'sse', or 'http'"): + with pytest.raises(ConfigError, match="kind"): load_config(cfg_path) -# --------------------------------------------------------------------------- -# 24. load_config validation errors -# --------------------------------------------------------------------------- - - -def test_config_file_not_found(tmp_path: Path) -> None: - with pytest.raises(ConfigError, match="Config file not found"): - load_config(tmp_path / "nonexistent.yml") - - -def test_config_malformed_yaml(tmp_path: Path) -> None: - cfg_path = _write_config(tmp_path, ":\n - [invalid yaml") - with pytest.raises(ConfigError, match="Failed to parse YAML"): - load_config(cfg_path) - - -def test_config_not_a_mapping(tmp_path: Path) -> None: - cfg_path = _write_config(tmp_path, "- a list\n- not a mapping\n") - with pytest.raises(ConfigError, match="Config file must be a YAML mapping"): - load_config(cfg_path) - - -def test_config_dependencies_not_a_mapping(tmp_path: Path) -> None: - cfg_path = _write_config( - tmp_path, +def test_merge_carries_global_target_definitions(tmp_path: Path) -> None: + project_path = _write_config(tmp_path, "targets:\n - shared-tool\n") + project = load_config(project_path) + global_path = tmp_path / "global.yml" + global_path.write_text( """\ -targets: - - claude -dependencies: "oops" +target_definitions: + shared-tool: + skills: + kind: copy-directory + path: .shared/skills """, + encoding="utf-8", ) - with pytest.raises(ConfigError, match="'dependencies' must be a mapping"): - load_config(cfg_path) + global_cfg = load_global_config(global_path) + assert global_cfg is not None + merged = merge_configs(project, global_cfg) + assert "shared-tool" in merged.target_definitions diff --git a/tests/test_deployer.py b/tests/test_deployer.py index 06caae6..46e6c21 100644 --- a/tests/test_deployer.py +++ b/tests/test_deployer.py @@ -2,36 +2,86 @@ from __future__ import annotations +import json +from dataclasses import dataclass +from dataclasses import field from pathlib import Path from unittest.mock import patch import pytest from agpack.config import DependencySource -from agpack.deployer import DeployError -from agpack.deployer import DeployResult -from agpack.deployer import _atomic_copy_file from agpack.deployer import cleanup_deployed_files -from agpack.deployer import deploy_agent -from agpack.deployer import deploy_command -from agpack.deployer import deploy_single_skill -from agpack.deployer import deploy_skill +from agpack.deployer import deploy_item +from agpack.deployer import detect_items +from agpack.deployer import sync_edit_resource +from agpack.errors import DeployError from agpack.fetcher import FetchResult +from agpack.kinds._shared import atomic_copy_file +from agpack.patch import Patch +from agpack.registry import load_all_builtins +from agpack.target_schema import TargetDef # --------------------------------------------------------------------------- # Helpers # --------------------------------------------------------------------------- -ALL_TARGETS = [ - "claude", - "opencode", - "codex", - "cursor", - "copilot", - "gemini", - "windsurf", - "antigravity", -] +_BUILTINS = load_all_builtins() + +ALL_TARGETS = sorted(_BUILTINS) + +SKILL_DIRS = {name: t.resources["skills"].path for name, t in _BUILTINS.items() if "skills" in t.resources} +COMMAND_DIRS = {name: t.resources["commands"].path for name, t in _BUILTINS.items() if "commands" in t.resources} +AGENT_DIRS = {name: t.resources["agents"].path for name, t in _BUILTINS.items() if "agents" in t.resources} + + +@dataclass +class _Deployed: + """Aggregate result of a detect → deploy run, matching the old API shape.""" + + files: list[str] = field(default_factory=list) + expanded_items: list[str] = field(default_factory=list) + + +def _resolve(names: list[str]) -> list[TargetDef]: + return [_BUILTINS[n] for n in names if n in _BUILTINS] + + +def _run(resource_type, fr, targets, project, **kwargs): # type: ignore[no-untyped-def] + """Mirror what the CLI does: detect items, then deploy each one. + + If no resolved target declares the resource type, return an empty result — there's nothing to detect or deploy. + The production CLI short-circuits the same way via the kind map. + """ + target_defs = _resolve(targets) + detect_resource = next( + (t.resources[resource_type] for t in target_defs if resource_type in t.resources), + None, + ) + if detect_resource is None: + return _Deployed() + items = detect_items(fr, detect_resource, resource_type) + files: list[str] = [] + for name, path in items: + files.extend(deploy_item(name, path, resource_type, target_defs, project, **kwargs)) + expanded = [name for name, _ in items] if len(items) > 1 else [] + return _Deployed(files=files, expanded_items=expanded) + + +def deploy_skill(fr, targets, project, **kwargs): # type: ignore[no-untyped-def] + return _run("skills", fr, targets, project, **kwargs) + + +def deploy_command(fr, targets, project, **kwargs): # type: ignore[no-untyped-def] + return _run("commands", fr, targets, project, **kwargs) + + +def deploy_agent(fr, targets, project, **kwargs): # type: ignore[no-untyped-def] + return _run("agents", fr, targets, project, **kwargs) + + +def deploy_single_skill(name, path, targets, project, dry_run, verbose): # type: ignore[no-untyped-def] + return deploy_item(name, path, "skills", _resolve(targets), project, dry_run, verbose) def _make_source(name: str = "my-skill") -> DependencySource: @@ -49,9 +99,7 @@ def _make_file_fetch( src.parent.mkdir(parents=True, exist_ok=True) src.write_text(content) return FetchResult( - source=DependencySource( - urls=[f"https://github.com/org/{source_name}"], path=source_name - ), + source=DependencySource(urls=[f"https://github.com/org/{source_name}"], path=source_name), local_path=src, resolved_ref="abc1234", ) @@ -93,11 +141,10 @@ def test_copies_directory_to_all_targets(self, tmp_path: Path) -> None: result = deploy_skill(fr, ALL_TARGETS, project) - assert isinstance(result, DeployResult) + assert isinstance(result, _Deployed) assert result.expanded_items == [] # Should produce files under every target's skill dir - from agpack.targets import SKILL_DIRS for target, base in SKILL_DIRS.items(): skill_md = project / base / "my-skill" / "SKILL.md" @@ -157,9 +204,7 @@ def test_dry_run_no_files_created(self, tmp_path: Path) -> None: def test_skill_name_derived_from_source_path(self, tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() - source = DependencySource( - urls=["https://github.com/org/repo"], path="skills/custom-name" - ) + source = DependencySource(urls=["https://github.com/org/repo"], path="skills/custom-name") src_dir = tmp_path / "src" / "custom-name" src_dir.mkdir(parents=True) (src_dir / "README.md").write_text("hi") @@ -184,11 +229,9 @@ def test_copies_to_supported_targets(self, tmp_path: Path) -> None: result = deploy_command(fr, ALL_TARGETS, project) - assert isinstance(result, DeployResult) + assert isinstance(result, _Deployed) assert result.expanded_items == [] - from agpack.targets import COMMAND_DIRS - # Only targets with entries in COMMAND_DIRS should have files assert len(result.files) == len(COMMAND_DIRS) for target, base in COMMAND_DIRS.items(): @@ -201,8 +244,8 @@ def test_skips_unsupported_targets(self, tmp_path: Path) -> None: project.mkdir() fr = _make_file_fetch(tmp_path, source_name="lint.md") - # codex and cursor don't support commands - result = deploy_command(fr, ["codex", "cursor"], project) + # codex doesn't support project-level commands + result = deploy_command(fr, ["codex"], project) assert result.files == [] @@ -230,11 +273,9 @@ def test_copies_to_supported_targets(self, tmp_path: Path) -> None: result = deploy_agent(fr, ALL_TARGETS, project) - assert isinstance(result, DeployResult) + assert isinstance(result, _Deployed) assert result.expanded_items == [] - from agpack.targets import AGENT_DIRS - assert len(result.files) == len(AGENT_DIRS) for target, base in AGENT_DIRS.items(): dst = project / base / "reviewer.md" @@ -246,8 +287,8 @@ def test_skips_unsupported_targets(self, tmp_path: Path) -> None: project.mkdir() fr = _make_file_fetch(tmp_path, source_name="reviewer.md") - # codex doesn't support agents - result = deploy_agent(fr, ["codex"], project) + # cursor / gemini / windsurf / antigravity don't expose project-level agents + result = deploy_agent(fr, ["cursor", "gemini", "windsurf"], project) assert result.files == [] @@ -256,7 +297,7 @@ def test_dry_run_no_files_created(self, tmp_path: Path) -> None: project.mkdir() fr = _make_file_fetch(tmp_path, source_name="reviewer.md") - result = deploy_agent(fr, ["claude", "cursor"], project, dry_run=True) + result = deploy_agent(fr, ["claude", "opencode"], project, dry_run=True) assert len(result.files) == 2 for rel in result.files: @@ -344,7 +385,7 @@ def test_file_exists_after_copy(self, tmp_path: Path) -> None: src.write_text("payload") dst = tmp_path / "out" / "dst.txt" - _atomic_copy_file(src, dst) + atomic_copy_file(src, dst) assert dst.exists() assert dst.read_text() == "payload" @@ -354,7 +395,7 @@ def test_creates_parent_directories(self, tmp_path: Path) -> None: src.write_text("data") dst = tmp_path / "a" / "b" / "c" / "file.txt" - _atomic_copy_file(src, dst) + atomic_copy_file(src, dst) assert dst.exists() assert dst.read_text() == "data" @@ -365,7 +406,7 @@ def test_overwrites_existing_file(self, tmp_path: Path) -> None: dst = tmp_path / "dst.txt" dst.write_text("old") - _atomic_copy_file(src, dst) + atomic_copy_file(src, dst) assert dst.read_text() == "new" @@ -445,8 +486,6 @@ def test_folder_of_skills_to_multiple_targets(self, tmp_path: Path) -> None: result = deploy_skill(fr, ALL_TARGETS, project) - from agpack.targets import SKILL_DIRS - # 3 files × 5 targets = 15 assert len(result.files) == 3 * len(SKILL_DIRS) assert result.expanded_items == ["skill-a", "skill-b"] @@ -472,14 +511,12 @@ def test_errors_on_empty_directory(self, tmp_path: Path) -> None: src = tmp_path / "src" / "empty" src.mkdir(parents=True) fr = FetchResult( - source=DependencySource( - urls=["https://github.com/org/empty"], path="empty" - ), + source=DependencySource(urls=["https://github.com/org/empty"], path="empty"), local_path=src, resolved_ref="abc1234", ) - with pytest.raises(DeployError, match="does not contain any skill folders"): + with pytest.raises(DeployError, match="does not contain any skills folders"): deploy_skill(fr, ["claude"], project) def test_errors_on_dir_with_only_empty_subdirs(self, tmp_path: Path) -> None: @@ -489,14 +526,12 @@ def test_errors_on_dir_with_only_empty_subdirs(self, tmp_path: Path) -> None: (src / "empty-a").mkdir(parents=True) (src / "empty-b").mkdir(parents=True) fr = FetchResult( - source=DependencySource( - urls=["https://github.com/org/parent"], path="parent" - ), + source=DependencySource(urls=["https://github.com/org/parent"], path="parent"), local_path=src, resolved_ref="abc1234", ) - with pytest.raises(DeployError, match="does not contain any skill folders"): + with pytest.raises(DeployError, match="does not contain any skills folders"): deploy_skill(fr, ["claude"], project) def test_single_skill_folder_still_works(self, tmp_path: Path) -> None: @@ -557,14 +592,12 @@ def test_errors_on_empty_directory(self, tmp_path: Path) -> None: src = tmp_path / "src" / "empty" src.mkdir(parents=True) fr = FetchResult( - source=DependencySource( - urls=["https://github.com/org/empty"], path="empty" - ), + source=DependencySource(urls=["https://github.com/org/empty"], path="empty"), local_path=src, resolved_ref="abc1234", ) - with pytest.raises(DeployError, match="does not contain any command files"): + with pytest.raises(DeployError, match="does not contain any commands files"): deploy_command(fr, ["claude"], project) def test_dry_run_with_directory(self, tmp_path: Path) -> None: @@ -593,9 +626,7 @@ def test_deploys_each_file_from_directory(self, tmp_path: Path) -> None: (src / "reviewer.md").write_text("# Reviewer") (src / "planner.md").write_text("# Planner") fr = FetchResult( - source=DependencySource( - urls=["https://github.com/org/agents"], path="agents" - ), + source=DependencySource(urls=["https://github.com/org/agents"], path="agents"), local_path=src, resolved_ref="abc1234", ) @@ -613,9 +644,7 @@ def test_deploys_files_from_subfolders(self, tmp_path: Path) -> None: (src / "group").mkdir(parents=True) (src / "group" / "reviewer.md").write_text("# Reviewer") fr = FetchResult( - source=DependencySource( - urls=["https://github.com/org/agents"], path="agents" - ), + source=DependencySource(urls=["https://github.com/org/agents"], path="agents"), local_path=src, resolved_ref="abc1234", ) @@ -632,14 +661,12 @@ def test_errors_on_empty_directory(self, tmp_path: Path) -> None: src = tmp_path / "src" / "empty" src.mkdir(parents=True) fr = FetchResult( - source=DependencySource( - urls=["https://github.com/org/empty"], path="empty" - ), + source=DependencySource(urls=["https://github.com/org/empty"], path="empty"), local_path=src, resolved_ref="abc1234", ) - with pytest.raises(DeployError, match="does not contain any agent files"): + with pytest.raises(DeployError, match="does not contain any agents files"): deploy_agent(fr, ["claude"], project) @@ -656,10 +683,10 @@ def test_cleans_up_temp_file_on_copy_failure(self, tmp_path: Path) -> None: dst = tmp_path / "out" / "dst.txt" with ( - patch("agpack.deployer.shutil.copy2", side_effect=OSError("no space")), + patch("agpack.kinds._shared.shutil.copy2", side_effect=OSError("no space")), pytest.raises(OSError, match="no space"), ): - _atomic_copy_file(src, dst) + atomic_copy_file(src, dst) # No temp files should be left behind leftover = list((tmp_path / "out").glob(".agpack-tmp-*")) @@ -681,9 +708,7 @@ def test_deploys_single_file_as_skill(self, tmp_path: Path) -> None: skill_file.parent.mkdir(parents=True) skill_file.write_text("# My Skill") - result = deploy_single_skill( - "my-skill", skill_file, ["claude"], project, dry_run=False, verbose=False - ) + result = deploy_single_skill("my-skill", skill_file, ["claude"], project, dry_run=False, verbose=False) assert len(result) == 1 deployed = project / result[0] @@ -691,9 +716,7 @@ def test_deploys_single_file_as_skill(self, tmp_path: Path) -> None: assert deployed.read_text() == "# My Skill" assert "my-skill" in str(deployed) - def test_deploys_single_file_skill_to_multiple_targets( - self, tmp_path: Path - ) -> None: + def test_deploys_single_file_skill_to_multiple_targets(self, tmp_path: Path) -> None: project = tmp_path / "project" project.mkdir() @@ -701,11 +724,7 @@ def test_deploys_single_file_skill_to_multiple_targets( skill_file.parent.mkdir(parents=True) skill_file.write_text("# My Skill") - result = deploy_single_skill( - "my-skill", skill_file, ALL_TARGETS, project, dry_run=False, verbose=False - ) - - from agpack.targets import SKILL_DIRS + result = deploy_single_skill("my-skill", skill_file, ALL_TARGETS, project, dry_run=False, verbose=False) assert len(result) == len(SKILL_DIRS) for rel in result: @@ -719,9 +738,68 @@ def test_dry_run_single_file_skill(self, tmp_path: Path) -> None: skill_file.parent.mkdir(parents=True) skill_file.write_text("# My Skill") - result = deploy_single_skill( - "my-skill", skill_file, ["claude"], project, dry_run=True, verbose=False - ) + result = deploy_single_skill("my-skill", skill_file, ["claude"], project, dry_run=True, verbose=False) assert len(result) == 1 assert not (project / result[0]).exists() + + +# --------------------------------------------------------------------------- +# sync_edit_resource — dedup of identical edit-file paths across targets +# --------------------------------------------------------------------------- + + +class TestSyncEditResourceDedup: + """Regression: two targets resolving to the same edit-file path used to apply every patch once per target, + double-appending list elements on every sync and leaving lockfile-orphaned entries. + """ + + def test_duplicate_target_does_not_double_append(self, tmp_path: Path) -> None: + project = tmp_path / "project" + project.mkdir() + # Two identical targets — simulates ``targets: [claude, claude]``. + targets = [_BUILTINS["claude"], _BUILTINS["claude"]] + + patch_append = Patch( + key="${bucket}.PreToolUse", + value={"matcher": "Write", "hooks": [{"type": "command", "command": "x"}]}, + strategy="append", + ) + + applied = sync_edit_resource( + resource_type="hooks", + desired=[patch_append], + applied_old=[], + targets=targets, + project_root=project, + ) + + # Exactly one applied record, exactly one item appended in the file. + assert len(applied) == 1 + settings = json.loads((project / ".claude/settings.json").read_text()) + assert len(settings["hooks"]["PreToolUse"]) == 1 + + def test_two_targets_same_path_apply_once(self, tmp_path: Path) -> None: + # Distinct target objects with the same `path:` for an edit-file resource — same dedup contract as the + # duplicate case. + project = tmp_path / "project" + project.mkdir() + targets = [_BUILTINS["claude"], _BUILTINS["claude"]] + + patch_replace = Patch( + key="${bucket}.filesystem", + value={"command": "npx"}, + strategy="replace", + ) + + applied = sync_edit_resource( + resource_type="mcp", + desired=[patch_replace], + applied_old=[], + targets=targets, + project_root=project, + ) + + assert len(applied) == 1 + mcp = json.loads((project / ".mcp.json").read_text()) + assert mcp == {"mcpServers": {"filesystem": {"command": "npx"}}} diff --git a/tests/test_envsubst.py b/tests/test_envsubst.py index 417d027..2b0cd78 100644 --- a/tests/test_envsubst.py +++ b/tests/test_envsubst.py @@ -1,4 +1,4 @@ -"""Tests for agpack.envsubst – .env loading and ${VAR} substitution.""" +"""Tests for ``.env`` loading and ``${VAR}`` substitution (lives in :mod:`agpack.config`).""" from __future__ import annotations @@ -8,12 +8,13 @@ from agpack.config import AgpackConfig from agpack.config import ConfigError +from agpack.config import DependencyEntry from agpack.config import DependencySource from agpack.config import GlobalConfig -from agpack.config import McpServer +from agpack.config import resolve_config from agpack.envsubst import load_dotenv -from agpack.envsubst import resolve_config from agpack.envsubst import resolve_env_vars +from agpack.patch import Patch # --------------------------------------------------------------------------- # 1. load_dotenv @@ -22,8 +23,7 @@ def test_load_dotenv_basic(tmp_path: Path) -> None: (tmp_path / ".env").write_text("FOO=bar\nBAZ=qux\n") - result = load_dotenv(tmp_path) - assert result == {"FOO": "bar", "BAZ": "qux"} + assert load_dotenv(tmp_path) == {"FOO": "bar", "BAZ": "qux"} def test_load_dotenv_with_double_quotes(tmp_path: Path) -> None: @@ -89,357 +89,177 @@ def test_resolve_mixed_literal_and_var() -> None: def test_resolve_missing_var_raises() -> None: - with pytest.raises(ConfigError, match="environment variable 'MISSING' is not set"): + with pytest.raises(ConfigError, match="variable 'MISSING' is not defined"): resolve_env_vars("${MISSING}", {}) -def test_resolve_missing_var_includes_context() -> None: - with pytest.raises(ConfigError, match="mcp server 'ctx7'"): - resolve_env_vars("${NOPE}", {}, context="mcp server 'ctx7'") - - def test_resolve_partial_missing_raises() -> None: with pytest.raises(ConfigError, match="'MISSING'"): resolve_env_vars("${EXISTS}-${MISSING}", {"EXISTS": "ok"}) # --------------------------------------------------------------------------- -# 3. resolve_config – MCP server env +# 3. resolve_config — dependency URL/path/ref # --------------------------------------------------------------------------- -def _make_config(mcp: list[McpServer] | None = None) -> AgpackConfig: - return AgpackConfig( - targets=["claude"], - mcp=mcp or [], - ) - - -def test_resolve_config_from_dotenv(tmp_path: Path) -> None: - (tmp_path / ".env").write_text("API_KEY=secret-from-dotenv\n") - server = McpServer(name="s", command="cmd", env={"API_KEY": "${API_KEY}"}) - config = _make_config([server]) - - resolve_config(config, tmp_path) - - assert config.mcp[0].env["API_KEY"] == "secret-from-dotenv" - - -def test_resolve_config_from_shell( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - monkeypatch.setenv("SHELL_VAR", "from-shell") - server = McpServer(name="s", command="cmd", env={"KEY": "${SHELL_VAR}"}) - config = _make_config([server]) - - resolve_config(config, tmp_path) - - assert config.mcp[0].env["KEY"] == "from-shell" - - -def test_resolve_config_dotenv_takes_precedence( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - monkeypatch.setenv("MY_VAR", "from-shell") - (tmp_path / ".env").write_text("MY_VAR=from-dotenv\n") - server = McpServer(name="s", command="cmd", env={"V": "${MY_VAR}"}) - config = _make_config([server]) - - resolve_config(config, tmp_path) - - assert config.mcp[0].env["V"] == "from-dotenv" - - -def test_resolve_config_no_substitution_needed(tmp_path: Path) -> None: - server = McpServer(name="s", command="cmd", env={"KEY": "plain-value"}) - config = _make_config([server]) - - resolve_config(config, tmp_path) - - assert config.mcp[0].env["KEY"] == "plain-value" +def _make_config(**deps: list[DependencyEntry]) -> AgpackConfig: + return AgpackConfig(targets=["claude"], dependencies=dict(deps)) -def test_resolve_config_missing_var_raises(tmp_path: Path) -> None: - server = McpServer(name="ctx7", command="cmd", env={"K": "${UNDEFINED}"}) - config = _make_config([server]) +def test_resolve_validates_but_does_not_mutate_url(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Templates with resolvable ${VAR}s stay verbatim on DependencySource. - with pytest.raises(ConfigError, match="'UNDEFINED'"): - resolve_config(config, tmp_path) - - -def test_resolve_config_empty_mcp_list(tmp_path: Path) -> None: - config = _make_config([]) - resolve_config(config, tmp_path) # should not raise - - -def test_resolve_config_multiple_servers( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - monkeypatch.setenv("TOKEN_A", "aaa") - monkeypatch.setenv("TOKEN_B", "bbb") - servers = [ - McpServer(name="a", command="cmd", env={"T": "${TOKEN_A}"}), - McpServer(name="b", command="cmd", env={"T": "${TOKEN_B}"}), - ] - config = _make_config(servers) - - resolve_config(config, tmp_path) - - assert config.mcp[0].env["T"] == "aaa" - assert config.mcp[1].env["T"] == "bbb" - - -# --------------------------------------------------------------------------- -# 4. resolve_config – dependency fields (url, path, ref) -# --------------------------------------------------------------------------- - - -def test_resolve_dependency_url( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: + The lockfile, progress output, and FetchError messages all read these fields, so substituting in place would leak + ${GITHUB_TOKEN}. Actual substitution happens inside fetch_dependency. + """ monkeypatch.setenv("GH_ORG", "my-org") dep = DependencySource(urls=["https://github.com/${GH_ORG}/repo"]) - config = _make_config() - config.skills = [dep] - + config = _make_config(skills=[dep]) resolve_config(config, tmp_path) - - assert config.skills[0].url == "https://github.com/my-org/repo" + # Template preserved — substitution is deferred to clone time. + assert config.dependencies["skills"][0].urls == [ # type: ignore[union-attr] + "https://github.com/${GH_ORG}/repo" + ] -def test_resolve_dependency_path( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_resolve_validates_but_does_not_mutate_path(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("SKILL_NAME", "my-skill") - dep = DependencySource( - urls=["https://github.com/org/repo"], path="skills/${SKILL_NAME}" - ) - config = _make_config() - config.skills = [dep] - + dep = DependencySource(urls=["https://github.com/org/repo"], path="skills/${SKILL_NAME}") + config = _make_config(skills=[dep]) resolve_config(config, tmp_path) - - assert config.skills[0].path == "skills/my-skill" + assert config.dependencies["skills"][0].path == "skills/${SKILL_NAME}" # type: ignore[union-attr] -def test_resolve_dependency_ref( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_resolve_validates_but_does_not_mutate_ref(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("TAG", "v2.0") dep = DependencySource(urls=["https://github.com/org/repo"], ref="${TAG}") - config = _make_config() - config.commands = [dep] - - resolve_config(config, tmp_path) - - assert config.commands[0].ref == "v2.0" - - -def test_resolve_dependency_no_vars_unchanged(tmp_path: Path) -> None: - dep = DependencySource(urls=["https://github.com/org/repo"], path="skills/foo") - config = _make_config() - config.agents = [dep] - + config = _make_config(commands=[dep]) resolve_config(config, tmp_path) + assert config.dependencies["commands"][0].ref == "${TAG}" # type: ignore[union-attr] - assert config.agents[0].url == "https://github.com/org/repo" - assert config.agents[0].path == "skills/foo" - - -def test_resolve_dependency_path_none_stays_none(tmp_path: Path) -> None: - dep = DependencySource(urls=["https://github.com/org/repo"]) - config = _make_config() - config.skills = [dep] +def test_resolve_validates_every_url_in_fallback_list(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Each URL in a fallback list is validated; none are mutated.""" + monkeypatch.setenv("GH_ORG", "my-org") + dep = DependencySource( + urls=[ + "https://github.com/${GH_ORG}/repo", + "git@github.com:${GH_ORG}/repo.git", + ], + ) + config = _make_config(skills=[dep]) resolve_config(config, tmp_path) + assert config.dependencies["skills"][0].urls == [ # type: ignore[union-attr] + "https://github.com/${GH_ORG}/repo", + "git@github.com:${GH_ORG}/repo.git", + ] - assert config.skills[0].path is None - - -def test_resolve_dependency_ref_none_stays_none(tmp_path: Path) -> None: - dep = DependencySource(urls=["https://github.com/org/repo"]) - config = _make_config() - config.skills = [dep] - - resolve_config(config, tmp_path) - assert config.skills[0].ref is None +def test_resolve_raises_for_missing_var(tmp_path: Path) -> None: + """An unresolvable ${VAR} fails at validate time, before any clones.""" + dep = DependencySource(urls=["https://github.com/${MISSING_VAR_XYZ}/repo"]) + config = _make_config(skills=[dep]) + with pytest.raises(Exception, match="MISSING_VAR_XYZ"): + resolve_config(config, tmp_path) # --------------------------------------------------------------------------- -# 5. resolve_config – MCP command, args, url fields +# 4. resolve_config — patches are NOT substituted at load time # --------------------------------------------------------------------------- -def test_resolve_mcp_command(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setenv("MCP_BIN", "/usr/local/bin/my-server") - server = McpServer(name="s", command="${MCP_BIN}") - config = _make_config([server]) - - resolve_config(config, tmp_path) - - assert config.mcp[0].command == "/usr/local/bin/my-server" - - -def test_resolve_mcp_args(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setenv("PORT", "9090") - server = McpServer(name="s", command="node", args=["--port", "${PORT}"]) - config = _make_config([server]) +def test_patches_pass_through_unchanged(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Patches keep their ${...} templates after resolve_config. + Substitution happens per-target at apply time so target ``vars`` can win over env vars on collision (see + test_patches.py). + """ + monkeypatch.setenv("MCP_BIN", "/usr/local/bin/my-server") + patch = Patch(key="${bucket}.s", value="${MCP_BIN}") + config = _make_config(mcp=[patch]) resolve_config(config, tmp_path) + # Templates intact: substitution is deferred to apply time. + assert config.dependencies["mcp"][0].key == "${bucket}.s" + assert config.dependencies["mcp"][0].value == "${MCP_BIN}" - assert config.mcp[0].args == ["--port", "9090"] - - -def test_resolve_mcp_url(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setenv("MCP_HOST", "private.example.com") - server = McpServer(name="s", type="sse", url="https://${MCP_HOST}/sse") - config = _make_config([server]) - - resolve_config(config, tmp_path) - assert config.mcp[0].url == "https://private.example.com/sse" +def test_resolve_config_returns_env_table(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """resolve_config returns the merged env table for downstream apply.""" + (tmp_path / ".env").write_text("API_KEY=secret\n") + monkeypatch.setenv("OTHER", "from-shell") + config = _make_config() + env = resolve_config(config, tmp_path) + assert env["API_KEY"] == "secret" + assert env["OTHER"] == "from-shell" # --------------------------------------------------------------------------- -# 6. resolve_config – three-tier .env resolution (project > global > shell) +# 5. Three-tier .env (project > global > shell) — verified via fetch deps # --------------------------------------------------------------------------- -def test_three_tier_project_dotenv_wins( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - """Project .env takes highest priority.""" +def test_three_tier_project_dotenv_wins(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("MY_VAR", "from-shell") - - # Global .env global_dir = tmp_path / "global" global_dir.mkdir() (global_dir / ".env").write_text("MY_VAR=from-global\n") global_cfg = GlobalConfig(config_dir=global_dir) - # Project .env project_dir = tmp_path / "project" project_dir.mkdir() (project_dir / ".env").write_text("MY_VAR=from-project\n") - server = McpServer(name="s", command="cmd", env={"V": "${MY_VAR}"}) - config = _make_config([server]) - - resolve_config(config, project_dir, global_config=global_cfg) + dep = DependencySource(urls=["https://example.com/${MY_VAR}"]) + config = _make_config(skills=[dep]) + env = resolve_config(config, project_dir, global_config=global_cfg) + # Precedence in the returned env table (consumed at clone time). + assert env["MY_VAR"] == "from-project" - assert config.mcp[0].env["V"] == "from-project" - -def test_three_tier_global_dotenv_fallback( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - """Global .env used when project .env doesn't define the var.""" +def test_three_tier_global_dotenv_fallback(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("MY_VAR", "from-shell") - global_dir = tmp_path / "global" global_dir.mkdir() (global_dir / ".env").write_text("MY_VAR=from-global\n") global_cfg = GlobalConfig(config_dir=global_dir) - project_dir = tmp_path / "project" project_dir.mkdir() - # No project .env - - server = McpServer(name="s", command="cmd", env={"V": "${MY_VAR}"}) - config = _make_config([server]) - resolve_config(config, project_dir, global_config=global_cfg) + dep = DependencySource(urls=["https://example.com/${MY_VAR}"]) + config = _make_config(skills=[dep]) + env = resolve_config(config, project_dir, global_config=global_cfg) + assert env["MY_VAR"] == "from-global" - assert config.mcp[0].env["V"] == "from-global" - -def test_three_tier_shell_fallback( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - """Shell env used when neither .env defines the var.""" +def test_three_tier_shell_fallback(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: monkeypatch.setenv("MY_VAR", "from-shell") - global_dir = tmp_path / "global" global_dir.mkdir() - # No global .env global_cfg = GlobalConfig(config_dir=global_dir) - project_dir = tmp_path / "project" project_dir.mkdir() - # No project .env - - server = McpServer(name="s", command="cmd", env={"V": "${MY_VAR}"}) - config = _make_config([server]) - resolve_config(config, project_dir, global_config=global_cfg) - - assert config.mcp[0].env["V"] == "from-shell" - - -def test_three_tier_no_global_config( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: - """When no global config is provided, only project .env + shell are used.""" - monkeypatch.setenv("MY_VAR", "from-shell") - (tmp_path / ".env").write_text("MY_VAR=from-project\n") - - server = McpServer(name="s", command="cmd", env={"V": "${MY_VAR}"}) - config = _make_config([server]) - - resolve_config(config, tmp_path) # no global_config - - assert config.mcp[0].env["V"] == "from-project" + dep = DependencySource(urls=["https://example.com/${MY_VAR}"]) + config = _make_config(skills=[dep]) + env = resolve_config(config, project_dir, global_config=global_cfg) + assert env["MY_VAR"] == "from-shell" def test_three_tier_global_env_applies_to_deps(tmp_path: Path) -> None: - """Global .env vars are available for dependency field substitution too.""" global_dir = tmp_path / "global" global_dir.mkdir() (global_dir / ".env").write_text("GH_ORG=my-global-org\n") global_cfg = GlobalConfig(config_dir=global_dir) - project_dir = tmp_path / "project" project_dir.mkdir() dep = DependencySource(urls=["https://github.com/${GH_ORG}/repo"]) - config = _make_config() - config.skills = [dep] - - resolve_config(config, project_dir, global_config=global_cfg) - - assert config.skills[0].url == "https://github.com/my-global-org/repo" - + config = _make_config(skills=[dep]) + env = resolve_config(config, project_dir, global_config=global_cfg) + assert env["GH_ORG"] == "my-global-org" -# --------------------------------------------------------------------------- -# 7. resolve_config – multiple URL substitution -# --------------------------------------------------------------------------- - -def test_resolve_multiple_urls(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: - monkeypatch.setenv("GH_ORG", "my-org") - dep = DependencySource( - urls=[ - "https://github.com/${GH_ORG}/repo", - "git@github.com:${GH_ORG}/repo.git", - ], - ) +def test_resolve_empty_config_no_op(tmp_path: Path) -> None: config = _make_config() - config.skills = [dep] - resolve_config(config, tmp_path) - - assert config.skills[0].urls == [ - "https://github.com/my-org/repo", - "git@github.com:my-org/repo.git", - ] - - -def test_resolve_single_url_unchanged(tmp_path: Path) -> None: - dep = DependencySource(urls=["https://github.com/org/repo"]) - config = _make_config() - config.skills = [dep] - - resolve_config(config, tmp_path) - - assert config.skills[0].urls == ["https://github.com/org/repo"] diff --git a/tests/test_fetcher.py b/tests/test_fetcher.py index 607a424..c2eb457 100644 --- a/tests/test_fetcher.py +++ b/tests/test_fetcher.py @@ -26,16 +26,12 @@ def _ok(stdout: str = "") -> subprocess.CompletedProcess[str]: """Return a successful CompletedProcess.""" - return subprocess.CompletedProcess( - args=["git"], returncode=0, stdout=stdout, stderr="" - ) + return subprocess.CompletedProcess(args=["git"], returncode=0, stdout=stdout, stderr="") def _fail(stderr: str = "error") -> subprocess.CompletedProcess[str]: """Return a failed CompletedProcess.""" - return subprocess.CompletedProcess( - args=["git"], returncode=1, stdout="", stderr=stderr - ) + return subprocess.CompletedProcess(args=["git"], returncode=1, stdout="", stderr=stderr) FAKE_SHA_FULL = "a" * 40 @@ -119,9 +115,7 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 assert "main" in clone_args @patch("agpack.fetcher._run_git") - def test_clone_fails_raises_fetch_error( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_clone_fails_raises_fetch_error(self, mock_git: MagicMock, tmp_path: Path) -> None: """When clone fails, FetchError is raised.""" source = DependencySource(urls=["https://github.com/owner/repo"], ref="main") @@ -134,13 +128,9 @@ def test_clone_fails_raises_fetch_error( fetch_dependency(source) @patch("agpack.fetcher._run_git") - def test_sparse_checkout_when_path_set( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_sparse_checkout_when_path_set(self, mock_git: MagicMock, tmp_path: Path) -> None: """When source.path is set, sparse checkout is attempted.""" - source = DependencySource( - urls=["https://github.com/owner/repo"], ref="main", path="skills/foo" - ) + source = DependencySource(urls=["https://github.com/owner/repo"], ref="main", path="skills/foo") def side_effect(args: list[str], cwd=None): # noqa: ARG001 if args[0] == "clone": @@ -170,20 +160,14 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 assert "--filter=blob:none" in clone_args # Verify sparse-checkout set was called - sparse_calls = [ - c for c in mock_git.call_args_list if c[0][0][0] == "sparse-checkout" - ] + sparse_calls = [c for c in mock_git.call_args_list if c[0][0][0] == "sparse-checkout"] assert len(sparse_calls) == 1 assert sparse_calls[0][0][0] == ["sparse-checkout", "set", "skills/foo"] @patch("agpack.fetcher._run_git") - def test_sparse_checkout_fallback_to_full_clone( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_sparse_checkout_fallback_to_full_clone(self, mock_git: MagicMock, tmp_path: Path) -> None: """When sparse checkout fails, a full clone is performed.""" - source = DependencySource( - urls=["https://github.com/owner/repo"], ref="main", path="lib/bar" - ) + source = DependencySource(urls=["https://github.com/owner/repo"], ref="main", path="lib/bar") clone_attempt = {"count": 0} def side_effect(args: list[str], cwd=None): # noqa: ARG001 @@ -195,11 +179,10 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 # First clone (sparse) succeeds clone_dir.mkdir(parents=True, exist_ok=True) return _ok() - else: - # Second clone (full) succeeds - clone_dir.mkdir(parents=True, exist_ok=True) - (clone_dir / "lib" / "bar").mkdir(parents=True, exist_ok=True) - return _ok() + # Second clone (full) succeeds + clone_dir.mkdir(parents=True, exist_ok=True) + (clone_dir / "lib" / "bar").mkdir(parents=True, exist_ok=True) + return _ok() if args[0] == "sparse-checkout": return _fail(stderr="sparse-checkout not supported") @@ -264,20 +247,14 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 assert len(fetch_calls) >= 1 assert sha in fetch_calls[0][0][0] - checkout_calls = [ - c for c in mock_git.call_args_list if c[0][0][0] == "checkout" - ] + checkout_calls = [c for c in mock_git.call_args_list if c[0][0][0] == "checkout"] assert len(checkout_calls) == 1 assert sha in checkout_calls[0][0][0] @patch("agpack.fetcher._run_git") - def test_path_not_found_raises_fetch_error( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_path_not_found_raises_fetch_error(self, mock_git: MagicMock, tmp_path: Path) -> None: """When source.path does not exist after clone, FetchError is raised.""" - source = DependencySource( - urls=["https://github.com/owner/repo"], ref="main", path="does/not/exist" - ) + source = DependencySource(urls=["https://github.com/owner/repo"], ref="main", path="does/not/exist") def side_effect(args: list[str], cwd=None): # noqa: ARG001 if args[0] == "clone": @@ -299,9 +276,7 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 fetch_dependency(source) @patch("agpack.fetcher._run_git") - def test_clone_dir_cleanup_on_fallback_url( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_clone_dir_cleanup_on_fallback_url(self, mock_git: MagicMock, tmp_path: Path) -> None: """When a fallback URL is tried, the previous clone dir is cleaned up.""" source = DependencySource( urls=["https://primary.com/repo", "https://fallback.com/repo"], @@ -364,9 +339,7 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 assert ["checkout", sha] in call_log @patch("agpack.fetcher._run_git") - def test_unshallow_fails_falls_back_to_fetch_origin( - self, mock_git: MagicMock - ) -> None: + def test_unshallow_fails_falls_back_to_fetch_origin(self, mock_git: MagicMock) -> None: """When unshallow fails (already full), falls back to fetch origin.""" sha = "b" * 40 call_log: list[list[str]] = [] @@ -444,9 +417,7 @@ def test_removes_temp_directory(self, tmp_path: Path) -> None: content_dir.mkdir(parents=True) (content_dir / "SKILL.md").write_text("hello") - source = DependencySource( - urls=["https://github.com/owner/repo"], path="skills/foo" - ) + source = DependencySource(urls=["https://github.com/owner/repo"], path="skills/foo") result = FetchResult( source=source, local_path=content_dir, @@ -533,9 +504,7 @@ class TestUrlFallback: """Tests for fallback URLs when the primary URL fails.""" @patch("agpack.fetcher._run_git") - def test_primary_succeeds_no_fallback_tried( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_primary_succeeds_no_fallback_tried(self, mock_git: MagicMock, tmp_path: Path) -> None: """When the primary URL works, fallback URLs are not attempted.""" source = DependencySource( urls=["https://github.com/owner/repo", "git@github.com:owner/repo.git"], @@ -564,9 +533,7 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 assert "https://github.com/owner/repo" in clone_calls[0][0][0] @patch("agpack.fetcher._run_git") - def test_primary_fails_fallback_succeeds( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_primary_fails_fallback_succeeds(self, mock_git: MagicMock, tmp_path: Path) -> None: """When the primary URL fails, the fallback URL is tried and succeeds.""" source = DependencySource( urls=["https://github.com/owner/repo", "git@github.com:owner/repo.git"], @@ -603,9 +570,7 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 assert "git@github.com:owner/repo.git" in clone_calls[1][0][0] @patch("agpack.fetcher._run_git") - def test_all_urls_fail_raises_last_error( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_all_urls_fail_raises_last_error(self, mock_git: MagicMock, tmp_path: Path) -> None: """When all URLs fail, the last FetchError is raised.""" source = DependencySource( urls=["https://github.com/owner/repo", "git@github.com:owner/repo.git"], @@ -621,9 +586,7 @@ def test_all_urls_fail_raises_last_error( fetch_dependency(source) @patch("agpack.fetcher._run_git") - def test_multiple_urls_tried_in_order( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_multiple_urls_tried_in_order(self, mock_git: MagicMock, tmp_path: Path) -> None: """Multiple URLs are tried in order until one succeeds.""" source = DependencySource( urls=[ @@ -683,9 +646,7 @@ def test_single_url_no_retry(self, mock_git: MagicMock, tmp_path: Path) -> None: assert len(clone_calls) == 1 @patch("agpack.fetcher._run_git") - def test_fallback_with_sparse_checkout( - self, mock_git: MagicMock, tmp_path: Path - ) -> None: + def test_fallback_with_sparse_checkout(self, mock_git: MagicMock, tmp_path: Path) -> None: """URL fallback works together with sparse checkout.""" source = DependencySource( urls=["https://github.com/owner/repo", "git@github.com:owner/repo.git"], @@ -718,3 +679,58 @@ def side_effect(args: list[str], cwd=None): # noqa: ARG001 result = fetch_dependency(source) assert result.local_path == tmp_path / "repo" / "skills" / "foo" + + +# --------------------------------------------------------------------------- +# Secret-leak guarantees: clone-time ${VAR} substitution + stderr scrubbing +# --------------------------------------------------------------------------- + + +class TestFetchSubstitutesTokensAtCloneTime: + """``${VAR}`` references in templates get resolved only for the git invocation; ``source`` is never mutated and + stderr from git is scrubbed before propagating into ``FetchError``. + """ + + @patch("agpack.fetcher._run_git") + def test_resolves_url_for_clone_but_leaves_template_intact(self, mock_git: MagicMock, tmp_path: Path) -> None: + source = DependencySource(urls=["https://x-access-token:${GH_TOKEN}@github.com/o/r"]) + + def side_effect(args, cwd=None): # noqa: ARG001 # mock must mirror _run_git's kwarg name + if args[0] == "clone": + Path(args[-1]).mkdir(parents=True, exist_ok=True) + return _ok() + if args[0] == "rev-parse": + return _ok(stdout=FAKE_SHA_FULL + "\n") + return _ok() + + mock_git.side_effect = side_effect + + with patch("agpack.fetcher.tempfile.mkdtemp", return_value=str(tmp_path)): + fetch_dependency(source, {"GH_TOKEN": "ghp_SECRET"}) + + clone_call = next(c for c in mock_git.call_args_list if c[0][0][0] == "clone") + assert "ghp_SECRET" in clone_call[0][0][-2] + + assert source.urls == ["https://x-access-token:${GH_TOKEN}@github.com/o/r"] + + @patch("agpack.fetcher.subprocess.run") + def test_clone_failure_redacts_token_from_error(self, mock_run: MagicMock, tmp_path: Path) -> None: + # Mock subprocess.run (not _run_git) so the redaction inside _run_git actually fires. + source = DependencySource(urls=["https://x-access-token:${GH_TOKEN}@github.com/o/r"]) + mock_run.return_value = subprocess.CompletedProcess( + args=["git", "clone"], + returncode=128, + stdout="", + stderr="fatal: unable to access 'https://x-access-token:ghp_SECRET@github.com/o/r/': 404", + ) + + with ( + patch("agpack.fetcher.tempfile.mkdtemp", return_value=str(tmp_path)), + pytest.raises(FetchError) as exc_info, + ): + fetch_dependency(source, {"GH_TOKEN": "ghp_SECRET"}) + + msg = str(exc_info.value) + assert "ghp_SECRET" not in msg + assert "x-access-token" not in msg + assert "github.com/o/r" in msg diff --git a/tests/test_integration.py b/tests/test_integration.py index 8d305ca..1d11bda 100644 --- a/tests/test_integration.py +++ b/tests/test_integration.py @@ -11,6 +11,9 @@ from click.testing import CliRunner from agpack.cli import main +from agpack.config import load_config +from agpack.errors import EditFileError +from agpack.target_schema import parse_target_def def _run_git(args: list[str], cwd: Path) -> None: @@ -106,18 +109,25 @@ def test_full_sync_flow(tmp_path: Path) -> None: ], "mcp": [ { - "name": "filesystem", - "command": "npx", - "args": ["-y", "@modelcontextprotocol/server-filesystem", "."], + # ${bucket} resolves per target: 'mcpServers' for claude, 'mcp' for opencode. One patch, both + # targets, correct bucket name on each. + "key": "${bucket}.filesystem", + "value": { + "command": "npx", + "args": [ + "-y", + "@modelcontextprotocol/server-filesystem", + ".", + ], + }, }, ], }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) - # Run sync runner = CliRunner() result = runner.invoke( main, @@ -126,40 +136,35 @@ def test_full_sync_flow(tmp_path: Path) -> None: ) assert result.exit_code == 0, f"sync failed:\n{result.output}" - assert "1 skills, 1 commands, 1 agents, 1 MCP servers" in result.output + assert "1 skills, 1 commands, 1 agents, 1 mcp" in result.output - # Verify skill files for target_dir in [".claude/skills/my-skill", ".opencode/skills/my-skill"]: skill_md = project_dir / target_dir / "SKILL.md" assert skill_md.exists(), f"Missing {skill_md}" assert "My Skill" in skill_md.read_text() - util_py = project_dir / target_dir / "helpers" / "util.py" assert util_py.exists(), f"Missing {util_py}" - # Verify command files assert (project_dir / ".claude/commands/review.md").exists() assert (project_dir / ".opencode/commands/review.md").exists() - - # Verify agent files assert (project_dir / ".claude/agents/backend-expert.md").exists() assert (project_dir / ".opencode/agents/backend-expert.md").exists() - # Verify MCP configs + # MCP — each target uses its own bucket name via ${bucket}. claude_mcp = json.loads((project_dir / ".mcp.json").read_text()) - assert "filesystem" in claude_mcp["mcpServers"] assert claude_mcp["mcpServers"]["filesystem"]["command"] == "npx" - opencode_mcp = json.loads((project_dir / "opencode.json").read_text()) - assert "filesystem" in opencode_mcp["mcp"] + assert opencode_mcp["mcp"]["filesystem"]["command"] == "npx" # Verify lockfile lockfile_path = project_dir / ".agpack.lock.yml" assert lockfile_path.exists() lockfile = yaml.safe_load(lockfile_path.read_text()) assert len(lockfile["installed"]) == 3 - assert len(lockfile["mcp"]) == 1 - assert lockfile["mcp"][0]["name"] == "filesystem" + edits = {e["resource_type"]: e for e in lockfile["edits"]} + assert "mcp" in edits + # The single patch was applied to two targets (claude + opencode). + assert len(edits["mcp"]["applied"]) == 2 def test_sync_cleanup_removed_dependency(tmp_path: Path) -> None: @@ -189,7 +194,7 @@ def test_sync_cleanup_removed_dependency(tmp_path: Path) -> None: } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -205,7 +210,7 @@ def test_sync_cleanup_removed_dependency(tmp_path: Path) -> None: # Remove the skill from config, keep command config["dependencies"]["skills"] = [] - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) # Second sync result = runner.invoke( @@ -243,7 +248,7 @@ def test_sync_dry_run(tmp_path: Path) -> None: } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -284,7 +289,7 @@ def test_status_command(tmp_path: Path) -> None: } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() @@ -316,6 +321,125 @@ def test_status_command(tmp_path: Path) -> None: assert "✓" in result.output +def test_status_shows_synced_for_var_patches(tmp_path: Path) -> None: + """Patches whose keys use ${var} must show as synced after a successful sync. + + Regression: status used to compare the unresolved ``dep.key`` (``${bucket}.fs``) against the resolved ``ap.key`` + (``mcpServers.fs``) recorded in the lockfile, so any patch using the documented ${bucket} pattern was permanently + rendered as "not yet synced". + """ + project_dir = tmp_path / "project" + project_dir.mkdir() + config = { + "targets": ["claude"], + "dependencies": { + "mcp": [ + { + "key": "${bucket}.filesystem", + "value": {"command": "npx", "args": ["@modelcontextprotocol/server-filesystem", "."]}, + }, + ], + }, + } + config_path = project_dir / "agpack.yml" + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + + runner = CliRunner() + sync_result = runner.invoke(main, ["sync", "--config", str(config_path)], catch_exceptions=False) + assert sync_result.exit_code == 0 + + # The patch landed in .mcp.json under the resolved key. + mcp_file = json.loads((project_dir / ".mcp.json").read_text()) + assert "filesystem" in mcp_file["mcpServers"] + + status_result = runner.invoke(main, ["status", "--config", str(config_path)], catch_exceptions=False) + assert status_result.exit_code == 0 + assert "✓" in status_result.output + assert "not yet synced" not in status_result.output + # The unresolved key is what the user wrote; status still surfaces it verbatim. + assert "${bucket}.filesystem" in status_result.output + + +def test_sync_does_not_write_resolved_secrets_to_lockfile(tmp_path: Path) -> None: + """A patch that interpolates ``${API_KEY}`` into its **value** must not write the resolved secret to the lockfile. + + The lockfile stores the resolved key plus a SHA256 hash of the resolved value, so even if the user commits the + lockfile to git, the actual value secret never lands on disk. (Patch keys are assumed to be structural — see + :class:`AppliedPatch` for the assumption that callers don't put secrets in keys.) + """ + project_dir = tmp_path / "project" + project_dir.mkdir() + secret = "sk-super-secret-token-do-not-leak" + (project_dir / ".env").write_text(f"API_KEY={secret}\n", encoding="utf-8") + config = { + "targets": ["claude"], + "dependencies": { + "mcp": [ + { + "key": "${bucket}.filesystem", + "value": { + "command": "npx", + "env": {"API_KEY": "${API_KEY}"}, + }, + }, + ], + }, + } + config_path = project_dir / "agpack.yml" + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + + runner = CliRunner() + result = runner.invoke(main, ["sync", "--config", str(config_path)], catch_exceptions=False) + assert result.exit_code == 0 + + # The resolved file contains the secret (that's the whole point of the sync). + mcp_file = (project_dir / ".mcp.json").read_text() + assert secret in mcp_file + + # But the lockfile must not — value never lands there, only its hash. + lockfile_text = (project_dir / ".agpack.lock.yml").read_text() + assert secret not in lockfile_text + # The resolved key (post ${bucket} substitution) is what gets recorded. + assert "mcpServers.filesystem" in lockfile_text + + +def test_status_marks_partially_applied_patch_as_unsynced(tmp_path: Path) -> None: + """When a patch is applied for one owning target but missing for another, status shows unsynced. + + Two targets owning the same resource type produce two AppliedPatch records (one per file). If the lockfile only + has one, the patch is *not* fully synced — status must reflect that. + """ + project_dir = tmp_path / "project" + project_dir.mkdir() + config = { + "targets": ["claude", "codex"], + "dependencies": { + "mcp": [ + { + "key": "${bucket}.filesystem", + "value": {"command": "npx"}, + }, + ], + }, + } + config_path = project_dir / "agpack.yml" + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + + runner = CliRunner() + sync_result = runner.invoke(main, ["sync", "--config", str(config_path)], catch_exceptions=False) + assert sync_result.exit_code == 0 + + lockfile_path = project_dir / ".agpack.lock.yml" + lockfile = yaml.safe_load(lockfile_path.read_text()) + # Drop one of the two recorded applied patches. + lockfile["edits"][0]["applied"] = lockfile["edits"][0]["applied"][:1] + lockfile_path.write_text(yaml.safe_dump(lockfile, sort_keys=False)) + + status_result = runner.invoke(main, ["status", "--config", str(config_path)], catch_exceptions=False) + assert status_result.exit_code == 0 + assert "not yet synced" in status_result.output + + def test_init_command(tmp_path: Path) -> None: """Test the init command with --config.""" runner = CliRunner() @@ -357,21 +481,22 @@ def test_sync_mcp_cleanup(tmp_path: Path) -> None: "dependencies": { "mcp": [ { - "name": "filesystem", - "command": "npx", - "args": ["-y", "@modelcontextprotocol/server-filesystem"], + "key": "mcpServers.filesystem", + "value": { + "command": "npx", + "args": ["-y", "@modelcontextprotocol/server-filesystem"], + }, }, { - "name": "other-server", - "command": "node", - "args": ["server.js"], + "key": "mcpServers.other-server", + "value": {"command": "node", "args": ["server.js"]}, }, ], }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -379,18 +504,19 @@ def test_sync_mcp_cleanup(tmp_path: Path) -> None: ["sync", "--config", str(config_path)], catch_exceptions=False, ) - assert result.exit_code == 0 + assert result.exit_code == 0, result.output - # Verify both servers exist mcp_data = json.loads((project_dir / ".mcp.json").read_text()) assert "filesystem" in mcp_data["mcpServers"] assert "other-server" in mcp_data["mcpServers"] - # Remove filesystem from config config["dependencies"]["mcp"] = [ - {"name": "other-server", "command": "node", "args": ["server.js"]} + { + "key": "mcpServers.other-server", + "value": {"command": "node", "args": ["server.js"]}, + } ] - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) # Re-sync result = runner.invoke( @@ -411,9 +537,7 @@ def test_sync_mcp_cleanup(tmp_path: Path) -> None: # --------------------------------------------------------------------------- -def test_sync_with_global_config( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_sync_with_global_config(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Global config dependencies are included in sync.""" bare_repo = _create_bare_repo(tmp_path) @@ -431,7 +555,7 @@ def test_sync_with_global_config( }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) # Set up project directory with skills only @@ -449,7 +573,7 @@ def test_sync_with_global_config( }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -483,7 +607,7 @@ def test_sync_no_global_flag(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) # Project with skills only @@ -501,7 +625,7 @@ def test_sync_no_global_flag(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -517,9 +641,7 @@ def test_sync_no_global_flag(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> assert not (project_dir / ".claude/agents/backend-expert.md").exists() -def test_sync_global_false_in_project( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_sync_global_false_in_project(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """'global: false' in project config prevents global config from loading.""" bare_repo = _create_bare_repo(tmp_path) @@ -537,7 +659,7 @@ def test_sync_global_false_in_project( }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) # Project config with global: false @@ -569,9 +691,7 @@ def test_sync_global_false_in_project( assert not (project_dir / ".claude/agents/backend-expert.md").exists() -def test_sync_global_mcp_merged( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_sync_global_mcp_merged(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Global MCP servers are merged and deployed alongside project ones.""" _create_bare_repo(tmp_path) @@ -582,18 +702,16 @@ def test_sync_global_mcp_merged( "dependencies": { "mcp": [ { - "name": "global-server", - "command": "node", - "args": ["global.js"], + "key": "mcpServers.global-server", + "value": {"command": "node", "args": ["global.js"]}, }, ], }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) - # Project with its own MCP server project_dir = tmp_path / "project" project_dir.mkdir() config = { @@ -601,15 +719,14 @@ def test_sync_global_mcp_merged( "dependencies": { "mcp": [ { - "name": "project-server", - "command": "npx", - "args": ["-y", "project-pkg"], + "key": "mcpServers.project-server", + "value": {"command": "npx", "args": ["-y", "project-pkg"]}, }, ], }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -624,9 +741,7 @@ def test_sync_global_mcp_merged( assert "global-server" in mcp_data["mcpServers"] -def test_sync_global_mcp_project_wins_duplicate( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_sync_global_mcp_project_wins_duplicate(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """When project and global define the same MCP server name, project wins.""" _create_bare_repo(tmp_path) @@ -636,15 +751,14 @@ def test_sync_global_mcp_project_wins_duplicate( "dependencies": { "mcp": [ { - "name": "shared", - "command": "node", - "args": ["global-version.js"], + "key": "mcpServers.shared", + "value": {"command": "node", "args": ["global-version.js"]}, }, ], }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) project_dir = tmp_path / "project" @@ -654,15 +768,14 @@ def test_sync_global_mcp_project_wins_duplicate( "dependencies": { "mcp": [ { - "name": "shared", - "command": "npx", - "args": ["project-version"], + "key": "mcpServers.shared", + "value": {"command": "npx", "args": ["project-version"]}, }, ], }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -702,9 +815,7 @@ def test_init_global(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: assert "\ntargets:" not in content -def test_init_global_already_exists( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_init_global_already_exists(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Test 'agpack init --global' when file already exists.""" global_path = tmp_path / "agpack.yml" global_path.write_text("existing content\n") @@ -720,9 +831,7 @@ def test_init_global_already_exists( assert global_path.read_text() == "existing content\n" -def test_status_with_global_config( - tmp_path: Path, monkeypatch: pytest.MonkeyPatch -) -> None: +def test_status_with_global_config(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: """Status command includes global config dependencies.""" bare_repo = _create_bare_repo(tmp_path) @@ -740,7 +849,7 @@ def test_status_with_global_config( }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) # Project with skills only @@ -758,7 +867,7 @@ def test_status_with_global_config( }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() @@ -790,7 +899,7 @@ def test_status_no_global_flag(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) }, } global_path = global_dir / "agpack.yml" - global_path.write_text(yaml.dump(global_config, default_flow_style=False)) + global_path.write_text(yaml.dump(global_config, default_flow_style=False, sort_keys=False)) monkeypatch.setenv("AGPACK_GLOBAL_CONFIG", str(global_path)) project_dir = tmp_path / "project" @@ -807,7 +916,7 @@ def test_status_no_global_flag(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -846,7 +955,7 @@ def test_sync_url_fallback(tmp_path: Path) -> None: }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( @@ -886,7 +995,7 @@ def test_sync_detect_failure_writes_partial_lockfile(tmp_path: Path) -> None: }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) # First sync succeeds, establishing a lockfile runner = CliRunner() @@ -897,11 +1006,14 @@ def test_sync_detect_failure_writes_partial_lockfile(tmp_path: Path) -> None: ) assert result.exit_code == 0 - # Now make detect_command_items raise on second sync - def failing_detect(fetch_result): # noqa: ARG001 - raise RuntimeError("detection failed") + # Now make item detection raise on the commands pass + def failing_detect(fetch_result, layout, resource_type): # noqa: ARG001 + if resource_type == "commands": + msg = "detection failed" + raise RuntimeError(msg) + return [(fetch_result.source.name, fetch_result.local_path)] - with patch("agpack.cli.detect_command_items", side_effect=failing_detect): + with patch("agpack.cli.detect_items", side_effect=failing_detect): result = runner.invoke( main, ["sync", "--config", str(config_path), "--no-global"], @@ -919,11 +1031,9 @@ def failing_detect(fetch_result): # noqa: ARG001 def test_sync_mcp_failure_writes_partial_lockfile(tmp_path: Path) -> None: - """When deploy_mcp_servers raises McpError, partial lockfile is written.""" + """When deploy_mcp_servers raises EditFileError, partial lockfile is written.""" from unittest.mock import patch - from agpack.mcp import McpError - bare_repo = _create_bare_repo(tmp_path) project_dir = tmp_path / "project" @@ -940,19 +1050,18 @@ def test_sync_mcp_failure_writes_partial_lockfile(tmp_path: Path) -> None: ], "mcp": [ { - "name": "bad-server", - "command": "npx", - "args": ["-y", "bad-server"], + "key": "mcpServers.bad-server", + "value": {"command": "npx", "args": ["-y", "bad-server"]}, }, ], }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) with patch( - "agpack.cli.deploy_mcp_servers", - side_effect=McpError("corrupt config file"), + "agpack.cli.sync_edit_resource", + side_effect=EditFileError("corrupt config file"), ): runner = CliRunner() result = runner.invoke( @@ -963,14 +1072,410 @@ def test_sync_mcp_failure_writes_partial_lockfile(tmp_path: Path) -> None: assert result.exit_code != 0 assert "corrupt config file" in result.output - # Partial lockfile should exist with the successfully synced skill + # Partial lockfile should exist with the successfully synced skill. lockfile_path = project_dir / ".agpack.lock.yml" assert lockfile_path.exists() lockfile = yaml.safe_load(lockfile_path.read_text()) assert len(lockfile["installed"]) == 1 - assert lockfile["installed"][0]["type"] == "skill" - # MCP section should be empty since deploy failed - assert lockfile.get("mcp", []) == [] + assert lockfile["installed"][0]["type"] == "skills" + assert lockfile.get("edits", []) == [] + + +def test_sync_with_claude_hooks_and_permissions(tmp_path: Path) -> None: + """End-to-end: hooks + permissions resources deploy realistic Claude Code settings.json content via the patch + model.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + + config = { + "targets": ["claude"], + "dependencies": { + "hooks": [ + { + "key": "${bucket}.PreToolUse", + "strategy": "append", + "value": { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + # $${} escapes — Claude Code resolves this at hook execution time, not agpack. + "command": ("$${CLAUDE_PROJECT_DIR}/.claude/hooks/block.sh"), + } + ], + }, + }, + ], + "permissions": [ + { + "key": "${bucket}.allow", + "strategy": "append", + "value": "Read(/etc/**)", + }, + ], + }, + } + config_path = project_dir / "agpack.yml" + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + + runner = CliRunner() + result = runner.invoke( + main, + ["sync", "--config", str(config_path), "--no-global"], + catch_exceptions=False, + ) + assert result.exit_code == 0, result.output + + settings = json.loads((project_dir / ".claude/settings.json").read_text()) + # ${bucket} resolved to "hooks" and "permissions" respectively; the runtime variable inside the command was + # preserved literally. + assert settings["hooks"]["PreToolUse"] == [ + { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "${CLAUDE_PROJECT_DIR}/.claude/hooks/block.sh", + } + ], + } + ] + assert settings["permissions"]["allow"] == ["Read(/etc/**)"] + + +def test_sync_with_target_definitions_overriding_builtin(tmp_path: Path) -> None: + """A target_definitions entry fully replaces the built-in of the same name.""" + bare_repo = _create_bare_repo(tmp_path) + project_dir = tmp_path / "project" + project_dir.mkdir() + + config = { + "targets": ["claude"], + "dependencies": { + "skills": [{"url": str(bare_repo), "path": "skills/my-skill"}], + }, + "target_definitions": { + "claude": { + "skills": {"kind": "copy-directory", "path": ".my-claude/skills"}, + }, + }, + } + config_path = project_dir / "agpack.yml" + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + + runner = CliRunner() + result = runner.invoke( + main, + ["sync", "--config", str(config_path), "--no-global"], + catch_exceptions=False, + ) + + assert result.exit_code == 0, result.output + # Files land at the overridden path, not at the built-in .claude/skills + assert (project_dir / ".my-claude/skills/my-skill/SKILL.md").exists() + assert not (project_dir / ".claude/skills").exists() + + +def test_sync_with_brand_new_custom_target(tmp_path: Path) -> None: + """A target name absent from built-ins is resolved from target_definitions.""" + bare_repo = _create_bare_repo(tmp_path) + project_dir = tmp_path / "project" + project_dir.mkdir() + + config = { + "targets": ["my-internal-tool"], + "dependencies": { + "skills": [{"url": str(bare_repo), "path": "skills/my-skill"}], + "mcp": [ + { + "key": "mcpServers.filesystem", + "value": {"command": "npx", "args": ["-y", "fs"]}, + }, + ], + }, + "target_definitions": { + "my-internal-tool": { + "skills": {"kind": "copy-directory", "path": ".myaitool/skills"}, + "mcp": { + "kind": "edit-file", + "path": ".myaitool/config.json", + }, + }, + }, + } + config_path = project_dir / "agpack.yml" + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) + + runner = CliRunner() + result = runner.invoke( + main, + ["sync", "--config", str(config_path), "--no-global"], + catch_exceptions=False, + ) + + assert result.exit_code == 0, result.output + assert (project_dir / ".myaitool/skills/my-skill/SKILL.md").exists() + mcp_cfg = json.loads((project_dir / ".myaitool/config.json").read_text()) + assert "filesystem" in mcp_cfg["mcpServers"] + + +def test_targets_list_shows_all_builtins(tmp_path: Path) -> None: + """`agpack targets list` shows the eight built-in targets.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + + runner = CliRunner() + result = runner.invoke( + main, + ["targets", "list", "--config", str(project_dir / "agpack.yml"), "--no-global"], + ) + assert result.exit_code == 0, result.output + + for name in [ + "claude", + "opencode", + "codex", + "cursor", + "copilot", + "gemini", + "windsurf", + "antigravity", + ]: + assert name in result.output + + +def test_targets_list_marks_user_override(tmp_path: Path) -> None: + """User-defined entries that share a name with a built-in are flagged.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + config_path = project_dir / "agpack.yml" + config_path.write_text( + yaml.dump( + { + "targets": ["claude"], + "target_definitions": { + "claude": { + "skills": { + "kind": "copy-directory", + "path": ".my-claude/skills", + }, + }, + "my-tool": { + "skills": { + "kind": "copy-directory", + "path": ".my-tool/skills", + }, + }, + }, + }, + default_flow_style=False, + ) + ) + + runner = CliRunner() + result = runner.invoke( + main, + ["targets", "list", "--config", str(config_path), "--no-global"], + ) + assert result.exit_code == 0, result.output + assert "overrides built-in" in result.output + assert "my-tool" in result.output + + +def test_targets_show_prints_yaml_for_builtin(tmp_path: Path) -> None: + """`agpack targets show ` prints a valid manifest as YAML.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + + runner = CliRunner() + result = runner.invoke( + main, + [ + "targets", + "show", + "claude", + "--config", + str(project_dir / "agpack.yml"), + "--no-global", + ], + ) + assert result.exit_code == 0, result.output + + # Output must be valid YAML that parses back into a TargetDef + parsed = parse_target_def(yaml.safe_load(result.output)) + assert parsed.resources["skills"].path == ".claude/skills" + + +def test_targets_show_uses_user_definition(tmp_path: Path) -> None: + """A user definition shadows the built-in when shown.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + config_path = project_dir / "agpack.yml" + config_path.write_text( + yaml.dump( + { + "targets": ["claude"], + "target_definitions": { + "claude": { + "skills": { + "kind": "copy-directory", + "path": ".my-claude/skills", + }, + }, + }, + }, + default_flow_style=False, + ) + ) + + runner = CliRunner() + result = runner.invoke( + main, + ["targets", "show", "claude", "--config", str(config_path), "--no-global"], + ) + assert result.exit_code == 0, result.output + + parsed = yaml.safe_load(result.output) + assert parsed["skills"]["path"] == ".my-claude/skills" + # User definition has no mcp block — the built-in's mcp must NOT leak in + assert "mcp" not in parsed + + +def test_targets_show_unknown_name_errors(tmp_path: Path) -> None: + project_dir = tmp_path / "project" + project_dir.mkdir() + + runner = CliRunner() + result = runner.invoke( + main, + [ + "targets", + "show", + "bogus-tool", + "--config", + str(project_dir / "agpack.yml"), + "--no-global", + ], + ) + assert result.exit_code != 0 + assert "Unknown target 'bogus-tool'" in result.output + + +def test_init_template_parses_when_uncommented(tmp_path: Path) -> None: + """The scaffolded agpack.yml must parse without errors after `init`.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + config_path = project_dir / "agpack.yml" + + runner = CliRunner() + init_result = runner.invoke(main, ["init", "--config", str(config_path)]) + assert init_result.exit_code == 0, init_result.output + assert config_path.exists() + + # The scaffold has only commented entries — load should report missing targets, since 'targets' is required. + with pytest.raises(Exception, match="targets"): + load_config(config_path) + + +def test_sync_warns_on_misspelled_resource_type(tmp_path: Path) -> None: + """A dependency key no target declares warns instead of silently dropping. + + Regression: open-ended resource type names make typos the most common failure mode. ``mpc:`` (typo of ``mcp:``) + used to produce a clean sync with zero entries deployed. + """ + bare_repo = _create_bare_repo(tmp_path) + project_dir = tmp_path / "project" + project_dir.mkdir() + config_path = project_dir / "agpack.yml" + config_path.write_text( + yaml.dump( + { + "targets": ["claude"], + "dependencies": { + # ``mpc`` instead of ``mcp`` — no target declares it. + "mpc": [ + { + "key": "mcpServers.fs", + "value": {"command": "npx"}, + } + ], + # A real dependency so sync has something to do. + "skills": [ + {"url": str(bare_repo), "path": "skills/my-skill"}, + ], + }, + }, + default_flow_style=False, + ) + ) + + runner = CliRunner() + result = runner.invoke( + main, + ["sync", "--config", str(config_path), "--no-global"], + catch_exceptions=False, + ) + assert result.exit_code == 0 + assert "warning" in result.output.lower() + assert "mpc" in result.output + # The hint must enumerate the resource types the user *could* have meant. + assert "mcp" in result.output + + +def test_sync_warns_on_duplicate_target(tmp_path: Path) -> None: + """``targets: [claude, claude]`` warns and dedups to one application.""" + bare_repo = _create_bare_repo(tmp_path) + project_dir = tmp_path / "project" + project_dir.mkdir() + config_path = project_dir / "agpack.yml" + config_path.write_text( + yaml.dump( + { + "targets": ["claude", "claude"], + "dependencies": { + "skills": [ + {"url": str(bare_repo), "path": "skills/my-skill"}, + ], + }, + }, + default_flow_style=False, + ) + ) + + runner = CliRunner() + result = runner.invoke( + main, + ["sync", "--config", str(config_path), "--no-global"], + catch_exceptions=False, + ) + assert result.exit_code == 0 + assert "warning" in result.output.lower() + assert "multiple times" in result.output.lower() + assert (project_dir / ".claude/skills/my-skill/SKILL.md").exists() + + +def test_sync_unknown_target_lists_options_in_error(tmp_path: Path) -> None: + """An unknown target name surfaces a CLI error mentioning both pools.""" + project_dir = tmp_path / "project" + project_dir.mkdir() + config_path = project_dir / "agpack.yml" + config_path.write_text( + yaml.dump( + {"targets": ["bogus-tool"], "dependencies": {}}, + default_flow_style=False, + ) + ) + + runner = CliRunner() + result = runner.invoke( + main, + ["sync", "--config", str(config_path), "--no-global"], + ) + + assert result.exit_code != 0 + assert "Unknown target 'bogus-tool'" in result.output + assert "Built-in targets:" in result.output + assert "target_definitions" in result.output def test_sync_url_multiple_fallbacks(tmp_path: Path) -> None: @@ -995,7 +1500,7 @@ def test_sync_url_multiple_fallbacks(tmp_path: Path) -> None: }, } config_path = project_dir / "agpack.yml" - config_path.write_text(yaml.dump(config, default_flow_style=False)) + config_path.write_text(yaml.dump(config, default_flow_style=False, sort_keys=False)) runner = CliRunner() result = runner.invoke( diff --git a/tests/test_lockfile.py b/tests/test_lockfile.py index 73c8eef..71b6483 100644 --- a/tests/test_lockfile.py +++ b/tests/test_lockfile.py @@ -1,106 +1,109 @@ -"""Tests for agpack.lockfile module.""" +"""Tests for agpack.lockfile.""" from __future__ import annotations from pathlib import Path -from unittest.mock import patch import pytest import yaml from agpack import __version__ from agpack.lockfile import LOCKFILE_NAME +from agpack.lockfile import AppliedPatch +from agpack.lockfile import EditLockEntry from agpack.lockfile import InstalledEntry from agpack.lockfile import Lockfile -from agpack.lockfile import McpLockEntry from agpack.lockfile import find_removed_dependencies -from agpack.lockfile import find_removed_mcp_servers from agpack.lockfile import read_lockfile from agpack.lockfile import write_lockfile -# --------------------------------------------------------------------------- -# Helpers -# --------------------------------------------------------------------------- + +def _write_raw(tmp_path: Path, content: str) -> Path: + p = tmp_path / LOCKFILE_NAME + p.write_text(content, encoding="utf-8") + return p def _make_lockfile_yaml(data: dict) -> str: return yaml.dump(data, default_flow_style=False, sort_keys=False) -def _write_raw(tmp_path: Path, content: str) -> Path: - path = tmp_path / LOCKFILE_NAME - path.write_text(content, encoding="utf-8") - return path - - # --------------------------------------------------------------------------- # read_lockfile # --------------------------------------------------------------------------- class TestReadLockfile: - def test_returns_none_when_no_lockfile_exists(self, tmp_path: Path): + def test_returns_none_when_no_lockfile_exists(self, tmp_path: Path) -> None: assert read_lockfile(tmp_path) is None - def test_parses_valid_lockfile(self, tmp_path: Path): + def test_parses_valid_lockfile(self, tmp_path: Path) -> None: data = { - "generated_at": "2025-01-01T00:00:00+00:00", - "agpack_version": "0.1.0", + "generated_at": "2026-05-22T00:00:00+00:00", + "agpack_version": __version__, "installed": [ { "url": "https://github.com/owner/repo", "resolved_ref": "abc123", - "type": "skill", + "type": "skills", "deployed_files": ["skills/foo.md"], "path": "skills/foo", }, + ], + "edits": [ { - "url": "https://gitlab.com/owner/other", - "resolved_ref": "def456", - "type": "command", - "deployed_files": [], + "resource_type": "mcp", + "applied": [ + { + "file_path": ".mcp.json", + "key": "mcpServers.fs", + "strategy": "replace", + "value_hash": "sha256:abc123", + }, + ], }, ], - "mcp": [ - {"name": "my-server", "targets": ["claude"]}, - ], } _write_raw(tmp_path, _make_lockfile_yaml(data)) lf = read_lockfile(tmp_path) assert lf is not None - assert lf.generated_at == "2025-01-01T00:00:00+00:00" - assert lf.agpack_version == "0.1.0" - - assert len(lf.installed) == 2 - first = lf.installed[0] - assert first.url == "https://github.com/owner/repo" - assert first.path == "skills/foo" - assert first.resolved_ref == "abc123" - assert first.type == "skill" - assert first.deployed_files == ["skills/foo.md"] - - second = lf.installed[1] - assert second.url == "https://gitlab.com/owner/other" - assert second.path is None - - assert len(lf.mcp) == 1 - assert lf.mcp[0].name == "my-server" - assert lf.mcp[0].targets == ["claude"] - - def test_returns_none_for_corrupt_yaml(self, tmp_path: Path): + assert lf.installed[0].type == "skills" + assert lf.installed[0].url == "https://github.com/owner/repo" + edit = lf.edits[0] + assert edit.resource_type == "mcp" + assert len(edit.applied) == 1 + assert edit.applied[0].file_path == ".mcp.json" + # Stored key is the resolved dotted path — what cleanup navigates to. + assert edit.applied[0].key == "mcpServers.fs" + assert edit.applied[0].strategy == "replace" + # Value never lands in the lockfile — only its SHA256 hash, so interpolated secrets never leak. + assert edit.applied[0].value_hash == "sha256:abc123" + + def test_returns_none_for_corrupt_yaml(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None: _write_raw(tmp_path, "{{{{not: valid: yaml::::") assert read_lockfile(tmp_path) is None + # Must warn loudly — without the lockfile agpack cannot clean up patches the user has since removed from + # agpack.yml. + out = capsys.readouterr().out + assert "warning" in out.lower() + assert "corrupt" in out.lower() - def test_returns_none_for_non_dict_yaml(self, tmp_path: Path): + def test_returns_none_for_non_dict_yaml(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None: _write_raw(tmp_path, "- just\n- a\n- list\n") assert read_lockfile(tmp_path) is None + out = capsys.readouterr().out + assert "warning" in out.lower() + assert "unexpected shape" in out.lower() - def test_handles_empty_file(self, tmp_path: Path): + def test_handles_empty_file(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None: _write_raw(tmp_path, "") assert read_lockfile(tmp_path) is None + # Empty file is also a shape problem — warn so the user knows we have no record of prior applied patches. + out = capsys.readouterr().out + assert "warning" in out.lower() - def test_skips_non_dict_entries_in_installed(self, tmp_path: Path): + def test_skips_non_dict_entries_in_installed(self, tmp_path: Path) -> None: data = { "generated_at": "", "agpack_version": "", @@ -109,10 +112,10 @@ def test_skips_non_dict_entries_in_installed(self, tmp_path: Path): { "url": "https://github.com/owner/repo", "resolved_ref": "abc", - "type": "skill", + "type": "skills", }, ], - "mcp": [], + "edits": [], } _write_raw(tmp_path, _make_lockfile_yaml(data)) lf = read_lockfile(tmp_path) @@ -120,141 +123,114 @@ def test_skips_non_dict_entries_in_installed(self, tmp_path: Path): assert len(lf.installed) == 1 assert lf.installed[0].url == "https://github.com/owner/repo" - def test_skips_non_dict_entries_in_mcp(self, tmp_path: Path): + def test_skips_non_dict_entries_in_edits(self, tmp_path: Path) -> None: data = { "generated_at": "", "agpack_version": "", "installed": [], - "mcp": [42, {"name": "srv", "targets": []}], + "edits": [42, {"resource_type": "mcp", "applied": []}], } _write_raw(tmp_path, _make_lockfile_yaml(data)) lf = read_lockfile(tmp_path) assert lf is not None - assert len(lf.mcp) == 1 - assert lf.mcp[0].name == "srv" + assert len(lf.edits) == 1 + assert lf.edits[0].resource_type == "mcp" # --------------------------------------------------------------------------- -# write_lockfile +# write_lockfile + round-trip # --------------------------------------------------------------------------- class TestWriteLockfile: - def test_creates_new_lockfile(self, tmp_path: Path): - lf = Lockfile( - installed=[ - InstalledEntry( - url="https://github.com/owner/repo", - path=None, - resolved_ref="abc123", - type="skill", - ), - ], - ) - write_lockfile(tmp_path, lf) - - path = tmp_path / LOCKFILE_NAME - assert path.exists() - data = yaml.safe_load(path.read_text(encoding="utf-8")) - assert data["installed"][0]["url"] == "https://github.com/owner/repo" - - def test_overwrites_existing_lockfile(self, tmp_path: Path): - _write_raw(tmp_path, "old: data\n") - - lf = Lockfile( - installed=[ - InstalledEntry( - url="https://github.com/new/repo", - path=None, - resolved_ref="new123", - type="agent", - ), - ], - ) + def test_creates_new_lockfile(self, tmp_path: Path) -> None: + lf = Lockfile(installed=[]) write_lockfile(tmp_path, lf) + assert (tmp_path / LOCKFILE_NAME).exists() - data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text(encoding="utf-8")) - assert "old" not in data - assert data["installed"][0]["url"] == "https://github.com/new/repo" - - def test_includes_generated_at_and_version(self, tmp_path: Path): + def test_includes_generated_at_and_version(self, tmp_path: Path) -> None: lf = Lockfile() write_lockfile(tmp_path, lf) - - data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text(encoding="utf-8")) + data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text()) assert data["agpack_version"] == __version__ - # generated_at should be a non-empty ISO timestamp assert data["generated_at"] - assert "T" in data["generated_at"] - def test_url_is_preserved_in_serialized_lockfile(self, tmp_path: Path): + def test_omits_path_when_none(self, tmp_path: Path) -> None: lf = Lockfile( installed=[ InstalledEntry( - url="https://github.com/owner/repo-a", + url="https://github.com/o/r", path=None, - resolved_ref="aaa", - type="skill", - ), - InstalledEntry( - url="https://gitlab.com/owner/repo-b", - path=None, - resolved_ref="bbb", - type="skill", + resolved_ref="abc", + type="skills", ), ], ) write_lockfile(tmp_path, lf) + data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text()) + assert "path" not in data["installed"][0] - data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text(encoding="utf-8")) - assert data["installed"][0]["url"] == "https://github.com/owner/repo-a" - assert data["installed"][1]["url"] == "https://gitlab.com/owner/repo-b" - - def test_omits_path_when_none(self, tmp_path: Path): + def test_writes_edits(self, tmp_path: Path) -> None: lf = Lockfile( - installed=[ - InstalledEntry( - url="https://github.com/owner/repo", - path=None, - resolved_ref="aaa", - type="skill", + edits=[ + EditLockEntry( + resource_type="mcp", + applied=[ + AppliedPatch( + file_path=".mcp.json", + key="mcpServers.fs", + strategy="replace", + value_hash="sha256:abc", + ), + ], ), ], ) write_lockfile(tmp_path, lf) + data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text()) + assert data["edits"][0]["resource_type"] == "mcp" + assert data["edits"][0]["applied"][0]["key"] == "mcpServers.fs" + assert data["edits"][0]["applied"][0]["value_hash"] == "sha256:abc" + # The raw resolved value never gets written to disk. + assert "value" not in data["edits"][0]["applied"][0] + # target_name was dropped — resolved keys make it unnecessary. + assert "target_name" not in data["edits"][0]["applied"][0] - data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text(encoding="utf-8")) - assert "path" not in data["installed"][0] - def test_includes_path_when_set(self, tmp_path: Path): - lf = Lockfile( +class TestRoundTrip: + def test_write_then_read_preserves_data(self, tmp_path: Path) -> None: + original = Lockfile( installed=[ InstalledEntry( - url="https://github.com/owner/repo", - path="subdir/thing", - resolved_ref="aaa", - type="skill", + url="https://github.com/o/r-a", + path="skills/foo", + resolved_ref="abc123", + type="skills", + deployed_files=["skills/foo.md"], ), ], - ) - write_lockfile(tmp_path, lf) - - data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text(encoding="utf-8")) - assert data["installed"][0]["path"] == "subdir/thing" - - def test_writes_mcp_entries(self, tmp_path: Path): - lf = Lockfile( - mcp=[ - McpLockEntry(name="srv-a", targets=["claude", "cursor"]), - McpLockEntry(name="srv-b", targets=[]), + edits=[ + EditLockEntry( + resource_type="settings", + applied=[ + AppliedPatch( + file_path=".claude/settings.json", + key="hooks.PreToolUse", + strategy="append", + value_hash="sha256:deadbeef", + ), + ], + ), ], ) - write_lockfile(tmp_path, lf) - - data = yaml.safe_load((tmp_path / LOCKFILE_NAME).read_text(encoding="utf-8")) - assert len(data["mcp"]) == 2 - assert data["mcp"][0] == {"name": "srv-a", "targets": ["claude", "cursor"]} - assert data["mcp"][1] == {"name": "srv-b", "targets": []} + write_lockfile(tmp_path, original) + restored = read_lockfile(tmp_path) + assert restored is not None + assert restored.installed[0].url == original.installed[0].url + assert restored.installed[0].type == "skills" + assert restored.edits[0].resource_type == "settings" + assert restored.edits[0].applied[0].key == original.edits[0].applied[0].key + assert restored.edits[0].applied[0].value_hash == original.edits[0].applied[0].value_hash # --------------------------------------------------------------------------- @@ -263,90 +239,42 @@ def test_writes_mcp_entries(self, tmp_path: Path): class TestFindRemovedDependencies: - def test_returns_removed_entries(self): + def test_returns_removed_entries(self) -> None: old = Lockfile( installed=[ InstalledEntry( - url="https://github.com/owner/keep", + url="https://github.com/o/r1", path=None, - resolved_ref="a", - type="skill", + resolved_ref="x", + type="skills", ), InstalledEntry( - url="https://github.com/owner/remove", + url="https://github.com/o/r2", path=None, - resolved_ref="b", - type="skill", - ), - InstalledEntry( - url="https://github.com/owner/also-remove", - path="sub", - resolved_ref="c", - type="command", + resolved_ref="x", + type="skills", ), ], ) - current = {"https://github.com/owner/keep"} - removed = find_removed_dependencies(old, current) - - assert len(removed) == 2 - identities = {e.identity for e in removed} - assert "https://github.com/owner/remove" in identities - assert "https://github.com/owner/also-remove::sub" in identities + removed = find_removed_dependencies(old, {"https://github.com/o/r1"}) + assert len(removed) == 1 + assert removed[0].url == "https://github.com/o/r2" - def test_returns_empty_when_nothing_removed(self): + def test_returns_empty_when_nothing_removed(self) -> None: old = Lockfile( installed=[ InstalledEntry( - url="https://github.com/owner/a", + url="https://github.com/o/r", path=None, resolved_ref="x", - type="skill", + type="skills", ), ], ) - current = {"https://github.com/owner/a"} - assert find_removed_dependencies(old, current) == [] - - def test_returns_empty_when_old_lockfile_is_none(self): - assert find_removed_dependencies(None, {"https://github.com/owner/a"}) == [] - - def test_returns_empty_for_empty_installed(self): - old = Lockfile(installed=[]) - assert find_removed_dependencies(old, set()) == [] - - -# --------------------------------------------------------------------------- -# find_removed_mcp_servers -# --------------------------------------------------------------------------- - - -class TestFindRemovedMcpServers: - def test_returns_removed_servers(self): - old = Lockfile( - mcp=[ - McpLockEntry(name="keep-srv", targets=["claude"]), - McpLockEntry(name="remove-srv", targets=[]), - ], - ) - current = {"keep-srv"} - removed = find_removed_mcp_servers(old, current) - - assert len(removed) == 1 - assert removed[0].name == "remove-srv" - - def test_returns_empty_when_nothing_removed(self): - old = Lockfile( - mcp=[McpLockEntry(name="srv", targets=[])], - ) - assert find_removed_mcp_servers(old, {"srv"}) == [] - - def test_returns_empty_when_old_lockfile_is_none(self): - assert find_removed_mcp_servers(None, {"srv"}) == [] + assert find_removed_dependencies(old, {"https://github.com/o/r"}) == [] - def test_returns_empty_for_empty_mcp(self): - old = Lockfile(mcp=[]) - assert find_removed_mcp_servers(old, set()) == [] + def test_returns_empty_when_old_lockfile_is_none(self) -> None: + assert find_removed_dependencies(None, {"x"}) == [] # --------------------------------------------------------------------------- @@ -355,114 +283,15 @@ def test_returns_empty_for_empty_mcp(self): class TestInstalledEntryIdentity: - def test_identity_without_path(self): - e = InstalledEntry( - url="https://github.com/owner/repo", - path=None, - resolved_ref="x", - type="skill", - ) - assert e.identity == "https://github.com/owner/repo" + def test_identity_without_path(self) -> None: + e = InstalledEntry(url="https://github.com/o/r", path=None, resolved_ref="x", type="skills") + assert e.identity == "https://github.com/o/r" - def test_identity_with_path(self): + def test_identity_with_path(self) -> None: e = InstalledEntry( - url="https://github.com/owner/repo", + url="https://github.com/o/r", path="sub/dir", resolved_ref="x", - type="skill", - ) - assert e.identity == "https://github.com/owner/repo::sub/dir" - - def test_identity_with_different_host(self): - e = InstalledEntry( - url="https://gitlab.com/owner/repo", - path=None, - resolved_ref="x", - type="skill", - ) - assert e.identity == "https://gitlab.com/owner/repo" - - def test_identity_with_different_host_and_path(self): - e = InstalledEntry( - url="https://gitlab.com/owner/repo", - path="p", - resolved_ref="x", - type="skill", + type="skills", ) - assert e.identity == "https://gitlab.com/owner/repo::p" - - -# --------------------------------------------------------------------------- -# Round-trip -# --------------------------------------------------------------------------- - - -class TestRoundTrip: - def test_write_then_read_preserves_data(self, tmp_path: Path): - original = Lockfile( - installed=[ - InstalledEntry( - url="https://github.com/owner/repo-a", - path="skills/foo", - resolved_ref="abc123", - type="skill", - deployed_files=["skills/foo.md", "skills/foo/bar.txt"], - ), - InstalledEntry( - url="https://gitlab.com/owner/repo-b", - path=None, - resolved_ref="def456", - type="command", - deployed_files=[], - ), - ], - mcp=[ - McpLockEntry(name="srv-1", targets=["claude", "cursor"]), - McpLockEntry(name="srv-2", targets=[]), - ], - ) - - write_lockfile(tmp_path, original) - restored = read_lockfile(tmp_path) - - assert restored is not None - assert restored.agpack_version == __version__ - assert restored.generated_at # non-empty - - assert len(restored.installed) == len(original.installed) - for orig, rest in zip(original.installed, restored.installed, strict=False): - assert rest.url == orig.url - assert rest.path == orig.path - assert rest.resolved_ref == orig.resolved_ref - assert rest.type == orig.type - assert rest.deployed_files == orig.deployed_files - assert rest.identity == orig.identity - - assert len(restored.mcp) == len(original.mcp) - for orig, rest in zip(original.mcp, restored.mcp, strict=False): - assert rest.name == orig.name - assert rest.targets == orig.targets - - -# --------------------------------------------------------------------------- -# Atomic write failure cleanup -# --------------------------------------------------------------------------- - - -class TestWriteLockfileAtomicFailure: - def test_cleans_up_temp_file_on_replace_failure(self, tmp_path: Path) -> None: - """When os.replace fails, the temp file is cleaned up and error re-raised.""" - lf = Lockfile() - - with ( - patch("agpack.lockfile.os.replace", side_effect=OSError("disk full")), - pytest.raises(OSError, match="disk full"), - ): - write_lockfile(tmp_path, lf) - - # No temp files should be left behind - leftover = list(tmp_path.glob(".agpack-lock-tmp-*")) - assert leftover == [] - - # The lockfile should not have been created - assert not (tmp_path / LOCKFILE_NAME).exists() + assert e.identity == "https://github.com/o/r::sub/dir" diff --git a/tests/test_mcp.py b/tests/test_mcp.py deleted file mode 100644 index f58e7ca..0000000 --- a/tests/test_mcp.py +++ /dev/null @@ -1,822 +0,0 @@ -"""Tests for agpack.mcp – MCP config merge / deploy / cleanup logic.""" - -from __future__ import annotations - -import json -import tomllib -from pathlib import Path -from unittest.mock import patch - -import pytest -import tomli_w - -from agpack.config import McpServer -from agpack.mcp import McpError -from agpack.mcp import _atomic_write -from agpack.mcp import _merge_json -from agpack.mcp import _merge_toml -from agpack.mcp import _remove_from_json -from agpack.mcp import _remove_from_toml -from agpack.mcp import cleanup_mcp_server -from agpack.mcp import deploy_mcp_servers - -# --------------------------------------------------------------------------- -# Helpers -# --------------------------------------------------------------------------- - - -def _stdio_server( - name: str = "my-server", - command: str = "npx", - args: list[str] | None = None, - env: dict[str, str] | None = None, -) -> McpServer: - return McpServer( - name=name, - type="stdio", - command=command, - args=args or ["-y", "my-server"], - env=env or {}, - ) - - -def _sse_server( - name: str = "remote", - url: str = "https://mcp.example.com/sse", -) -> McpServer: - return McpServer(name=name, type="sse", url=url) - - -def _read_json(path: Path) -> dict: - return json.loads(path.read_text(encoding="utf-8")) - - -def _read_toml(path: Path) -> dict: - return tomllib.loads(path.read_text(encoding="utf-8")) - - -# --------------------------------------------------------------------------- -# 1. Create new JSON config file with stdio server -# --------------------------------------------------------------------------- - - -class TestCreateJsonStdio: - def test_creates_file_with_stdio_server(self, tmp_path: Path) -> None: - server = _stdio_server(env={"API_KEY": "secret"}) - deploy_mcp_servers([server], ["claude"], tmp_path) - - cfg = _read_json(tmp_path / ".mcp.json") - assert "mcpServers" in cfg - srv = cfg["mcpServers"]["my-server"] - assert srv["command"] == "npx" - assert srv["args"] == ["-y", "my-server"] - assert srv["env"] == {"API_KEY": "secret"} - - def test_stdio_server_omits_empty_args_and_env(self, tmp_path: Path) -> None: - server = McpServer(name="bare", type="stdio", command="run") - deploy_mcp_servers([server], ["claude"], tmp_path) - - srv = _read_json(tmp_path / ".mcp.json")["mcpServers"]["bare"] - assert "args" not in srv - assert "env" not in srv - - -# --------------------------------------------------------------------------- -# 2. Create new JSON config file with SSE server -# --------------------------------------------------------------------------- - - -class TestCreateJsonSse: - def test_creates_file_with_sse_server(self, tmp_path: Path) -> None: - server = _sse_server() - deploy_mcp_servers([server], ["claude"], tmp_path) - - cfg = _read_json(tmp_path / ".mcp.json") - srv = cfg["mcpServers"]["remote"] - assert srv == {"url": "https://mcp.example.com/sse", "type": "sse"} - - -# --------------------------------------------------------------------------- -# 3. Merge into existing JSON config – add new server, preserve existing -# --------------------------------------------------------------------------- - - -class TestMergeJsonAddServer: - def test_preserves_existing_and_adds_new(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - config_path.write_text( - json.dumps( - { - "mcpServers": { - "existing": {"command": "old-cmd", "args": ["--flag"]}, - } - }, - indent=2, - ) - + "\n", - encoding="utf-8", - ) - - deploy_mcp_servers([_stdio_server()], ["claude"], tmp_path) - - cfg = _read_json(config_path) - assert "existing" in cfg["mcpServers"] - assert "my-server" in cfg["mcpServers"] - # Original untouched - assert cfg["mcpServers"]["existing"]["command"] == "old-cmd" - - def test_preserves_non_mcp_keys(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - config_path.write_text( - json.dumps({"otherSetting": True, "mcpServers": {}}), - encoding="utf-8", - ) - - deploy_mcp_servers([_stdio_server()], ["claude"], tmp_path) - - cfg = _read_json(config_path) - assert cfg["otherSetting"] is True - - -# --------------------------------------------------------------------------- -# 4. Merge into existing JSON config – overwrite existing server -# --------------------------------------------------------------------------- - - -class TestMergeJsonOverwrite: - def test_overwrites_existing_server_entry(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - config_path.write_text( - json.dumps( - { - "mcpServers": { - "my-server": {"command": "old-cmd"}, - } - } - ), - encoding="utf-8", - ) - - deploy_mcp_servers([_stdio_server()], ["claude"], tmp_path) - - srv = _read_json(config_path)["mcpServers"]["my-server"] - assert srv["command"] == "npx" - - -# --------------------------------------------------------------------------- -# 5. Create new TOML config file (codex format) -# --------------------------------------------------------------------------- - - -class TestCreateToml: - def test_creates_toml_file(self, tmp_path: Path) -> None: - server = _stdio_server() - deploy_mcp_servers([server], ["codex"], tmp_path) - - config_path = tmp_path / ".codex" / "config.toml" - assert config_path.exists() - - cfg = _read_toml(config_path) - assert "mcp_servers" in cfg - srv = cfg["mcp_servers"]["my-server"] - assert srv["command"] == "npx" - assert srv["args"] == ["-y", "my-server"] - - -# --------------------------------------------------------------------------- -# 6. Merge into existing TOML config -# --------------------------------------------------------------------------- - - -class TestMergeToml: - def test_preserves_existing_and_adds_new(self, tmp_path: Path) -> None: - config_dir = tmp_path / ".codex" - config_dir.mkdir() - config_path = config_dir / "config.toml" - config_path.write_text( - '[mcp_servers.existing]\ncommand = "old-cmd"\n', - encoding="utf-8", - ) - - deploy_mcp_servers([_stdio_server()], ["codex"], tmp_path) - - cfg = _read_toml(config_path) - assert "existing" in cfg["mcp_servers"] - assert "my-server" in cfg["mcp_servers"] - assert cfg["mcp_servers"]["existing"]["command"] == "old-cmd" - - def test_preserves_non_mcp_keys_in_toml(self, tmp_path: Path) -> None: - config_dir = tmp_path / ".codex" - config_dir.mkdir() - config_path = config_dir / "config.toml" - config_path.write_text( - 'model = "o3"\n\n[mcp_servers]\n', - encoding="utf-8", - ) - - deploy_mcp_servers([_stdio_server()], ["codex"], tmp_path) - - cfg = _read_toml(config_path) - assert cfg["model"] == "o3" - - -# --------------------------------------------------------------------------- -# 7. deploy_mcp_servers with multiple targets -# --------------------------------------------------------------------------- - - -class TestDeployMultipleTargets: - def test_writes_to_all_requested_targets(self, tmp_path: Path) -> None: - server = _stdio_server() - targets = ["claude", "cursor", "codex"] - result = deploy_mcp_servers([server], targets, tmp_path) - - assert ".mcp.json" in result["my-server"] - assert ".cursor/mcp.json" in result["my-server"] - assert ".codex/config.toml" in result["my-server"] - - # Verify each file actually exists and has the server - claude_cfg = _read_json(tmp_path / ".mcp.json") - assert "my-server" in claude_cfg["mcpServers"] - - cursor_cfg = _read_json(tmp_path / ".cursor" / "mcp.json") - assert "my-server" in cursor_cfg["mcpServers"] - - codex_cfg = _read_toml(tmp_path / ".codex" / "config.toml") - assert "my-server" in codex_cfg["mcp_servers"] - - def test_skips_unknown_targets(self, tmp_path: Path) -> None: - server = _stdio_server() - result = deploy_mcp_servers([server], ["claude", "nonexistent"], tmp_path) - - assert result["my-server"] == [".mcp.json"] - - def test_deploys_multiple_servers(self, tmp_path: Path) -> None: - servers = [_stdio_server(), _sse_server()] - result = deploy_mcp_servers(servers, ["claude"], tmp_path) - - cfg = _read_json(tmp_path / ".mcp.json") - assert "my-server" in cfg["mcpServers"] - assert "remote" in cfg["mcpServers"] - assert "my-server" in result - assert "remote" in result - - -# --------------------------------------------------------------------------- -# 8. deploy_mcp_servers dry_run mode -# --------------------------------------------------------------------------- - - -class TestDeployDryRun: - def test_dry_run_does_not_create_files(self, tmp_path: Path) -> None: - server = _stdio_server() - result = deploy_mcp_servers( - [server], ["claude", "codex"], tmp_path, dry_run=True - ) - - assert ".mcp.json" in result["my-server"] - assert ".codex/config.toml" in result["my-server"] - - # No files should be written - assert not (tmp_path / ".mcp.json").exists() - assert not (tmp_path / ".codex" / "config.toml").exists() - - def test_dry_run_does_not_modify_existing(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - original = json.dumps({"mcpServers": {}}) - config_path.write_text(original, encoding="utf-8") - - deploy_mcp_servers([_stdio_server()], ["claude"], tmp_path, dry_run=True) - - assert config_path.read_text(encoding="utf-8") == original - - -# --------------------------------------------------------------------------- -# 9. cleanup_mcp_server – removes from JSON file -# --------------------------------------------------------------------------- - - -class TestCleanupJson: - def test_removes_server_from_json(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - config_path.write_text( - json.dumps( - {"mcpServers": {"my-server": {"command": "npx"}}}, - indent=2, - ) - + "\n", - encoding="utf-8", - ) - - cleanup_mcp_server( - "my-server", - [".mcp.json"], - tmp_path, - targets=["claude"], - ) - - cfg = _read_json(config_path) - assert "my-server" not in cfg["mcpServers"] - - def test_noop_when_server_not_present(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - original = {"mcpServers": {"other": {"command": "x"}}} - config_path.write_text(json.dumps(original, indent=2) + "\n", encoding="utf-8") - - cleanup_mcp_server( - "nonexistent", - [".mcp.json"], - tmp_path, - targets=["claude"], - ) - - assert _read_json(config_path) == original - - def test_noop_when_file_missing(self, tmp_path: Path) -> None: - # Should not raise - cleanup_mcp_server( - "my-server", - [".mcp.json"], - tmp_path, - targets=["claude"], - ) - - -# --------------------------------------------------------------------------- -# 10. cleanup_mcp_server – removes from TOML file -# --------------------------------------------------------------------------- - - -class TestCleanupToml: - def test_removes_server_from_toml(self, tmp_path: Path) -> None: - config_dir = tmp_path / ".codex" - config_dir.mkdir() - config_path = config_dir / "config.toml" - config_path.write_text( - '[mcp_servers.my-server]\ncommand = "npx"\nargs = ["-y"]\n', - encoding="utf-8", - ) - - cleanup_mcp_server( - "my-server", - [".codex/config.toml"], - tmp_path, - targets=["codex"], - ) - - cfg = _read_toml(config_path) - assert "my-server" not in cfg.get("mcp_servers", {}) - - -# --------------------------------------------------------------------------- -# 11. cleanup_mcp_server – preserves other servers -# --------------------------------------------------------------------------- - - -class TestCleanupPreservesOthers: - def test_preserves_other_servers_in_json(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - config_path.write_text( - json.dumps( - { - "mcpServers": { - "keep-me": {"command": "keep"}, - "remove-me": {"command": "bye"}, - } - }, - indent=2, - ) - + "\n", - encoding="utf-8", - ) - - cleanup_mcp_server( - "remove-me", - [".mcp.json"], - tmp_path, - targets=["claude"], - ) - - cfg = _read_json(config_path) - assert "keep-me" in cfg["mcpServers"] - assert "remove-me" not in cfg["mcpServers"] - - def test_preserves_other_servers_in_toml(self, tmp_path: Path) -> None: - config_dir = tmp_path / ".codex" - config_dir.mkdir() - config_path = config_dir / "config.toml" - config_path.write_text( - '[mcp_servers.keep-me]\ncommand = "keep"\n\n' - '[mcp_servers.remove-me]\ncommand = "bye"\n', - encoding="utf-8", - ) - - cleanup_mcp_server( - "remove-me", - [".codex/config.toml"], - tmp_path, - targets=["codex"], - ) - - cfg = _read_toml(config_path) - assert "keep-me" in cfg["mcp_servers"] - assert "remove-me" not in cfg["mcp_servers"] - - -# --------------------------------------------------------------------------- -# 12. Different servers_key per target -# --------------------------------------------------------------------------- - - -class TestServersKeyPerTarget: - @pytest.mark.parametrize( - ("target", "config_rel", "servers_key"), - [ - ("claude", ".mcp.json", "mcpServers"), - ("opencode", "opencode.json", "mcp"), - ("cursor", ".cursor/mcp.json", "mcpServers"), - ("copilot", ".vscode/mcp.json", "servers"), - ("gemini", ".gemini/settings.json", "mcpServers"), - ], - ) - def test_json_targets_use_correct_key( - self, - tmp_path: Path, - target: str, - config_rel: str, - servers_key: str, - ) -> None: - server = _stdio_server() - deploy_mcp_servers([server], [target], tmp_path) - - cfg = _read_json(tmp_path / config_rel) - assert servers_key in cfg, f"Expected key '{servers_key}' in {config_rel}" - assert "my-server" in cfg[servers_key] - - -# --------------------------------------------------------------------------- -# 13. Opencode-specific MCP format -# --------------------------------------------------------------------------- - - -# --------------------------------------------------------------------------- -# 13a. Copilot/VS Code-specific MCP format -# --------------------------------------------------------------------------- - - -class TestCopilotFormat: - def test_stdio_server_includes_type_field(self, tmp_path: Path) -> None: - server = _stdio_server(env={"API_KEY": "secret"}) - deploy_mcp_servers([server], ["copilot"], tmp_path) - - srv = _read_json(tmp_path / ".vscode" / "mcp.json")["servers"]["my-server"] - assert srv["type"] == "stdio" - assert srv["command"] == "npx" - assert srv["args"] == ["-y", "my-server"] - assert srv["env"] == {"API_KEY": "secret"} - - def test_sse_server_includes_type_field(self, tmp_path: Path) -> None: - server = _sse_server() - deploy_mcp_servers([server], ["copilot"], tmp_path) - - srv = _read_json(tmp_path / ".vscode" / "mcp.json")["servers"]["remote"] - assert srv == {"url": "https://mcp.example.com/sse", "type": "sse"} - - -# --------------------------------------------------------------------------- -# 14. Opencode-specific MCP format -# --------------------------------------------------------------------------- - - -class TestOpencodeFormat: - def test_stdio_server_uses_local_type_and_array_command( - self, tmp_path: Path - ) -> None: - server = _stdio_server(env={"API_KEY": "secret"}) - deploy_mcp_servers([server], ["opencode"], tmp_path) - - srv = _read_json(tmp_path / "opencode.json")["mcp"]["my-server"] - assert srv["type"] == "local" - assert srv["command"] == ["npx", "-y", "my-server"] - assert srv["environment"] == {"API_KEY": "secret"} - assert "args" not in srv - assert "env" not in srv - - def test_stdio_server_omits_empty_environment(self, tmp_path: Path) -> None: - server = McpServer(name="bare", type="stdio", command="run") - deploy_mcp_servers([server], ["opencode"], tmp_path) - - srv = _read_json(tmp_path / "opencode.json")["mcp"]["bare"] - assert srv["type"] == "local" - assert srv["command"] == ["run"] - assert "environment" not in srv - - def test_sse_server_uses_remote_type(self, tmp_path: Path) -> None: - server = _sse_server() - deploy_mcp_servers([server], ["opencode"], tmp_path) - - srv = _read_json(tmp_path / "opencode.json")["mcp"]["remote"] - assert srv == {"type": "remote", "url": "https://mcp.example.com/sse"} - - def test_codex_uses_mcp_servers_key(self, tmp_path: Path) -> None: - deploy_mcp_servers([_stdio_server()], ["codex"], tmp_path) - cfg = _read_toml(tmp_path / ".codex" / "config.toml") - assert "mcp_servers" in cfg - assert "my-server" in cfg["mcp_servers"] - - def test_deploy_and_cleanup_roundtrip(self, tmp_path: Path) -> None: - """Deploy to all targets then clean up – verify all files are cleaned.""" - server = _stdio_server() - targets = ["claude", "opencode", "cursor", "copilot", "codex", "gemini"] - result = deploy_mcp_servers([server], targets, tmp_path) - - cleanup_mcp_server( - "my-server", - result["my-server"], - tmp_path, - targets=targets, - ) - - # Every target config should have the server removed - for _target, rel in [ - ("claude", ".mcp.json"), - ("opencode", "opencode.json"), - ("cursor", ".cursor/mcp.json"), - ("copilot", ".vscode/mcp.json"), - ("gemini", ".gemini/settings.json"), - ]: - cfg = _read_json(tmp_path / rel) - for key in cfg.values(): - if isinstance(key, dict): - assert "my-server" not in key - - codex_cfg = _read_toml(tmp_path / ".codex" / "config.toml") - assert "my-server" not in codex_cfg.get("mcp_servers", {}) - - -# --------------------------------------------------------------------------- -# 15. _merge_json / _merge_toml with corrupt files -# --------------------------------------------------------------------------- - - -class TestMergeCorruptFiles: - def test_merge_json_corrupt_file_raises(self, tmp_path: Path) -> None: - config_path = tmp_path / "bad.json" - config_path.write_text("not valid json {{{{", encoding="utf-8") - - with pytest.raises(McpError, match="Failed to read"): - _merge_json(config_path, "mcpServers", {"s": {"command": "x"}}) - - def test_merge_toml_corrupt_file_raises(self, tmp_path: Path) -> None: - config_path = tmp_path / "bad.toml" - config_path.write_text("= invalid toml", encoding="utf-8") - - with pytest.raises(McpError, match="Failed to read"): - _merge_toml(config_path, "mcp_servers", {"s": {"command": "x"}}) - - -# --------------------------------------------------------------------------- -# 16. _remove_from_json – fuzzy key matching -# --------------------------------------------------------------------------- - - -class TestRemoveFromJsonFuzzyKeys: - @pytest.mark.parametrize("key", ["mcpServers", "mcp", "servers"]) - def test_removes_server_under_various_keys(self, tmp_path: Path, key: str) -> None: - config_path = tmp_path / "config.json" - data = {key: {"my-server": {"command": "npx"}, "keep": {"command": "y"}}} - config_path.write_text(json.dumps(data), encoding="utf-8") - - _remove_from_json(config_path, "my-server", dry_run=False) - - result = json.loads(config_path.read_text(encoding="utf-8")) - assert "my-server" not in result[key] - assert "keep" in result[key] - - def test_noop_when_server_not_found(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.json" - original = {"mcpServers": {"other": {"command": "x"}}} - config_path.write_text(json.dumps(original), encoding="utf-8") - - _remove_from_json(config_path, "nonexistent", dry_run=False) - - assert json.loads(config_path.read_text(encoding="utf-8")) == original - - def test_dry_run_skips_removal(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.json" - original = {"mcpServers": {"my-server": {"command": "x"}}} - config_path.write_text(json.dumps(original), encoding="utf-8") - - _remove_from_json(config_path, "my-server", dry_run=True) - - assert json.loads(config_path.read_text(encoding="utf-8")) == original - - def test_corrupt_file_returns_silently(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.json" - config_path.write_text("not json!", encoding="utf-8") - - # Should not raise - _remove_from_json(config_path, "my-server", dry_run=False) - - -# --------------------------------------------------------------------------- -# 17. _remove_from_toml – fuzzy key matching -# --------------------------------------------------------------------------- - - -class TestRemoveFromTomlFuzzyKeys: - def test_removes_server_under_mcp_servers_key(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.toml" - data = { - "mcp_servers": {"my-server": {"command": "npx"}, "keep": {"command": "y"}} - } - config_path.write_text(tomli_w.dumps(data), encoding="utf-8") - - _remove_from_toml(config_path, "my-server", dry_run=False) - - result = tomllib.loads(config_path.read_text(encoding="utf-8")) - assert "my-server" not in result["mcp_servers"] - assert "keep" in result["mcp_servers"] - - def test_noop_when_server_not_found(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.toml" - data = {"mcp_servers": {"other": {"command": "x"}}} - config_path.write_text(tomli_w.dumps(data), encoding="utf-8") - - _remove_from_toml(config_path, "nonexistent", dry_run=False) - - result = tomllib.loads(config_path.read_text(encoding="utf-8")) - assert result["mcp_servers"] == {"other": {"command": "x"}} - - def test_dry_run_skips_removal(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.toml" - data = {"mcp_servers": {"my-server": {"command": "x"}}} - config_path.write_text(tomli_w.dumps(data), encoding="utf-8") - - _remove_from_toml(config_path, "my-server", dry_run=True) - - result = tomllib.loads(config_path.read_text(encoding="utf-8")) - assert "my-server" in result["mcp_servers"] - - def test_corrupt_file_returns_silently(self, tmp_path: Path) -> None: - config_path = tmp_path / "config.toml" - config_path.write_text("= invalid toml", encoding="utf-8") - - # Should not raise - _remove_from_toml(config_path, "my-server", dry_run=False) - - -# --------------------------------------------------------------------------- -# 18. cleanup_mcp_server – unknown target config (infer from extension) -# --------------------------------------------------------------------------- - - -class TestCleanupUnknownTarget: - def test_cleanup_json_file_with_unknown_target(self, tmp_path: Path) -> None: - """When target config is not found, format is inferred from .json extension.""" - config_path = tmp_path / "custom.json" - data = {"mcpServers": {"my-server": {"command": "npx"}}} - config_path.write_text(json.dumps(data), encoding="utf-8") - - cleanup_mcp_server( - "my-server", - ["custom.json"], - tmp_path, - targets=[], # no targets match - ) - - result = json.loads(config_path.read_text(encoding="utf-8")) - assert "my-server" not in result["mcpServers"] - - def test_cleanup_toml_file_with_unknown_target(self, tmp_path: Path) -> None: - """When target config is not found, format is inferred from .toml extension.""" - config_dir = tmp_path / "custom" - config_dir.mkdir() - config_path = config_dir / "config.toml" - data = {"mcp_servers": {"my-server": {"command": "npx"}}} - config_path.write_text(tomli_w.dumps(data), encoding="utf-8") - - cleanup_mcp_server( - "my-server", - ["custom/config.toml"], - tmp_path, - targets=[], # no targets match - ) - - result = tomllib.loads(config_path.read_text(encoding="utf-8")) - assert "my-server" not in result["mcp_servers"] - - -# --------------------------------------------------------------------------- -# 19. cleanup_mcp_server – dry-run with known target -# --------------------------------------------------------------------------- - - -class TestCleanupDryRun: - def test_dry_run_does_not_remove(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - data = {"mcpServers": {"my-server": {"command": "npx"}}} - config_path.write_text(json.dumps(data, indent=2) + "\n", encoding="utf-8") - - cleanup_mcp_server( - "my-server", - [".mcp.json"], - tmp_path, - targets=["claude"], - dry_run=True, - ) - - result = _read_json(config_path) - assert "my-server" in result["mcpServers"] - - -# --------------------------------------------------------------------------- -# 20. deploy_mcp_servers – error wrapping -# --------------------------------------------------------------------------- - - -class TestDeployErrorHandling: - def test_non_mcp_error_wrapped(self, tmp_path: Path) -> None: - """Non-McpError exceptions from file writes are wrapped in McpError.""" - server = _stdio_server() - - with ( - patch("agpack.mcp._merge_json", side_effect=OSError("disk full")), - pytest.raises(McpError, match="Failed to write MCP config.*disk full"), - ): - deploy_mcp_servers([server], ["claude"], tmp_path) - - def test_mcp_error_re_raised_directly(self, tmp_path: Path) -> None: - """McpError from _merge_json is re-raised without wrapping.""" - server = _stdio_server() - - with ( - patch( - "agpack.mcp._merge_json", - side_effect=McpError("corrupt config"), - ), - pytest.raises(McpError, match="corrupt config"), - ): - deploy_mcp_servers([server], ["claude"], tmp_path) - - -# --------------------------------------------------------------------------- -# 21. _remove_server_from_json / _remove_server_from_toml – corrupt files -# --------------------------------------------------------------------------- - - -# --------------------------------------------------------------------------- -# 20b. _atomic_write failure cleanup -# --------------------------------------------------------------------------- - - -class TestAtomicWriteFailure: - def test_cleans_up_temp_file_on_replace_failure(self, tmp_path: Path) -> None: - """When os.replace fails, the temp file is cleaned up and error re-raised.""" - target = tmp_path / "output.json" - - with ( - patch("agpack.mcp.os.replace", side_effect=OSError("disk full")), - pytest.raises(OSError, match="disk full"), - ): - _atomic_write(target, '{"test": true}\n') - - # No temp files should be left behind - leftover = list(tmp_path.glob(".agpack-mcp-*")) - assert leftover == [] - - # The target file should not have been created - assert not target.exists() - - -# --------------------------------------------------------------------------- -# 21. _remove_server_from_json / _remove_server_from_toml – corrupt files -# --------------------------------------------------------------------------- - - -class TestRemoveServerCorruptFiles: - def test_remove_server_from_corrupt_json(self, tmp_path: Path) -> None: - config_path = tmp_path / ".mcp.json" - config_path.write_text("not json!", encoding="utf-8") - - # Should not raise — silently returns - cleanup_mcp_server( - "my-server", - [".mcp.json"], - tmp_path, - targets=["claude"], - ) - - def test_remove_server_from_corrupt_toml(self, tmp_path: Path) -> None: - config_dir = tmp_path / ".codex" - config_dir.mkdir() - config_path = config_dir / "config.toml" - config_path.write_text("= invalid toml", encoding="utf-8") - - # Should not raise — silently returns - cleanup_mcp_server( - "my-server", - [".codex/config.toml"], - tmp_path, - targets=["codex"], - ) diff --git a/tests/test_patches.py b/tests/test_patches.py new file mode 100644 index 0000000..5ec3083 --- /dev/null +++ b/tests/test_patches.py @@ -0,0 +1,821 @@ +"""Tests for the edit-file Patch engine.""" + +from __future__ import annotations + +import json +import tomllib +from pathlib import Path +from unittest.mock import patch as mock_patch + +import pytest + +from agpack.errors import ConfigError +from agpack.errors import EditFileError +from agpack.kinds import EditFileResource +from agpack.kinds import infer_config_format +from agpack.kinds._shared import _atomic_write +from agpack.kinds.edit_file import _apply_patch +from agpack.kinds.edit_file import _undo_resolved +from agpack.lockfile import AppliedPatch +from agpack.patch import Patch +from agpack.patch import value_hash + + +def _undo(root: dict[str, object], patch: Patch) -> bool: + """Test-only convenience: undo a :class:`Patch` against ``root`` in place.""" + return _undo_resolved(root, patch.strategy, patch.key, value_hash(patch.value)) + + +# --------------------------------------------------------------------------- +# infer_config_format +# --------------------------------------------------------------------------- + + +def test_infer_format_json() -> None: + assert infer_config_format(".mcp.json") == "json" + assert infer_config_format("nested/path/foo.JSON") == "json" + + +def test_infer_format_toml() -> None: + assert infer_config_format(".codex/config.toml") == "toml" + + +def test_infer_format_unknown_extension_raises() -> None: + with pytest.raises(EditFileError, match="cannot infer"): + infer_config_format(".mcp.yaml") + + +# --------------------------------------------------------------------------- +# _apply_patch — replace +# --------------------------------------------------------------------------- + + +def test_apply_replace_at_leaf() -> None: + root: dict = {} + _apply_patch(root, Patch(key="a", value=1)) + assert root == {"a": 1} + + +def test_apply_replace_creates_nested_dicts() -> None: + root: dict = {} + _apply_patch(root, Patch(key="a.b.c", value=42)) + assert root == {"a": {"b": {"c": 42}}} + + +def test_apply_replace_overwrites_existing() -> None: + root: dict = {"a": {"b": "old"}} + _apply_patch(root, Patch(key="a.b", value="new")) + assert root == {"a": {"b": "new"}} + + +def test_apply_replace_preserves_siblings() -> None: + root: dict = {"a": {"keep": 1}} + _apply_patch(root, Patch(key="a.new", value=2)) + assert root == {"a": {"keep": 1, "new": 2}} + + +def test_apply_replace_through_non_dict_raises() -> None: + root: dict = {"a": "scalar"} + with pytest.raises(EditFileError, match="non-dict"): + _apply_patch(root, Patch(key="a.b", value=1)) + + +# --------------------------------------------------------------------------- +# _apply_patch — dotted-key escaping +# --------------------------------------------------------------------------- + + +class TestDottedKeyEscape: + """``\\.`` and ``\\\\`` let users address keys that contain dots.""" + + def test_literal_dot_in_segment(self) -> None: + root: dict = {} + _apply_patch(root, Patch(key="mcpServers.example\\.com", value="x")) + assert root == {"mcpServers": {"example.com": "x"}} + + def test_dot_only_in_leaf(self) -> None: + root: dict = {"mcpServers": {}} + _apply_patch(root, Patch(key="mcpServers.a\\.b\\.c", value=1)) + assert root == {"mcpServers": {"a.b.c": 1}} + + def test_literal_backslash_in_segment(self) -> None: + root: dict = {} + _apply_patch(root, Patch(key="a\\\\b.c", value=1)) + assert root == {"a\\b": {"c": 1}} + + def test_backslash_followed_by_separator(self) -> None: + """``\\\\.`` is a literal backslash and then a separator.""" + root: dict = {} + _apply_patch(root, Patch(key="a\\\\.b", value=1)) + assert root == {"a\\": {"b": 1}} + + def test_empty_key_rejected(self) -> None: + with pytest.raises(EditFileError, match="non-empty"): + _apply_patch({}, Patch(key="", value=1)) + + def test_empty_segment_rejected(self) -> None: + with pytest.raises(EditFileError, match="empty segment"): + _apply_patch({}, Patch(key="a..b", value=1)) + + def test_trailing_dot_rejected(self) -> None: + with pytest.raises(EditFileError, match="empty segment"): + _apply_patch({}, Patch(key="a.", value=1)) + + def test_leading_dot_rejected(self) -> None: + with pytest.raises(EditFileError, match="empty segment"): + _apply_patch({}, Patch(key=".a", value=1)) + + def test_cleanup_understands_escapes(self) -> None: + """Round-trip: a key applied with an escaped dot can also be undone with the same escaped key.""" + root: dict = {} + _apply_patch(root, Patch(key="mcpServers.example\\.com", value="x")) + assert _undo(root, Patch(key="mcpServers.example\\.com", value="x")) + assert root == {"mcpServers": {}} + + +# --------------------------------------------------------------------------- +# _apply_patch — append +# --------------------------------------------------------------------------- + + +def test_apply_append_creates_missing_list() -> None: + root: dict = {} + _apply_patch(root, Patch(key="hooks", value="x", strategy="append")) + assert root == {"hooks": ["x"]} + + +def test_apply_append_extends_existing_list() -> None: + root: dict = {"hooks": ["a"]} + _apply_patch(root, Patch(key="hooks", value="b", strategy="append")) + assert root == {"hooks": ["a", "b"]} + + +def test_apply_append_nested_path() -> None: + root: dict = {} + _apply_patch( + root, + Patch( + key="hooks.PreToolUse", + value={"matcher": "Write", "hooks": [{"type": "command"}]}, + strategy="append", + ), + ) + assert root["hooks"]["PreToolUse"] == [{"matcher": "Write", "hooks": [{"type": "command"}]}] + + +def test_apply_append_on_non_list_raises() -> None: + root: dict = {"hooks": "not a list"} + with pytest.raises(EditFileError, match="non-list"): + _apply_patch(root, Patch(key="hooks", value="x", strategy="append")) + + +# --------------------------------------------------------------------------- +# _undo_resolved — replace +# --------------------------------------------------------------------------- + + +def test_cleanup_replace_deletes_leaf() -> None: + root: dict = {"a": {"b": 1, "c": 2}} + changed = _undo(root, Patch(key="a.b", value=1)) + assert changed is True + assert root == {"a": {"c": 2}} + + +def test_cleanup_replace_missing_is_noop() -> None: + root: dict = {"a": {}} + changed = _undo(root, Patch(key="a.b", value=1)) + assert changed is False + assert root == {"a": {}} + + +def test_cleanup_replace_missing_intermediate_is_noop() -> None: + root: dict = {} + changed = _undo(root, Patch(key="x.y.z", value=1)) + assert changed is False + + +# --------------------------------------------------------------------------- +# _undo_resolved — append +# --------------------------------------------------------------------------- + + +def test_cleanup_append_removes_first_match() -> None: + root: dict = {"hooks": ["a", "b", "c"]} + changed = _undo(root, Patch(key="hooks", value="b", strategy="append")) + assert changed is True + assert root == {"hooks": ["a", "c"]} + + +def test_cleanup_append_deep_equality() -> None: + root: dict = { + "hooks": { + "PreToolUse": [ + {"matcher": "Read", "hooks": []}, + {"matcher": "Write", "hooks": [{"type": "command", "command": "x"}]}, + ] + } + } + changed = _undo( + root, + Patch( + key="hooks.PreToolUse", + value={"matcher": "Write", "hooks": [{"type": "command", "command": "x"}]}, + strategy="append", + ), + ) + assert changed is True + assert root["hooks"]["PreToolUse"] == [{"matcher": "Read", "hooks": []}] + + +def test_cleanup_append_no_match_is_noop() -> None: + root: dict = {"hooks": ["a"]} + changed = _undo(root, Patch(key="hooks", value="z", strategy="append")) + assert changed is False + assert root == {"hooks": ["a"]} + + +# --------------------------------------------------------------------------- +# EditFileResource.apply_patches — JSON file end-to-end +# --------------------------------------------------------------------------- + + +def _read_json(path: Path) -> dict: + return json.loads(path.read_text(encoding="utf-8")) + + +def _read_toml(path: Path) -> dict: + return tomllib.loads(path.read_text(encoding="utf-8")) + + +class TestApplyPatchesJson: + def test_creates_file_when_missing(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + resource.apply_patches( + [ + Patch( + key="mcpServers.fs", + value={"command": "npx", "args": ["-y", "fs"]}, + ) + ], + tmp_path, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg == {"mcpServers": {"fs": {"command": "npx", "args": ["-y", "fs"]}}} + + def test_preserves_existing_unrelated_keys(self, tmp_path: Path) -> None: + (tmp_path / ".mcp.json").write_text( + json.dumps({"$schema": "x", "mcpServers": {"old": {"command": "old"}}}), + encoding="utf-8", + ) + resource = EditFileResource(path=".mcp.json") + resource.apply_patches( + [Patch(key="mcpServers.new", value={"command": "new"})], + tmp_path, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg["$schema"] == "x" + assert cfg["mcpServers"]["old"]["command"] == "old" + assert cfg["mcpServers"]["new"]["command"] == "new" + + def test_dry_run_does_not_write(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + resource.apply_patches( + [Patch(key="mcpServers.fs", value={"command": "npx"})], + tmp_path, + dry_run=True, + ) + assert not (tmp_path / ".mcp.json").exists() + + def test_append_creates_list_and_extends(self, tmp_path: Path) -> None: + (tmp_path / "settings.json").write_text("{}", encoding="utf-8") + resource = EditFileResource(path="settings.json") + resource.apply_patches( + [ + Patch( + key="hooks.PreToolUse", + value={"matcher": "Write", "hooks": [{"type": "command"}]}, + strategy="append", + ), + Patch( + key="permissions.allow", + value="Read(/etc/**)", + strategy="append", + ), + ], + tmp_path, + ) + cfg = _read_json(tmp_path / "settings.json") + assert cfg["hooks"]["PreToolUse"] == [{"matcher": "Write", "hooks": [{"type": "command"}]}] + assert cfg["permissions"]["allow"] == ["Read(/etc/**)"] + + +class TestApplyPatchesToml: + def test_writes_toml_inferred_from_extension(self, tmp_path: Path) -> None: + resource = EditFileResource(path="config.toml") + resource.apply_patches( + [Patch(key="mcp_servers.fs", value={"command": "npx"})], + tmp_path, + ) + cfg = _read_toml(tmp_path / "config.toml") + assert cfg == {"mcp_servers": {"fs": {"command": "npx"}}} + + +# --------------------------------------------------------------------------- +# EditFileResource.cleanup_patches +# --------------------------------------------------------------------------- + + +class TestCleanupPatches: + def test_undoes_replace(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + applied = resource.apply_patches( + [ + Patch(key="mcpServers.fs", value={"command": "npx"}), + Patch(key="mcpServers.other", value={"command": "stay"}), + ], + tmp_path, + ) + # Clean up just the first one — pass the AppliedPatch the apply call returned. + first = next(ap for ap in applied if ap.key == "mcpServers.fs") + resource.cleanup_patches([first], tmp_path) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg == {"mcpServers": {"other": {"command": "stay"}}} + + def test_undoes_append(self, tmp_path: Path) -> None: + resource = EditFileResource(path="settings.json") + applied = resource.apply_patches( + [ + Patch(key="hooks.PreToolUse", value={"matcher": "Write"}, strategy="append"), + Patch(key="hooks.PreToolUse", value={"matcher": "Read"}, strategy="append"), + ], + tmp_path, + ) + # Drop the Write entry; the Read entry stays. Match by hash, which is what cleanup uses internally. + write_hash = value_hash({"matcher": "Write"}) + write_entry = next(ap for ap in applied if ap.value_hash == write_hash) + resource.cleanup_patches([write_entry], tmp_path) + cfg = _read_json(tmp_path / "settings.json") + assert cfg == {"hooks": {"PreToolUse": [{"matcher": "Read"}]}} + + def test_cleanup_missing_file_is_noop(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + # File doesn't exist — should not raise. + resource.cleanup_patches( + [ + AppliedPatch( + file_path=".mcp.json", + key="mcpServers.x", + strategy="replace", + value_hash=value_hash({}), + ) + ], + tmp_path, + ) + + +# --------------------------------------------------------------------------- +# Cleanup semantics — delete-on-removal, format preservation, idempotency +# --------------------------------------------------------------------------- + + +class TestCleanupSemantics: + """Removing a patch deletes the leaf — symmetric with copy-kind cleanup. + + If the user had data at the key before agpack first applied a ``replace``, that value was overwritten on apply. + agpack does not try to remember it and put it back; the same way ``cp foo bar`` overwrites whatever was at ``bar``. + """ + + def test_cleanup_deletes_replace_leaf(self, tmp_path: Path) -> None: + (tmp_path / ".mcp.json").write_text("{}", encoding="utf-8") + resource = EditFileResource(path=".mcp.json") + applied = resource.sync_patches( + applied_old=[], + desired_new=[Patch(key="mcpServers.new", value={"command": "x"})], + project_root=tmp_path, + ) + resource.sync_patches( + applied_old=applied, + desired_new=[], + project_root=tmp_path, + ) + cfg = json.loads((tmp_path / ".mcp.json").read_text()) + assert "new" not in cfg.get("mcpServers", {}) + + def test_cleanup_drops_pre_existing_value_too(self, tmp_path: Path) -> None: + """When the user had a value at a key before agpack first patched it, that value was overwritten on apply and + cleanup does not magically restore it.""" + (tmp_path / ".mcp.json").write_text( + json.dumps({"mcpServers": {"fs": {"command": "user-tool"}}}), + encoding="utf-8", + ) + resource = EditFileResource(path=".mcp.json") + applied = resource.sync_patches( + applied_old=[], + desired_new=[Patch(key="mcpServers.fs", value={"command": "agpack"})], + project_root=tmp_path, + ) + resource.sync_patches( + applied_old=applied, + desired_new=[], + project_root=tmp_path, + ) + cfg = json.loads((tmp_path / ".mcp.json").read_text()) + assert "fs" not in cfg.get("mcpServers", {}) + + def test_value_change_keeps_only_latest_value(self, tmp_path: Path) -> None: + """Updating a patch's value writes the new value; the lockfile only tracks what's currently applied.""" + (tmp_path / ".mcp.json").write_text("{}", encoding="utf-8") + resource = EditFileResource(path=".mcp.json") + first = resource.sync_patches( + applied_old=[], + desired_new=[Patch(key="mcpServers.fs", value={"command": "v1"})], + project_root=tmp_path, + ) + second = resource.sync_patches( + applied_old=first, + desired_new=[Patch(key="mcpServers.fs", value={"command": "v2"})], + project_root=tmp_path, + ) + # Lockfile records the new value's hash, not the value itself. + from agpack.patch import value_hash + + assert second[0].value_hash == value_hash({"command": "v2"}) + cfg = json.loads((tmp_path / ".mcp.json").read_text()) + assert cfg["mcpServers"]["fs"] == {"command": "v2"} + + +class TestTomlPreservation: + """tomlkit retains comments and ordering on untouched sections.""" + + def test_comments_survive_round_trip(self, tmp_path: Path) -> None: + original = ( + "# leading comment\n" + "[mcp_servers]\n" + "# inline comment\n" + 'existing = { command = "old" }\n' + "\n" + "[other]\n" + "# unrelated\n" + 'key = "value"\n' + ) + (tmp_path / "config.toml").write_text(original, encoding="utf-8") + resource = EditFileResource(path="config.toml") + resource.sync_patches( + applied_old=[], + desired_new=[Patch(key="mcp_servers.new", value={"command": "x"})], + project_root=tmp_path, + ) + after = (tmp_path / "config.toml").read_text() + assert "# leading comment" in after + assert "# inline comment" in after + assert "# unrelated" in after + assert "existing" in after + + +class TestIdempotency: + """No-op syncs must not rewrite the file.""" + + def test_resync_identical_state_does_not_touch_file(self, tmp_path: Path) -> None: + (tmp_path / ".mcp.json").write_text("{}", encoding="utf-8") + resource = EditFileResource(path=".mcp.json") + applied = resource.sync_patches( + applied_old=[], + desired_new=[Patch(key="mcpServers.fs", value={"command": "x"})], + project_root=tmp_path, + ) + path = tmp_path / ".mcp.json" + mtime_before = path.stat().st_mtime_ns + # Re-running with the same desired state must produce no write. + with mock_patch( + "agpack.kinds._shared._atomic_write", + side_effect=AssertionError("file was rewritten despite no change"), + ): + resource.sync_patches( + applied_old=applied, + desired_new=[Patch(key="mcpServers.fs", value={"command": "x"})], + project_root=tmp_path, + ) + assert path.stat().st_mtime_ns == mtime_before + + def test_empty_sync_does_not_create_file(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + resource.sync_patches( + applied_old=[], + desired_new=[], + project_root=tmp_path, + ) + assert not (tmp_path / ".mcp.json").exists() + + def test_fresh_clone_with_committed_lockfile_reapplies_replace(self, tmp_path: Path) -> None: + """Lockfile says applied, destination file is missing (e.g. fresh clone of a repo that committed the lock): + sync must reconstruct the file. The "unchanged" diff branch calls ``_apply_patch`` so the entry actually + lands on disk.""" + resource = EditFileResource(path=".mcp.json") + patch = Patch(key="mcpServers.fs", value={"command": "npx"}) + applied_old = [ + AppliedPatch( + file_path=".mcp.json", + key="mcpServers.fs", + strategy="replace", + value_hash=value_hash({"command": "npx"}), + ) + ] + assert not (tmp_path / ".mcp.json").exists() # fresh clone — file absent + result = resource.sync_patches( + applied_old=applied_old, + desired_new=[patch], + project_root=tmp_path, + ) + # The diff sees "unchanged" (same hash on both sides) and carries forward the AppliedPatch. + assert len(result) == 1 + assert result[0].value_hash == applied_old[0].value_hash + # But the file is now actually written with the patch applied. + assert json.loads((tmp_path / ".mcp.json").read_text()) == {"mcpServers": {"fs": {"command": "npx"}}} + + def test_fresh_clone_with_committed_lockfile_reapplies_append(self, tmp_path: Path) -> None: + """Same scenario for append patches: stored entry, file missing, sync recreates the list.""" + resource = EditFileResource(path=".claude/settings.json") + patch = Patch( + key="hooks.PreToolUse", + value={"matcher": "Write", "hooks": [{"type": "command"}]}, + strategy="append", + ) + applied_old = [ + AppliedPatch( + file_path=".claude/settings.json", + key="hooks.PreToolUse", + strategy="append", + value_hash=value_hash(patch.value), + ) + ] + resource.sync_patches( + applied_old=applied_old, + desired_new=[patch], + project_root=tmp_path, + ) + cfg = json.loads((tmp_path / ".claude/settings.json").read_text()) + assert cfg == {"hooks": {"PreToolUse": [{"matcher": "Write", "hooks": [{"type": "command"}]}]}} + + def test_apply_patch_append_idempotent(self) -> None: + """Direct ``_apply_patch`` call with an append whose value already exists in the list is a no-op.""" + root: dict = {"hooks": ["a", "b"]} + _apply_patch(root, Patch(key="hooks", value="b", strategy="append")) + # Hash-match — no duplicate added. + assert root == {"hooks": ["a", "b"]} + + +# --------------------------------------------------------------------------- +# Error paths +# --------------------------------------------------------------------------- + + +class TestCollisionDetection: + """Two patches that resolve to the same identity must be rejected — silent last-write-wins is the bug we ship + this check to avoid. The check lives at apply time because parse time doesn't have target vars yet.""" + + def test_literal_duplicate_replace_rejected(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + with pytest.raises(EditFileError, match=r"patches at indices 0 and 1 resolve to the same identity"): + resource.apply_patches( + [ + Patch(key="mcpServers.fs", value={"command": "v1"}), + Patch(key="mcpServers.fs", value={"command": "v2"}), + ], + tmp_path, + ) + + def test_literal_duplicate_append_rejected(self, tmp_path: Path) -> None: + """Two appends with identical (key, value) — would dedup to one in the diff dict.""" + resource = EditFileResource(path="settings.json") + with pytest.raises(EditFileError, match=r"strategy=append"): + resource.apply_patches( + [ + Patch(key="permissions.allow", value="Read(/etc)", strategy="append"), + Patch(key="permissions.allow", value="Read(/etc)", strategy="append"), + ], + tmp_path, + ) + + def test_resolved_collision_via_substitution_rejected(self, tmp_path: Path) -> None: + """Different unresolved keys that resolve to the same key collide too. This is the case parse time *cannot* + catch — only apply time has the target's ``vars`` to do the substitution.""" + resource = EditFileResource(path=".mcp.json", vars={"bucket": "mcpServers"}) + with pytest.raises( + EditFileError, + match=r"unresolved keys: 'mcpServers\.fs' and '\$\{bucket\}\.fs'", + ): + resource.apply_patches( + [ + Patch(key="mcpServers.fs", value={"command": "v1"}), + Patch(key="${bucket}.fs", value={"command": "v2"}), + ], + tmp_path, + ) + + def test_appends_with_distinct_values_do_not_collide(self, tmp_path: Path) -> None: + """Two appends at the same key with different values are distinct list elements — both must land. + + Order is currently *not* guaranteed because :meth:`EditFileResource.sync_patches` iterates the diff via + ``set.difference``; that's a separate latent issue worth fixing, but it's out of scope here. + """ + resource = EditFileResource(path="settings.json") + resource.apply_patches( + [ + Patch(key="permissions.allow", value="Read(/etc)", strategy="append"), + Patch(key="permissions.allow", value="Write(/tmp)", strategy="append"), + ], + tmp_path, + ) + cfg = _read_json(tmp_path / "settings.json") + assert sorted(cfg["permissions"]["allow"]) == ["Read(/etc)", "Write(/tmp)"] + + +class TestErrors: + def test_top_level_must_be_mapping(self, tmp_path: Path) -> None: + (tmp_path / "settings.json").write_text("[1, 2, 3]", encoding="utf-8") + resource = EditFileResource(path="settings.json") + with pytest.raises(EditFileError, match="top-level must be a mapping"): + resource.apply_patches([Patch(key="x", value=1)], tmp_path) + + def test_corrupt_json_raises_on_apply(self, tmp_path: Path) -> None: + (tmp_path / ".mcp.json").write_text("not json!", encoding="utf-8") + resource = EditFileResource(path=".mcp.json") + with pytest.raises(EditFileError, match="Failed to read"): + resource.apply_patches([Patch(key="x", value=1)], tmp_path) + + def test_oserror_on_write_wrapped(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + with ( + mock_patch("agpack.kinds._shared._atomic_write", side_effect=OSError("disk full")), + pytest.raises(EditFileError, match="Failed to write.*disk full"), + ): + resource.apply_patches([Patch(key="x", value=1)], tmp_path) + + +# --------------------------------------------------------------------------- +# _atomic_write failure cleanup +# --------------------------------------------------------------------------- + + +class TestVariableSubstitution: + """${name} in patch keys and values resolves at apply time, with the target's own ``vars`` taking precedence over + env_vars.""" + + def test_target_var_substituted_in_key(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json", vars={"bucket": "mcpServers"}) + resource.apply_patches( + [Patch(key="${bucket}.fs", value={"command": "npx"})], + tmp_path, + env_vars={}, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg == {"mcpServers": {"fs": {"command": "npx"}}} + + def test_env_var_substituted_in_value(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + resource.apply_patches( + [ + Patch( + key="mcpServers.fs", + value={"env": {"API_KEY": "${API_KEY}"}}, + ) + ], + tmp_path, + env_vars={"API_KEY": "secret"}, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg["mcpServers"]["fs"]["env"]["API_KEY"] == "secret" + + def test_target_var_overrides_env_var(self, tmp_path: Path) -> None: + """Same-name collision: target wins.""" + resource = EditFileResource(path=".mcp.json", vars={"bucket": "from-target"}) + resource.apply_patches( + [Patch(key="${bucket}.fs", value={"command": "x"})], + tmp_path, + env_vars={"bucket": "from-env"}, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert "from-target" in cfg + assert "from-env" not in cfg + + def test_substitutes_recursively_in_value(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json", vars={"bucket": "mcpServers"}) + resource.apply_patches( + [ + Patch( + key="${bucket}.fs", + value={ + "command": "node", + "args": ["--port", "${PORT}"], + "env": {"TOKEN": "${TOKEN}"}, + }, + ) + ], + tmp_path, + env_vars={"PORT": "9090", "TOKEN": "abc"}, + ) + cfg = _read_json(tmp_path / ".mcp.json") + srv = cfg["mcpServers"]["fs"] + assert srv["args"] == ["--port", "9090"] + assert srv["env"]["TOKEN"] == "abc" + + def test_missing_var_raises(self, tmp_path: Path) -> None: + resource = EditFileResource(path=".mcp.json") + with pytest.raises(ConfigError, match="'UNDEFINED' is not defined"): + resource.apply_patches( + [Patch(key="${UNDEFINED}.x", value=1)], + tmp_path, + env_vars={}, + ) + + def test_no_env_vars_argument_defaults_to_empty(self, tmp_path: Path) -> None: + """env_vars=None is equivalent to {}; target vars still resolve.""" + resource = EditFileResource(path=".mcp.json", vars={"bucket": "x"}) + resource.apply_patches( + [Patch(key="${bucket}.fs", value={})], + tmp_path, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg == {"x": {"fs": {}}} + + def test_dollar_dollar_escapes_to_literal_dollar(self, tmp_path: Path) -> None: + """$${X} writes ${X} literally — needed for runtime vars (e.g. Claude Code's ${CLAUDE_PROJECT_DIR} inside hook + commands).""" + resource = EditFileResource(path="settings.json") + resource.apply_patches( + [ + Patch( + key="hooks.PreToolUse", + strategy="append", + value={ + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "$${CLAUDE_PROJECT_DIR}/.claude/block.sh", + } + ], + }, + ), + ], + tmp_path, + env_vars={}, + ) + cfg = _read_json(tmp_path / "settings.json") + cmd = cfg["hooks"]["PreToolUse"][0]["hooks"][0]["command"] + assert cmd == "${CLAUDE_PROJECT_DIR}/.claude/block.sh" + + def test_escape_alongside_substitution(self, tmp_path: Path) -> None: + """$$ and ${} can appear in the same string.""" + resource = EditFileResource(path=".mcp.json", vars={"bucket": "mcpServers"}) + resource.apply_patches( + [ + Patch( + key="${bucket}.fs", + value="literal $${X} and substituted ${SUB}", + ), + ], + tmp_path, + env_vars={"SUB": "OK"}, + ) + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg["mcpServers"]["fs"] == "literal ${X} and substituted OK" + + def test_resolved_patches_returned(self, tmp_path: Path) -> None: + """Returned AppliedPatch stores the **resolved** key plus a hash of the resolved value. + + Values never land in the lockfile — only their SHA256 hash — so interpolated secrets (``${API_KEY}``) stay + out of committed lockfiles. Keys are assumed to be structural, not secret. + """ + from agpack.patch import value_hash + + resource = EditFileResource(path=".mcp.json", vars={"bucket": "mcpServers"}) + applied = resource.apply_patches( + [ + Patch(key="${bucket}.fs", value={"env": {"K": "${K}"}}), + ], + tmp_path, + env_vars={"K": "v"}, + ) + assert len(applied) == 1 + assert applied[0].key == "mcpServers.fs" + assert applied[0].value_hash == value_hash({"env": {"K": "v"}}) + # The file itself has the resolved key and value. + cfg = _read_json(tmp_path / ".mcp.json") + assert cfg["mcpServers"]["fs"]["env"]["K"] == "v" + + +class TestAtomicWriteFailure: + def test_cleans_up_temp_file_on_replace_failure(self, tmp_path: Path) -> None: + target = tmp_path / "output.json" + with ( + mock_patch( + "agpack.kinds._shared.os.replace", + side_effect=OSError("disk full"), + ), + pytest.raises(OSError, match="disk full"), + ): + _atomic_write(target, '{"test": true}\n') + leftover = list(tmp_path.glob(".agpack-edit-*")) + assert leftover == [] + assert not target.exists() diff --git a/tests/test_registry.py b/tests/test_registry.py new file mode 100644 index 0000000..62dd3cb --- /dev/null +++ b/tests/test_registry.py @@ -0,0 +1,45 @@ +"""Tests for the built-in target registry.""" + +from __future__ import annotations + +import pytest + +from agpack.registry import list_builtins +from agpack.registry import load_all_builtins +from agpack.registry import load_builtin +from agpack.target_schema import TargetSchemaError + +EXPECTED_BUILTINS = { + "antigravity", + "claude", + "codex", + "copilot", + "cursor", + "gemini", + "opencode", + "windsurf", +} + + +def test_list_builtins_returns_all_shipped_targets() -> None: + assert set(list_builtins()) == EXPECTED_BUILTINS + + +def test_list_builtins_is_sorted() -> None: + names = list_builtins() + assert names == sorted(names) + + +@pytest.mark.parametrize("name", sorted(EXPECTED_BUILTINS)) +def test_each_builtin_loads(name: str) -> None: + # Just verify the manifest parses successfully. + load_builtin(name) + + +def test_load_builtin_unknown_raises() -> None: + with pytest.raises(TargetSchemaError, match="No built-in target"): + load_builtin("nonexistent") + + +def test_load_all_builtins_returns_full_map() -> None: + assert set(load_all_builtins()) == EXPECTED_BUILTINS diff --git a/tests/test_target_schema.py b/tests/test_target_schema.py new file mode 100644 index 0000000..415490a --- /dev/null +++ b/tests/test_target_schema.py @@ -0,0 +1,154 @@ +"""Tests for the target manifest parser and validator.""" + +from __future__ import annotations + +import pytest + +from agpack.kinds import CopyDirectoryResource +from agpack.kinds import CopyFileResource +from agpack.kinds import EditFileResource +from agpack.target_schema import TargetDef +from agpack.target_schema import TargetSchemaError +from agpack.target_schema import parse_target_def + + +def test_minimal_target_parses() -> None: + target = parse_target_def({}) + assert target.resources == {} + + +def test_full_target_parses() -> None: + raw = { + "skills": {"kind": "copy-directory", "path": ".demo/skills"}, + "commands": {"kind": "copy-file", "path": ".demo/commands"}, + "agents": {"kind": "copy-file", "path": ".demo/agents"}, + "mcp": {"kind": "edit-file", "path": ".demo/mcp.json"}, + "settings": {"kind": "edit-file", "path": ".demo/settings.json"}, + } + + target = parse_target_def(raw) + + assert isinstance(target, TargetDef) + assert target.resources["skills"] == CopyDirectoryResource(path=".demo/skills") + assert target.resources["commands"] == CopyFileResource(path=".demo/commands") + assert target.resources["agents"] == CopyFileResource(path=".demo/agents") + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.path == ".demo/mcp.json" + assert mcp.format == "json" + + +def test_non_mapping_raises() -> None: + with pytest.raises(TargetSchemaError, match="expected a mapping"): + parse_target_def("not a dict") + + +def test_arbitrary_resource_type_parses() -> None: + """Resource type names are open — any name with a valid kind works.""" + target = parse_target_def({"rules": {"kind": "copy-file", "path": ".mytool/rules"}}) + assert target.resources["rules"] == CopyFileResource(path=".mytool/rules") + + +def test_non_mapping_resource_value_raises() -> None: + with pytest.raises(TargetSchemaError, match="expected a mapping"): + parse_target_def({"bogus": 1}) + + +def test_empty_string_resource_key_raises() -> None: + with pytest.raises(TargetSchemaError, match="keys must be non-empty"): + parse_target_def({"": {"kind": "copy-file", "path": ".x"}}) + + +def test_missing_kind_raises() -> None: + with pytest.raises(TargetSchemaError, match="kind"): + parse_target_def({"skills": {"path": ".x"}}) + + +def test_invalid_kind_raises() -> None: + with pytest.raises(TargetSchemaError, match="kind"): + parse_target_def({"skills": {"kind": "weird", "path": ".x"}}) + + +def test_legacy_layout_field_is_rejected() -> None: + """The old 'layout: directory|file' form must point users at 'kind:'.""" + with pytest.raises(TargetSchemaError, match="deprecated.*kind"): + parse_target_def({"skills": {"layout": "directory", "path": ".x"}}) + + +def test_legacy_merge_field_is_rejected() -> None: + """The old 'merge:' block on edit-file resources is rejected with a hint.""" + with pytest.raises(TargetSchemaError, match="removed.*Patches"): + parse_target_def( + { + "mcp": { + "kind": "edit-file", + "path": ".x.json", + "merge": {"servers_key": "mcpServers"}, + } + } + ) + + +def test_missing_resource_path_raises() -> None: + with pytest.raises(TargetSchemaError, match="path"): + parse_target_def({"skills": {"kind": "copy-file"}}) + + +def test_resource_unknown_field_raises() -> None: + with pytest.raises(TargetSchemaError, match="unknown keys"): + parse_target_def({"skills": {"kind": "copy-file", "path": ".x", "extra": 1}}) + + +def test_edit_file_unknown_path_extension_raises() -> None: + """Path without .json/.toml suffix can't infer format.""" + with pytest.raises(TargetSchemaError, match="cannot infer"): + parse_target_def({"mcp": {"kind": "edit-file", "path": ".my-mcp-config"}}) + + +def test_edit_file_with_vars_parses() -> None: + target = parse_target_def( + { + "mcp": { + "kind": "edit-file", + "path": ".mcp.json", + "vars": {"bucket": "mcpServers"}, + } + } + ) + mcp = target.resources["mcp"] + assert isinstance(mcp, EditFileResource) + assert mcp.vars == {"bucket": "mcpServers"} + + +def test_edit_file_vars_must_be_mapping() -> None: + with pytest.raises(TargetSchemaError, match="vars: must be a mapping"): + parse_target_def( + { + "mcp": { + "kind": "edit-file", + "path": ".mcp.json", + "vars": ["not", "a", "mapping"], + } + } + ) + + +def test_edit_file_var_value_must_be_string() -> None: + with pytest.raises(TargetSchemaError, match="value must be a string"): + parse_target_def( + { + "mcp": { + "kind": "edit-file", + "path": ".mcp.json", + "vars": {"bucket": 42}, + } + } + ) + + +def test_context_appears_in_error() -> None: + with pytest.raises(TargetSchemaError, match="my-source.skills.kind"): + parse_target_def( + {"skills": {"kind": "weird", "path": ".x"}}, + context="my-source", + ) diff --git a/uv.lock b/uv.lock index 99423e1..8b0385d 100644 --- a/uv.lock +++ b/uv.lock @@ -9,7 +9,7 @@ dependencies = [ { name = "click" }, { name = "pyyaml" }, { name = "rich" }, - { name = "tomli-w" }, + { name = "tomlkit" }, ] [package.dev-dependencies] @@ -27,7 +27,7 @@ requires-dist = [ { name = "click", specifier = ">=8.0" }, { name = "pyyaml", specifier = ">=6.0" }, { name = "rich", specifier = ">=13.0" }, - { name = "tomli-w", specifier = ">=1.0" }, + { name = "tomlkit", specifier = ">=0.13" }, ] [package.metadata.requires-dev] @@ -660,12 +660,12 @@ wheels = [ ] [[package]] -name = "tomli-w" -version = "1.2.0" +name = "tomlkit" +version = "0.15.0" source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/19/75/241269d1da26b624c0d5e110e8149093c759b7a286138f4efd61a60e75fe/tomli_w-1.2.0.tar.gz", hash = "sha256:2dd14fac5a47c27be9cd4c976af5a12d87fb1f0b4512f81d69cce3b35ae25021", size = 7184, upload-time = "2025-01-15T12:07:24.262Z" } +sdist = { url = "https://files.pythonhosted.org/packages/51/db/03eaf4331631ef6b27d6e3c9b68c54dc6f0d63d87201fed600cc409307fd/tomlkit-0.15.0.tar.gz", hash = "sha256:7d1a9ecba3086638211b13814ea79c90dd54dd11993564376f3aa92271f5c7a3", size = 161875, upload-time = "2026-05-10T07:38:22.245Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/c7/18/c86eb8e0202e32dd3df50d43d7ff9854f8e0603945ff398974c1d91ac1ef/tomli_w-1.2.0-py3-none-any.whl", hash = "sha256:188306098d013b691fcadc011abd66727d3c414c571bb01b1a174ba8c983cf90", size = 6675, upload-time = "2025-01-15T12:07:22.074Z" }, + { url = "https://files.pythonhosted.org/packages/6a/43/8bd850ee71a191bf072e31302c73a66be413fecdd98fdcd111ecbcce13ca/tomlkit-0.15.0-py3-none-any.whl", hash = "sha256:4dbc8f0fc024412b57ced8757ac7461305126a648ff8c2c807fcb8e133a78738", size = 41328, upload-time = "2026-05-10T07:38:23.517Z" }, ] [[package]]