Skip to content

feat: Spark major version upgrade skill#113

Open
csmith49 wants to merge 4 commits intomainfrom
fix/spark-migration
Open

feat: Spark major version upgrade skill#113
csmith49 wants to merge 4 commits intomainfrom
fix/spark-migration

Conversation

@csmith49
Copy link
Contributor

This PR adds a skill (registered to the large-codebase marketplace) that walks an agent through the process of upgrading major versions of Spark.

The skill queries Apache's migration guide, identifies the necessary changes, and ensures the project builds and all tests pass before completing.

csmith49 and others added 3 commits March 17, 2026 12:20
…tration

- SKILL.md: 6-phase upgrade workflow (inventory, build, API, config, SQL, test)
  covering Spark 2.x→3.x and 3.x→4.x with actionable checklists
- README.md: human-readable overview with trigger keywords and examples
- references/api-changes.md: removed/deprecated API catalog with before/after code
- references/config-changes.md: config property rename/removal mapping
- Registered in marketplaces/large-codebase.json

Co-authored-by: openhands <openhands@all-hands.dev>
Replace references/api-changes.md and references/config-changes.md with
the canonical upstream URL:
https://spark.apache.org/docs/latest/migration-guide.html

Co-authored-by: openhands <openhands@all-hands.dev>
The guide is most relevant during inventory — read it before making changes.

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Contributor

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Acceptable - Comprehensive skill for a real problem. Verify marketplace registration to avoid CI failures.

"evaluation",
"reporting"
]
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Important: Per AGENTS.md CI guidance, "every directory under skills/ should be listed in marketplaces/default.json." This PR only adds the skill to large-codebase.json. Verify whether it also needs to be in default.json to prevent CI failures, or if specialized marketplace registration is sufficient.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guidance is out-dated -- skills must be in some marketplace. I've updated the AGENTS.md file accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants