Add EventLogURI to commandContext and update Spark event log directory handling

[sanketjadhavSF]

Description:
Currently we put all spark job logs inside separate directories for respective {job_id} due to this Spark History Server does not able to read event logs and it requires all the logs consolidated into single directory in order to render all the spark applications.

This pull request updates how the Spark event log directory is configured for Spark History Server in the sparkeks command implementation. The main change is to use the new EventLogURI field in the commandContext struct instead of the previous logURI field.

Configuration improvements:

• Added EventLogURI field to the commandContext struct to explicitly specify the event log URI for Spark jobs.
• Updated logic in applySparkOperatorConfig to use commandContext.EventLogURI for setting the Spark event log directory, replacing the previous use of logURI.

[Copilot]

Pull request overview

Updates Spark event log directory configuration in the sparkeks command so Spark History Server can read consolidated event logs, by introducing an explicit EventLogURI in commandContext.

Changes:

• Added EventLogURI to commandContext for explicitly configuring the Spark event log base URI.
• Updated applySparkOperatorConfig to set spark.eventLog.dir from commandContext.EventLogURI instead of execCtx.logURI.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.