Conversation
|
augment review |
🤖 Augment PR SummarySummary: This PR adds RPM generation and publishing alongside the existing Changes:
Technical Notes: RPM building is performed through 🤖 Was this summary useful? React with 👍 or 👎 |
| "-gp" | "--generate-pax") | ||
| generatePax=true | ||
| ;; | ||
| "-gr" | "--generate-rpm") |
There was a problem hiding this comment.
--generate-rpm currently only flips generateRPM=true, but the RPM creation logic is nested under the pax-generation flow, so using --generate-rpm by itself appears to do nothing (and won’t hit the “ensure --generate-pax” error). Consider either enforcing the pairing or making the behavior explicit to users.
Severity: medium
Other Locations
bin/zopen-build:2222
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-build
Outdated
| if ${generateRPM}; then | ||
| if [ -f "${paxFileName}" ]; then | ||
| printHeader "Generating RPM from ${ZOPEN_INSTALL_DIR}" | ||
| rpm_deps=$(echo "${ZOPEN_RUNTIME_DEPS}" | xargs -n1 | sort -u | xargs) |
There was a problem hiding this comment.
cicd/publish.groovy
Outdated
|
|
||
| # PAX file should be a copied artifact | ||
| PAX=`find . -type f -path "*install/*zos.pax.Z"` | ||
| RPM=`find rpmbuild/RPMS -type f -name "*.rpm"` |
There was a problem hiding this comment.
RPM is populated via find, which can return 0 or multiple matches; in the multi-match case, [ -f "$RPM" ], basename, and github-release --file "$RPM" will fail or pick an unintended path.
Severity: high
Other Locations
cicd/publish.groovy:37cicd/publish.groovy:66cicd/publish.groovy:210
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
| if [[ "$source_file" == *.pax.Z ]]; then | ||
| cat >> "$output_file" << 'EOF' | ||
| # Extract compressed pax archive | ||
| pax -rzf %{SOURCE0} |
There was a problem hiding this comment.
The generated spec runs pax -r... %{SOURCE0} after cd into %{_builddir}/%{name}-%{version}; %{SOURCE0} is typically located in %{_sourcedir}, so extraction may fail at build time.
Severity: high
Other Locations
bin/zopen-pax2rpm:338
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
|
|
||
| if [ -n "$FOUND_PATH" ]; then | ||
| FOUND_PATH=${FOUND_PATH#./} | ||
| if [[ "$FOUND_PATH" == */pyz ]]; then |
There was a problem hiding this comment.
bin/zopen-pax2rpm
Outdated
|
|
||
| # Run rpmlint | ||
| local rpmlint_output | ||
| rpmlint_output=$(rpmlint "$spec_file" 2>&1) |
There was a problem hiding this comment.
| with the .rej extension. | ||
| -g, --get-source get the source and apply patch without building. | ||
| -gp, --generate-pax generate a pax.Z file based on the install contents. | ||
| -gr, --generate-rpm generate an RPM package from the pax archive. |
There was a problem hiding this comment.
This PR also adds a top-level zopen-build file that appears as a binary/empty file in the checkout; if it’s intended as an entrypoint wrapper, it may be an accidentally committed artifact or missing content.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
|
|
||
| mkdir -p "$buildroot"/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS} | ||
|
|
||
| if [ ! -f "$HOME/.rpmmacros" ]; then |
There was a problem hiding this comment.
If the user already has ~/.rpmmacros, rpmbuild may still use an existing %_topdir and ignore the script’s chosen --buildroot, which can make later steps (like finding artifacts under $buildroot/RPMS) inconsistent or fail unexpectedly. Consider ensuring the requested build root is honored even when ~/.rpmmacros already exists (or at least warning the user).
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
cicd/publish.groovy
Outdated
| PAX=`find . -type f -path "*install/*zos.pax.Z"` | ||
| METADATA=`find . -type f -path "*install/metadata.json"` | ||
| PAX=`find . -type f -path "*install/*zos.pax.Z" | head -n 1` | ||
| RPM=`find rpmbuild/RPMS -type f -name "*.rpm" | head -n 1` |
There was a problem hiding this comment.
find ... | head -n 1 will silently pick an arbitrary RPM if more than one is present (e.g., multiple output RPMs/arches), which can cause CI to upload the wrong asset without failing. Consider asserting there’s exactly one match or selecting deterministically.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
bin/zopen-pax2rpm
Outdated
| # This will check syntax without actually building | ||
| # Use --nodeps to check syntax/prep logic without requiring local RPM DB to have all deps | ||
| local temp_output=$(mktemp) | ||
| if rpmbuild --define "_topdir $buildroot" -bp --nodeps "$spec_file" &> "$temp_output"; then |
There was a problem hiding this comment.
validate_spec_syntax calls rpmbuild --define "_topdir $buildroot" ..., but $buildroot isn’t set in this function (the script otherwise uses BUILDROOT/function parameters). This makes --validate potentially run against the wrong topdir (or an empty one) and fail unexpectedly.
Severity: medium
Other Locations
bin/zopen-pax2rpm:657
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| local date=$(date "+%a %b %d %Y") | ||
|
|
||
| # Analyze the pax contents first (populate global DIR_STRUCTURE) | ||
| analyze_pax_contents "$source_file" |
There was a problem hiding this comment.
generate_spec calls analyze_pax_contents unguarded, but analyze_pax_contents can return 1 on an empty/failed listing; with set -e this will abort the whole script. That seems to contradict the “using default structure” warning path and can prevent spec generation in edge cases.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
bin/zopen-build
Outdated
| if [ -f "${paxFileName}" ]; then | ||
| printHeader "Generating RPM from ${ZOPEN_INSTALL_DIR}" | ||
| rpm_deps=$(echo "${ZOPEN_RUNTIME_DEPS}" | xargs -n1 | sort -u | xargs) | ||
| PATH="${ZOPEN_ROOTFS}/usr/local/bin:${PATH}" "${MYDIR}/zopen-pax2rpm" "${paxFileName}" --summary "${ZOPEN_NAME} package" --requires "${rpm_deps}" --build |
There was a problem hiding this comment.
zopen-pax2rpm defaults --buildroot to ${HOME}/rpmbuild, but cicd/publish.groovy later searches for RPMs under ./rpmbuild/RPMS, so the publish step may not find the built RPM depending on HOME. Consider aligning the output location between build and publish (e.g., by explicitly setting the buildroot used here).
Severity: high
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
| mkdir -p "$buildroot"/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS} | ||
|
|
||
| if [ ! -f "$HOME/.rpmmacros" ]; then | ||
| cat > "$HOME/.rpmmacros" << EOF |
There was a problem hiding this comment.
setup_rpmbuild writes ~/.rpmmacros, which can fail in CI where $HOME is read-only and can also have surprising side effects for users. Since the script already passes --define "_topdir ...", consider making the ~/.rpmmacros write best-effort (or avoiding it) so builds don't fail unnecessarily.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
| # Function to analyze pax contents and categorize files | ||
| analyze_pax_contents() { | ||
| local pax_file="$1" | ||
| local temp_dir=$(mktemp -d) |
There was a problem hiding this comment.
analyze_pax_contents creates temp_dir via mktemp -d but never uses it, and with set -e a missing/failed mktemp would abort the entire run. Consider removing this dependency or guarding it so content analysis can't hard-fail for an unused temp directory.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
cicd/publish.groovy
Outdated
| exit 1 | ||
| fi | ||
|
|
||
| RPM=`find rpmbuild/RPMS -type f -name "*.rpm"` |
There was a problem hiding this comment.
Publish now hard-fails unless exactly one RPM exists under rpmbuild/RPMS; this will break publish runs for ports/builds that still only produce a pax artifact (or produce multiple RPM subpackages). Consider confirming the Port-Build job always generates exactly one RPM before enforcing this check.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| with the .rej extension. | ||
| -g, --get-source get the source and apply patch without building. | ||
| -gp, --generate-pax generate a pax.Z file based on the install contents. | ||
| -gr, --generate-rpm generate an RPM package from the pax archive. |
There was a problem hiding this comment.
|
augment review |
| # Parse command line options | ||
| while [ $# -gt 0 ]; do | ||
| case "$1" in | ||
| --name) |
There was a problem hiding this comment.
Option parsing assumes every flag has a value (e.g., --name), and with set -e a missing $2 will cause a confusing shift error instead of a clear message. This can make CLI failures hard to diagnose for users/CI.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
|
|
||
| # Check for rpmbuild if building or validating | ||
| if [ "$BUILD_RPM" = true ] || [ "$VALIDATE_SPEC" = true ]; then | ||
| echo "DEBUG: Current PATH is $PATH" >&2 |
There was a problem hiding this comment.
check_required_tools always prints DEBUG: Current PATH... whenever --build or --validate is set, even when --verbose isn’t enabled. This can clutter normal output and makes “debug” logging hard to interpret.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
bin/zopen-pax2rpm
Outdated
| echo "# Pax file contents (first 100 entries):" >&2 | ||
|
|
||
| if [[ "$pax_file" == *.pax.Z ]]; then | ||
| pax -rvzf "$pax_file" 2>&1 | head -100 |
There was a problem hiding this comment.
bin/zopen-pax2rpm:213: list_pax_contents uses pax -r..., which extracts files into the current working directory (and piping to head can interrupt extraction), even though the function says it’s listing contents. Consider switching this helper to pax’s list mode so it can’t mutate the caller’s workspace.
Severity: high
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-build
Outdated
| if [ -f "${paxFileName}" ]; then | ||
| printHeader "Generating RPM from ${ZOPEN_INSTALL_DIR}" | ||
| rpm_deps=$(echo "${ZOPEN_RUNTIME_DEPS}" | xargs -n1 | sort -u | xargs) | ||
| if ! runAndLog "PATH=\"${ZOPEN_ROOTFS}/usr/local/bin:${PATH}\" \"${MYDIR}/zopen-pax2rpm\" \"${paxFileName}\" --summary \"${ZOPEN_NAME} package\" --requires \"${rpm_deps}\" --build --buildroot \"${ZOPEN_ROOT}/rpmbuild\""; then |
There was a problem hiding this comment.
bin/zopen-build:2229: This always passes --requires "${rpm_deps}"; if ZOPEN_RUNTIME_DEPS is empty/unset in some build contexts, that expands to an empty value and zopen-pax2rpm will fail option parsing (--requires demands a non-empty argument). Consider only including --requires when there’s at least one dependency to emit.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
bin/zopen-pax2rpm
Outdated
| --url "https://www.ibm.com" | ||
|
|
||
| EOF | ||
| exit 1 |
bin/zopen-pax2rpm
Outdated
| fi | ||
|
|
||
| # Check for uncompress if dealing with .Z files | ||
| if [[ "$pax_file" == *.pax.Z ]] && ! command -v uncompress &> /dev/null; then |
There was a problem hiding this comment.
check_required_tools() hard-requires uncompress for *.pax.Z, but this script never invokes uncompress directly and relies on pax -z for (de)compression. This can cause a false “missing tool” failure on systems where pax handles the archive without uncompress (or uses gzip instead).
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
| RELEASE="1" | ||
| LICENSE="Proprietary" | ||
| BUILD_ARCH=$(uname -m 2>/dev/null || echo "s390x") | ||
| PACKAGER_NAME="${USER}" |
There was a problem hiding this comment.
PACKAGER_NAME/PACKAGER_EMAIL are derived from $USER, which is often unset in non-interactive CI environments. If these end up empty, the generated spec’s %changelog header line may be malformed and could trigger rpmbuild/rpmlint warnings or failures.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
| ME=$(basename "$0") | ||
| MYDIR="$( cd "$(dirname "$0")" >/dev/null 2>&1 && pwd -P )" | ||
| INCDIR="${MYDIR}/../include" | ||
| if ! [ -d "${INCDIR}" ] && ! [ -f "${INCDIR}/common.sh" ]; then |
There was a problem hiding this comment.
bin/zopen-pax2rpm:30: The setupMyself guard only errors when both the include dir is missing and common.sh is missing (&&), so if the directory exists but common.sh is absent/mispackaged the script will fall through and fail later at the . with a less clear error.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| if [ -n "${rpm_deps}" ]; then | ||
| cmd="${cmd} --requires \"${rpm_deps}\"" | ||
| fi | ||
| if ! runAndLog "${cmd}"; then |
There was a problem hiding this comment.
bin/zopen-build:2233: runAndLog executes via eval, so building cmd from variables like paxFileName/ZOPEN_NAME means any unexpected shell metacharacters or quotes in those values can break the command line or be interpreted as additional shell syntax.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
1f61ea5 to
34d084a
Compare
|
augment review |
| fi | ||
| echo "Detected nested root: $ROOT_DIR" | ||
| mkdir -p %{buildroot}%{install_dir} | ||
| cp -RP "$ROOT_DIR"/* %{buildroot}%{install_dir}/ |
There was a problem hiding this comment.
bin/zopen-pax2rpm:279 The generated %install snippet copies with "$ROOT_DIR"/* (and later "$SUB_DIR"/* / *), which will skip dotfiles like .env/.depsenv and can also fail when a directory contains only dotfiles (glob expands to a literal *). This risks producing RPMs missing critical runtime files or causing build-time copy failures.
Other locations where this applies: bin/zopen-pax2rpm:285, bin/zopen-pax2rpm:290
Severity: high
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
| generate_files_list() { | ||
| cat << 'EOF' | ||
| %defattr(-,root,root,-) | ||
| %{install_dir} |
There was a problem hiding this comment.
bin/zopen-pax2rpm:308 The generated %files list only contains %{install_dir}, which typically won’t include the directory contents; rpmbuild commonly errors with “Installed (but unpackaged) file(s) found” (or produces an RPM with an empty payload). Consider ensuring the %files section captures the installed tree under %{buildroot}%{install_dir}.
Severity: high
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
| # Default values | ||
| RELEASE="1" | ||
| LICENSE="Proprietary" | ||
| BUILD_ARCH=$(uname -m 2>/dev/null || echo "s390x") |
There was a problem hiding this comment.
bin/zopen-pax2rpm:41 BUILD_ARCH is taken directly from uname -m, but some environments return values that RPM may not recognize as valid arches (which can make BuildArch: invalid or mislabel the artifact). Consider normalizing/mapping the output to RPM architecture names expected by your target build environment.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
|
augment review |
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
16e647f to
703956e
Compare
bin/zopen-pax2rpm
Outdated
| is_os_layout=false | ||
|
|
||
| # Analyze each file | ||
| echo "$pax_listing" | while IFS= read -r line; do |
There was a problem hiding this comment.
analyze_pax_contents sets has_* flags inside a while loop that’s fed by a pipeline (echo ... | while ...), which runs in a subshell under POSIX sh; those assignments won’t persist, so the later "Detected structure" output will likely always be empty/misleading.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
|
augment review |
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
| if ${generateRPM}; then | ||
| if [ -f "${paxFileName}" ]; then | ||
| printHeader "Generating RPM from ${ZOPEN_INSTALL_DIR}" | ||
| rpm_deps=$(echo "${ZOPEN_RUNTIME_DEPS}" | xargs -n1 | sort -u | xargs) |
There was a problem hiding this comment.
rpm_deps is normalized via xargs -n1 | sort -u, which splits dependency expressions on whitespace; this will break versioned deps like oef >= 1.1.0 into separate tokens and can produce an invalid --requires/Requires: value in the generated spec.
Severity: medium
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
bin/zopen-pax2rpm
Outdated
| fi | ||
| else | ||
| # Check if _topdir is already defined and different | ||
| existing_topdir=$(grep "%_topdir" "$HOME/.rpmmacros" | awk '{print $2}') |
There was a problem hiding this comment.
In setup_rpmbuild, if ~/.rpmmacros exists but doesn’t define %_topdir, existing_topdir becomes empty and the script still emits a warning claiming _topdir is set to '', which is misleading and can confuse debugging.
Severity: low
🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
|
augment review |
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
Signed-off-by: TejaswiniIBM <tejaswini.r1@ibm.com>
What type of PR is this? (check all applicable)
Category
Description
Added support to generate and publish RPM
Related Issues
[optional] Are there any post-deployment tasks or follow-up actions required?