Skip to content

Drop generating index.html generation#5171

Open
marbre wants to merge 1 commit into
mainfrom
users/marbre/native-linux-drop-index-uploading
Open

Drop generating index.html generation#5171
marbre wants to merge 1 commit into
mainfrom
users/marbre/native-linux-drop-index-uploading

Conversation

@marbre
Copy link
Copy Markdown
Member

@marbre marbre commented May 11, 2026

index.html files are generated server-side, making local generation superfluous.

`index.html` files are generated server-side, making local generation
superfluous.
)

# Skip index.html generation for run_id is not passed. (handled by Lambda function)
if not args.run_id:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if check i added last day to issue you saw in https://therock-nightly-artifacts.s3.amazonaws.com/25355852546-linux/index.html.
Now the newer builds are showing fine. Could you confirm?

https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/index.html

If we remove the index completely the single arch (release_portable_linux ) workflow may not have index files created. The current check is making sure the index is created only for single arch workflow.

Copy link
Copy Markdown
Member Author

@marbre marbre May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This if check i added last day to issue you saw in https://therock-nightly-artifacts.s3.amazonaws.com/25355852546-linux/index.html. Now the newer builds are showing fine. Could you confirm?

https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/index.html

If we remove the index completely the single arch (release_portable_linux ) workflow may not have index files created. The current check is making sure the index is created only for single arch workflow.

I think they always should as indexing server side doesn't distinguish between multi-arch or single arch. The index you link above, https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/index.html, does indeed not list the packages directory but it the subdir is indexed, see https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/packages/index.html.

If https://therock-nightly-artifacts.s3.amazonaws.com/25355852546-linux/index.html is the index created by the script this patch tries to modify I know at least understand why it looks so broken ;) Server-side indexing might have a minor bug with not listing packages but creating it for single-arch only like in this script is the wrong fix and makes debugging even harder.

Actually, I am not even sure we mix things up. This script should only index the packages buckets. The artifacts buckets are index by a different script. If this script is also trying to index the artifacts bucket this is then indeed breaking some things and is purely wrong.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the current scenario. ci and multi arch builds.
native packages are created -> uploaded to therock--artifacts -> promoted to therock--packages bucket. In this flow the index files are getting created by server side lamda fn in therock--artifacts. The new check i added will make sure build_tools/packaging/linux/upload_package_repo.py is not invoked in this flow at build time.

In single multi arch following is the flow.
native packages are created -> uploaded to therock--packages bucket directly. In this flow the index files are getting created by build_tools/packaging/linux/upload_package_repo.py

So if we remove the indexing completely from upload_package_repo.py, the single arch builds will be affected. May be we can remove the file once we stop creating single arch builds.

Copy link
Copy Markdown
Member Author

@marbre marbre May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have one script to index therock-*-artifacts. We have another script to index therock-*-packages. The server-side indexing doesn't care if a single-family for multi-arch job uploads to a bucket. So we should be able to safely drop this. @araravik-psd, please confirm or correct me if that isn't the case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: TODO

Development

Successfully merging this pull request may close these issues.

2 participants