Drop generating index.html generation#5171
Conversation
`index.html` files are generated server-side, making local generation superfluous.
| ) | ||
|
|
||
| # Skip index.html generation for run_id is not passed. (handled by Lambda function) | ||
| if not args.run_id: |
There was a problem hiding this comment.
This if check i added last day to issue you saw in https://therock-nightly-artifacts.s3.amazonaws.com/25355852546-linux/index.html.
Now the newer builds are showing fine. Could you confirm?
https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/index.html
If we remove the index completely the single arch (release_portable_linux ) workflow may not have index files created. The current check is making sure the index is created only for single arch workflow.
There was a problem hiding this comment.
This if check i added last day to issue you saw in https://therock-nightly-artifacts.s3.amazonaws.com/25355852546-linux/index.html. Now the newer builds are showing fine. Could you confirm?
https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/index.html
If we remove the index completely the single arch (release_portable_linux ) workflow may not have index files created. The current check is making sure the index is created only for single arch workflow.
I think they always should as indexing server side doesn't distinguish between multi-arch or single arch. The index you link above, https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/index.html, does indeed not list the packages directory but it the subdir is indexed, see https://therock-nightly-artifacts.s3.amazonaws.com/25648442805-linux/packages/index.html.
If https://therock-nightly-artifacts.s3.amazonaws.com/25355852546-linux/index.html is the index created by the script this patch tries to modify I know at least understand why it looks so broken ;) Server-side indexing might have a minor bug with not listing packages but creating it for single-arch only like in this script is the wrong fix and makes debugging even harder.
Actually, I am not even sure we mix things up. This script should only index the packages buckets. The artifacts buckets are index by a different script. If this script is also trying to index the artifacts bucket this is then indeed breaking some things and is purely wrong.
There was a problem hiding this comment.
This is the current scenario. ci and multi arch builds.
native packages are created -> uploaded to therock--artifacts -> promoted to therock--packages bucket. In this flow the index files are getting created by server side lamda fn in therock--artifacts. The new check i added will make sure build_tools/packaging/linux/upload_package_repo.py is not invoked in this flow at build time.
In single multi arch following is the flow.
native packages are created -> uploaded to therock--packages bucket directly. In this flow the index files are getting created by build_tools/packaging/linux/upload_package_repo.py
So if we remove the indexing completely from upload_package_repo.py, the single arch builds will be affected. May be we can remove the file once we stop creating single arch builds.
There was a problem hiding this comment.
We have one script to index therock-*-artifacts. We have another script to index therock-*-packages. The server-side indexing doesn't care if a single-family for multi-arch job uploads to a bucket. So we should be able to safely drop this. @araravik-psd, please confirm or correct me if that isn't the case.
index.htmlfiles are generated server-side, making local generation superfluous.