-
Notifications
You must be signed in to change notification settings - Fork 358
Feature/unified multiarch #752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tdamir
wants to merge
14
commits into
mostlygeek:main
Choose a base branch
from
tdamir:feature/unified-multiarch
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
0d9d6c1
docker: update Dockerfile and build scripts for multi-architecture su…
tdamir 83788fa
ci: enhance multi-architecture support in Docker workflow
tdamir cd077a0
ci: add conditional for multi-arch manifest creation in workflow
tdamir 2fedd31
Consolidate the platform→runner/arch mapping via matrix.include
tdamir a72f2cd
ci: limit parallel jobs to 2 in Docker build workflow
tdamir b4b45f8
ci: update unified Docker workflow for multi-architecture support and…
tdamir c7fd01e
ci: simplify conditions for merge-manifests job and enhance CUDA arch…
tdamir d7a420b
ci: remove max-parallel limit from build job strategy in unified Dock…
tdamir 52ccb19
ci: remove platform inclusion for multi-architecture in Docker workflow
tdamir 063e6f6
Add cuda13.
tdamir 95de083
ci: update backend condition to support cuda13 in installation scripts
tdamir 9dc0a6c
Remove newer architectures from CUDA 12
tdamir 5538f1d
Merge branch 'mostlygeek:main' into feature/unified-multiarch
tdamir ea87c5a
Merge branch 'mostlygeek:main' into feature/unified-multiarch
tdamir File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| #!/bin/bash | ||
| # | ||
| # Build script for unified container with version pinning | ||
| # | ||
| # Usage: | ||
| # ./build-image-rootless.sh --cuda # Build CUDA image | ||
| # ./build-image-rootless.sh --vulkan # Build Vulkan image | ||
| # ./build-image-rootless.sh --cuda --no-cache # Build without cache | ||
| # LLAMA_REF=b1234 ./build-image-rootless.sh --vulkan # Pin llama.cpp to a commit hash | ||
| # LLAMA_REF=v1.2.3 ./build-image-rootless.sh --cuda # Pin llama.cpp to a tag | ||
| # WHISPER_REF=v1.0.0 ./build-image-rootless.sh --vulkan # Pin whisper.cpp to a tag | ||
| # SD_REF=master ./build-image-rootless.sh --cuda # Pin stable-diffusion.cpp to a branch | ||
| # LS_VERSION=170 ./build-image-rootless.sh --cuda # Override llama-swap version | ||
| # IK_LLAMA_REF=main ./build-image-rootless.sh --cuda # Pin ik_llama.cpp to main branch (CUDA only) | ||
| # | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| BACKEND="" | ||
| NO_CACHE=false | ||
|
|
||
| for arg in "$@"; do | ||
| case $arg in | ||
| --cuda) | ||
| BACKEND="cuda" | ||
| ;; | ||
| --cuda13) | ||
| BACKEND="cuda13" | ||
| ;; | ||
| --vulkan) | ||
| BACKEND="vulkan" | ||
| ;; | ||
| --no-cache) | ||
| NO_CACHE=true | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
| ;; | ||
| --help|-h) | ||
| echo "Usage: ./build-image-rootless.sh --cuda|--vulkan" | ||
| echo "" | ||
| echo "Options:" | ||
| echo " --cuda Build CUDA image (NVIDIA GPUs)" | ||
| echo " --cuda13 Build CUDA 13 image (NVIDIA GPUs)" | ||
| echo " --vulkan Build Vulkan image (AMD GPUs and compatible hardware)" | ||
| echo " --no-cache Force rebuild without using Docker cache" | ||
| echo " --help, -h Show this help message" | ||
| echo "" | ||
| echo "Environment variables:" | ||
| echo " DOCKER_IMAGE_TAG Set custom image tag (default: llama-swap:unified-cuda or llama-swap:unified-vulkan)" | ||
| exit 0 | ||
| ;; | ||
| esac | ||
| done | ||
|
|
||
| if [[ -z "$BACKEND" ]]; then | ||
| echo "Error: No backend specified. Please use --cuda, --cuda13, or --vulkan." | ||
| echo "" | ||
| echo "Usage: ./build-image-rootless.sh --cuda|--cuda13|--vulkan" | ||
| exit 1 | ||
| fi | ||
|
|
||
| ARCH=$(uname -m) | ||
| case "$ARCH" in | ||
| x86_64) ARCH="amd64" ;; | ||
| aarch64|arm64) ARCH="arm64" ;; | ||
| *) echo "FATAL: Unsupported architecture: $ARCH" >&2; exit 1 ;; | ||
| esac | ||
|
|
||
| DOCKER_IMAGE_TAG="${DOCKER_IMAGE_TAG:-llama-swap:unified-${BACKEND}-${ARCH}}" | ||
|
|
||
| echo "" | ||
| echo "==========================================" | ||
| echo "Building rootless image..." | ||
| echo "==========================================" | ||
| echo "" | ||
|
|
||
| ROOTLESS_TAG="${DOCKER_IMAGE_TAG}-rootless" | ||
| BUILD_ARGS=() | ||
| if [[ "$NO_CACHE" == "true" ]]; then | ||
| BUILD_ARGS+=(--no-cache) | ||
| fi | ||
| docker buildx build "${BUILD_ARGS[@]}" --load -t "${ROOTLESS_TAG}" - <<EOF | ||
| FROM ${DOCKER_IMAGE_TAG} | ||
| USER root | ||
| RUN groupadd --system --gid 10001 llama-swap && \\ | ||
| useradd --system --uid 10001 --gid 10001 \\ | ||
| --home /app --shell /sbin/nologin llama-swap && \\ | ||
| chown -R 10001:10001 /etc/llama-swap /models | ||
| USER 10001 | ||
| EOF | ||
|
|
||
| echo "Rootless image built: ${ROOTLESS_TAG}" | ||
|
|
||
| echo "" | ||
| echo "==========================================" | ||
| echo "Build complete!" | ||
| echo "==========================================" | ||
| echo "" | ||
| echo "Image tags:" | ||
| echo " ${ROOTLESS_TAG}" | ||
| echo "" | ||
| echo " llama-swap: $(docker run --rm --entrypoint cat "${DOCKER_IMAGE_TAG}" /versions.txt | grep llama-swap | cut -d' ' -f2-)" | ||
| echo "" | ||
| if [[ "$BACKEND" == "vulkan" ]]; then | ||
| echo "Run with:" | ||
| echo " docker run -it --rm --device /dev/dri:/dev/dri ${ROOTLESS_TAG}" | ||
| echo "" | ||
| echo "Note: For AMD GPUs, you may also need:" | ||
| echo " docker run -it --rm --device /dev/dri:/dev/dri --group-add video ${ROOTLESS_TAG}" | ||
| else | ||
| echo "Run with:" | ||
| echo " docker run -it --rm --gpus all ${ROOTLESS_TAG}" | ||
| fi | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.