Skip to content

GHA Windows DinD runner scale set #5

Description

@jrbe228

Per the discussion here, we eventually got a working solution :)
Runner pods are scheduled on an isolated node pool to mitigate potential security issues related to hostProcess Daemon containers. Assuming someone already has GHA ARC controller / listeners installed...

kubectl apply -f dockerd-daemonset.yaml --namespace "windows-dind-rss"

dockerd-daemonset.yaml:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: dockerd-22
  namespace: "windows-dind-rss"
spec:
  selector:
    matchLabels:
      app: dockerd
  template:
    metadata:
      labels:
        app: dockerd
    spec:
      tolerations:
        - key: "windows-dind"
          operator: "Equal"
          value: "true"
          effect: "NoSchedule"
      nodeSelector:
        "nodetype": "windows-dind-22"
      containers:
      - name: dockerd
        image: lippertmarkus/dockerd:20.10.23
        workingDir: "C:\\hpc\\docker"
      securityContext:
        windowsOptions:
          hostProcess: true
          runAsUserName: "NT AUTHORITY\\SYSTEM"
      hostNetwork: true

helm install "arc-win-dind-22" --debug --namespace "windows-dind-rss" --set githubConfigUrl="https://github.com/my-org/my-repo/" --set githubConfigSecret.github_token="ghp_***********************" -f values-ars-win-22.yaml oci://ghcr.io/actions/actions-runner-controller-charts/gha-runner-scale-set --version 0.9.0

values-ars-win-22.yaml:

## ex: https://github.com/myorg/myrepo or https://github.com/myorg
githubConfigUrl: ""

## githubConfigSecret is the k8s secrets to use when auth with GitHub API.
## You can choose to use GitHub App or a PAT token
githubConfigSecret:
  ### GitHub PAT Configuration
  github_token: ""

listenerTemplate:
  spec:
    containers:
    # Use this section to append additional configuration to the listener container.
    # If you change the name of the container, the configuration will not be applied to the listener,
    # and it will be treated as a side-car container.
    - name: listener
      securityContext:
        runAsUser: 1000
    tolerations:
      - key: "os_type"
        operator: "Equal"
        value: "linux"
        effect: "NoSchedule"
    nodeSelector:
      #nodetype: linux
      kubernetes.io/os: linux

## template is the PodSpec for each runner Pod
template:
  metadata:
    labels:
      app: actions-runner
  spec:
    containers:
    - name: runner
      #Per this example - https://www.kenmuse.com/blog/windows-runners-on-actions-runner-controller/
      image: ghcr.io/kenmuse/arc-windows-runner:v2.321.0
      command: ["cmd.exe", "/c", "\\home\\runner\\run.cmd"]
      
      imagePullPolicy: Always
      resources:
        requests:
          cpu: "0.125"
          memory: "64Mi"
        limits:
          cpu: "0.5"
          memory: "4096Mi"
      volumeMounts:
      - mountPath: \\.\pipe\docker_engine
        name: dockersock
      securityContext:
        windowsOptions:
          runAsUserName: "ContainerAdministrator"
    volumes:
    - name: dockersock
      hostPath:
        path: \\.\pipe\docker_engine
        type: null
    tolerations:
      - key: "windows-dind"
        operator: "Equal"
        value: "true"
        effect: "NoSchedule"
    nodeSelector:
      nodetype: "windows-dind-22"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions