auth: impersonation between provider identities

[matheuscscp]

The AssumeRole and Impersonate APIs are native AWS and GCP APIs, respectively, that allow for a native identity to impersonate another:

• AWS identities to "assume" an AWS IAM Role
• GCP identities to "impersonate" a GCP IAM Service Account

The users of these two clouds are very used to these operations, and various opensource projects support performing them. In AWS, for example, AssumeRole is a very common pattern for cross-account access:

fluxcd/flux2#5708

While we do support cross-account access for AWS (docs), the current implementation has limitations.

Azure

Azure does not support this native operation. Impersonation is only supported via OIDC federation, which we already support.

Newly supported AWS use cases

EKS Pod Identity (i.e. controller permissions) calling AssumeRole

EKS Pod Identity only allows same-account IAM Roles to be assigned to EKS pods. But when calling AssumeRole, the target role can belong to any account. Config:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: my-app
  annotations:
    aws.auth.fluxcd.io/assume-role: |
      roleARN: arn:aws:iam::${TARGET_ACCOUNT}:role/${TARGET_ACCOUNT_ROLE}

The above ServiceAccount should be used via .spec.serviceAccountName (or .spec.kubeConfig.configMapRef -> .data.serviceAccountName) in Flux resources.

The advantage this approach has over this one is that each Flux object can have its own target role to assume. The approach described in this link supports only a single target role that is associated directly with the EKS Pod Identity entry created for the Flux controller pod.

Note: By design, this approach is not supported under workload identity multi-tenancy lockdown. Lockdown isolation is based on Kubernetes namespaces. The impersonation relationship between EKS Pod Identity and a target role has zero namespaces involved, so we can't allow it when lockdown is enabled.

Controller-level IRSA calling AssumeRole

This approach is similar to the above, except that the controller permissions come from IRSA and not EKS Pod Identity. Similarly, this approach is not supported under workload identity multi-tenancy lockdown for the same reason as the above. The ServiceAccount for the Flux object is identical.

Object-level IRSA calling AssumeRole

This is the strongest approach, as it is supported both with and without workload identity multi-tenancy lockdown. Config:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: my-app
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::${SOURCE_ACCOUNT}:role/${SOURCE_ACCOUNT_ROLE}
    aws.auth.fluxcd.io/assume-role: |
      roleARN: arn:aws:iam::${TARGET_ACCOUNT}:role/${TARGET_ACCOUNT_ROLE}
      useServiceAccount: true # not needed when multi-tenancy lockdown is enabled

In this approach, we first exchange a ServiceAccount token for AWS creds representing the source account role, then we call AssumeRole for the target account role.

In my personal opinion, nothing beats the simplicity of the first example in this approach, but the EKS Pod Identity fans won't like it 🤷

Newly supported GCP use cases

Controller-level WIF calling Impersonate

In this approach, configure WIF for the Flux controller pod, then use the following ServiceAccount config for Flux objects:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: my-app
  annotations:
    gcp.auth.fluxcd.io/impersonate: |
      gcpServiceAccount: ${TARGET_SA_NAME}@${TARGET_PROJECT_ID}.iam.gserviceaccount.com

Similarly to the controller-level approaches in AWS, this approach is not supported under workload identity multi-tenancy lockdown.

Object-level WIF calling Impersonate

In this approach, use the following ServiceAccount config for Flux objects:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-app
  namespace: my-app
  annotations:
    iam.gke.io/gcp-service-account: ${SOURCE_SA_NAME}@${SOURCE_PROJECT_ID}.iam.gserviceaccount.com # optional
    gcp.auth.fluxcd.io/impersonate: |
      gcpServiceAccount: ${TARGET_SA_NAME}@${TARGET_PROJECT_ID}.iam.gserviceaccount.com
      useServiceAccount: true # not needed when multi-tenancy lockdown is enabled

[matheuscscp]

Integration tests are passing:

• AWS: https://github.com/fluxcd/pkg/actions/runs/21782546436
• Azure: https://github.com/fluxcd/pkg/actions/runs/21782547816
• GCP: https://github.com/fluxcd/pkg/actions/runs/21782549712

[matheuscscp]

I added integration tests for this feature (f0b372a). All tests are passing:

• AWS: https://github.com/fluxcd/pkg/actions/runs/21787777229
• Azure: https://github.com/fluxcd/pkg/actions/runs/21787779682
• GCP: https://github.com/fluxcd/pkg/actions/runs/21787781396