feat(k8s): expose UDP ports on cloud-compose via mixed-protocol LB#363
Open
smecsia wants to merge 5 commits into
Open
feat(k8s): expose UDP ports on cloud-compose via mixed-protocol LB#363smecsia wants to merge 5 commits into
smecsia wants to merge 5 commits into
Conversation
sc parses the compose file and generates its own k8s Deployment +
Service via Pulumi; during that translation the port protocol was
discarded. toRunPorts mapped every compose port to a bare int, so
"7882:7882/udp" silently became a TCP port and UDP never reached the
pod. This blocks deploying a WebRTC media server (e.g. LiveKit SFU),
where signaling is TCP (wss) but media is UDP.
Thread the port protocol end-to-end:
- CloudRunContainer.Ports is now []k8s.ContainerPort{Port, Protocol}
instead of []int; toRunPorts preserves the compose protocol
(udp -> "UDP"; tcp/unspecified -> "").
- New ports.go helpers set Protocol on ContainerPortArgs and
ServicePortArgs, but only for non-TCP protocols, so TCP-only specs
stay byte-for-byte identical (TCP keeps http-<port> names; UDP gets
udp-<port>).
- autoTCPReadinessProbe no longer attaches a TCP/HTTP probe to a
UDP-only port (resolves the old "// TODO: support non-http ports").
- serviceType is now expressible via cloudExtras and threaded
CloudExtras -> DeploymentConfig -> kubeArgs, letting a service turn
its own Service into a LoadBalancer (required for protocols Caddy
cannot proxy).
A service exposing both TCP and UDP emits ONE mixed-protocol
LoadBalancer Service (GA since k8s 1.26, supported on recent GKE)
rather than splitting into two Services, so a single external IP
fronts the whole service. Announcing that IP in ICE candidates
(LiveKit rtc.use_external_ip / node_ip) is the app's job, not sc's.
Backward compatible: ports with no protocol behave exactly as before,
and serviceType defaults to ClusterIP.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Ruslan <ruslan.mikailov@fulldive.com>
smecsia
requested review from
Cre-eD,
Laboratory and
universe-ops
as code owners
Semgrep Scan ResultsRepository:
Scanned at 2026-07-15 19:08 UTC |
Security Scan ResultsRepository:
Scanned at 2026-07-15 19:08 UTC |
📊 Statement coverageMeasured on the documented included set (see
Baseline: |
Contributor
Author
|
Heads-up for reviewers: the only red check here — govulncheck — is not caused by this change. It's GO-2026-5856 in the Go stdlib ( Fix is in #364 ( Everything else on this PR is green (build all platforms, tests, CodeQL, semgrep, secret scan, SBOM, coverage, DCO; commit verified). |
smecsia
added a commit
that referenced
this pull request
Jul 14, 2026
## What Bumps the `go` directive in `go.mod` from `1.26.4` to `1.26.5`. ## Why `govulncheck` reads the stdlib version it scans from the `go` directive. Go **1.26.4** is affected by **GO-2026-5856** (`crypto/tls`), fixed in **go1.26.5**. The finding is *reachable* through several pre-existing paths, so the reachability gate now fails on **every Go-touching PR** until the toolchain is bumped: ``` Vulnerability #1: GO-2026-5856 (crypto/tls@go1.26.4 → fixed in go1.26.5) #1 pkg/util/exec.go … tls.Conn.Handshake #2 pkg/assistant/mcp/server.go … tls.Conn.HandshakeContext #3 pkg/assistant/llm/ollama.go … tls.Conn.Read #4 pkg/api/git/repo.go … tls.Conn.Write #5 pkg/clouds/pulumi/mongodb/drop_db.go … tls.Dial #6 pkg/assistant/llm/openai.go … tls.Dialer.DialContext ``` Bumping the directive clears the advisory fleet-wide (unblocks #363 and any other open Go PR). `go1.26.5` is released; `GOTOOLCHAIN=auto` already resolves the toolchain, and the other workflows read the version from `go.mod` / auto-upgrade — so **no source change** is needed. ## Scope One line in `go.mod`. No code changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Ilya Sadykov <smecsia@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
PR #362 threaded UDP protocol + a stack-level serviceType, but the Service was still built solely from the ingress container's ports, and on GKE a mixed TCP+UDP LoadBalancer failed to provision (LoadBalancerMixedProtocolNotSupported). Both blocked self-hosting a LiveKit SFU alongside the HTTP backend in one stack. - Service ports now include non-ingress run containers' ports (they share the pod's network namespace and the Service selects the pod), so a sidecar SFU's 7880/7882 land on the same Service as backend:3000. Ingress ports win on a port-number clash; ports are deduped. - When the emitted LoadBalancer Service mixes TCP and UDP, annotate it with cloud.google.com/l4-rbs: "enabled" so GKE uses the RBS/NEG-based NetLB, which supports mixed protocols (verified live: one external IP fronting 7880/7881 TCP + 7882/UDP). No config passthrough exists for Service annotations, so this must be set by sc. Tests: dedupePorts, hasMixedProtocols, and a non-ingress-UDP Service case. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
The Service applied args.ExternalTrafficPolicy (simple_container.go) and the DeploymentArgs carried it, but nothing populated it for a client stack — the field only existed on CaddyConfig, and CloudExtras had no way to set it. So `cloudExtras.externalTrafficPolicy` was silently dropped and the Service stayed Cluster. Thread it like serviceType: CloudExtras -> DeploymentConfig -> kubeArgs -> Service. Needed for self-hosted WebRTC media behind the LB: externalTrafficPolicy: Local preserves the LB IP (and client source IP) on the return path, so clients accept the SFU downlink. Cluster SNATs to a node IP, which WebRTC rejects (dtls timeout, call drops ~2 min in). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
The preceding commit wired CloudExtras.ExternalTrafficPolicy through the custom-stack path (kube_run.go) but the GKE Autopilot path (gke_autopilot_stack.go) only forwarded ServiceType, so the Service's externalTrafficPolicy was silently dropped to the default "Cluster" on Autopilot. That SNATs to a node IP and breaks WebRTC return media — exactly what "Local" fixes — so the setting must reach the Autopilot kubeArgs too. Mirror the ServiceType line; no other behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: Ilya Sadykov <smecsia@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Re-established on an SC-owned branch so CI runs with full secret access (the original fork PR #362 could not run secret-gated jobs). Original author preserved: @RuslanMikailov (
Ruslan <ruslan.mikailov@fulldive.com>). Supersedes #362.What
scparses the compose file and generates its own k8s Deployment + Service via Pulumi; during that translation the port protocol was discarded.toRunPortsmapped every compose port to a bareint, so"7882:7882/udp"silently became a TCP port and UDP never reached the pod. This blocks a WebRTC media server (e.g. LiveKit SFU), where signaling is TCP (wss) but media is UDP.Change
Thread the port protocol end-to-end:
CloudRunContainer.Portsis now[]k8s.ContainerPort{Port, Protocol}instead of[]int;toRunPortspreserves the compose protocol (udp→"UDP"; tcp/unspecified →"").ports.gohelpers setProtocolonContainerPortArgs/ServicePortArgs, only for non-TCP protocols, so TCP-only specs stay byte-for-byte identical (TCP keepshttp-<port>names; UDP getsudp-<port>).autoTCPReadinessProbeno longer attaches a TCP/HTTP probe to a UDP-only port (resolves the old// TODO: support non-http ports).serviceTypeis now expressible viacloudExtrasand threadedCloudExtras → DeploymentConfig → kubeArgs, letting a service turn its own Service into aLoadBalancer(required for protocols Caddy cannot proxy).A service exposing both TCP and UDP emits ONE mixed-protocol LoadBalancer Service (GA since k8s 1.26) rather than two Services, so a single external IP fronts the whole service. Announcing that IP in ICE candidates (LiveKit
rtc.use_external_ip/node_ip) is the app's job, notsc's.Backward compatible: ports with no protocol behave exactly as before;
serviceTypedefaults toClusterIP.🤖 Generated with Claude Code