Skip to content

KUBERNETES_CHECK.md

Latest commit

 

History

History
309 lines (217 loc) · 7.59 KB

KUBERNETES_CHECK.md

File metadata and controls

309 lines (217 loc) · 7.59 KB
Raw

Kubernetes Connection Verification Guide

This project was originally deployed to a Kubernetes cluster via GitLab CI. Use this guide to verify if your Kubernetes credentials still work.

Prerequisites

  • kubectl CLI installed (Download)
  • Kubernetes service account token from your previous GitLab deployment

Step 1: Check if You Have Kubernetes Access

Option A: Using Existing Kubeconfig

If you previously configured kubectl:

# Check current context
kubectl config current-context

# List all pods in staging namespace
kubectl get pods -n power-ranger-staging

# List all pods in production namespace  
kubectl get pods -n power-ranger-prod

Expected Output (if connected):

NAME                        READY   STATUS    RESTARTS   AGE
posts-api-xxxxxxxxx-xxxxx   1/1     Running   0          5d
posts-db-0                  1/1     Running   0          5d

If you see: error: You must be logged in to the server - Connection expired, proceed to Step 2.

Step 2: Reconnect to Kubernetes (If Access Expired)

Get Your Service Account Token

You need the Kubernetes service account token that was stored in GitLab CI variables.

From your GitLab project:

  1. Go to: Settings → CI/CD → Variables
  2. Find: KUBE_TOKEN (for staging) or KUBE_TOKEN_PROD (for production)
  3. Copy the token value

Configure kubectl with Token

# Set variables
$KUBE_SERVER = "https://kubernetes.devops.cs.ut.ee"
$KUBE_NAMESPACE = "power-ranger-staging"  # or "power-ranger-prod"
$KUBE_TOKEN = "your-token-here"  # Replace with actual token

# Create kubeconfig
kubectl config set-cluster devops-cluster `
  --server=$KUBE_SERVER `
  --insecure-skip-tls-verify=true

kubectl config set-credentials gitlab-ci `
  --token=$KUBE_TOKEN

kubectl config set-context devops `
  --cluster=devops-cluster `
  --user=gitlab-ci `
  --namespace=$KUBE_NAMESPACE

kubectl config use-context devops

Test Connection

# Check connection
kubectl get pods -n $KUBE_NAMESPACE

# Check deployments
kubectl get deployments -n $KUBE_NAMESPACE

# Check services
kubectl get services -n $KUBE_NAMESPACE

# Check ingress
kubectl get ingress -n $KUBE_NAMESPACE

Step 3: Check Deployment Status

Backend API Status

# Get all resources in namespace
kubectl get all -n power-ranger-staging

# Check API pods
kubectl get pods -n power-ranger-staging -l app=posts-api

# Check pod logs
kubectl logs -n power-ranger-staging -l app=posts-api --tail=50

# Check deployment details
kubectl describe deployment posts-api -n power-ranger-staging

# Check service
kubectl get service posts-api -n power-ranger-staging

Frontend Status

# Check frontend pods
kubectl get pods -n power-ranger-staging -l app=frontend

# Check frontend logs
kubectl logs -n power-ranger-staging -l app=frontend --tail=50

# Check frontend service
kubectl get service frontend -n power-ranger-staging

Database Status (CloudNativePG)

# Check database cluster
kubectl get cluster -n power-ranger-staging

# Check database pods
kubectl get pods -n power-ranger-staging -l cnpg.io/cluster

# Check database status
kubectl describe cluster posts-db -n power-ranger-staging

Check Website Access

# Get ingress URL
kubectl get ingress -n power-ranger-staging

# Test with curl
curl https://power-ranger-staging.kubernetes.devops.cs.ut.ee/livez

Or open in browser:

Step 4: Common Issues & Solutions

Issue: "error: You must be logged in"

Solution: Your token expired or is invalid.

  • Get a new token from GitLab CI variables
  • Reconfigure kubectl (see Step 2)

Issue: "Error from server (Forbidden)"

Solution: Your service account doesn't have proper permissions.

  • Check RBAC configuration in k8s-staging/cicd-rbac.yaml
  • Verify service account exists: 
    kubectl get serviceaccount gitlab-ci -n power-ranger-staging

Issue: Pods not running

# Check pod status
kubectl get pods -n power-ranger-staging

# Check pod events
kubectl describe pod <pod-name> -n power-ranger-staging

# Check pod logs
kubectl logs <pod-name> -n power-ranger-staging

Issue: Can't access website (404/502)

# Check ingress configuration
kubectl describe ingress -n power-ranger-staging

# Check if NGINX ingress is installed
kubectl get pods -n ingress-nginx

# Check backend service endpoint
kubectl get endpoints posts-api -n power-ranger-staging

Helm Deployments (If Using Helm)

Check Helm releases

# List all releases in namespace
helm list -n power-ranger-staging

# Get release status
helm status posts -n power-ranger-staging

# Get release values
helm get values posts -n power-ranger-staging

Redeploy with Helm

# Upgrade/install backend
helm upgrade --install posts ./helm/posts `
  -f ./helm/posts/values-staging.yaml `
  --namespace power-ranger-staging

# Upgrade/install frontend
helm upgrade --install frontend ./frontend-main/helm/frontend `
  -f ./frontend-main/helm/frontend/values.yaml `
  --namespace power-ranger-staging

Quick Health Check Script

Save this as check-k8s.ps1:

param(
    [string]$Namespace = "power-ranger-staging"
)

Write-Host "Checking Kubernetes resources in $Namespace..." -ForegroundColor Cyan

Write-Host "`n=== PODS ===" -ForegroundColor Yellow
kubectl get pods -n $Namespace

Write-Host "`n=== DEPLOYMENTS ===" -ForegroundColor Yellow
kubectl get deployments -n $Namespace

Write-Host "`n=== SERVICES ===" -ForegroundColor Yellow
kubectl get services -n $Namespace

Write-Host "`n=== INGRESS ===" -ForegroundColor Yellow
kubectl get ingress -n $Namespace

Write-Host "`n=== HPA ===" -ForegroundColor Yellow
kubectl get hpa -n $Namespace

Write-Host "`n=== DATABASE ===" -ForegroundColor Yellow
kubectl get cluster -n $Namespace

Write-Host "`n=== POD HEALTH ===" -ForegroundColor Yellow
$unhealthyPods = kubectl get pods -n $Namespace -o json | ConvertFrom-Json | 
    Select-Object -ExpandProperty items | 
    Where-Object { $_.status.phase -ne "Running" }

if ($unhealthyPods) {
    Write-Host "WARNING: Found unhealthy pods!" -ForegroundColor Red
    $unhealthyPods | ForEach-Object { 
        Write-Host "  - $($_.metadata.name): $($_.status.phase)" 
    }
} else {
    Write-Host "All pods are healthy!" -ForegroundColor Green
}

Run with: .\check-k8s.ps1 -Namespace power-ranger-staging

Need Access from Scratch?

If you don't have the tokens or access anymore:

  1. Contact your cluster administrator to get a new service account
  2. Use the RBAC files in this repo:
    • Backend: k8s-staging/cicd-rbac.yaml
    • Frontend: frontend-main/k8s-staging/cicd-serviceaccount.yaml
  3. Apply them to create service account: 
    kubectl apply -f k8s-staging/cicd-rbac.yaml
kubectl apply -f frontend-main/k8s-staging/cicd-serviceaccount.yaml
  4. Get the token: 
    kubectl get secret -n power-ranger-staging | findstr gitlab-ci
kubectl describe secret <secret-name> -n power-ranger-staging

Summary Checklist

  • kubectl installed and working
  • Can list pods: kubectl get pods -n power-ranger-staging
  • Backend pods are running
  • Frontend pods are running
  • Database cluster is healthy
  • Ingress is configured correctly
  • Website accessible at staging/production URL

If all checks pass, your Kubernetes deployment is still active! ✅