Kubernetes for Automation Workloads: Production Deployment Guide

As automation workloads scale from simple scripts to enterprise-grade orchestration platforms, Kubernetes provides the ideal platform for deploying, scaling, and managing automation tools like n8n, Airflow, and custom automation services. Kubernetes offers automated scaling, self-healing, service discovery, and declarative configuration that transforms how automation engineers deploy and operate their workflow automation platforms. This guide covers Kubernetes deployment patterns specifically tailored for automation workloads, from basic n8n deployments to complex multi-tenant automation platforms.

Why Kubernetes for Automation Workloads?

Kubernetes offers several advantages for automation platform deployment:

• Horizontal scaling — Automatically scale automation workers based on queue depth
• High availability — Self-healing with automatic pod restart and node failover
• Resource efficiency — Share cluster resources across multiple automation tools
• Declarative configuration — Version-controlled infrastructure as code
• Service discovery — Automatic DNS and load balancing for automation services
• Multi-cloud portability — Run automation workloads across any cloud or on-prem

Basic n8n Kubernetes Deployment

Simple n8n Deployment Manifest

# n8n-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: n8n
  namespace: automation
spec:
  replicas: 2
  selector:
    matchLabels:
      app: n8n
  template:
    metadata:
      labels:
        app: n8n
    spec:
      containers:
      - name: n8n
        image: n8nio/n8n
        ports:
        - containerPort: 5678
        env:
        - name: N8N_BASIC_AUTH_ACTIVE
          value: "true"
        - name: N8N_BASIC_AUTH_USER
          valueFrom:
            secretKeyRef:
              name: n8n-secrets
              key: username
        - name: N8N_BASIC_AUTH_PASSWORD
          valueFrom:
            secretKeyRef:
              name: n8n-secrets
              key: password
        - name: DB_TYPE
          value: "postgresdb"
        - name: DB_POSTGRESDB_HOST
          value: "postgres-service"
        - name: DB_POSTGRESDB_DATABASE
          value: "n8n"
        - name: DB_POSTGRESDB_USER
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: username
        - name: DB_POSTGRESDB_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: password
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "1Gi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /healthz
            port: 5678
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /healthz
            port: 5678
          initialDelaySeconds: 5
          periodSeconds: 5

Service and Ingress Configuration

# n8n-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: n8n-service
  namespace: automation
spec:
  selector:
    app: n8n
  ports:
  - port: 80
    targetPort: 5678
  type: ClusterIP

# n8n-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: n8n-ingress
  namespace: automation
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  tls:
  - hosts:
    - n8n.example.com
    secretName: n8n-tls
  rules:
  - host: n8n.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: n8n-service
            port:
              number: 80

Production n8n Kubernetes Architecture

Queue Mode with Multiple Workers

# n8n-queue-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: n8n-web
  namespace: automation
spec:
  replicas: 2
  selector:
    matchLabels:
      app: n8n
      component: web
  template:
    metadata:
      labels:
        app: n8n
        component: web
    spec:
      containers:
      - name: n8n
        image: n8nio/n8n
        env:
        - name: EXECUTIONS_MODE
          value: "queue"
        - name: N8N_PROTOCOL
          value: "https"
        - name: N8N_HOST
          value: "n8n.example.com"
        # ... other environment variables
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: n8n-worker
  namespace: automation
spec:
  replicas: 3
  selector:
    matchLabels:
      app: n8n
      component: worker
  template:
    metadata:
      labels:
        app: n8n
        component: worker
    spec:
      containers:
      - name: n8n-worker
        image: n8nio/n8n
        command: ["n8n", "worker"]
        env:
        - name: EXECUTIONS_MODE
          value: "queue"
        - name: QUEUE_BULL_REDIS_HOST
          value: "redis-service"
        # ... other environment variables
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "1000m"

Horizontal Pod Autoscaler Configuration

# n8n-hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: n8n-worker-hpa
  namespace: automation
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: n8n-worker
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  - type: Pods
    pods:
      metric:
        name: queue_length
      target:
        type: AverageValue
        averageValue: "100"

Database and Cache Configuration

PostgreSQL StatefulSet

# postgres-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
  namespace: automation
spec:
  serviceName: postgres-service
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:15
        env:
        - name: POSTGRES_DB
          value: "n8n"
        - name: POSTGRES_USER
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: username
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: postgres-secrets
              key: password
        ports:
        - containerPort: 5432
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
        resources:
          requests:
            memory: "2Gi"
            cpu: "500m"
          limits:
            memory: "4Gi"
            cpu: "1000m"
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 50Gi

Redis Deployment for Queue Mode

# redis-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: automation
spec:
  replicas: 3
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:7-alpine
        command: ["redis-server"]
        args: ["--appendonly", "yes", "--requirepass", "$(REDIS_PASSWORD)"]
        env:
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              name: redis-secrets
              key: password
        ports:
        - containerPort: 6379
        volumeMounts:
        - name: redis-data
          mountPath: /data
---
apiVersion: v1
kind: Service
metadata:
  name: redis-service
  namespace: automation
spec:
  selector:
    app: redis
  ports:
  - port: 6379
    targetPort: 6379
  type: ClusterIP

Monitoring and Observability

Prometheus Monitoring Configuration

# n8n-service-monitor.yaml
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: n8n-monitor
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: n8n
  endpoints:
  - port: http-metrics
    interval: 30s
    path: /metrics
  namespaceSelector:
    matchNames:
    - automation

# Custom metrics for n8n
apiVersion: v1
kind: ConfigMap
metadata:
  name: n8n-metrics-config
  namespace: automation
data:
  custom-metrics.yaml: |
    groups:
    - name: n8n_metrics
      rules:
      - record: n8n:workflow_executions:rate5m
        expr: rate(n8n_workflow_executions_total[5m])
      - record: n8n:workflow_duration:quantile
        expr: histogram_quantile(0.95, rate(n8n_workflow_duration_seconds_bucket[5m]))
      - record: n8n:queue_length:avg5m
        expr: avg_over_time(n8n_queue_length[5m])
      - record: n8n:error_rate:ratio
        expr: rate(n8n_workflow_errors_total[5m]) / rate(n8n_workflow_executions_total[5m])

Grafana Dashboard Configuration

# n8n-grafana-dashboard.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: n8n-grafana-dashboard
  namespace: monitoring
  labels:
    grafana_dashboard: "1"
data:
  n8n-dashboard.json: |
    {
      "dashboard": {
        "title": "n8n Automation Platform",
        "panels": [
          {
            "title": "Workflow Executions",
            "type": "graph",
            "targets": [{
              "expr": "rate(n8n_workflow_executions_total[5m])",
              "legendFormat": "{{workflow}}"
            }]
          },
          {
            "title": "Queue Length",
            "type": "stat",
            "targets": [{
              "expr": "n8n_queue_length"
            }]
          },
          {
            "title": "Error Rate",
            "type": "gauge",
            "targets": [{
              "expr": "n8n:error_rate:ratio * 100",
              "format": "percent"
            }]
          }
        ]
      }
    }

Security Best Practices

Pod Security Standards

# pod-security-policy.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: n8n-restricted
spec:
  privileged: false
  allowPrivilegeEscalation: false
  requiredDropCapabilities:
    - ALL
  volumes:
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    - 'persistentVolumeClaim'
  hostNetwork: false
  hostIPC: false
  hostPID: false
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      - min: 1
        max: 65535

Network Policies

# network-policy.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: n8n-network-policy
  namespace: automation
spec:
  podSelector:
    matchLabels:
      app: n8n
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: ingress-nginx
    ports:
    - protocol: TCP
      port: 5678
  egress:
  - to:
    - podSelector:
        matchLabels:
          app: postgres
    ports:
    - protocol: TCP
      port: 5432
  - to:
    - podSelector:
        matchLabels:
          app: redis
    ports:
    - protocol: TCP
      port: 6379
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8
        - 172.16.0.0/12
        - 192.168.0.0/16
    ports:
    - protocol: TCP
      port: 443
    - protocol: TCP
      port: 80

CI/CD Pipeline for Automation Workloads

GitOps with ArgoCD

# argocd-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: n8n-automation
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/n8n-k8s-config.git
    targetRevision: HEAD
    path: k8s/
  destination:
    server: https://kubernetes.default.svc
    namespace: automation
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
    - CreateNamespace=true

Getting Started with Kubernetes for Automation

1. Set up Kubernetes cluster — Managed (EKS, GKE, AKS) or self-managed
2. Install essential tools — kubectl, Helm, kustomize
3. Configure namespace — Create dedicated namespace for automation workloads
4. Set up storage — Configure persistent volumes for databases
5. Deploy n8n — Start with basic deployment, then add production features
6. Configure ingress — Set up HTTPS with cert-manager
7. Implement monitoring — Deploy Prometheus, Grafana, and custom metrics
8. Set up CI/CD — Automated deployment pipeline
9. Test scaling — Verify HPA and cluster autoscaling
10. Implement backup — Regular backups of databases and configurations

Kubernetes provides the ideal platform for running production automation workloads at scale. By leveraging Kubernetes' automated scaling, self-healing, and declarative configuration capabilities, automation engineers can deploy and operate n8n and other automation tools with enterprise-grade reliability and scalability. The key to success is starting with a solid foundation and gradually adding production features as your automation platform grows.