Increasing the request to 256Mi caused backend pods to be Pending with no
node assignment — the scheduler couldn't fit them alongside MySQL (512Mi
request) and existing pods on the on-premise nodes.
Memory REQUEST drives scheduling (how much the node reserves).
Memory LIMIT drives OOMKill (the actual cap at runtime).
Keep request at 128Mi so pods schedule, limit at 512Mi so Node.js +
Socket.io + MySQL pool don't get OOMKilled on startup.
Also add terminationGracePeriodSeconds: 15 so pods from failed/previous
builds release their node slot quickly instead of blocking new pod scheduling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Backend was OOMKilled during rolling update startup (Node.js + Socket.io +
MySQL pool exceeds 256Mi). Raised limit to 512Mi and request to 256Mi.
Jenkinsfile: show kubectl get pods immediately after apply so pod state
is visible in build logs. Added full diagnostics (describe + logs) in
post.failure block so the root cause of any future rollout failure is
visible without needing to SSH into the cluster.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Database fixes:
- Add hostPath.type=DirectoryOrCreate so kubelet auto-creates /mnt/data/mysql
- Add fsGroup=999 so MySQL process can write to the hostPath volume
- Add MYSQL_ROOT_HOST=% to allow backend pods to authenticate as root
- Fix liveness/readiness probes to include credentials (-p$MYSQL_ROOT_PASSWORD)
- Increase probe initialDelaySeconds (30/60s) for slow first-run init
- Add 15s grace sleep in backend initContainer after MySQL TCP is up
- Add persistentVolumeReclaimPolicy=Retain to prevent accidental data loss
- Explicit accessModes+resources in PVC patch to avoid list merge ambiguity
- Add nodeAffinity comment in PV for multi-node cluster guidance
Ingress/nginx fixes:
- Remove broken rewrite-target=/ that was rewriting all paths (incl /api) to /
- Route /socket.io directly to backend for WebSocket support
- Add /socket.io/ proxy location to both nginx.conf and K8s ConfigMap
Frontend fix:
- Persist currentUser to localStorage on login so page refresh no longer
clears session and redirects users back to the login page
Tooling:
- Add k8s/overlays/on-premise/deploy.sh for one-command deployment
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
