Erich Blume 67883950c3 C2(deploy-infra-alerting): finalize rewrite cards as historical docs

Remove all Mikado frontmatter (status, branch, requires) from chain
cards. Rename docs/how-to/alerts/ to docs/how-to/runbooks/ and update
all runbook_url references. Add changelog fragment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-03-22 14:40:21 -07:00

1.8 KiB

Raw Blame History

title

modified

Runbook: ArgoCD App Out of Sync

Alert name: ArgoCDAppOutOfSync

An ArgoCD application has been out of sync for 30+ minutes. This means the live state in Kubernetes differs from what's declared in Git.

Diagnostic Steps

Check which app is out of sync — the name label in the alert tells you:
```
argocd app get <app-name>
```
View the diff:
```
argocd app diff <app-name>
```
Check if it's a branch revision issue — during C1/C2 work, apps may be pointed at a feature branch. After merge, they need to be reset to main:
```
argocd app get <app-name> -o json | python3 -c "import json,sys; print(json.load(sys.stdin)['spec']['source']['targetRevision'])"
```
Check ArgoCD UI — https://argocd.ops.eblu.me — look for sync errors or degraded status.

Common Causes

Forgot to sync after push — ArgoCD uses manual sync; changes require explicit argocd app sync
Branch revision not reset after PR merge — app still points at a deleted branch
Kustomize/manifest error — invalid YAML or unsatisfiable resource requirements
Pruning needed — old ConfigMaps from configMapGenerator need pruning

Resolution

# Simple sync
argocd app sync <app-name>

# If pruning is needed
argocd app sync <app-name> --prune

# If stuck on a deleted branch
argocd app set <app-name> --revision main
argocd app sync <app-name>

Silencing

During active C1/C2 development, apps may intentionally be out of sync:

Grafana → Alerting → Silences → Create Silence
Match alertname = ArgoCDAppOutOfSync and name = <app-name>

argocd — ArgoCD reference
deploy-infra-alerting — Alerting pipeline overview

1.8 KiB Raw Blame History