Add Reference section with 24 technical reference cards
Phase 2 of documentation restructuring. Creates docs/reference/ with: Services (16): - alloy, argocd, borgmatic, 1password, forgejo, grafana - jellyfin, kiwix, loki, miniflux, navidrome, postgresql - prometheus, teslamate, transmission, zot Infrastructure (3): - hosts - Device inventory - tailscale - ACLs, groups, tags - routing - DNS domains and port mappings Kubernetes (2): - cluster - Minikube specs - apps - ArgoCD application registry Storage (2): - sifaka - Synology NAS configuration - backups - Backup policy All cards use wiki-links for cross-referencing and include YAML frontmatter with title and tags for Quartz. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
parent
16fba7f1e7
commit
ce1f696bd8
26 changed files with 1435 additions and 6 deletions
|
|
@ -64,14 +64,16 @@ The documentation is being restructured to follow the [Diataxis](https://diataxi
|
|||
|
||||
**Docs URL:** https://docs.ops.eblu.me
|
||||
|
||||
### Phase 2: Reference
|
||||
### Phase 2: Reference (Complete)
|
||||
Information-oriented technical descriptions. Built first so other docs can link to reference material.
|
||||
|
||||
- [ ] Create `reference/` directory
|
||||
- [ ] Service reference pages (migrate from zk cards)
|
||||
- [ ] Infrastructure inventory
|
||||
- [ ] Configuration reference
|
||||
- [ ] API/CLI reference for mise tasks
|
||||
- [x] Create `reference/` directory with index
|
||||
- [x] Service reference pages (16 services: alloy, argocd, borgmatic, 1password, forgejo, grafana, jellyfin, kiwix, loki, miniflux, navidrome, postgresql, prometheus, teslamate, transmission, zot)
|
||||
- [x] Infrastructure inventory (hosts, tailscale, routing)
|
||||
- [x] Kubernetes reference (cluster, apps)
|
||||
- [x] Storage reference (sifaka, backups)
|
||||
|
||||
**Reference URL:** https://docs.ops.eblu.me/reference/
|
||||
|
||||
### Phase 3: Tutorials
|
||||
Learning-oriented content for getting started.
|
||||
|
|
|
|||
1
docs/changelog.d/docs-phase-2-reference.feature.md
Normal file
1
docs/changelog.d/docs-phase-2-reference.feature.md
Normal file
|
|
@ -0,0 +1 @@
|
|||
Add Reference section with 24 technical reference cards covering services, infrastructure, kubernetes, and storage
|
||||
54
docs/reference/index.md
Normal file
54
docs/reference/index.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
title: Reference
|
||||
tags:
|
||||
- reference
|
||||
---
|
||||
|
||||
# Reference
|
||||
|
||||
Technical specifications, inventories, and configuration details for BlumeOps infrastructure.
|
||||
|
||||
## Services
|
||||
|
||||
Individual service reference cards with URLs, configuration, and operational details.
|
||||
|
||||
| Service | Description | Location |
|
||||
|---------|-------------|----------|
|
||||
| [[services/alloy\|Alloy]] | Observability collector (metrics & logs) | indri + k8s |
|
||||
| [[services/argocd\|ArgoCD]] | GitOps continuous delivery | k8s |
|
||||
| [[services/borgmatic\|Borgmatic]] | Backup system | indri |
|
||||
| [[services/1password\|1Password]] | Secrets management | cloud + k8s |
|
||||
| [[services/forgejo\|Forgejo]] | Git forge & CI/CD | indri |
|
||||
| [[services/grafana\|Grafana]] | Dashboards & visualization | k8s |
|
||||
| [[services/jellyfin\|Jellyfin]] | Media server | indri |
|
||||
| [[services/kiwix\|Kiwix]] | Offline Wikipedia & ZIM archives | k8s |
|
||||
| [[services/loki\|Loki]] | Log aggregation | k8s |
|
||||
| [[services/miniflux\|Miniflux]] | RSS feed reader | k8s |
|
||||
| [[services/navidrome\|Navidrome]] | Music streaming | k8s |
|
||||
| [[services/postgresql\|PostgreSQL]] | Database cluster | k8s |
|
||||
| [[services/prometheus\|Prometheus]] | Metrics collection | k8s |
|
||||
| [[services/teslamate\|TeslaMate]] | Tesla data logger | k8s |
|
||||
| [[services/transmission\|Transmission]] | BitTorrent daemon | k8s |
|
||||
| [[services/zot\|Zot]] | Container registry | indri |
|
||||
|
||||
## Infrastructure
|
||||
|
||||
Host inventory and network configuration.
|
||||
|
||||
- [[infrastructure/hosts\|Hosts]] - Device inventory (indri, gilbert, sifaka, etc.)
|
||||
- [[infrastructure/tailscale\|Tailscale]] - ACLs, groups, tags
|
||||
- [[infrastructure/routing\|Routing]] - DNS domains, port mappings
|
||||
|
||||
## Kubernetes
|
||||
|
||||
Cluster configuration and application registry.
|
||||
|
||||
- [[kubernetes/cluster\|Cluster]] - Minikube specs, storage, networking
|
||||
- [[kubernetes/apps\|Apps]] - ArgoCD application registry
|
||||
|
||||
## Storage
|
||||
|
||||
Network storage and backup configuration.
|
||||
|
||||
- [[storage/sifaka\|Sifaka]] - Synology NAS configuration
|
||||
- [[storage/backups\|Backups]] - Backup policy and schedule
|
||||
71
docs/reference/infrastructure/hosts.md
Normal file
71
docs/reference/infrastructure/hosts.md
Normal file
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
title: Host Inventory
|
||||
tags:
|
||||
- infrastructure
|
||||
---
|
||||
|
||||
# Host Inventory
|
||||
|
||||
All devices connected via [Tailscale](https://login.tailscale.com/) tailnet `tail8d86e.ts.net`.
|
||||
|
||||
## Devices
|
||||
|
||||
| Host | Description | Notes |
|
||||
|------|-------------|-------|
|
||||
| **Indri** | Mac Mini M1, 2020 | Primary server, 2TB internal disk |
|
||||
| **[[storage/sifaka\|Sifaka]]** | Synology NAS | 10.9TB RAID 5, backup target |
|
||||
| **Gilbert** | 13" MacBook Air M4, 2025 | Primary workstation |
|
||||
| **Mouse** | 13" MacBook Air M2 | Allison's laptop |
|
||||
| **UniFi** | UniFi Express 7 | Home WiFi network |
|
||||
| **Dwarf** | iPad Air | Employer-provided, off tailnet |
|
||||
|
||||
## Indri Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Model** | Mac mini M1, 2020 (Macmini9,1) |
|
||||
| **Storage** | 2TB internal SSD |
|
||||
| **macOS** | 15.7.3 (Sequoia) |
|
||||
| **Role** | Primary server |
|
||||
| **Tailscale IP** | 100.98.163.89 |
|
||||
|
||||
### Services Hosted
|
||||
|
||||
**Native (via Ansible):**
|
||||
- [[services/forgejo\|Forgejo]] - Git forge
|
||||
- [[services/zot\|Zot]] - Container registry
|
||||
- [[services/jellyfin\|Jellyfin]] - Media server
|
||||
- [[services/borgmatic\|Borgmatic]] - Backup system
|
||||
- [[services/alloy\|Alloy]] - Metrics/logs collector
|
||||
- Caddy - Reverse proxy
|
||||
|
||||
**Kubernetes (via minikube):**
|
||||
- [[kubernetes/apps\|All k8s applications]]
|
||||
|
||||
### Sleep Prevention
|
||||
|
||||
Indri uses Amphetamine (App Store) to prevent sleep. Configuration:
|
||||
- Start Session At Launch: enabled
|
||||
- Default Duration: indefinite
|
||||
- Allow Closed-Display Sleep: enabled
|
||||
|
||||
## Gilbert Details
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Model** | 13" MacBook Air M4, 2025 |
|
||||
| **Role** | Development workstation |
|
||||
| **User** | eblume |
|
||||
|
||||
### Development Tools
|
||||
|
||||
Managed via `Brewfile` and `mise.toml`.
|
||||
|
||||
Fish abbreviations:
|
||||
- `ki` -> `kubectl --context=minikube-indri`
|
||||
- `k9i` -> `k9s --context=minikube-indri`
|
||||
|
||||
## Related
|
||||
|
||||
- [[infrastructure/tailscale\|Tailscale]] - Network configuration
|
||||
- [[storage/sifaka\|Sifaka]] - NAS details
|
||||
79
docs/reference/infrastructure/routing.md
Normal file
79
docs/reference/infrastructure/routing.md
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
---
|
||||
title: Service Routing
|
||||
tags:
|
||||
- infrastructure
|
||||
- network
|
||||
---
|
||||
|
||||
# Service Routing
|
||||
|
||||
Services are accessible via two DNS domains with different reachability.
|
||||
|
||||
## DNS Domains
|
||||
|
||||
| Domain | Proxy | Reachable From |
|
||||
|--------|-------|----------------|
|
||||
| `*.ops.eblu.me` | Caddy on indri | k8s pods, docker containers, tailnet clients |
|
||||
| `*.tail8d86e.ts.net` | Tailscale MagicDNS | Tailnet clients only |
|
||||
|
||||
**Use `*.ops.eblu.me`** for services that need pod-to-service communication.
|
||||
|
||||
## Caddy Services (`*.ops.eblu.me`)
|
||||
|
||||
DNS points to indri's Tailscale IP (100.98.163.89). TLS via Let's Encrypt (ACME DNS-01 with Gandi).
|
||||
|
||||
| Service | URL | Description |
|
||||
|---------|-----|-------------|
|
||||
| Homepage | https://go.ops.eblu.me | Service dashboard |
|
||||
| [[services/forgejo\|Forgejo]] | https://forge.ops.eblu.me | Git hosting (SSH: 2222) |
|
||||
| [[services/zot\|Zot]] | https://registry.ops.eblu.me | Container registry |
|
||||
| [[services/grafana\|Grafana]] | https://grafana.ops.eblu.me | Dashboards |
|
||||
| [[services/argocd\|ArgoCD]] | https://argocd.ops.eblu.me | GitOps CD |
|
||||
| [[services/prometheus\|Prometheus]] | https://prometheus.ops.eblu.me | Metrics |
|
||||
| [[services/loki\|Loki]] | https://loki.ops.eblu.me | Logs |
|
||||
| [[services/miniflux\|Miniflux]] | https://feed.ops.eblu.me | RSS reader |
|
||||
| [[services/kiwix\|Kiwix]] | https://kiwix.ops.eblu.me | Offline Wikipedia |
|
||||
| [[services/transmission\|Transmission]] | https://torrent.ops.eblu.me | BitTorrent |
|
||||
| [[services/teslamate\|TeslaMate]] | https://tesla.ops.eblu.me | Tesla logger |
|
||||
| [[services/navidrome\|Navidrome]] | https://dj.ops.eblu.me | Music streaming |
|
||||
| [[services/jellyfin\|Jellyfin]] | https://jellyfin.ops.eblu.me | Media server |
|
||||
| [[services/postgresql\|PostgreSQL]] | pg.ops.eblu.me:5432 | Database |
|
||||
| [[storage/sifaka\|Sifaka]] | https://nas.ops.eblu.me | NAS dashboard |
|
||||
|
||||
## Tailscale-Only Services
|
||||
|
||||
| Service | URL | Description |
|
||||
|---------|-----|-------------|
|
||||
| Kubernetes | https://k8s.tail8d86e.ts.net | Minikube API |
|
||||
|
||||
## Port Map (Indri)
|
||||
|
||||
| Port | Service | Protocol | Binding | Notes |
|
||||
|------|---------|----------|---------|-------|
|
||||
| 443 | Caddy | HTTPS | 0.0.0.0 | Reverse proxy |
|
||||
| 2222 | Caddy L4 | TCP | 0.0.0.0 | SSH proxy to Forgejo |
|
||||
| 5432 | Caddy L4 | TCP | 0.0.0.0 | PostgreSQL proxy |
|
||||
| 2200 | Forgejo SSH | TCP | localhost | Built-in SSH server |
|
||||
| 3001 | Forgejo | HTTP | localhost | Web UI |
|
||||
| 5050 | Zot | HTTP | localhost | Registry API |
|
||||
| 8096 | Jellyfin | HTTP | localhost | Media server |
|
||||
| 44491 | K8s API | HTTPS | 0.0.0.0 | Minikube API server |
|
||||
|
||||
## Adding New Services
|
||||
|
||||
### Indri Services (via Caddy)
|
||||
1. Host service on localhost
|
||||
2. Add to `ansible/roles/caddy/defaults/main.yml`
|
||||
3. Run `mise run provision-indri -- --tags caddy`
|
||||
|
||||
### K8s Services (via Tailscale Ingress)
|
||||
1. Create manifests in `argocd/manifests/<service>/`
|
||||
2. Add ArgoCD Application in `argocd/apps/`
|
||||
3. Add Tailscale Ingress annotation
|
||||
4. Add Caddy proxy entry
|
||||
5. Sync via ArgoCD
|
||||
|
||||
## Related
|
||||
|
||||
- [[infrastructure/tailscale\|Tailscale]] - ACL configuration
|
||||
- [[infrastructure/hosts\|Hosts]] - Where services run
|
||||
67
docs/reference/infrastructure/tailscale.md
Normal file
67
docs/reference/infrastructure/tailscale.md
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
---
|
||||
title: Tailscale
|
||||
tags:
|
||||
- infrastructure
|
||||
- network
|
||||
---
|
||||
|
||||
# Tailscale
|
||||
|
||||
Tailnet `tail8d86e.ts.net` provides secure networking for all BlumeOps infrastructure.
|
||||
|
||||
## ACL Management
|
||||
|
||||
ACLs managed via Pulumi in `pulumi/policy.hujson`.
|
||||
|
||||
```bash
|
||||
mise run tailnet-preview # Preview changes
|
||||
mise run tailnet-up # Apply changes
|
||||
```
|
||||
|
||||
## Groups
|
||||
|
||||
| Group | Members | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `group:allisonflix` | admin, member | [[services/jellyfin\|Jellyfin]] media access |
|
||||
|
||||
## Device Tags
|
||||
|
||||
| Tag | Devices | Purpose |
|
||||
|-----|---------|---------|
|
||||
| `tag:homelab` | indri | Server infrastructure |
|
||||
| `tag:nas` | sifaka | Network-attached storage |
|
||||
| `tag:blumeops` | indri, sifaka | Pulumi IaC managed resources |
|
||||
| `tag:registry` | indri | Container registry access |
|
||||
| `tag:k8s-api` | indri | Kubernetes API server access |
|
||||
|
||||
**Important:** Don't tag user-owned devices (like gilbert). Tagging converts them to "tagged devices" which lose user identity and break user-based SSH rules.
|
||||
|
||||
## Access Matrix
|
||||
|
||||
| Source | Kiwix | Forge | PyPI | Miniflux | PostgreSQL | NAS | Grafana | Loki |
|
||||
|--------|-------|-------|------|----------|------------|-----|---------|------|
|
||||
| `autogroup:admin` | Y | Y | Y | Y | Y | Y | Y | Y |
|
||||
| `autogroup:member` | Y | Y | Y | Y | Y | - | - | - |
|
||||
| `tag:homelab` | - | - | - | - | - | Y | - | - |
|
||||
|
||||
- **Admins** - full access to all services
|
||||
- **Members** - member services only, no Grafana/Loki/NAS
|
||||
|
||||
## SSH Access
|
||||
|
||||
| Source | Destinations | Auth |
|
||||
|--------|--------------|------|
|
||||
| `autogroup:member` | `autogroup:self` | check |
|
||||
| `autogroup:admin` | `tag:homelab` | check (12h) |
|
||||
| `autogroup:admin` | `tag:nas` | check (12h) |
|
||||
|
||||
## OAuth Credentials
|
||||
|
||||
Pulumi uses OAuth client from 1Password (blumeops vault):
|
||||
- Scopes: acl, dns, devices, services
|
||||
- Auto-applies `tag:blumeops` to IaC-managed resources
|
||||
|
||||
## Related
|
||||
|
||||
- [[infrastructure/routing\|Routing]] - Service URLs
|
||||
- [[infrastructure/hosts\|Hosts]] - Device inventory
|
||||
65
docs/reference/kubernetes/apps.md
Normal file
65
docs/reference/kubernetes/apps.md
Normal file
|
|
@ -0,0 +1,65 @@
|
|||
---
|
||||
title: ArgoCD Applications
|
||||
tags:
|
||||
- kubernetes
|
||||
- argocd
|
||||
---
|
||||
|
||||
# ArgoCD Applications
|
||||
|
||||
Registry of all applications deployed via [[services/argocd\|ArgoCD]].
|
||||
|
||||
## Application Registry
|
||||
|
||||
| App | Namespace | Path/Source | Service |
|
||||
|-----|-----------|-------------|---------|
|
||||
| `apps` | argocd | `argocd/apps/` | App-of-apps root |
|
||||
| `argocd` | argocd | `argocd/manifests/argocd/` | [[services/argocd\|ArgoCD]] |
|
||||
| `tailscale-operator` | tailscale | `argocd/manifests/tailscale-operator/` | Tailscale k8s operator |
|
||||
| `1password-connect` | 1password | `argocd/manifests/1password-connect/` | [[services/1password\|1Password]] |
|
||||
| `external-secrets` | external-secrets | Helm chart | [[services/1password\|1Password]] |
|
||||
| `external-secrets-config` | external-secrets | `argocd/manifests/external-secrets-config/` | [[services/1password\|1Password]] |
|
||||
| `cloudnative-pg` | cnpg-system | Helm chart (forge mirror) | PostgreSQL operator |
|
||||
| `blumeops-pg` | databases | `argocd/manifests/databases/` | [[services/postgresql\|PostgreSQL]] |
|
||||
| `prometheus` | monitoring | `argocd/manifests/prometheus/` | [[services/prometheus\|Prometheus]] |
|
||||
| `loki` | monitoring | `argocd/manifests/loki/` | [[services/loki\|Loki]] |
|
||||
| `grafana` | monitoring | Helm chart (forge mirror) | [[services/grafana\|Grafana]] |
|
||||
| `grafana-config` | monitoring | `argocd/manifests/grafana-config/` | [[services/grafana\|Grafana]] |
|
||||
| `alloy-k8s` | alloy | `argocd/manifests/alloy-k8s/` | [[services/alloy\|Alloy]] |
|
||||
| `kube-state-metrics` | monitoring | `argocd/manifests/kube-state-metrics/` | K8s metrics |
|
||||
| `miniflux` | miniflux | `argocd/manifests/miniflux/` | [[services/miniflux\|Miniflux]] |
|
||||
| `kiwix` | kiwix | `argocd/manifests/kiwix/` | [[services/kiwix\|Kiwix]] |
|
||||
| `torrent` | torrent | `argocd/manifests/torrent/` | [[services/transmission\|Transmission]] |
|
||||
| `navidrome` | navidrome | `argocd/manifests/navidrome/` | [[services/navidrome\|Navidrome]] |
|
||||
| `teslamate` | teslamate | `argocd/manifests/teslamate/` | [[services/teslamate\|TeslaMate]] |
|
||||
| `forgejo-runner` | forgejo-runner | `argocd/manifests/forgejo-runner/` | [[services/forgejo\|Forgejo]] CI |
|
||||
|
||||
## Sync Policies
|
||||
|
||||
| Application | Policy | Rationale |
|
||||
|-------------|--------|-----------|
|
||||
| `apps` | Automated | Picks up new Application manifests |
|
||||
| All others | Manual | Explicit control over deployments |
|
||||
|
||||
## Common Commands
|
||||
|
||||
```bash
|
||||
argocd app list # List all apps
|
||||
argocd app get <app> # Get details
|
||||
argocd app diff <app> # Preview changes
|
||||
argocd app sync <app> # Deploy changes
|
||||
```
|
||||
|
||||
## PR Workflow
|
||||
|
||||
1. Create feature branch, modify manifests
|
||||
2. Push to forge
|
||||
3. Sync apps application: `argocd app sync apps`
|
||||
4. Point service at branch: `argocd app set <service> --revision feature/branch`
|
||||
5. Test: `argocd app sync <service>`
|
||||
6. After merge, reset: `argocd app set <service> --revision main`
|
||||
|
||||
## Related
|
||||
|
||||
- [[services/argocd\|ArgoCD]] - GitOps platform details
|
||||
- [[kubernetes/cluster\|Cluster]] - Kubernetes infrastructure
|
||||
73
docs/reference/kubernetes/cluster.md
Normal file
73
docs/reference/kubernetes/cluster.md
Normal file
|
|
@ -0,0 +1,73 @@
|
|||
---
|
||||
title: Kubernetes Cluster
|
||||
tags:
|
||||
- kubernetes
|
||||
---
|
||||
|
||||
# Kubernetes Cluster
|
||||
|
||||
Single-node Minikube cluster running on [[infrastructure/hosts\|Indri]].
|
||||
|
||||
## Cluster Specifications
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Driver** | docker |
|
||||
| **Container Runtime** | docker |
|
||||
| **Kubernetes Version** | v1.34.0 |
|
||||
| **CPUs** | 6 |
|
||||
| **Memory** | 11GB |
|
||||
| **Disk** | 200GB |
|
||||
| **API Server** | https://k8s.tail8d86e.ts.net |
|
||||
|
||||
**Prerequisites:** Docker Desktop with at least 12GB memory allocated.
|
||||
|
||||
## Remote Access
|
||||
|
||||
From gilbert:
|
||||
|
||||
```bash
|
||||
mise run ensure-minikube-indri-kubectl-config
|
||||
```
|
||||
|
||||
Fish abbreviations:
|
||||
- `ki` -> `kubectl --context=minikube-indri`
|
||||
- `k9i` -> `k9s --context=minikube-indri`
|
||||
|
||||
## Volume Mounting
|
||||
|
||||
Pods mount NFS directly from [[storage/sifaka\|Sifaka]]:
|
||||
|
||||
```yaml
|
||||
volumes:
|
||||
- name: torrents
|
||||
nfs:
|
||||
server: sifaka
|
||||
path: /volume1/torrents
|
||||
```
|
||||
|
||||
Docker NATs outbound traffic through indri's LAN IP (192.168.1.50), allowing access to Sifaka's NFS exports.
|
||||
|
||||
## Registry Mirror
|
||||
|
||||
Containerd uses [[services/zot\|Zot]] as a pull-through cache:
|
||||
- Endpoint: `host.minikube.internal:5050`
|
||||
- Config: `/etc/containerd/certs.d/<registry>/hosts.toml`
|
||||
|
||||
Mirrors configured: `registry.ops.eblu.me`, `docker.io`, `ghcr.io`, `quay.io`
|
||||
|
||||
## Useful Commands (on indri)
|
||||
|
||||
```bash
|
||||
minikube status # Cluster status
|
||||
minikube start # Start cluster
|
||||
minikube stop # Stop cluster
|
||||
minikube ssh # SSH into node
|
||||
minikube logs # View logs
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [[kubernetes/apps\|Apps]] - ArgoCD applications
|
||||
- [[services/argocd\|ArgoCD]] - GitOps deployment
|
||||
- [[services/zot\|Zot]] - Registry mirror
|
||||
58
docs/reference/services/1password.md
Normal file
58
docs/reference/services/1password.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
title: 1Password
|
||||
tags:
|
||||
- service
|
||||
- secrets
|
||||
---
|
||||
|
||||
# 1Password
|
||||
|
||||
Root credential store for all BlumeOps secrets, synced to Kubernetes via External Secrets Operator.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
1Password Cloud
|
||||
|
|
||||
v
|
||||
1Password Connect (namespace: 1password)
|
||||
|
|
||||
v
|
||||
External Secrets Operator (namespace: external-secrets)
|
||||
|
|
||||
v
|
||||
Native Kubernetes Secrets
|
||||
```
|
||||
|
||||
## Vault
|
||||
|
||||
The `blumeops` vault contains all infrastructure credentials.
|
||||
|
||||
## Kubernetes Integration
|
||||
|
||||
**ClusterSecretStore:** `onepassword-blumeops`
|
||||
|
||||
Services reference 1Password items via `ExternalSecret` manifests. Example: `argocd/manifests/devpi/external-secret.yaml`
|
||||
|
||||
## CLI Usage
|
||||
|
||||
```bash
|
||||
# Get a secret field
|
||||
op --vault blumeops item get <item-id> --fields <field> --reveal
|
||||
|
||||
# Inject into a template
|
||||
op inject -i secret.yaml.tpl | kubectl apply -f -
|
||||
```
|
||||
|
||||
## Bootstrap (Disaster Recovery)
|
||||
|
||||
1. Create Connect server: `op connect server create blumeops --vaults blumeops`
|
||||
2. Create token: `op connect token create blumeops --server <id> --vault blumeops`
|
||||
3. Store credentials in 1Password item "1Password Connect"
|
||||
4. Apply bootstrap secret to k8s
|
||||
5. Sync apps: 1password-connect, external-secrets-crds, external-secrets, external-secrets-config
|
||||
|
||||
## Related
|
||||
|
||||
- [[argocd\|ArgoCD]] - Uses secrets for git access
|
||||
- [[postgresql\|PostgreSQL]] - Database credentials
|
||||
52
docs/reference/services/alloy.md
Normal file
52
docs/reference/services/alloy.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
title: Grafana Alloy
|
||||
tags:
|
||||
- service
|
||||
- observability
|
||||
---
|
||||
|
||||
# Grafana Alloy
|
||||
|
||||
Unified observability collector for metrics and logs with two deployments:
|
||||
1. **Indri (host)** - System metrics and service logs from macOS host
|
||||
2. **Kubernetes (DaemonSet)** - Automatic pod log collection and service health probes
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Indri Binary** | `~/.local/bin/alloy` |
|
||||
| **Indri Config** | `~/.config/grafana-alloy/config.alloy` |
|
||||
| **K8s Namespace** | `alloy` |
|
||||
| **K8s Image** | `grafana/alloy:v1.8.2` |
|
||||
| **ArgoCD App** | `alloy-k8s` |
|
||||
|
||||
## Metrics Collected
|
||||
|
||||
### From Indri
|
||||
- System metrics via `prometheus.exporter.unix`
|
||||
- Textfile collector: `minikube.prom`, `borgmatic.prom`, `zot.prom`, `jellyfin.prom`
|
||||
- Zot registry metrics from `http://localhost:5050/metrics`
|
||||
- Pushed to [[prometheus\|Prometheus]] via remote_write
|
||||
|
||||
### From Kubernetes
|
||||
- All pod logs via `loki.source.kubernetes`
|
||||
- Service health probes: miniflux, kiwix, transmission, devpi, argocd
|
||||
|
||||
## Logs Collected
|
||||
|
||||
**Brew services:** forgejo, tailscale
|
||||
|
||||
**mcquack LaunchAgents:** alloy, borgmatic, zot, jellyfin
|
||||
|
||||
Logs pushed to [[loki\|Loki]] at `https://loki.tail8d86e.ts.net/loki/api/v1/push`.
|
||||
|
||||
## Why Built from Source
|
||||
|
||||
The Homebrew bottle uses `CGO_ENABLED=0`, which breaks Tailscale MagicDNS. Building with `CGO_ENABLED=1` uses the macOS native resolver.
|
||||
|
||||
## Related
|
||||
|
||||
- [[prometheus\|Prometheus]] - Metrics storage
|
||||
- [[loki\|Loki]] - Log storage
|
||||
- [[grafana\|Grafana]] - Visualization
|
||||
50
docs/reference/services/argocd.md
Normal file
50
docs/reference/services/argocd.md
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
---
|
||||
title: ArgoCD
|
||||
tags:
|
||||
- service
|
||||
- gitops
|
||||
---
|
||||
|
||||
# ArgoCD
|
||||
|
||||
GitOps continuous delivery platform for the [[kubernetes/cluster\|Kubernetes cluster]].
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://argocd.ops.eblu.me |
|
||||
| **Tailscale URL** | https://argocd.tail8d86e.ts.net |
|
||||
| **Namespace** | `argocd` |
|
||||
| **Git Source** | `ssh://forgejo@indri.tail8d86e.ts.net:2200/eblume/blumeops.git` |
|
||||
| **Manifests Path** | `argocd/` |
|
||||
|
||||
## Sync Policy
|
||||
|
||||
| Application | Sync Policy | Rationale |
|
||||
|-------------|-------------|-----------|
|
||||
| `apps` | Automated | Picks up new Application manifests |
|
||||
| All workloads | Manual | Explicit control over deployments |
|
||||
|
||||
## CLI Commands
|
||||
|
||||
```bash
|
||||
# Login
|
||||
argocd login argocd.ops.eblu.me --username admin --password "$(op ...)"
|
||||
|
||||
# Common operations
|
||||
argocd app list
|
||||
argocd app diff <app>
|
||||
argocd app sync <app>
|
||||
argocd app get <app>
|
||||
```
|
||||
|
||||
## Credentials
|
||||
|
||||
- Admin password: 1Password (blumeops vault)
|
||||
- Git deploy key (SSH): 1Password
|
||||
|
||||
## Related
|
||||
|
||||
- [[kubernetes/apps\|Apps]] - Full application registry
|
||||
- [[forgejo\|Forgejo]] - Git source
|
||||
60
docs/reference/services/borgmatic.md
Normal file
60
docs/reference/services/borgmatic.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
title: Borgmatic
|
||||
tags:
|
||||
- service
|
||||
- backup
|
||||
---
|
||||
|
||||
# Borgmatic
|
||||
|
||||
Daily backup system using Borg backup, running on indri.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Install** | mise (pipx) |
|
||||
| **Config** | `~/.config/borgmatic/config.yaml` |
|
||||
| **Schedule** | Daily at 2:00 AM |
|
||||
| **Repository** | `/Volumes/backups/borg/` on [[storage/sifaka\|Sifaka]] |
|
||||
|
||||
## What Gets Backed Up
|
||||
|
||||
**Directories:**
|
||||
- `~/code/personal/zk` - Zettelkasten
|
||||
- `/opt/homebrew/var/forgejo` - Git forge data
|
||||
- `~/.config/borgmatic` - Borgmatic config
|
||||
- `~/Documents` - Personal documents
|
||||
- `~/Pictures` - Photos
|
||||
|
||||
**Databases:**
|
||||
- `miniflux` on [[postgresql\|PostgreSQL]]
|
||||
- `teslamate` on [[postgresql\|PostgreSQL]]
|
||||
|
||||
**Not backed up (by design):**
|
||||
- ZIM archives (re-downloadable)
|
||||
- Prometheus metrics (ephemeral)
|
||||
- Loki logs (ephemeral)
|
||||
|
||||
## Retention Policy
|
||||
|
||||
| Period | Count |
|
||||
|--------|-------|
|
||||
| Daily | 7 |
|
||||
| Monthly | 12 |
|
||||
| Yearly | 1000 |
|
||||
|
||||
## Monitoring
|
||||
|
||||
Metrics exposed via textfile collector to [[prometheus\|Prometheus]]:
|
||||
- `borgmatic_up` - Repository accessibility
|
||||
- `borgmatic_last_archive_timestamp` - Last backup time
|
||||
- `borgmatic_repo_deduplicated_size_bytes` - Disk usage
|
||||
|
||||
Dashboard: "Borgmatic Backups" in [[grafana\|Grafana]]
|
||||
|
||||
## Related
|
||||
|
||||
- [[storage/backups\|Backups]] - Full backup policy
|
||||
- [[storage/sifaka\|Sifaka]] - Backup target
|
||||
- [[postgresql\|PostgreSQL]] - Database backups
|
||||
58
docs/reference/services/forgejo.md
Normal file
58
docs/reference/services/forgejo.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
title: Forgejo
|
||||
tags:
|
||||
- service
|
||||
- git
|
||||
- cicd
|
||||
---
|
||||
|
||||
# Forgejo
|
||||
|
||||
Git forge and CI/CD platform. **Primary source of truth for blumeops** (mirrored to GitHub).
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://forge.ops.eblu.me |
|
||||
| **SSH** | `ssh://forgejo@forge.ops.eblu.me:2222` |
|
||||
| **Local Ports** | 3001 (HTTP), 2200 (SSH) |
|
||||
| **Config** | `ansible/roles/forgejo/templates/app.ini.j2` |
|
||||
|
||||
## Repositories
|
||||
|
||||
| Repo | Description |
|
||||
|------|-------------|
|
||||
| `eblume/blumeops` | Infrastructure as code (primary) |
|
||||
| `eblume/alloy` | Grafana Alloy fork (CGO build) |
|
||||
| `eblume/tesla_auth` | Tesla OAuth helper |
|
||||
| Helm chart mirrors | cloudnative-pg-charts, grafana-helm-charts |
|
||||
|
||||
## CI/CD (Forgejo Actions)
|
||||
|
||||
**Runner:** Kubernetes pod with Docker-in-Docker sidecar
|
||||
- Namespace: `forgejo-runner`
|
||||
- Labels: `k8s`
|
||||
- ArgoCD app: `forgejo-runner`
|
||||
|
||||
**Workflows:** `.forgejo/workflows/`
|
||||
- `build-container.yaml` - Container image builds on tag
|
||||
|
||||
**Container release:**
|
||||
```bash
|
||||
mise run container-list # List containers
|
||||
mise run container-release runner v1.0.0 # Tag and build
|
||||
```
|
||||
|
||||
## Ansible Management
|
||||
|
||||
```bash
|
||||
mise run provision-indri -- --tags forgejo
|
||||
```
|
||||
|
||||
Secrets fetched from 1Password: `lfs-jwt-secret`, `internal-token`, `oauth2-jwt-secret`, `runner_reg`
|
||||
|
||||
## Related
|
||||
|
||||
- [[argocd\|ArgoCD]] - Uses Forgejo as git source
|
||||
- [[zot\|Zot]] - Container registry for built images
|
||||
50
docs/reference/services/grafana.md
Normal file
50
docs/reference/services/grafana.md
Normal file
|
|
@ -0,0 +1,50 @@
|
|||
---
|
||||
title: Grafana
|
||||
tags:
|
||||
- service
|
||||
- observability
|
||||
---
|
||||
|
||||
# Grafana
|
||||
|
||||
Dashboards and visualization for BlumeOps observability.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://grafana.ops.eblu.me |
|
||||
| **Tailscale URL** | https://grafana.tail8d86e.ts.net |
|
||||
| **Namespace** | `monitoring` |
|
||||
| **Helm Chart** | grafana (mirrored to forge) |
|
||||
| **Values** | `argocd/manifests/grafana/values.yaml` |
|
||||
|
||||
## Datasources
|
||||
|
||||
| Name | Type | Target |
|
||||
|------|------|--------|
|
||||
| Prometheus | prometheus | `prometheus.monitoring.svc.cluster.local:9090` |
|
||||
| Loki | loki | `loki.monitoring.svc.cluster.local:3100` |
|
||||
| TeslaMate | postgres | `blumeops-pg-rw.databases.svc.cluster.local:5432` |
|
||||
|
||||
## Dashboard Provisioning
|
||||
|
||||
Dashboards are ConfigMaps with label `grafana_dashboard: "1"`.
|
||||
|
||||
Location: `argocd/manifests/grafana-config/dashboards/`
|
||||
|
||||
Optional annotation: `grafana_folder: "FolderName"`
|
||||
|
||||
## Key Dashboards
|
||||
|
||||
- macOS System - Host metrics for indri
|
||||
- Minikube - Kubernetes cluster overview
|
||||
- Borgmatic Backups - Backup status and trends
|
||||
- Services Health - HTTP probe results
|
||||
- TeslaMate (18 dashboards) - Vehicle data
|
||||
|
||||
## Related
|
||||
|
||||
- [[prometheus\|Prometheus]] - Metrics datasource
|
||||
- [[loki\|Loki]] - Logs datasource
|
||||
- [[alloy\|Alloy]] - Data collector
|
||||
51
docs/reference/services/jellyfin.md
Normal file
51
docs/reference/services/jellyfin.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
title: Jellyfin
|
||||
tags:
|
||||
- service
|
||||
- media
|
||||
---
|
||||
|
||||
# Jellyfin
|
||||
|
||||
Open-source media server running natively on indri for VideoToolbox hardware transcoding.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://jellyfin.ops.eblu.me |
|
||||
| **Local Port** | 8096 |
|
||||
| **Data** | `~/Library/Application Support/jellyfin` |
|
||||
| **Media** | `/Volumes/allisonflix` (NFS from sifaka) |
|
||||
| **LaunchAgent** | `mcquack.jellyfin` |
|
||||
|
||||
## Hardware Transcoding
|
||||
|
||||
Apple VideoToolbox on M1 Mac Mini.
|
||||
|
||||
| Codec | Support |
|
||||
|-------|---------|
|
||||
| H.264 encode/decode | Hardware |
|
||||
| HEVC (H.265) encode/decode | Hardware |
|
||||
| AV1 decode | Software (requires M3+) |
|
||||
| HDR to SDR tone mapping | VPP (hardware) |
|
||||
|
||||
Concurrent 4K streams with HDR tonemapping: ~3
|
||||
|
||||
## Configuration
|
||||
|
||||
Dashboard > Playback:
|
||||
1. Hardware Acceleration: Apple VideoToolbox
|
||||
2. Allow hardware encoding: Enabled
|
||||
3. VPP Tone mapping: Enabled
|
||||
|
||||
## Observability
|
||||
|
||||
- Metrics: `jellyfin_metrics` ansible role
|
||||
- Logs: Forwarded via [[alloy\|Alloy]]
|
||||
- Dashboard: "Jellyfin Media Server" in [[grafana\|Grafana]]
|
||||
|
||||
## Related
|
||||
|
||||
- [[navidrome\|Navidrome]] - Music streaming
|
||||
- [[storage/sifaka\|Sifaka]] - Media storage
|
||||
52
docs/reference/services/kiwix.md
Normal file
52
docs/reference/services/kiwix.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
title: Kiwix
|
||||
tags:
|
||||
- service
|
||||
- knowledge
|
||||
---
|
||||
|
||||
# Kiwix
|
||||
|
||||
Offline Wikipedia and ZIM archive server.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://kiwix.ops.eblu.me |
|
||||
| **Tailscale URL** | https://kiwix.tail8d86e.ts.net |
|
||||
| **Namespace** | `kiwix` |
|
||||
| **Image** | `ghcr.io/kiwix/kiwix-serve:3.8.1` |
|
||||
| **Storage** | NFS from [[storage/sifaka\|Sifaka]] (`/volume1/torrents`) |
|
||||
|
||||
## Architecture
|
||||
|
||||
| Component | Purpose |
|
||||
|-----------|---------|
|
||||
| kiwix-serve | Serves ZIM files on port 80 |
|
||||
| torrent-sync | Sidecar syncing ZIM torrents to [[transmission\|Transmission]] |
|
||||
| zim-watcher | CronJob (hourly) to restart on new ZIMs |
|
||||
|
||||
## Configured Archives
|
||||
|
||||
- Wikipedia top 1M English articles with images
|
||||
- Project Gutenberg (60,000+ books)
|
||||
- iFixit repair guides
|
||||
- Stack Exchange (SuperUser, Math, etc.)
|
||||
- LibreTexts textbooks
|
||||
- DevDocs developer documentation
|
||||
|
||||
Full list: `argocd/manifests/kiwix/configmap-zim-torrents.yaml`
|
||||
|
||||
## Adding Archives
|
||||
|
||||
1. Edit `configmap-zim-torrents.yaml`
|
||||
2. Add torrent URL from https://download.kiwix.org/zim/
|
||||
3. Sync: `argocd app sync kiwix`
|
||||
4. Torrent-sync adds to [[transmission\|Transmission]]
|
||||
5. zim-watcher restarts kiwix when download completes
|
||||
|
||||
## Related
|
||||
|
||||
- [[transmission\|Transmission]] - Downloads ZIM files
|
||||
- [[storage/sifaka\|Sifaka]] - ZIM storage
|
||||
51
docs/reference/services/loki.md
Normal file
51
docs/reference/services/loki.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
title: Loki
|
||||
tags:
|
||||
- service
|
||||
- observability
|
||||
---
|
||||
|
||||
# Loki
|
||||
|
||||
Log aggregation system for BlumeOps infrastructure.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://loki.ops.eblu.me |
|
||||
| **Tailscale URL** | https://loki.tail8d86e.ts.net |
|
||||
| **Namespace** | `monitoring` |
|
||||
| **Image** | `grafana/loki:3.4.2` |
|
||||
| **Storage** | 50Gi PVC |
|
||||
| **Retention** | 31 days |
|
||||
|
||||
## Architecture
|
||||
|
||||
- Single-node deployment with filesystem storage
|
||||
- TSDB index with 24h period
|
||||
- Logs collected by [[alloy\|Alloy]] and pushed via Loki API
|
||||
- Queried via [[grafana\|Grafana]]
|
||||
|
||||
## Log Sources
|
||||
|
||||
**From Indri (via Alloy):**
|
||||
- forgejo, tailscale (brew services)
|
||||
- alloy, borgmatic, zot, jellyfin (LaunchAgents)
|
||||
|
||||
**From Kubernetes (via Alloy DaemonSet):**
|
||||
- All pods in all namespaces
|
||||
|
||||
## Query Examples (LogQL)
|
||||
|
||||
```logql
|
||||
{service="forgejo"} # All forgejo logs
|
||||
{service="borgmatic", stream="stderr"} # Borgmatic errors
|
||||
{host="indri"} |= "error" # All logs containing "error"
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [[alloy\|Alloy]] - Log collector
|
||||
- [[grafana\|Grafana]] - Log visualization
|
||||
- [[prometheus\|Prometheus]] - Metrics counterpart
|
||||
49
docs/reference/services/miniflux.md
Normal file
49
docs/reference/services/miniflux.md
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
---
|
||||
title: Miniflux
|
||||
tags:
|
||||
- service
|
||||
- rss
|
||||
---
|
||||
|
||||
# Miniflux
|
||||
|
||||
Minimalist RSS/Atom feed reader.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://feed.ops.eblu.me |
|
||||
| **Tailscale URL** | https://feed.tail8d86e.ts.net |
|
||||
| **Namespace** | `miniflux` |
|
||||
| **Image** | `ghcr.io/miniflux/miniflux:latest` |
|
||||
| **Database** | [[postgresql\|PostgreSQL]] |
|
||||
|
||||
## Features
|
||||
|
||||
- Keyboard shortcuts for efficient reading
|
||||
- Fever and Google Reader API compatible
|
||||
- Mobile-friendly web interface
|
||||
- OPML import/export
|
||||
- Content scraping for full articles
|
||||
|
||||
## Database
|
||||
|
||||
Uses CloudNativePG cluster at `pg.ops.eblu.me`.
|
||||
|
||||
Database user password stored in `blumeops-pg-app` secret (auto-generated by CNPG).
|
||||
|
||||
## Backup
|
||||
|
||||
Feed subscriptions and read state backed up via [[borgmatic\|Borgmatic]] PostgreSQL hook.
|
||||
|
||||
## Health Check
|
||||
|
||||
```bash
|
||||
curl https://feed.ops.eblu.me/healthcheck
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [[postgresql\|PostgreSQL]] - Database backend
|
||||
- [[borgmatic\|Borgmatic]] - Data backup
|
||||
52
docs/reference/services/navidrome.md
Normal file
52
docs/reference/services/navidrome.md
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
title: Navidrome
|
||||
tags:
|
||||
- service
|
||||
- media
|
||||
---
|
||||
|
||||
# Navidrome
|
||||
|
||||
Self-hosted music streaming server.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://dj.ops.eblu.me |
|
||||
| **Tailscale URL** | https://dj.tail8d86e.ts.net |
|
||||
| **Namespace** | `navidrome` |
|
||||
| **Manifests** | `argocd/manifests/navidrome/` |
|
||||
|
||||
## Storage
|
||||
|
||||
| Mount | Type | Source | Access |
|
||||
|-------|------|--------|--------|
|
||||
| /music | NFS PV | sifaka:/volume1/music | Read-only |
|
||||
| /data | Local PVC (10Gi) | minikube storage | Read-write |
|
||||
|
||||
The `/data` directory contains SQLite database, configuration, and cache.
|
||||
|
||||
## Configuration
|
||||
|
||||
| Variable | Value |
|
||||
|----------|-------|
|
||||
| `ND_SCANSCHEDULE` | 1h |
|
||||
| `ND_LOGLEVEL` | info |
|
||||
| `ND_MUSICFOLDER` | /music |
|
||||
| `ND_DATAFOLDER` | /data |
|
||||
|
||||
## Initial Setup
|
||||
|
||||
On first access, Navidrome prompts to create an admin user. No default credentials.
|
||||
|
||||
## Verify NFS Mount
|
||||
|
||||
```bash
|
||||
kubectl --context=minikube-indri -n navidrome exec deploy/navidrome -- ls /music
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [[jellyfin\|Jellyfin]] - Video streaming
|
||||
- [[storage/sifaka\|Sifaka]] - Music storage
|
||||
68
docs/reference/services/postgresql.md
Normal file
68
docs/reference/services/postgresql.md
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
---
|
||||
title: PostgreSQL
|
||||
tags:
|
||||
- service
|
||||
- database
|
||||
---
|
||||
|
||||
# PostgreSQL
|
||||
|
||||
Database cluster via CloudNativePG operator.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | `tcp://pg.ops.eblu.me:5432` |
|
||||
| **Metrics** | `http://cnpg-metrics.tail8d86e.ts.net:9187/metrics` |
|
||||
| **Namespace** | `databases` |
|
||||
| **Cluster** | `blumeops-pg` |
|
||||
| **Operator** | CloudNativePG |
|
||||
|
||||
## Databases
|
||||
|
||||
| Database | Owner | Purpose |
|
||||
|----------|-------|---------|
|
||||
| miniflux | miniflux | [[miniflux\|Miniflux]] feed data |
|
||||
| teslamate | teslamate | [[teslamate\|TeslaMate]] vehicle data |
|
||||
|
||||
## Users
|
||||
|
||||
| User | Role | Purpose |
|
||||
|------|------|---------|
|
||||
| postgres | superuser | CNPG internal |
|
||||
| miniflux | app owner | Owns miniflux database |
|
||||
| teslamate | superuser | TeslaMate (needs extensions) |
|
||||
| eblume | superuser | Admin access |
|
||||
| borgmatic | pg_read_all_data | [[borgmatic\|Backup]] access |
|
||||
|
||||
## Quick Connect
|
||||
|
||||
```bash
|
||||
PGPASSWORD=$(op --vault blumeops item get <item-id> --fields password --reveal) \
|
||||
psql -h pg.ops.eblu.me -U eblume -d miniflux
|
||||
```
|
||||
|
||||
## Backup
|
||||
|
||||
Backed up via [[borgmatic\|Borgmatic]] `postgresql_databases` hook.
|
||||
|
||||
Borgmatic streams `pg_dump` directly to Borg (no intermediate files, no downtime).
|
||||
|
||||
## Credentials
|
||||
|
||||
**1Password items:**
|
||||
- `guxu3j7ajhjyey6xxl2ovsl2ui` - eblume password
|
||||
- `mw2bv5we7woicjza7hc6s44yvy` - borgmatic password
|
||||
|
||||
**CNPG-managed secrets:**
|
||||
- `blumeops-pg-app` - miniflux user
|
||||
- `blumeops-pg-eblume` - eblume superuser
|
||||
- `blumeops-pg-borgmatic` - borgmatic backup user
|
||||
- `blumeops-pg-teslamate` - teslamate user
|
||||
|
||||
## Related
|
||||
|
||||
- [[miniflux\|Miniflux]] - Feed reader database
|
||||
- [[teslamate\|TeslaMate]] - Vehicle data database
|
||||
- [[borgmatic\|Borgmatic]] - Database backup
|
||||
54
docs/reference/services/prometheus.md
Normal file
54
docs/reference/services/prometheus.md
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
title: Prometheus
|
||||
tags:
|
||||
- service
|
||||
- observability
|
||||
---
|
||||
|
||||
# Prometheus
|
||||
|
||||
Metrics storage and querying for BlumeOps infrastructure.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://prometheus.ops.eblu.me |
|
||||
| **Tailscale URL** | https://prometheus.tail8d86e.ts.net |
|
||||
| **Namespace** | `monitoring` |
|
||||
| **Image** | `prom/prometheus:v3.2.1` |
|
||||
| **Storage** | 50Gi PVC |
|
||||
|
||||
## Data Sources
|
||||
|
||||
### Remote Write (from Alloy)
|
||||
- Indri system metrics via [[alloy\|Alloy]] remote_write
|
||||
- Textfile metrics: minikube, borgmatic, zot, jellyfin
|
||||
|
||||
### Scrape Targets
|
||||
| Target | Metrics |
|
||||
|--------|---------|
|
||||
| `sifaka:9100` | [[storage/sifaka\|Sifaka]] NAS (node_exporter) |
|
||||
| `cnpg-metrics.tail8d86e.ts.net:9187` | [[postgresql\|CloudNativePG]] metrics |
|
||||
| `kube-state-metrics.monitoring.svc:8080` | Kubernetes resource metrics |
|
||||
|
||||
## Query API
|
||||
|
||||
```bash
|
||||
# Check targets
|
||||
curl -s https://prometheus.ops.eblu.me/api/v1/targets | jq '.data.activeTargets[].scrapeUrl'
|
||||
```
|
||||
|
||||
## ArgoCD Management
|
||||
|
||||
```bash
|
||||
argocd app sync prometheus
|
||||
```
|
||||
|
||||
Manifests: `argocd/manifests/prometheus/`
|
||||
|
||||
## Related
|
||||
|
||||
- [[alloy\|Alloy]] - Metrics collector
|
||||
- [[grafana\|Grafana]] - Visualization
|
||||
- [[loki\|Loki]] - Logs counterpart
|
||||
58
docs/reference/services/teslamate.md
Normal file
58
docs/reference/services/teslamate.md
Normal file
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
title: TeslaMate
|
||||
tags:
|
||||
- service
|
||||
- vehicle
|
||||
---
|
||||
|
||||
# TeslaMate
|
||||
|
||||
Self-hosted Tesla data logger collecting vehicle telemetry from the Tesla Owner API.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://tesla.ops.eblu.me |
|
||||
| **Tailscale URL** | https://tesla.tail8d86e.ts.net |
|
||||
| **Namespace** | `teslamate` |
|
||||
| **Image** | `teslamate/teslamate:2.2.0` |
|
||||
| **Database** | [[postgresql\|PostgreSQL]] |
|
||||
|
||||
## Data Collected
|
||||
|
||||
- Battery level, state of charge, range estimates
|
||||
- Charging sessions (location, energy, cost, duration)
|
||||
- Drives (distance, efficiency, routes)
|
||||
- Climate/HVAC usage
|
||||
- Software update history
|
||||
- Vampire drain analysis
|
||||
- Vehicle states (asleep, driving, charging, online)
|
||||
|
||||
## Grafana Dashboards
|
||||
|
||||
18 dashboards in the "TeslaMate" folder:
|
||||
- Overview, Charges, Drives, Efficiency, States
|
||||
- Battery Health, Vampire Drain, Statistics
|
||||
- Charge Level, Locations, Trip, Mileage
|
||||
- Drive Stats, Charging Stats, Projected Range
|
||||
- Timeline, Updates, Visited
|
||||
|
||||
Dashboards use PostgreSQL datasource (not Prometheus).
|
||||
|
||||
## Authentication
|
||||
|
||||
Uses Tesla Owner API via OAuth:
|
||||
1. Access https://tesla.ops.eblu.me
|
||||
2. Click "Sign in with Tesla"
|
||||
3. Tokens encrypted with ENCRYPTION_KEY
|
||||
|
||||
## Credentials
|
||||
|
||||
**1Password:** `TeslaMate` item with `db_password` and `api_enc_key`
|
||||
|
||||
## Related
|
||||
|
||||
- [[postgresql\|PostgreSQL]] - Data storage
|
||||
- [[grafana\|Grafana]] - Dashboards
|
||||
- [[borgmatic\|Borgmatic]] - Database backup
|
||||
53
docs/reference/services/transmission.md
Normal file
53
docs/reference/services/transmission.md
Normal file
|
|
@ -0,0 +1,53 @@
|
|||
---
|
||||
title: Transmission
|
||||
tags:
|
||||
- service
|
||||
- torrent
|
||||
---
|
||||
|
||||
# Transmission
|
||||
|
||||
BitTorrent daemon, primarily for downloading ZIM archives for [[kiwix\|Kiwix]].
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://torrent.ops.eblu.me |
|
||||
| **Tailscale URL** | https://torrent.tail8d86e.ts.net |
|
||||
| **Namespace** | `torrent` |
|
||||
| **Image** | `lscr.io/linuxserver/transmission:latest` |
|
||||
| **Storage** | NFS PVC from [[storage/sifaka\|Sifaka]] |
|
||||
|
||||
## Storage Layout
|
||||
|
||||
NFS share on sifaka (`/volume1/torrents`):
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `/downloads/` | Active downloads and metadata |
|
||||
| `/downloads/complete/` | Completed downloads |
|
||||
| `/config/` | Transmission configuration |
|
||||
| `/watch/` | Watch directory for .torrent files |
|
||||
|
||||
[[kiwix\|Kiwix]] reads from `/downloads/complete/` to serve ZIM archives.
|
||||
|
||||
## Integration with Kiwix
|
||||
|
||||
The Kiwix deployment includes a torrent-sync sidecar that:
|
||||
1. Reads ZIM torrent list from ConfigMap
|
||||
2. Adds missing torrents via RPC
|
||||
3. Runs on startup and every 30 minutes
|
||||
|
||||
When downloads complete, the zim-watcher CronJob detects new ZIMs and restarts Kiwix.
|
||||
|
||||
## Monitoring
|
||||
|
||||
Basic uptime via blackbox probe in [[alloy\|Alloy]] k8s (Services Health dashboard).
|
||||
|
||||
Web UI shows: active/seeding/paused counts, speeds, disk usage.
|
||||
|
||||
## Related
|
||||
|
||||
- [[kiwix\|Kiwix]] - ZIM archive consumer
|
||||
- [[storage/sifaka\|Sifaka]] - Download storage
|
||||
66
docs/reference/services/zot.md
Normal file
66
docs/reference/services/zot.md
Normal file
|
|
@ -0,0 +1,66 @@
|
|||
---
|
||||
title: Zot
|
||||
tags:
|
||||
- service
|
||||
- registry
|
||||
---
|
||||
|
||||
# Zot
|
||||
|
||||
OCI-native container registry providing pull-through cache and private image storage.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **URL** | https://registry.ops.eblu.me |
|
||||
| **Local Port** | 5050 |
|
||||
| **Data** | `~/zot` |
|
||||
| **Config** | `~/.config/zot/config.json` |
|
||||
| **LaunchAgent** | mcquack |
|
||||
|
||||
## Namespace Convention
|
||||
|
||||
| Path | Source |
|
||||
|------|--------|
|
||||
| `registry.ops.eblu.me/docker.io/*` | Cached from Docker Hub |
|
||||
| `registry.ops.eblu.me/ghcr.io/*` | Cached from GHCR |
|
||||
| `registry.ops.eblu.me/quay.io/*` | Cached from Quay |
|
||||
| `registry.ops.eblu.me/blumeops/*` | Private images |
|
||||
|
||||
## Pull-Through Cache
|
||||
|
||||
When [[kubernetes/cluster\|minikube]] pulls an image:
|
||||
1. Containerd checks zot first (`host.minikube.internal:5050`)
|
||||
2. If cached, returns immediately
|
||||
3. If not, zot fetches from upstream, caches, returns
|
||||
|
||||
## Private Images
|
||||
|
||||
```bash
|
||||
# Build and push from gilbert
|
||||
podman build -t registry.ops.eblu.me/blumeops/myapp:v1 .
|
||||
podman push registry.ops.eblu.me/blumeops/myapp:v1
|
||||
|
||||
# Use in k8s manifest
|
||||
image: registry.ops.eblu.me/blumeops/myapp:v1
|
||||
```
|
||||
|
||||
## Security Model
|
||||
|
||||
Network access only (no authentication). Defense is the Tailscale ACL boundary.
|
||||
|
||||
## Useful Commands
|
||||
|
||||
```bash
|
||||
# List all images
|
||||
curl -s http://indri:5050/v2/_catalog | jq
|
||||
|
||||
# List tags
|
||||
curl -s http://indri:5050/v2/blumeops/devpi/tags/list | jq
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [[forgejo\|Forgejo]] - Container build CI
|
||||
- [[kubernetes/cluster\|Cluster]] - Registry consumer
|
||||
84
docs/reference/storage/backups.md
Normal file
84
docs/reference/storage/backups.md
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
---
|
||||
title: Backup Policy
|
||||
tags:
|
||||
- storage
|
||||
- backup
|
||||
---
|
||||
|
||||
# Backup Policy
|
||||
|
||||
Daily automated backups from [[infrastructure/hosts\|Indri]] to [[storage/sifaka\|Sifaka]] NAS.
|
||||
|
||||
## Schedule
|
||||
|
||||
| Time | Frequency | System |
|
||||
|------|-----------|--------|
|
||||
| 2:00 AM | Daily | [[services/borgmatic\|Borgmatic]] |
|
||||
|
||||
## What Gets Backed Up
|
||||
|
||||
### Directories
|
||||
|
||||
| Path | Description | Priority |
|
||||
|------|-------------|----------|
|
||||
| `~/code/personal/zk` | Zettelkasten notes | Critical |
|
||||
| `/opt/homebrew/var/forgejo` | Git repositories | Critical |
|
||||
| `~/.config/borgmatic` | Backup config | High |
|
||||
| `~/Documents` | Personal documents | High |
|
||||
| `~/Pictures` | Photos | Medium |
|
||||
|
||||
### Databases
|
||||
|
||||
| Database | Host | Method |
|
||||
|----------|------|--------|
|
||||
| miniflux | [[services/postgresql\|pg.ops.eblu.me]] | pg_dump stream |
|
||||
| teslamate | [[services/postgresql\|pg.ops.eblu.me]] | pg_dump stream |
|
||||
|
||||
## What Is NOT Backed Up
|
||||
|
||||
| Data | Reason |
|
||||
|------|--------|
|
||||
| ZIM archives (`~/transmission/`) | Re-downloadable via torrent |
|
||||
| Prometheus metrics | Ephemeral, in k8s PVC |
|
||||
| Loki logs | Ephemeral, in k8s PVC |
|
||||
| devpi cache | Re-fetchable from PyPI |
|
||||
|
||||
## Retention Policy
|
||||
|
||||
| Period | Retention |
|
||||
|--------|-----------|
|
||||
| Daily | 7 backups |
|
||||
| Monthly | 12 backups |
|
||||
| Yearly | 1000 backups |
|
||||
|
||||
## Backup Target
|
||||
|
||||
Repository: `/Volumes/backups/borg/` on [[storage/sifaka\|Sifaka]]
|
||||
|
||||
## Monitoring
|
||||
|
||||
Metrics exposed to [[services/prometheus\|Prometheus]]:
|
||||
- `borgmatic_up` - Repository accessible
|
||||
- `borgmatic_last_archive_timestamp` - Last backup time
|
||||
- `borgmatic_repo_deduplicated_size_bytes` - Disk usage
|
||||
|
||||
Dashboard: "Borgmatic Backups" in [[services/grafana\|Grafana]]
|
||||
|
||||
## Recovery
|
||||
|
||||
```bash
|
||||
# List archives
|
||||
ssh indri 'mise x -- borgmatic list'
|
||||
|
||||
# Extract specific path from latest
|
||||
ssh indri 'mise x -- borgmatic extract --archive latest --path /some/path'
|
||||
|
||||
# Check repository health
|
||||
ssh indri 'mise x -- borgmatic check'
|
||||
```
|
||||
|
||||
## Related
|
||||
|
||||
- [[services/borgmatic\|Borgmatic]] - Backup system details
|
||||
- [[storage/sifaka\|Sifaka]] - Backup storage
|
||||
- [[services/postgresql\|PostgreSQL]] - Database backups
|
||||
51
docs/reference/storage/sifaka.md
Normal file
51
docs/reference/storage/sifaka.md
Normal file
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
title: Sifaka NAS
|
||||
tags:
|
||||
- storage
|
||||
---
|
||||
|
||||
# Sifaka NAS
|
||||
|
||||
Synology NAS providing network storage and backup target.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Dashboard** | https://nas.ops.eblu.me |
|
||||
| **Model** | Synology |
|
||||
| **Storage** | 10.9TB RAID 5 |
|
||||
| **Role** | Backup target, media storage |
|
||||
|
||||
## Network Shares
|
||||
|
||||
| Share | Path | Purpose | Consumers |
|
||||
|-------|------|---------|-----------|
|
||||
| backups | `/volume1/backups` | Borg backup repository | [[services/borgmatic\|Borgmatic]] |
|
||||
| torrents | `/volume1/torrents` | ZIM downloads | [[services/kiwix\|Kiwix]], [[services/transmission\|Transmission]] |
|
||||
| music | `/volume1/music` | Music library | [[services/navidrome\|Navidrome]] |
|
||||
| allisonflix | `/volume1/allisonflix` | Video library | [[services/jellyfin\|Jellyfin]] |
|
||||
| photos | `/volume1/photos` | Photo library | Immich |
|
||||
|
||||
## NFS Exports
|
||||
|
||||
| Export | Allowed Clients | Purpose |
|
||||
|--------|-----------------|---------|
|
||||
| `/volume1/torrents` | 192.168.1.0/24, 100.64.0.0/10 | k8s pods via Docker NAT |
|
||||
| `/volume1/music` | 192.168.1.0/24, 100.64.0.0/10 | k8s pods via Docker NAT |
|
||||
|
||||
## Monitoring
|
||||
|
||||
Node exporter running in Docker container, scraped by [[services/prometheus\|Prometheus]] at `sifaka:9100`.
|
||||
|
||||
## Tailscale
|
||||
|
||||
- Tag: `tag:nas`
|
||||
- ACL: `tag:homelab` can access for backups
|
||||
|
||||
## Related
|
||||
|
||||
- [[storage/backups\|Backups]] - Backup policy
|
||||
- [[services/borgmatic\|Borgmatic]] - Backup system
|
||||
- [[services/jellyfin\|Jellyfin]] - Media consumer
|
||||
- [[services/navidrome\|Navidrome]] - Music consumer
|
||||
Loading…
Add table
Add a link
Reference in a new issue