gohoarder/deployments/kubernetes/README.md

Kubernetes Deployment Guide

This directory contains Kubernetes manifests for deploying GoHoarder in a production environment.

Architecture Overview

┌─────────────────────────────────────────────────────┐
│                   Kubernetes Pod                     │
│  ┌────────────────────────────────────────────────┐ │
│  │            GoHoarder Container                 │ │
│  │  ┌──────────────────────────────────────────┐ │ │
│  │  │  Pattern-Based Credential Store          │ │ │
│  │  │  ├─ github.com/myorg/* → token_A         │ │ │
│  │  │  ├─ gitlab.com/team/* → token_B          │ │ │
│  │  │  └─ * → fallback_token                   │ │ │
│  │  └──────────────────────────────────────────┘ │ │
│  └────────────────────────────────────────────────┘ │
│                                                       │
│  Mounted Volumes:                                    │
│  • config.yaml (ConfigMap)                           │
│  • git-credentials.json (Secret)                     │
│  • /var/lib/gohoarder/cache (PVC)                    │
│  • /var/lib/gohoarder (PVC for metadata DB)          │
└─────────────────────────────────────────────────────┘

Files

• secret-git-credentials.yaml - Git credentials for private repositories
• configmap-config.yaml - Application configuration
• pvc.yaml - Persistent volume claims for cache and metadata
• deployment.yaml - Main application deployment
• service.yaml - Service and optional ingress configuration

Quick Start

1. Configure Git Credentials

Edit secret-git-credentials.yaml and replace the placeholder tokens with your actual tokens:

{
  "credentials": [
    {
      "pattern": "github.com/mycompany/*",
      "host": "github.com",
      "username": "oauth2",
      "token": "ghp_YOUR_ACTUAL_TOKEN_HERE"
    }
  ]
}

Pattern Matching Examples:

• github.com/myorg/* - Matches all repos under myorg
• github.com/myorg/specific-repo - Matches only specific-repo
• gitlab.com/backend-team/* - Matches all GitLab repos under backend-team
• * - Fallback pattern (matches everything)

Credential Priority:

1. Most specific pattern wins (longest match)
2. Fallback credential ("fallback": true)
3. System git config (if no matches)

2. Customize Configuration

Edit configmap-config.yaml to adjust:

• Cache size (max_size_bytes)
• Security scanning settings
• Upstream registries
• Logging level

3. Deploy to Kubernetes

# Create namespace (optional)
kubectl create namespace gohoarder

# Apply manifests
kubectl apply -f pvc.yaml
kubectl apply -f secret-git-credentials.yaml
kubectl apply -f configmap-config.yaml
kubectl apply -f deployment.yaml
kubectl apply -f service.yaml

# Verify deployment
kubectl get pods -l app=gohoarder
kubectl logs -l app=gohoarder -f

4. Configure Go Client

# Set GOPROXY environment variable
export GOPROXY=http://gohoarder.default.svc.cluster.local:8080/go,direct

# Or in your Dockerfile
ENV GOPROXY=http://gohoarder.default.svc.cluster.local:8080/go,direct

# Test with a private module
go get github.com/mycompany/private-module@latest

Advanced Configuration

Using External Secrets Operator (ESO)

If you're using External Secrets Operator, uncomment the ExternalSecret section in secret-git-credentials.yaml and configure your SecretStore:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: gohoarder-git-credentials
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: SecretStore
  target:
    name: gohoarder-git-credentials
  data:
  - secretKey: credentials.json
    remoteRef:
      key: secret/gohoarder/git-credentials

Storage Classes

For production deployments, specify appropriate storage classes:

# In pvc.yaml
storageClassName: fast-ssd  # For cache (needs fast I/O)
storageClassName: standard  # For metadata (smaller, less critical)

Horizontal Pod Autoscaling

kubectl autoscale deployment gohoarder \
  --cpu-percent=70 \
  --min=2 \
  --max=10

Monitoring

Check health and metrics:

# Health check
kubectl port-forward svc/gohoarder 8080:8080
curl http://localhost:8080/health

# Metrics (Prometheus format)
curl http://localhost:8080/metrics

Multi-Organization Setup

Example 1: Multiple GitHub Organizations

{
  "credentials": [
    {
      "pattern": "github.com/company-frontend/*",
      "host": "github.com",
      "username": "oauth2",
      "token": "ghp_frontend_team_token"
    },
    {
      "pattern": "github.com/company-backend/*",
      "host": "github.com",
      "username": "oauth2",
      "token": "ghp_backend_team_token"
    },
    {
      "pattern": "github.com/company-infra/*",
      "host": "github.com",
      "username": "oauth2",
      "token": "ghp_infra_team_token"
    },
    {
      "pattern": "*",
      "host": "*",
      "username": "oauth2",
      "token": "ghp_readonly_default_token",
      "fallback": true
    }
  ]
}

Example 2: GitHub + GitLab

{
  "credentials": [
    {
      "pattern": "github.com/myorg/*",
      "host": "github.com",
      "username": "oauth2",
      "token": "ghp_github_token"
    },
    {
      "pattern": "gitlab.com/myteam/*",
      "host": "gitlab.com",
      "username": "oauth2",
      "token": "glpat_gitlab_token"
    }
  ]
}

Example 3: Enterprise GitHub

{
  "credentials": [
    {
      "pattern": "github.enterprise.com/engineering/*",
      "host": "github.enterprise.com",
      "username": "oauth2",
      "token": "ghp_enterprise_token"
    }
  ]
}

Security Best Practices

1. Token Scoping: Use fine-grained personal access tokens with minimal permissions

   • GitHub: Only repo scope needed for private repos
   • GitLab: Only read_repository scope needed

2. Secret Rotation: Regularly rotate tokens

   # Update secret
   kubectl create secret generic gohoarder-git-credentials \
     --from-file=credentials.json=./new-credentials.json \
     --dry-run=client -o yaml | kubectl apply -f -
   
   # Restart pods to pick up new credentials
   kubectl rollout restart deployment gohoarder
   

3. RBAC: Limit who can read the secret

   kubectl create role secret-reader \
     --verb=get,list \
     --resource=secrets \
     --resource-name=gohoarder-git-credentials
   

4. Audit Logging: Enable Kubernetes audit logging for secret access

5. Network Policies: Restrict which pods can access GoHoarder

   apiVersion: networking.k8s.io/v1
   kind: NetworkPolicy
   metadata:
     name: allow-from-build-namespace
   spec:
     podSelector:
       matchLabels:
         app: gohoarder
     ingress:
     - from:
       - namespaceSelector:
           matchLabels:
             name: build-namespace
   

Troubleshooting

Check if credentials are loaded

# Check logs for credential loading
kubectl logs -l app=gohoarder | grep "Loaded git credentials"

# Expected output:
# {"level":"info","file":"/etc/gohoarder/git-credentials.json","credentials":3,"message":"Loaded git credentials from file"}
# {"level":"debug","pattern":"github.com/myorg/*","host":"github.com","message":"Registered credential pattern"}

Test credential pattern matching

# Enable debug logging
kubectl set env deployment/gohoarder LOG_LEVEL=debug

# Watch logs during a go get request
kubectl logs -l app=gohoarder -f

Common Issues

Issue: git clone failed: authentication required

• Cause: No matching credential pattern
• Solution: Check pattern syntax in credentials.json, ensure it matches the module path

Issue: Failed to load git credentials

• Cause: Secret not mounted or JSON syntax error
• Solution: Verify secret exists and JSON is valid

  kubectl get secret gohoarder-git-credentials
  kubectl get secret gohoarder-git-credentials -o jsonpath='{.data.credentials\.json}' | base64 -d | jq .
  

Issue: Module fetch slow

• Cause: Git clone timeout or large repository
• Solution: Increase timeout in config.yaml or use upstream proxy for public modules

Performance Tuning

Cache Configuration

cache:
  max_size_bytes: 107374182400  # 100GB for large organizations
  default_ttl: 168h              # 7 days for stable modules

Resource Limits

For high-traffic deployments:

resources:
  requests:
    memory: "2Gi"
    cpu: "1000m"
  limits:
    memory: "8Gi"
    cpu: "4000m"

Replicas

Run multiple replicas for high availability:

spec:
  replicas: 3

Backup and Recovery

Backup Metadata Database

# Backup SQLite database
kubectl exec -it deployment/gohoarder -- \
  sqlite3 /var/lib/gohoarder/gohoarder.db ".backup /tmp/backup.db"

kubectl cp gohoarder-pod:/tmp/backup.db ./gohoarder-backup-$(date +%Y%m%d).db

Restore from Backup

kubectl cp ./gohoarder-backup-20260102.db gohoarder-pod:/var/lib/gohoarder/gohoarder.db
kubectl rollout restart deployment gohoarder

Integration Examples

CI/CD Pipeline (GitHub Actions)

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3

    - name: Set up Go
      uses: actions/setup-go@v4
      with:
        go-version: '1.21'

    - name: Configure GOPROXY
      run: |
        echo "GOPROXY=http://gohoarder.company.internal:8080/go,direct" >> $GITHUB_ENV

    - name: Build
      run: go build ./...

Dockerfile

FROM golang:1.21-alpine

# Configure proxy
ENV GOPROXY=http://gohoarder.default.svc.cluster.local:8080/go,direct
ENV GONOPROXY=none
ENV GONOSUMDB=github.com/yourcompany

WORKDIR /app
COPY . .
RUN go build -o myapp ./cmd/myapp

CMD ["/app/myapp"]

Support

For issues or questions:

• Check logs: kubectl logs -l app=gohoarder
• Enable debug logging: Set logging.level: debug in ConfigMap
• Review credential patterns in Secret