MAJOR PERFORMANCE IMPROVEMENT: Reduced frontend Docker build time
from ~20 minutes to ~30 seconds by building the SPA once on the
runner instead of twice (amd64 + arm64) in Docker.
Changes:
1. Release workflow now builds frontend on CI runner (native, fast)
2. Frontend artifact uploaded and downloaded in release job
3. Dockerfile.frontend simplified to just copy pre-built files
4. Multi-arch Docker build is now just copying files into nginx
Before:
- Docker builds frontend 2x (amd64 + arm64 with QEMU emulation)
- Each: pnpm install + pnpm build = ~10 min per arch
- Total: ~20 minutes for frontend image
After:
- Build frontend 1x on runner = ~2 min (native)
- Docker just copies files = ~30 sec (both architectures)
- Total: ~2.5 minutes for frontend image
Impact:
- 8x faster frontend builds
- Total release time reduced from ~25 min to ~7 min
- Lower resource usage (no QEMU emulation)
Files changed:
- .github/workflows/release.yaml: Enable node build
- Dockerfile.frontend: Remove build stage, expect pre-built files
- .goreleaser.yaml: Copy frontend/dist instead of full source
Eliminated duplicate nginx containers by merging gateway reverse proxy
functionality into the frontend container. This simplifies deployment
and reduces resource usage.
Architecture changes:
- Frontend now serves both static files AND reverse proxies to backend
- Single nginx container handles all HTTP routing
- Gateway container removed from builds and Helm chart
Dockerfile.frontend changes:
- Added upstream backend configuration
- Added proxy locations for /api, /health, /metrics, /npm, /pypi, /go, /ws
- Added rate limiting for API and downloads
- Added WebSocket support
- Configurable via BACKEND_HOST and BACKEND_PORT env vars
Helm chart changes:
- Updated frontend deployment to configure backend connection
- Simplified ingress to single route (all traffic → frontend)
- Frontend proxies backend requests internally
- Removed separate frontend/api ingress configurations
GoReleaser changes:
- Removed gohoarder-gateway Docker build
- Now builds: server, scanner, migrate, frontend (4 images)
Benefits:
- Fewer containers to manage
- Reduced complexity in Docker Compose and Kubernetes
- Single point of configuration for routing
- Better resource utilization
GoReleaser creates a temporary build context with only binaries, but our
Dockerfiles are multi-stage builds that need the full source code to
compile. Added extra_files to copy necessary directories.
Files copied per image:
- gohoarder-server: go.mod, go.sum, cmd, pkg, internal, config
- gohoarder-scanner: go.mod, go.sum, cmd, pkg, internal, config
- gohoarder-migrate: go.mod, go.sum, cmd, pkg, internal, migrations
- gohoarder-frontend: frontend/ directory (Node.js source)
- gohoarder-gateway: no extra files needed (static config)
This fixes the build context error:
"Seems like you tried to copy a file that is not available in the
build context."
Docker SBOM attestations require docker-container driver which is not
available in the default Docker driver used by GoReleaser. Disabled
SBOM generation for all Docker images to prevent build failures.
Error fixed:
- "Attestation is not supported for the docker driver"
Applied to all Docker images:
- gohoarder-server
- gohoarder-frontend
- gohoarder-scanner
- gohoarder-gateway
- gohoarder-migrate
Problem:
- linux_amd64 runner was trying to build BOTH:
- linux/amd64 (native - OK)
- linux/arm64 (cross-compile with CGO - FAILS)
- Error: gcc_arm64.S assembler errors when cross-compiling ARM64 on x86_64
- Workflow default platforms only include linux/amd64, not linux/arm64
Solution:
- Added linux/arm64 to ignore list in both builds
- Only build linux/amd64 binaries (native compilation on ubuntu-latest)
- Docker images still provide linux/arm64 via multi-stage builds
- Users get ARM64 support through Docker, not standalone binaries
Build matrix now:
- ✅ darwin/arm64 (macOS Apple Silicon) - native on macos-latest
- ✅ linux/amd64 (Linux x86_64) - native on ubuntu-latest
- ❌ linux/arm64 (skipped for binaries, available in Docker)
This eliminates CGO cross-compilation while maintaining full platform support
via Docker multi-arch images.
Problem:
- With builds: skip: true, no artifacts were created
- GoReleaser wasn't creating GitHub releases or tags
- Helm chart workflow wasn't triggered (depends on tags)
- No downloadable binaries for users
Solution:
- Enabled builds for both gohoarder and migrate binaries
- CGO_ENABLED=1 for SQLite support
- Added fts5 tag for full-text search
- Builds run natively per platform in split/merge workflow:
- darwin/arm64 (Apple Silicon Macs)
- linux/amd64 (x86_64 Linux)
- linux/arm64 (ARM64 Linux)
- Ignored darwin/amd64 (Intel Macs) to limit build matrix
How it works:
1. Split phase: Each platform builds natively (no cross-compilation)
2. Merge phase: Combines all artifacts, creates release, builds Docker images
3. Docker images still use multi-stage builds (independent of binaries)
4. GitHub release created with tags
5. Helm chart workflow triggered
Benefits:
- Downloadable binaries for all platforms
- Archives created automatically
- GitHub releases with proper tags
- Helm charts published
- Docker images built separately with multi-stage builds
Problem:
- Used incorrect field names (use: buildx, build_flag_templates) not supported in GoReleaser v2.13.2
- GitHub Actions workflow using non-CGO release workflow
- Docker builds failing due to invalid configuration
Solution:
- Updated dockers_v2 configuration with correct field names:
- Removed unsupported `use: buildx` field
- Changed `build_flag_templates` to `build_args` (map format)
- Kept `platforms` for multi-arch support (linux/amd64, linux/arm64)
- Updated GitHub Actions workflow to use go-release-cgo.yaml for CGO support
- Build args now passed correctly to Docker builds for version info
Changes:
- .goreleaser.yaml: Fixed all Docker image configurations
- .github/workflows/release.yaml: Changed to go-release-cgo.yaml workflow
Validation:
- goreleaser check: PASSED ✓
- Configuration validated with GoReleaser Pro v2.13.2
References:
- GoReleaser dockers_v2 docs: https://goreleaser.com/customization/dockers_v2/
Problem:
- GoReleaser Pro features (use: buildx, build_flag_templates) not available in free version
- CI/CD failing with "field not found in type config.DockerV2" errors
Solution:
- Split each Docker image into separate amd64 and arm64 builds
- Use goarch field to specify architecture
- Use build_flags instead of build_flag_templates
- Add docker_manifests section to combine arch-specific images into multi-arch manifests
Changes:
- Each service now has two Docker image definitions (amd64 and arm64)
- Images tagged with architecture suffix (e.g., v1.0.0-amd64, v1.0.0-arm64)
- Docker manifests combine them into unified tags (e.g., v1.0.0, latest)
- Users can pull multi-arch images normally, Docker will select correct arch
Result:
- Works with free GoReleaser version
- Maintains multi-architecture support
- Multi-stage Dockerfiles compile for each architecture natively
Problem:
- Enabling CGO_ENABLED=1 for SQLite support caused cross-compilation failures
- ARM64 assembly errors when building from amd64 host
- Cross-compilation with CGO requires architecture-specific toolchains
Solution:
- Converted all Dockerfiles to multi-stage builds
- Binaries now compile inside Docker using native platform builders
- Used --platform flag to build for target architecture natively
- Removed binary builds from .goreleaser.yaml (skip: true)
- Updated dockers_v2 to use buildx with multi-platform support
Changes:
- .goreleaser.yaml: Skip standalone builds, use Docker buildx
- Dockerfile.server: Multi-stage build with CGO
- Dockerfile.scanner: Multi-stage build with CGO
- Dockerfile.migrate: Multi-stage build with CGO
Benefits:
- No cross-compilation needed (each platform builds natively)
- Docker buildx handles multi-platform builds automatically
- SQLite support working with CGO enabled
- Cleaner separation between build and runtime environments
Changes:
- Set CGO_ENABLED=1 for gohoarder main binary in .goreleaser.yaml
- Add sqlite-libs and musl to Dockerfile.server
- Add sqlite-libs and musl to Dockerfile.scanner
All Go binaries that interact with SQLite now have CGO enabled:
✅ gohoarder (main binary) - used by server and scanner
✅ migrate (migration tool)
Runtime containers include necessary C libraries:
✅ Dockerfile.server - SQLite runtime support
✅ Dockerfile.scanner - SQLite runtime support
✅ Dockerfile.migrate - SQLite runtime support
This fixes: 'Binary was compiled with CGO_ENABLED=0, go-sqlite3 requires cgo'
Changes:
- Set CGO_ENABLED=1 for migrate build in .goreleaser.yaml
- Add sqlite-libs and musl runtime dependencies to Dockerfile.migrate
This fixes the migration error: 'Binary was compiled with CGO_ENABLED=0,
go-sqlite3 requires cgo to work'
- [x] Implement GORM V2 metadata store with SQLite, PostgreSQL, and MySQL support
- [x] Add database migration system using gormigrate for schema versioning
- [x] Create migration CLI tool with support for migrate, rollback, and status commands
- [x] Add Docker support for migration container (Dockerfile.migrate)
- [x] Implement automatic partition management for PostgreSQL time-series tables
- [x] Add background aggregation worker for download statistics
- [x] Support connection pooling configuration (max_open_conns, max_idle_conns, conn_max_lifetime)
- [x] Add blocking mechanism based on vulnerability thresholds in stats and handlers
- [x] Update Helm charts with migration init containers and multi-database configuration
- [x] Replace deprecated SQLite store with optimized GORM implementation
- [x] Add comprehensive integration tests for MySQL and PostgreSQL
- [x] Update frontend to display blocked packages and storage utilization
- [x] Add goreleaser configuration for migrate binary and container image
- [x] Update configuration examples with database backend options and recommendations
lukaszraczylo