MAJOR PERFORMANCE IMPROVEMENT: Reduced frontend Docker build time
from ~20 minutes to ~30 seconds by building the SPA once on the
runner instead of twice (amd64 + arm64) in Docker.
Changes:
1. Release workflow now builds frontend on CI runner (native, fast)
2. Frontend artifact uploaded and downloaded in release job
3. Dockerfile.frontend simplified to just copy pre-built files
4. Multi-arch Docker build is now just copying files into nginx
Before:
- Docker builds frontend 2x (amd64 + arm64 with QEMU emulation)
- Each: pnpm install + pnpm build = ~10 min per arch
- Total: ~20 minutes for frontend image
After:
- Build frontend 1x on runner = ~2 min (native)
- Docker just copies files = ~30 sec (both architectures)
- Total: ~2.5 minutes for frontend image
Impact:
- 8x faster frontend builds
- Total release time reduced from ~25 min to ~7 min
- Lower resource usage (no QEMU emulation)
Files changed:
- .github/workflows/release.yaml: Enable node build
- Dockerfile.frontend: Remove build stage, expect pre-built files
- .goreleaser.yaml: Copy frontend/dist instead of full source
Eliminated duplicate nginx containers by merging gateway reverse proxy
functionality into the frontend container. This simplifies deployment
and reduces resource usage.
Architecture changes:
- Frontend now serves both static files AND reverse proxies to backend
- Single nginx container handles all HTTP routing
- Gateway container removed from builds and Helm chart
Dockerfile.frontend changes:
- Added upstream backend configuration
- Added proxy locations for /api, /health, /metrics, /npm, /pypi, /go, /ws
- Added rate limiting for API and downloads
- Added WebSocket support
- Configurable via BACKEND_HOST and BACKEND_PORT env vars
Helm chart changes:
- Updated frontend deployment to configure backend connection
- Simplified ingress to single route (all traffic → frontend)
- Frontend proxies backend requests internally
- Removed separate frontend/api ingress configurations
GoReleaser changes:
- Removed gohoarder-gateway Docker build
- Now builds: server, scanner, migrate, frontend (4 images)
Benefits:
- Fewer containers to manage
- Reduced complexity in Docker Compose and Kubernetes
- Single point of configuration for routing
- Better resource utilization
CRITICAL FIX: arm64 images were getting amd64 binaries causing
"exec format error" when running on ARM platforms.
Root cause:
- Dockerfiles had ARG TARGETOS=linux and ARG TARGETARCH=amd64
- FROM --platform=$TARGETOS/$TARGETARCH used these defaults
- buildx couldn't override them, so all builds used amd64
Solution:
- Removed --platform from FROM statement
- Let buildx automatically handle platform selection
- Declared ARGs AFTER FROM to receive buildx values
- buildx automatically sets TARGETPLATFORM, TARGETOS, TARGETARCH
Now each platform builds native binaries:
- linux/amd64 → amd64 binary
- linux/arm64 → arm64 binary
This fixes: exec /usr/local/bin/migrate: exec format error
go.mod requires go >= 1.25.5 but Dockerfiles were using golang:1.23-alpine.
Updated all Go-based Dockerfiles to use golang:1.25-alpine.
Affected files:
- Dockerfile.server
- Dockerfile.scanner
- Dockerfile.migrate
This fixes the build error:
"go: go.mod requires go >= 1.25.5 (running go 1.23.12; GOTOOLCHAIN=local)"
GoReleaser creates a temporary build context with only binaries, but our
Dockerfiles are multi-stage builds that need the full source code to
compile. Added extra_files to copy necessary directories.
Files copied per image:
- gohoarder-server: go.mod, go.sum, cmd, pkg, internal, config
- gohoarder-scanner: go.mod, go.sum, cmd, pkg, internal, config
- gohoarder-migrate: go.mod, go.sum, cmd, pkg, internal, migrations
- gohoarder-frontend: frontend/ directory (Node.js source)
- gohoarder-gateway: no extra files needed (static config)
This fixes the build context error:
"Seems like you tried to copy a file that is not available in the
build context."
Docker SBOM attestations require docker-container driver which is not
available in the default Docker driver used by GoReleaser. Disabled
SBOM generation for all Docker images to prevent build failures.
Error fixed:
- "Attestation is not supported for the docker driver"
Applied to all Docker images:
- gohoarder-server
- gohoarder-frontend
- gohoarder-scanner
- gohoarder-gateway
- gohoarder-migrate
Removed local workflow copy and reverted to using the shared workflow
from lukaszraczylo/shared-actions@main, which now has the proper
if condition to prevent the release job from being skipped.
Problem:
- Release job (merge phase) was being skipped even though all builds succeeded
- The shared workflow's release job has `needs: [version, build]` but no `if` condition
- Without `if: always()`, GitHub Actions skips the job if dependencies have non-success status
- Frontend job being skipped caused downstream effects
Solution:
- Created local copy of go-release-cgo.yaml workflow
- Added explicit condition to release job:
if: |
always() &&
needs.version.result == 'success' &&
needs.build.result == 'success'
- Updated release.yaml to use local workflow instead of remote
This ensures the release/merge phase runs as long as version and build jobs succeed,
regardless of optional jobs like frontend being skipped.
Note: This is a workaround until the fix is merged upstream in shared-actions repo.
Problem:
- linux_amd64 runner was trying to build BOTH:
- linux/amd64 (native - OK)
- linux/arm64 (cross-compile with CGO - FAILS)
- Error: gcc_arm64.S assembler errors when cross-compiling ARM64 on x86_64
- Workflow default platforms only include linux/amd64, not linux/arm64
Solution:
- Added linux/arm64 to ignore list in both builds
- Only build linux/amd64 binaries (native compilation on ubuntu-latest)
- Docker images still provide linux/arm64 via multi-stage builds
- Users get ARM64 support through Docker, not standalone binaries
Build matrix now:
- ✅ darwin/arm64 (macOS Apple Silicon) - native on macos-latest
- ✅ linux/amd64 (Linux x86_64) - native on ubuntu-latest
- ❌ linux/arm64 (skipped for binaries, available in Docker)
This eliminates CGO cross-compilation while maintaining full platform support
via Docker multi-arch images.
Problem:
- CGO builds failing in CI with "cannot find sqlite3.h"
- go-release-cgo.yaml workflow looks for workflow-prepare.sh to install deps
- Script was missing, causing build failures
Solution:
- Created workflow-prepare.sh to install SQLite development headers
- Platform-specific installation:
- Linux (Ubuntu/Debian): libsqlite3-dev via apt-get
- Linux (RHEL/CentOS): sqlite-devel via yum
- Linux (Alpine): sqlite-dev via apk
- macOS: sqlite3 via Homebrew (if needed)
- Windows: Downloads SQLite amalgamation, sets CGO_CFLAGS/CGO_LDFLAGS
- Includes verification step to confirm SQLite availability
This script is automatically called by the shared GitHub Actions workflow
before running GoReleaser builds with CGO_ENABLED=1.
Problem:
- With builds: skip: true, no artifacts were created
- GoReleaser wasn't creating GitHub releases or tags
- Helm chart workflow wasn't triggered (depends on tags)
- No downloadable binaries for users
Solution:
- Enabled builds for both gohoarder and migrate binaries
- CGO_ENABLED=1 for SQLite support
- Added fts5 tag for full-text search
- Builds run natively per platform in split/merge workflow:
- darwin/arm64 (Apple Silicon Macs)
- linux/amd64 (x86_64 Linux)
- linux/arm64 (ARM64 Linux)
- Ignored darwin/amd64 (Intel Macs) to limit build matrix
How it works:
1. Split phase: Each platform builds natively (no cross-compilation)
2. Merge phase: Combines all artifacts, creates release, builds Docker images
3. Docker images still use multi-stage builds (independent of binaries)
4. GitHub release created with tags
5. Helm chart workflow triggered
Benefits:
- Downloadable binaries for all platforms
- Archives created automatically
- GitHub releases with proper tags
- Helm charts published
- Docker images built separately with multi-stage builds
Problem:
- Used incorrect field names (use: buildx, build_flag_templates) not supported in GoReleaser v2.13.2
- GitHub Actions workflow using non-CGO release workflow
- Docker builds failing due to invalid configuration
Solution:
- Updated dockers_v2 configuration with correct field names:
- Removed unsupported `use: buildx` field
- Changed `build_flag_templates` to `build_args` (map format)
- Kept `platforms` for multi-arch support (linux/amd64, linux/arm64)
- Updated GitHub Actions workflow to use go-release-cgo.yaml for CGO support
- Build args now passed correctly to Docker builds for version info
Changes:
- .goreleaser.yaml: Fixed all Docker image configurations
- .github/workflows/release.yaml: Changed to go-release-cgo.yaml workflow
Validation:
- goreleaser check: PASSED ✓
- Configuration validated with GoReleaser Pro v2.13.2
References:
- GoReleaser dockers_v2 docs: https://goreleaser.com/customization/dockers_v2/
Problem:
- GoReleaser Pro features (use: buildx, build_flag_templates) not available in free version
- CI/CD failing with "field not found in type config.DockerV2" errors
Solution:
- Split each Docker image into separate amd64 and arm64 builds
- Use goarch field to specify architecture
- Use build_flags instead of build_flag_templates
- Add docker_manifests section to combine arch-specific images into multi-arch manifests
Changes:
- Each service now has two Docker image definitions (amd64 and arm64)
- Images tagged with architecture suffix (e.g., v1.0.0-amd64, v1.0.0-arm64)
- Docker manifests combine them into unified tags (e.g., v1.0.0, latest)
- Users can pull multi-arch images normally, Docker will select correct arch
Result:
- Works with free GoReleaser version
- Maintains multi-architecture support
- Multi-stage Dockerfiles compile for each architecture natively
Problem:
- Enabling CGO_ENABLED=1 for SQLite support caused cross-compilation failures
- ARM64 assembly errors when building from amd64 host
- Cross-compilation with CGO requires architecture-specific toolchains
Solution:
- Converted all Dockerfiles to multi-stage builds
- Binaries now compile inside Docker using native platform builders
- Used --platform flag to build for target architecture natively
- Removed binary builds from .goreleaser.yaml (skip: true)
- Updated dockers_v2 to use buildx with multi-platform support
Changes:
- .goreleaser.yaml: Skip standalone builds, use Docker buildx
- Dockerfile.server: Multi-stage build with CGO
- Dockerfile.scanner: Multi-stage build with CGO
- Dockerfile.migrate: Multi-stage build with CGO
Benefits:
- No cross-compilation needed (each platform builds natively)
- Docker buildx handles multi-platform builds automatically
- SQLite support working with CGO enabled
- Cleaner separation between build and runtime environments
Changes:
- Set CGO_ENABLED=1 for gohoarder main binary in .goreleaser.yaml
- Add sqlite-libs and musl to Dockerfile.server
- Add sqlite-libs and musl to Dockerfile.scanner
All Go binaries that interact with SQLite now have CGO enabled:
✅ gohoarder (main binary) - used by server and scanner
✅ migrate (migration tool)
Runtime containers include necessary C libraries:
✅ Dockerfile.server - SQLite runtime support
✅ Dockerfile.scanner - SQLite runtime support
✅ Dockerfile.migrate - SQLite runtime support
This fixes: 'Binary was compiled with CGO_ENABLED=0, go-sqlite3 requires cgo'
Changes:
- Set CGO_ENABLED=1 for migrate build in .goreleaser.yaml
- Add sqlite-libs and musl runtime dependencies to Dockerfile.migrate
This fixes the migration error: 'Binary was compiled with CGO_ENABLED=0,
go-sqlite3 requires cgo to work'
Add max_cache_size and blocked_packages fields to all Stats mock objects in:
- Dashboard.spec.ts
- Stats.spec.ts
This fixes TypeScript compilation errors in the build process.
Backend:
- Add blocked_packages count to stats API by checking vulnerabilities against thresholds
- Add max_cache_size to stats API from config
- Add isBlocked field to package API responses
Frontend:
- Add blocked_packages and max_cache_size to Stats interface
- Add blocked packages counter card to stats page with fa-hand icon
- Add storage usage progress bar with color coding (green/yellow/orange/red)
- Add /blocked-packages route that filters vulnerable packages by isBlocked
- Update VulnerabilityBadge to show BLOCKED badge with fa-hand icon
- Fix TypeScript imports for useRoute in VulnerablePackages
Features:
- Stats page now shows blocked packages count (clickable)
- Storage display shows usage vs max with visual progress bar
- Blocked packages view accessible from stats page
- All blocked indicators use fa-hand icon instead of fa-ban
- [x] Implement GORM V2 metadata store with SQLite, PostgreSQL, and MySQL support
- [x] Add database migration system using gormigrate for schema versioning
- [x] Create migration CLI tool with support for migrate, rollback, and status commands
- [x] Add Docker support for migration container (Dockerfile.migrate)
- [x] Implement automatic partition management for PostgreSQL time-series tables
- [x] Add background aggregation worker for download statistics
- [x] Support connection pooling configuration (max_open_conns, max_idle_conns, conn_max_lifetime)
- [x] Add blocking mechanism based on vulnerability thresholds in stats and handlers
- [x] Update Helm charts with migration init containers and multi-database configuration
- [x] Replace deprecated SQLite store with optimized GORM implementation
- [x] Add comprehensive integration tests for MySQL and PostgreSQL
- [x] Update frontend to display blocked packages and storage utilization
- [x] Add goreleaser configuration for migrate binary and container image
- [x] Update configuration examples with database backend options and recommendations
- [x] Add explicit security context with fsGroup and runAsUser to frontend deployment
- [x] Add initContainer to copy nginx static files and config to writable volumes
- [x] Add security context to initContainer with capability restrictions
- [x] Add runAsUser to frontend container security context
- [x] Add emptyDir volumes for nginx HTML and conf directories
- [x] Replace template includes with explicit volumeMounts in scanner deployment
- [x] Conditionally mount trivy cache volume in scanner deployment
- [x] Replace template includes with explicit volumeMounts in server deployment
lukaszraczylo