mirror of
https://github.com/lukaszraczylo/kubemirror.git
synced 2026-06-05 22:43:51 +00:00
Reliabity improvements.
This commit is contained in:
+98
-24
@@ -19,8 +19,9 @@ The test suite uses a **data-driven framework** approach where test scenarios ar
|
||||
|
||||
1. **common.sh**: Base utilities (logging, assertions, cleanup)
|
||||
2. **test-framework.sh**: DRY test framework functions (resource creation, updates, verification)
|
||||
3. **test-comprehensive.sh**: Comprehensive test scenarios using the framework
|
||||
4. **run-all-tests.sh**: Main test runner (builds binary, starts controller, runs tests)
|
||||
3. **test-comprehensive.sh**: Comprehensive test scenarios using the framework (supports selective execution)
|
||||
4. **test-parallel.sh**: Parallel test runner for faster execution (batches independent tests)
|
||||
5. **run-all-tests.sh**: Main test runner (builds binary, starts controller, runs tests)
|
||||
|
||||
### Test Framework Functions
|
||||
|
||||
@@ -47,7 +48,7 @@ verify_orphan_cleanup <type> <name> <namespace1> <namespace2> ...
|
||||
|
||||
## Comprehensive Test Suite
|
||||
|
||||
The comprehensive test suite (`test-comprehensive.sh`) covers **22 systematic scenarios**:
|
||||
The comprehensive test suite (`test-comprehensive.sh`) covers **30 systematic scenarios**:
|
||||
|
||||
### Source Lifecycle Scenarios
|
||||
|
||||
@@ -88,11 +89,23 @@ The comprehensive test suite (`test-comprehensive.sh`) covers **22 systematic sc
|
||||
|
||||
### Resource Types
|
||||
|
||||
All scenarios tested with both **Secrets** and **ConfigMaps**.
|
||||
23. **Mixed resource types**: ConfigMaps alongside Secrets
|
||||
24. **Custom Resource (Traefik Middleware)**: CRD mirroring
|
||||
|
||||
### Transformation Scenarios (24-30)
|
||||
|
||||
25. **Static value transformation**: Replace data values with static strings
|
||||
26. **Template transformation**: Use Go templates with context variables
|
||||
27. **Merge transformation**: Merge new data into existing fields
|
||||
28. **Delete transformation**: Remove specific fields
|
||||
29. **Multiple transformations**: Combine multiple rules
|
||||
30. **Strict mode**: Fail on transformation errors vs skip
|
||||
|
||||
All basic scenarios tested with both **Secrets** and **ConfigMaps**.
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Run Complete Test Suite
|
||||
### Run Complete Test Suite (Sequential)
|
||||
|
||||
```bash
|
||||
cd e2e
|
||||
@@ -103,22 +116,55 @@ This will:
|
||||
1. Check you're on docker-desktop context
|
||||
2. Build the KubeMirror binary
|
||||
3. Start the controller in background
|
||||
4. Run comprehensive test scenarios (22+ scenarios)
|
||||
4. Run comprehensive test scenarios (all 30 scenarios sequentially)
|
||||
5. Report detailed results with pass/fail for each
|
||||
6. Clean up all resources automatically
|
||||
|
||||
### Run Individual Test Scenarios
|
||||
**Performance**: ~5-7 minutes for all 30 scenarios
|
||||
|
||||
### Run Complete Test Suite (Parallel) - **FASTER** ⚡
|
||||
|
||||
```bash
|
||||
cd e2e
|
||||
# Start controller first
|
||||
../kubemirror --max-targets=100 --worker-threads=5 > /tmp/kubemirror-test.log 2>&1 &
|
||||
|
||||
# Run tests in parallel batches
|
||||
./test-parallel.sh
|
||||
```
|
||||
|
||||
This runs independent tests in parallel batches:
|
||||
- **Sequential**: Scenarios 1-11 (core lifecycle - must run sequentially)
|
||||
- **Parallel Batch 1**: Scenarios 12-15 (namespace labels)
|
||||
- **Parallel Batch 2**: Scenarios 16-19 (deletion scenarios)
|
||||
- **Parallel Batch 3**: Scenarios 20-23 (mixed resources)
|
||||
- **Parallel Batch 4**: Scenarios 24-27 (transformations part 1)
|
||||
- **Parallel Batch 5**: Scenarios 28-30 (transformations part 2)
|
||||
|
||||
**Performance**: ~3-4 minutes (40-50% faster than sequential)
|
||||
|
||||
### Run Selective Scenarios
|
||||
|
||||
Run only specific scenarios for faster iteration during development:
|
||||
|
||||
```bash
|
||||
# Must have KubeMirror controller running first
|
||||
cd /Users/nvm/Documents/projects/private/kube-mirror
|
||||
./kubemirror --max-targets=100 --worker-threads=5 > /tmp/kubemirror-test.log 2>&1 &
|
||||
|
||||
# Then run the test
|
||||
# Run only transformation tests (scenarios 24-30)
|
||||
cd e2e
|
||||
./test-comprehensive.sh
|
||||
./test-comprehensive.sh 24 25 26 27 28 29 30
|
||||
|
||||
# Run only specific scenarios
|
||||
./test-comprehensive.sh 1 2 3
|
||||
|
||||
# Run single scenario for debugging
|
||||
./test-comprehensive.sh 24
|
||||
```
|
||||
|
||||
**Performance**: <1 minute for a few scenarios
|
||||
|
||||
## Test Output
|
||||
|
||||
Each test produces colored output:
|
||||
@@ -162,12 +208,17 @@ All tests passed!
|
||||
|
||||
## Test Resources
|
||||
|
||||
Tests create temporary resources:
|
||||
- **Namespaces**: `e2e-*` prefixed
|
||||
- **Secrets**: `test-*` prefixed in default namespace
|
||||
- **ConfigMaps**: `test-*` prefixed in default namespace
|
||||
Tests create temporary resources with clear naming for isolation:
|
||||
- **Source Namespace**: `kubemirror-e2e-source` (dedicated namespace for all test source resources)
|
||||
- **Target Namespaces**: `kubemirror-e2e-*` prefixed (ns-1, ns-2, app-1, db-1, etc.)
|
||||
- **Secrets**: `test-*` prefixed in source namespace
|
||||
- **ConfigMaps**: `test-*` prefixed in source namespace
|
||||
- **CRDs**: Traefik Middleware resources for CRD testing
|
||||
|
||||
All resources are cleaned up automatically on test completion.
|
||||
All resources are cleaned up automatically on test completion, including:
|
||||
- Automatic finalizer removal from source resources (prevents hanging deletions)
|
||||
- Cascade deletion of all target namespaces
|
||||
- Cleanup on test interruption (SIGINT/SIGTERM)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
@@ -292,7 +343,9 @@ verify_orphan_cleanup secret my-secret orphan-ns1 orphan-ns2
|
||||
| Pattern matching | 3 | New namespace creation, pattern changes, mixed explicit+pattern |
|
||||
| 'all' keyword opt-in | 4 | No label, add label, remove label, change true→false |
|
||||
| Edge cases | 5 | Namespace deletion, recreation, source deletion, target recreation |
|
||||
| **Total** | **22** | **All with Secrets and ConfigMaps** |
|
||||
| Resource types | 2 | Mixed ConfigMaps, Custom Resource (Traefik Middleware) |
|
||||
| Transformations | 7 | Static value, template, merge, delete, multiple, strict mode |
|
||||
| **Total** | **30** | **Comprehensive coverage with multiple resource types** |
|
||||
|
||||
## Test Methodology
|
||||
|
||||
@@ -372,23 +425,44 @@ kind delete cluster --name kubemirror-test
|
||||
|
||||
## Performance Notes
|
||||
|
||||
- Comprehensive test suite: ~3-5 minutes
|
||||
- Controller startup: ~10 seconds
|
||||
- Resource reconciliation: typically <5 seconds per operation
|
||||
- Total assertions: 60+ across all scenarios
|
||||
- Each scenario includes setup, action, verification, and cleanup phases
|
||||
### Test Execution Times
|
||||
- **Sequential execution** (test-comprehensive.sh): ~5-7 minutes for all 30 scenarios
|
||||
- **Parallel execution** (test-parallel.sh): ~3-4 minutes for all 30 scenarios (40-50% faster)
|
||||
- **Selective execution** (few scenarios): <1 minute
|
||||
- **Controller startup**: ~10 seconds
|
||||
- **Resource reconciliation**: typically <5 seconds per operation
|
||||
|
||||
### Test Coverage
|
||||
- **Total scenarios**: 30 comprehensive scenarios
|
||||
- **Total assertions**: 100+ across all scenarios
|
||||
- **Resource types tested**: Secrets, ConfigMaps, Traefik Middlewares (CRDs)
|
||||
- **Each scenario includes**: Setup, action, verification, and cleanup phases
|
||||
|
||||
### Optimization Tips
|
||||
- Use `test-parallel.sh` for full test runs (40-50% faster)
|
||||
- Use selective execution during development: `./test-comprehensive.sh 24 25 26`
|
||||
- Run only affected scenarios after code changes
|
||||
- Parallel execution is safe - batches ensure test independence
|
||||
|
||||
## Test Isolation and Cleanup
|
||||
|
||||
- **Automatic cleanup**: All resources cleaned up via trap handlers
|
||||
- **Namespace isolation**: Tests use `e2e-*` prefixed namespaces
|
||||
- **Sequential execution**: Tests run sequentially to avoid race conditions
|
||||
- **Namespace isolation**:
|
||||
- Dedicated source namespace: `kubemirror-e2e-source`
|
||||
- Target namespaces: `kubemirror-e2e-*` prefixed
|
||||
- No pollution of `default` namespace
|
||||
- **Execution modes**:
|
||||
- Sequential: All scenarios run in order (test-comprehensive.sh with no args)
|
||||
- Parallel: Independent scenarios batched (test-parallel.sh)
|
||||
- Selective: Run specific scenarios (test-comprehensive.sh 24 25 26)
|
||||
- **Idempotent**: Tests can be re-run without manual cleanup
|
||||
- **Resource labeling**: Test resources labeled for easy identification
|
||||
- **Resource labeling**: Test resources labeled `test-resource: e2e` for easy identification
|
||||
- **Finalizer handling**: Automatic finalizer removal prevents stuck resource deletions
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Tests assume clean docker-desktop cluster (or equivalent local cluster)
|
||||
- Some scenarios require waiting for reconciliation (30s default timeout)
|
||||
- Tests are sequential (not parallel) to ensure deterministic behavior
|
||||
- Controller must be stopped between runs if running manually (run-all-tests.sh handles this)
|
||||
- Parallel execution requires sufficient cluster resources (5-6 tests may run concurrently)
|
||||
- Some scenarios depend on previous state (scenarios 1-11 must run sequentially)
|
||||
|
||||
+889
-122
File diff suppressed because it is too large
Load Diff
Executable
+207
@@ -0,0 +1,207 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Parallel E2E Test Runner for KubeMirror
|
||||
# Runs independent tests in parallel batches for faster execution
|
||||
#
|
||||
# Usage:
|
||||
# ./test-parallel.sh # Run all tests in parallel batches
|
||||
#
|
||||
# Performance:
|
||||
# - Sequential execution: ~5-7 minutes for all 30 scenarios
|
||||
# - Parallel execution: ~3-4 minutes (batched by independence)
|
||||
#
|
||||
# Batching Strategy:
|
||||
# - Sequential: Scenarios 1-11 (core lifecycle - must run sequentially)
|
||||
# - Parallel Batch 1: Scenarios 12-15 (namespace labels)
|
||||
# - Parallel Batch 2: Scenarios 16-19 (deletion scenarios)
|
||||
# - Parallel Batch 3: Scenarios 20-23 (mixed resources)
|
||||
# - Parallel Batch 4: Scenarios 24-27 (transformations part 1)
|
||||
# - Parallel Batch 5: Scenarios 28-30 (transformations part 2)
|
||||
#
|
||||
# Individual scenario logs: e2e/.test-scenario-{num}.log
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
source "$SCRIPT_DIR/common.sh"
|
||||
|
||||
TEST_NAME="Parallel E2E Tests"
|
||||
|
||||
# Colors
|
||||
CYAN='\033[0;36m'
|
||||
NC='\033[0m'
|
||||
|
||||
# Batch execution tracking
|
||||
BATCH_RESULTS=()
|
||||
TOTAL_START_TIME=$(date +%s)
|
||||
|
||||
# Run a single scenario by number
|
||||
run_scenario() {
|
||||
local scenario_num=$1
|
||||
local log_file="$SCRIPT_DIR/.test-scenario-${scenario_num}.log"
|
||||
|
||||
echo -e "${CYAN}[BATCH]${NC} Starting scenario $scenario_num in background"
|
||||
|
||||
# Run the specific scenario using the comprehensive test script
|
||||
# Pass scenario number as argument to run only that scenario
|
||||
bash "$SCRIPT_DIR/test-comprehensive.sh" "$scenario_num" > "$log_file" 2>&1
|
||||
local exit_code=$?
|
||||
|
||||
# Store result
|
||||
echo "$scenario_num:$exit_code" >> "$SCRIPT_DIR/.batch-results.tmp"
|
||||
|
||||
if [ $exit_code -eq 0 ]; then
|
||||
echo -e "${GREEN}[PASS]${NC} Scenario $scenario_num completed successfully"
|
||||
else
|
||||
echo -e "${RED}[FAIL]${NC} Scenario $scenario_num failed (exit code: $exit_code)"
|
||||
fi
|
||||
|
||||
return $exit_code
|
||||
}
|
||||
|
||||
# Run multiple scenarios in parallel
|
||||
run_parallel_batch() {
|
||||
local batch_name=$1
|
||||
shift
|
||||
local scenarios=("$@")
|
||||
|
||||
echo ""
|
||||
echo -e "${CYAN}======================================${NC}"
|
||||
echo -e "${CYAN}Batch: $batch_name${NC}"
|
||||
echo -e "${CYAN}Scenarios: ${scenarios[*]}${NC}"
|
||||
echo -e "${CYAN}======================================${NC}"
|
||||
|
||||
local batch_start=$(date +%s)
|
||||
|
||||
# Clear batch results file
|
||||
> "$SCRIPT_DIR/.batch-results.tmp"
|
||||
|
||||
# Start all scenarios in background
|
||||
local pids=()
|
||||
for scenario in "${scenarios[@]}"; do
|
||||
run_scenario "$scenario" &
|
||||
pids+=($!)
|
||||
done
|
||||
|
||||
# Wait for all to complete
|
||||
local batch_failed=0
|
||||
for pid in "${pids[@]}"; do
|
||||
wait $pid || batch_failed=1
|
||||
done
|
||||
|
||||
local batch_end=$(date +%s)
|
||||
local batch_duration=$((batch_end - batch_start))
|
||||
|
||||
echo -e "${CYAN}Batch '$batch_name' completed in ${batch_duration}s${NC}"
|
||||
|
||||
return $batch_failed
|
||||
}
|
||||
|
||||
# Main execution
|
||||
main() {
|
||||
log_info "Starting $TEST_NAME"
|
||||
log_info "This runner executes independent tests in parallel for faster results"
|
||||
|
||||
# Ensure controller is running
|
||||
if ! pgrep -f "kubemirror" > /dev/null; then
|
||||
log_fail "KubeMirror controller is not running!"
|
||||
log_info "Please start the controller first: ./e2e/run-all-tests.sh"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Clean up any previous test artifacts
|
||||
log_info "Cleaning up previous test artifacts"
|
||||
rm -f "$SCRIPT_DIR/.test-scenario-"*.log
|
||||
rm -f "$SCRIPT_DIR/.batch-results.tmp"
|
||||
|
||||
local overall_failed=0
|
||||
|
||||
# ========================================================================
|
||||
# SEQUENTIAL BATCH: Core Lifecycle (Scenarios 1-11)
|
||||
# These tests modify the same resource and MUST run sequentially
|
||||
# ========================================================================
|
||||
echo ""
|
||||
echo -e "${CYAN}======================================${NC}"
|
||||
echo -e "${CYAN}Sequential Batch: Core Lifecycle${NC}"
|
||||
echo -e "${CYAN}Scenarios: 1-11 (sequential execution required)${NC}"
|
||||
echo -e "${CYAN}======================================${NC}"
|
||||
|
||||
local seq_start=$(date +%s)
|
||||
bash "$SCRIPT_DIR/test-comprehensive.sh" 1 2 3 4 5 6 7 8 9 10 11
|
||||
if [ $? -ne 0 ]; then
|
||||
overall_failed=1
|
||||
log_fail "Sequential batch (1-11) failed"
|
||||
fi
|
||||
local seq_end=$(date +%s)
|
||||
echo -e "${CYAN}Sequential batch completed in $((seq_end - seq_start))s${NC}"
|
||||
|
||||
# ========================================================================
|
||||
# PARALLEL BATCH 1: Namespace Label Tests (Scenarios 12-15)
|
||||
# ========================================================================
|
||||
run_parallel_batch "Namespace Labels" 12 13 14 15
|
||||
[ $? -ne 0 ] && overall_failed=1
|
||||
|
||||
# ========================================================================
|
||||
# PARALLEL BATCH 2: Deletion Scenarios (Scenarios 16-19)
|
||||
# ========================================================================
|
||||
run_parallel_batch "Deletion Scenarios" 16 17 18 19
|
||||
[ $? -ne 0 ] && overall_failed=1
|
||||
|
||||
# ========================================================================
|
||||
# PARALLEL BATCH 3: Mixed Resources (Scenarios 20-23)
|
||||
# ========================================================================
|
||||
run_parallel_batch "Mixed Resources" 20 21 22 23
|
||||
[ $? -ne 0 ] && overall_failed=1
|
||||
|
||||
# ========================================================================
|
||||
# PARALLEL BATCH 4: Transformations Part 1 (Scenarios 24-27)
|
||||
# ========================================================================
|
||||
run_parallel_batch "Transformations 1" 24 25 26 27
|
||||
[ $? -ne 0 ] && overall_failed=1
|
||||
|
||||
# ========================================================================
|
||||
# PARALLEL BATCH 5: Transformations Part 2 (Scenarios 28-30)
|
||||
# ========================================================================
|
||||
run_parallel_batch "Transformations 2" 28 29 30
|
||||
[ $? -ne 0 ] && overall_failed=1
|
||||
|
||||
# ========================================================================
|
||||
# SUMMARY
|
||||
# ========================================================================
|
||||
local total_end=$(date +%s)
|
||||
local total_duration=$((total_end - TOTAL_START_TIME))
|
||||
|
||||
echo ""
|
||||
echo -e "${CYAN}======================================${NC}"
|
||||
echo -e "${CYAN}Test Execution Summary${NC}"
|
||||
echo -e "${CYAN}======================================${NC}"
|
||||
echo -e "Total execution time: ${total_duration}s"
|
||||
echo -e "Sequential scenarios: 1-11"
|
||||
echo -e "Parallel batches: 5 batches (scenarios 12-30)"
|
||||
|
||||
if [ $overall_failed -eq 0 ]; then
|
||||
echo -e "${GREEN}All test batches PASSED!${NC}"
|
||||
|
||||
# Show individual scenario results from logs
|
||||
echo ""
|
||||
echo "Individual scenario results:"
|
||||
for i in {1..30}; do
|
||||
if [ -f "$SCRIPT_DIR/.test-scenario-${i}.log" ]; then
|
||||
if grep -q "PASS.*Scenario.*${i}" "$SCRIPT_DIR/.test-scenario-${i}.log" 2>/dev/null; then
|
||||
echo -e " Scenario $i: ${GREEN}PASS${NC}"
|
||||
else
|
||||
echo -e " Scenario $i: ${RED}FAIL${NC}"
|
||||
fi
|
||||
fi
|
||||
done
|
||||
else
|
||||
echo -e "${RED}Some test batches FAILED!${NC}"
|
||||
echo ""
|
||||
echo "Check individual scenario logs in: $SCRIPT_DIR/.test-scenario-*.log"
|
||||
fi
|
||||
|
||||
# Cleanup temp files
|
||||
rm -f "$SCRIPT_DIR/.batch-results.tmp"
|
||||
|
||||
exit $overall_failed
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user