test(lifecycle): add setup-teardown + run-20x harness scaffolding
Phase 4 of the v1.7.52 container excellence plan: a release-gate harness
that loops the bats suite N times in a row, with teardown between
iterations, and reports a pass/fail tally.
* setup-teardown.sh — clears /tmp/archy-rpc-session-* between runs so
iteration N+1 doesn't reuse a logged-out cookie from iteration N.
Idempotent; safe to run anytime. Designed to grow as we add suites
that leave other transient state.
* run-20x.sh — wraps run.sh in a loop of ARCHY_ITERATIONS (default 20).
Tracks per-iteration pass/fail with wall-clock timing, prints a
results block, exits non-zero on any failure. Honors ARCHY_FAIL_FAST
for short-circuit during dev.
Suggested release-gate command:
ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 \
tests/lifecycle/run-20x.sh
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
85
tests/lifecycle/run-20x.sh
Executable file
85
tests/lifecycle/run-20x.sh
Executable file
@@ -0,0 +1,85 @@
|
||||
#!/usr/bin/env bash
|
||||
# tests/lifecycle/run-20x.sh — loop the lifecycle harness N times.
|
||||
#
|
||||
# Each iteration: setup-teardown → run.sh (with the same args you'd pass
|
||||
# to run.sh) → setup-teardown. Tallies pass/fail per iteration and prints a
|
||||
# summary at the end. Returns non-zero if any iteration failed.
|
||||
#
|
||||
# Env:
|
||||
# ARCHY_ITERATIONS (default: 20)
|
||||
# ARCHY_FAIL_FAST=1 stop on first failed iteration
|
||||
# plus everything run.sh / lib/rpc.bash respects
|
||||
# (ARCHY_PASSWORD, ARCHY_HOST, ARCHY_SCHEME, ARCHY_ALLOW_DESTRUCTIVE,
|
||||
# ARCHY_ALLOW_CASCADE_DESTRUCTIVE, ARCHY_ALLOW_NOAUTH)
|
||||
#
|
||||
# Usage:
|
||||
# tests/lifecycle/run-20x.sh # 20× full bats/ suite
|
||||
# ARCHY_ITERATIONS=5 tests/lifecycle/run-20x.sh # 5× full suite
|
||||
# tests/lifecycle/run-20x.sh bitcoin-knots # 20× a single suite
|
||||
#
|
||||
# Suggested release-gate invocation:
|
||||
# ARCHY_PASSWORD=password123 ARCHY_ALLOW_DESTRUCTIVE=1 \
|
||||
# tests/lifecycle/run-20x.sh
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$HERE"
|
||||
|
||||
ITER="${ARCHY_ITERATIONS:-20}"
|
||||
if ! [[ "$ITER" =~ ^[1-9][0-9]*$ ]]; then
|
||||
echo "ARCHY_ITERATIONS must be a positive integer, got: $ITER" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
passed=0
|
||||
failed=0
|
||||
failures=()
|
||||
start=$(date +%s)
|
||||
|
||||
# One initial teardown so a previous run's cookies don't poison iteration 1.
|
||||
./setup-teardown.sh
|
||||
|
||||
for i in $(seq 1 "$ITER"); do
|
||||
echo
|
||||
echo "═══ iteration $i / $ITER ═══"
|
||||
iter_start=$(date +%s)
|
||||
|
||||
if ./run.sh "$@"; then
|
||||
iter_end=$(date +%s)
|
||||
passed=$((passed + 1))
|
||||
echo "── iteration $i: PASS ($((iter_end - iter_start))s) ──"
|
||||
else
|
||||
rc=$?
|
||||
iter_end=$(date +%s)
|
||||
failed=$((failed + 1))
|
||||
failures+=("$i")
|
||||
echo "── iteration $i: FAIL (exit=$rc, $((iter_end - iter_start))s) ──"
|
||||
if [[ "${ARCHY_FAIL_FAST:-0}" == "1" ]]; then
|
||||
echo "ARCHY_FAIL_FAST=1, stopping early"
|
||||
break
|
||||
fi
|
||||
fi
|
||||
|
||||
# Teardown between iterations so iteration N+1 starts with a clean
|
||||
# session-cookie state regardless of what iteration N did.
|
||||
./setup-teardown.sh
|
||||
done
|
||||
|
||||
end=$(date +%s)
|
||||
|
||||
echo
|
||||
echo "════════════════════════════════════════"
|
||||
echo " RESULTS"
|
||||
echo " iterations: $((passed + failed)) / $ITER"
|
||||
echo " passed: $passed"
|
||||
echo " failed: $failed"
|
||||
if (( failed > 0 )); then
|
||||
echo " failed at: ${failures[*]}"
|
||||
fi
|
||||
echo " wall time: $((end - start))s"
|
||||
echo "════════════════════════════════════════"
|
||||
|
||||
if (( failed > 0 )); then
|
||||
exit 1
|
||||
fi
|
||||
38
tests/lifecycle/setup-teardown.sh
Executable file
38
tests/lifecycle/setup-teardown.sh
Executable file
@@ -0,0 +1,38 @@
|
||||
#!/usr/bin/env bash
|
||||
# tests/lifecycle/setup-teardown.sh
|
||||
#
|
||||
# Cleanup helper used between lifecycle test iterations. Run before AND after
|
||||
# a full bats pass (run-20x.sh handles this). Idempotent — safe to run any
|
||||
# time, on any host.
|
||||
#
|
||||
# Removes:
|
||||
# - /tmp/archy-rpc-session-* — stale RPC session cookies from earlier runs.
|
||||
# If absent we'd reuse a session that was logged out by an auth.logout
|
||||
# test, then the next iteration would silently 401.
|
||||
# - Per-bats-run scratch files our tests may leave behind (none yet, but
|
||||
# this is the place when we add them).
|
||||
#
|
||||
# Does NOT touch:
|
||||
# - Real archipelago state (state.json, secrets, packages.json).
|
||||
# - Running containers — destructive container teardown is the test's
|
||||
# responsibility, not the harness's. We only clean the harness's own
|
||||
# transient state.
|
||||
# - SSH known_hosts, archipelago configs, etc.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
cleaned=0
|
||||
|
||||
# Match the pattern from lib/rpc.bash:26
|
||||
session_glob="/tmp/archy-rpc-session-*"
|
||||
# shellcheck disable=SC2086 # we want word-splitting on the glob
|
||||
for f in $session_glob; do
|
||||
if [[ -f "$f" ]]; then
|
||||
rm -f "$f"
|
||||
cleaned=$((cleaned + 1))
|
||||
fi
|
||||
done
|
||||
|
||||
if [[ "${ARCHY_TEARDOWN_VERBOSE:-0}" == "1" ]]; then
|
||||
echo "setup-teardown: cleaned $cleaned stale session file(s)"
|
||||
fi
|
||||
Reference in New Issue
Block a user