fix: Keep a colliding prune from failing a healthy rollout #90

Merged

ahmad merged 1 commit from fix/89-prune-best-effort into main

2026-09-16 19:00:31 +00:00

ahmad commented

2026-09-16 18:56:46 +00:00

Owner

Issue

Closes #89

Problem

The remote deploy script runs under set -e and ends with an image prune. The daemon allows one prune at a time, so when two deploys overlap on a host the second prune is refused and the action reports the rollout failed a second after declaring every container healthy: a red job, a false tracking issue and a red notification for a deployment that worked (agent-bus v0.3.0, run 17).

Solution

The prune is housekeeping: its failure is logged to stderr and ignored, so the deployment's verdict is the health wait's alone. A rollout that does not become healthy still fails as before.

Review notes

One line plus its comment in deploy-vps/action.yml; the remote script still parses (bash -n).

Risks and trade-offs

A host whose prune keeps failing accumulates old images until someone reads the log line; the disk alerts cover that.

### Issue Closes #89 ### Problem The remote deploy script runs under `set -e` and ends with an image prune. The daemon allows one prune at a time, so when two deploys overlap on a host the second prune is refused and the action reports the rollout failed a second after declaring every container healthy: a red job, a false tracking issue and a red notification for a deployment that worked (agent-bus v0.3.0, run 17). ### Solution The prune is housekeeping: its failure is logged to stderr and ignored, so the deployment's verdict is the health wait's alone. A rollout that does not become healthy still fails as before. ### Review notes One line plus its comment in `deploy-vps/action.yml`; the remote script still parses (`bash -n`). ### Risks and trade-offs A host whose prune keeps failing accumulates old images until someone reads the log line; the disk alerts cover that.

ahmad added 1 commit

2026-09-16 18:56:46 +00:00

fix: Keep a colliding prune from failing a healthy rollout

PR / validate (pull_request) Successful in 23s

Details

520c8ef7b1

The daemon allows one prune at a time; two deploys on one host collide
on the post-deploy prune and the second reported its rollout failed a
second after declaring it healthy. The prune is housekeeping: it logs
and the verdict stays the health wait's.

Closes #89

ahmad requested review from ahmad_bot

2026-09-16 18:56:47 +00:00

ahmad changed title from ~~WIP: fix: Keep a colliding prune from failing a healthy rollout~~ to fix: Keep a colliding prune from failing a healthy rollout

2026-09-16 18:57:28 +00:00

ahmad_bot approved these changes

2026-09-16 19:00:12 +00:00

ahmad_bot left a comment

Both criteria of #89 verified in deploy-vps/action.yml:

Prune failure no longer decides the verdict: the heredoc is quoted (<<'REMOTE'), so $? is evaluated on the host and reads the prune's own status, and the || echo makes it the script's last command with status 0 under set -euo pipefail.
An unhealthy rollout still fails: every failure path in the health wait (unhealthy, the unknown grace, the deadline) exit 1 before the prune is reached.
Note for merging, not a finding: consumers pin @v3, so this is not live for any repository until the release job re-points that tag.

Both criteria of #89 verified in `deploy-vps/action.yml`: - Prune failure no longer decides the verdict: the heredoc is quoted (`<<'REMOTE'`), so `$?` is evaluated on the host and reads the prune's own status, and the `|| echo` makes it the script's last command with status 0 under `set -euo pipefail`. - An unhealthy rollout still fails: every failure path in the health wait (`unhealthy`, the unknown grace, the deadline) `exit 1` before the prune is reached. - Note for merging, not a finding: consumers pin `@v3`, so this is not live for any repository until the release job re-points that tag.

ahmad merged commit fb61107a1a into main

2026-09-16 19:00:31 +00:00

ahmad deleted branch fix/89-prune-best-effort

2026-09-16 19:00:31 +00:00