The service update path stopped and renamed the old container aside before the new one was confirmed running, but only wired up rollback if createContainer threw or the 5s health check failed. A throw from newContainer.start() itself (bad device/GPU config, host port already bound, image incompatibility) bubbled straight to the outer catch, which returned a generic 400 and never restored the old container, leaving the service down. Retrying then wedged: the failed new container still held the service name, so the next attempt's rename to `<name>_old` collided with the leftover from the first attempt and threw the same error every time. - Wrap newContainer.start() so a start failure removes the half-created container and rolls back to the previous one. - Clear any stale `<name>_old` before renaming so retries can't collide. - Dedupe the three rollback sites into a single rollbackToOld() helper (also removes a non-null assertion in the create-failure path that could itself throw when no `_old` existed). Refs #949 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| controllers | ||
| exceptions | ||
| jobs | ||
| middleware | ||
| models | ||
| services | ||
| utils | ||
| validators | ||