Skip to content

Deployment safety and rollback

DeploymentsImpreza documentationRead as Markdown ↗
Requires agent 0.6.0 or newer. Existing installations must be updated by the customer. See agent updates for the check and update commands.

A failed image download or build should not remove an application that is already serving customers. The updated Docker executor prepares all images before replacing containers. Portal, REST and MCP deployments use this same executor.

Concurrent redeploy protection: custom-app redeploy reserves the application transactionally for every source mode. Overlapping requests or a changed saved configuration return 409 CONFLICT; refresh the app before retrying. Current server ownership is checked again before queuing. A queue-write failure rolls back the reservation and environment edits.

What happens during redeploy

  1. Save the previous compose.yaml, .env and startup policy.
  2. Validate the new Compose configuration, pull service images and build services that declare a build.
  3. If preparation fails, restore that configuration and leave the existing containers in place. The attempt remains failed, with a diagnostic explaining what was preserved or whether restoration failed.
  4. If preparation succeeds, run docker compose up -d --no-build --pull never, then the existing startup checks and lifecycle steps.

Automatic recovery of a previous version

Before changing a healthy existing runtime, the updated agent saves its resolved configuration and the exact image IDs of its containers. A mutable tag such as latest cannot silently change this recovery target. If replacement fails, the new containers crash-loop, Docker health checks miss the startup deadline, or the install hook fails, the agent attempts to restore that version.

Recovery uses local images, preserves volumes and does not repeat installation hooks. The previous containers must pass Docker startup checks before recovery is reported as successful. If an image is missing or recovery fails, the runtime is reported as unknown. First installations and already-broken runtimes have no eligible previous version and retain the existing failure policy.

Required healthy startup (agent 0.6.3+)

Required healthy start: opt in with require_healthy_start=true, an explicit healthcheck_path and build_strategy=node_npm. In the portal, enable Require a healthy start. Agent 0.6.3 or newer must be reported in Servers; update it explicitly and wait for the next heartbeat. startup_timeout_seconds accepts an integer from 30 to 600, default 60. The budget starts after containers are created and includes the stable health observation; Git fetch, dependency/image builds and recovery have separate time limits. Every running service must report Docker healthy and at least one serving container must exist. HTTP redirects, authentication errors and timeouts do not pass the configured 2xx probe. When the first install fails, its containers are removed and persistent volumes are preserved; the failure includes diagnostic logs. A failed replacement recovers a verified healthy previous release when available. Recovery and manual rollback use the saved release startup policy, so a slow healthy release retains its original deadline. Failure recovery does not undo database writes or mutable data. The control plane refuses incompatible agents at creation, never dispatches the policy without startup-health-v1 support in the current poll, and accepts success only with a matching health confirmation. Previews inherit the policy; redeploy keeps the snapshot. Deployment reads expose startup_health. Choose a new deployment to change the saved policy. Omission or false preserves legacy behavior; a startup timeout requires the option enabled. The public option is for generated Node builds. This does not provide zero-downtime switching or continuous automatic rollback after startup. Local MCP input support requires 0.20.0+. Existing agents update only when the customer runs the updater.

Using it through portal or MCP

Use the existing redeploy action, impreza_redeploy_deployment, or POST /v1/platform/deployments/custom/{id}/redeploy. No new arguments are required. On failure, inspect the reported error, correct the image, configuration or build, and retry. Check the application itself before interpreting a failed attempt as an outage.

Open Deployment history on an app card to see its ten most recent deployment attempts. The attempt remains failed when a rollback restores the previous version. The history records that recovery separately. These are historical results, not a live uptime monitor.

Failed attempts include a diagnosis with a likely cause, a description of the recorded evidence and a next action. Supported cases include Git authentication, missing branches, SSH host identity, source retrieval, Compose validation, image pull/build, startup and storage failures. Unknown errors remain unclassified. Evidence is summarized without raw logs or credentials; this is not a live health check or proof of a root cause. A failed attempt can coexist with a successfully restored previous version.

The same history is available as release_history in GET /v1/platform/deployments/{id}. Through MCP, find that read with impreza_api_search and call it using impreza_api_call. History excludes environment variables, snapshot contents and credentials. Older agents have no release or rollback metadata.

Manual release selection

In Deployment history, managers can choose Restore this release for entries with rollback_supported: true and confirm the interruption. REST clients use POST /v1/platform/deployments/{id}/rollback, manage scope and {"target_version":"rel_...","confirm":true}. The target is a snapshot ID, not a semantic version.

MCP uses impreza_rollback_deployment with deployment_id and target_version. Remote MCP requires its two-call confirm_token flow; local MCP requires confirm: true after customer confirmation. Use agent 0.6.0 or newer and impreza-mcp 0.12.0 or newer for the local connector.

The support flag records a release-v1 agent report, not current availability. An agent downgrade cannot claim success without the matching release and startup receipt. Expired snapshots, unavailable immutable images, changed ports, storage or routing are rejected before replacing containers. The current runtime must pass startup checks so it can be saved as a recovery target if the chosen version fails.

Manual rollback does not rerun hooks or revert database/mutable data. A rejected attempt can set the control-plane status to failed while the containers remain available. Desired control-plane configuration is kept for the next redeploy.

Retention

The agent keeps five local configuration snapshots per deployment, with owner-only permissions, and tags their images for retention. Uninstall removes those snapshots and releases their private image tags even when app data is retained. The control-plane history can outlive those snapshots, so a historical release ID is not a guarantee that the version can still be restored. External image pruning can remove unused images.

What this does not guarantee

This is not blue-green deployment, database rollback or crash recovery. Replacement and recovery can interrupt traffic. Builds consume VPS resources. Mutable bind-mounted files, build contexts, databases and lifecycle side effects are not reverted. Custom lifecycle health scripts and HTTPS probes retain their advisory behavior; Docker startup checks do not prove that every business operation works. Control-plane variables remain the desired values for the next attempt.

Reproducing the integration test

Use an isolated disposable Linux VPS with Docker and Docker Compose. In the DevKit agent module, run IMPREZA_DOCKER_TEST=1 go test -v ./internal/executor -run TestDockerPreparationPreservesRunningDeployment -count=1.

The test deploys an HTTP service, provokes invalid Compose, an unavailable image tag and a failing Dockerfile, and checks that the same container keeps serving the old version. It also verifies restored configuration and persistent data, then successfully builds and deploys the corrected version. Its own containers are removed at cleanup.

Run IMPREZA_DOCKER_TEST=1 go test -v ./internal/executor -run TestDockerAutomaticRollback -count=1 for crash-loop, unhealthy-container, install-hook and replacement failures. It verifies the old HTTP response, immutable image identity, persistent data and literal environment values, then checks missing-image recovery, retention permissions and a successful new deployment.

Exact Git revisions

Agents 0.6.1 and later honor the full Git commit supplied by a push webhook or manifest build context. If the branch has advanced, the agent fetches and checks out the requested commit. Invalid or unavailable commits fail before container replacement; the current application remains running. When no commit is supplied, deployment follows the selected branch. This does not pin external image tags, dependencies or database state.

Unsupported commands (agent 0.6.2+)

Agents 0.6.2 and later reject unsupported command kinds with status failed and an explicit diagnostic, instead of reporting simulated success. No operation is performed and the polling loop continues with subsequent commands. Queued agent_upgrade remains unsupported; use the customer-initiated update command. A failed command report does not itself mean the running application is unhealthy.

Apps created from an archive can select a retained source version for redeploy. The selected source can differ from the running release after failed replacement or rollback; consult release history before retrying.

Read current runtime health and deployment results separately. A failed deployment can leave the previous app running; old observations do not confirm current health.

To stop queued work or preparation before container replacement, see Cancel a deployment.

Read Deployment progress and interrupted agents for step reports, persistent result delivery and operations that require reconciliation.

Agent 0.6.7+ can restore previous configuration after a restart at a completed preparation checkpoint, verify unchanged containers and close the interrupted attempt without executing it again. Unconfirmed external work and replacement uncertainty still require review. See safe preparation reconciliation.

Agent 0.6.8+ adds supervised pull/build completion after an agent restart. Wait for the original worker and the final reconciled result before retrying.

Agent 0.6.9+ adds supervised replacement after an agent restart: the original runtime worker completes once and its final result is delivered without repeating containers or hooks. Missing or invalid final receipts still require support.

Security boundaries

The control plane checks new custom manifests, signed Git deliveries and confirmed MCP actions before dispatching deployment or account operations.

  • Custom Compose: inspection refuses YAML it cannot interpret safely, aliases, external includes, elevated container or build privileges, privileged lifecycle hooks and host file inputs. Use deployment-scoped named volumes for data, runtime variables for environment settings, inline configuration, and a Dockerfile to include project files. Absolute and relative host binds are refused. The Compose importer retains its platform-generated volume names. Existing running containers are not changed; a new deployment request is checked against the updated policy.
  • Signed Git webhooks: duplicate detection uses verified signed content for a 24-hour window. Changing a delivery header does not create a new operation. For the watched branch, the delivery receipt and deployment job commit together; a queue failure returns an error and permits retry of that same delivery. A preview receipt is retained when its outcome is uncertain, so inspect the preview before requesting another operation. Static-token triggers remain repeatable. This is bounded replay protection, not indefinite exactly-once delivery.
  • MCP confirmation: if confirmation consumption cannot be stored, CONFIRMATION_UNAVAILABLE means nothing was dispatched. Retry the same confirmed request after recovery; request a new confirmation if the previous one has expired. An already consumed token cannot authorize another call.

These checks do not provide a general container sandbox or outbound network policy. Read Compose import for its supported subset and agent transport updates for client-side protection.

Preview protection and network controls

Available with agent 0.6.16+. Local MCP protected previews require 0.38.0+; the portal and hosted MCP also expose this option. Update existing agents explicitly. Protected previews use HTTPS and a generated password shown once; only its bcrypt hash is retained. Create them explicitly through the portal or impreza_create_preview with protect: true. Save the password immediately. Repeating creation does not reveal it again. Webhook previews remain onion-only. Protected previews have no direct published application port; subsequent route changes retain authentication. HTTPS hostnames are visible in Certificate Transparency and are not anonymous.

The agent egress baseline applies to forwarded IPv4 traffic leaving standard Docker bridges. It blocks outbound mail ports, private/metadata destinations and limits new connections while preserving established flows, same-bridge service connections and inbound published-port traffic. For forwarded traffic it preserves downstream operator firewall decisions. The link-local metadata range (169.254.0.0/16) is blocked on every forwarded path, not only the default bridge names. This IPv4 policy does not cover host networking, custom bridge interface names for the other blocks, or edge/network-level abuse controls. Connections from containers to the server itself are covered separately, from agent 0.6.21: see connections from containers to the server itself. Agent 0.6.20+ adds the separate IPv6 baseline described below. Installation failure is reported; this baseline is not a complete network sandbox.

See traffic switching and database backups and restore for authorization, recovery and availability limits.

Source review before builds

Available with agent 0.6.20+. Source deployments produce an advisory review before the image build. Open the deployment history in My Apps, or read release_history[].source_scan from the custom deployment inspection API/MCP response. Image-only installs have no source tree to review. Warnings do not block a deployment.

The agent checks the fetched source directory, not persisted application data or injected runtime/build secrets. It looks for private-key headers, selected AWS/GCP credential patterns, database URLs containing passwords and environment files. Reports contain bounded paths, dependency names, versions and advisory identifiers; never matching file contents or credential values. Review possible secrets, remove them from source and rotate exposed credentials.

The scanner matches explicit versions and OSV version intervals from package-lock.json, composer.lock and go.sum, using npm, Composer and Go ordering. Its advisory database selects GitHub-reviewed advisories and the official Go vulnerability database through OSV.dev, with source attribution. Withdrawn records are excluded from new snapshots. Unsupported or inconsistent upstream data marks coverage as partial; unknown severity stays unknown. A matching dependency is a reason to review the advisory, not proof that vulnerable code executes. In particular, go.sum may retain unused versions.

The agent verifies an Ed25519 signature before accepting the database. Each candidate database release binds a review to its exact contents and previous signed release; anomalous coverage changes require explicit acknowledgment before signing. The agent checks for updates at startup and every six hours, using HTTPS and the agent's configured Tor/SOCKS transport without direct fallback. No source code, secrets or dependency inventory is sent to advisory providers. Invalid signatures, redirects, interrupted downloads and older revisions are refused; the last verified copy remains usable. Missing trust/data gives dependency_status: unavailable; expired data gives stale. No unsigned fallback is accepted. Neither an empty report nor scanned: true certifies safe dependencies.

Administrators can inspect or refresh the published signed database with sudo impreza-agent advisories status and sudo impreza-agent advisories update. These commands update advisory data only, preserve applications and credentials, and do not update the agent binary. Older agents require the explicit agent update first.

The publisher collects daily under a reviewed coverage policy. New unsupported records or anomalous coverage stop publication for review; the signing process has no network access. The agent includes its independently pinned public key, and advisories status exposes trusted_key_sha256 for verification. The signed feed and agent are publicly available; existing agents require an explicit update. See advisory updates and the expected key fingerprint.

Limits: 60 seconds, 2 MiB per file, 64 MiB total, 20,000 tree entries, 100 findings and 50 skipped paths. Symlinks and special files are refused. Check skipped files and truncated: partial results require additional review. The portal displays this metadata as text, and the API filters agent-controlled report fields before storage and presentation.

IPv6 forwarding baseline

Available with agent 0.6.20+. The IPv6 extension filters forwarded traffic originating at bridge ports, including custom bridge names. It blocks outbound mail ports and private/link-local metadata destinations, limits new connections, and preserves inbound traffic, established connections and same-bridge communication. Resolver exceptions apply only to configured resolvers. In IPv4, the link-local metadata range is blocked on every forwarded path and before established connections; the private-range and mail-port blocks keep the docker0 and br-* scope.

The baseline is advisory on installation failure: the agent records the error and remains available. Connections from containers to the server itself are covered by the host input rule (agent 0.6.21+). The baseline does not cover host networking, alternate container network drivers, every abuse scenario or edge firewalls. Inspect the recorded IPv6 status instead of assuming that an agent update installed the rules successfully. For stricter per-app runtime routing, see the separate Tor egress option, which refuses to start when its isolation prerequisites fail.

Connections from containers to the server itself (agent 0.6.21+, revised in 0.6.25)

The agent adds a host input rule, IMPREZA-EGRESS-HOST, for IPv4 and IPv6. From agent 0.6.25 it is the last rule of the server's INPUT chain, so your firewall's own rules come first and always win; the agent keeps it last, including after ufw enable and reboots. Connections from Docker bridges (docker0 and br-*) that nobody accepted are dropped, except established flows, ICMP, and the ports this server publishes through Docker on a non-loopback address. Traffic from other interfaces, including your SSH and management access, is not affected.

Allow a host port for containers with sudo impreza-agent egress host allow tcp/PORT, or turn this part off for the whole server with sudo impreza-agent egress host disable (enable, remove and status are the other subcommands). Both persist across reboots and agent updates. On hosts managed by firewalld, Docker places its bridges in the docker zone; use the agent exception rather than a zone rule. Container traffic to the link-local metadata range (169.254.0.0/16) stays blocked.

Agents 0.6.21 to 0.6.24 placed the rule first in INPUT, before your firewall's own rules, so an allow rule you added for the container address range did not apply to these connections. Patterns that stopped working on those versions include host.docker.internal or host-gateway; a database, cache, mail relay or metrics exporter running directly on the server; a service listening on a Docker bridge gateway address; and a container on the reverse proxy's network calling the server's own public address or domain. Existing connections kept working, so the effect could appear only after an application reconnected or restarted, or after a reboot. Update those servers to 0.6.25.

The agent records the rule state in /var/lib/impreza-agent/egress.json (v4_host and v6_host) and, from 0.6.25, your exceptions in /var/lib/impreza-agent/egress-host-policy.json (root only). It writes one journal line when the rule set changes (journalctl -t impreza-agent), with counts only and no address.

Release notes before 0.6.25 did not mention this rule, and this page used to say the baseline did not cover host input.