Docker Swarm Agent

The Status200 Docker Swarm agent monitors a Docker Swarm cluster: its nodes, services, tasks, stacks, overlay networks, secrets, configs and volumes, plus each container's metrics and logs. You run it with Docker Compose on a swarm manager node, and the cluster appears in Status200 with all of them. Docker Swarm monitors alert on what arrives.

How it works#

The agent is three containers, all defined in one docker-compose.yml:

1. Collector: an OpenTelemetry Collector that reads docker_stats for each container's metrics, tails the containers' log files, and tails the inventory snapshot file. It stamps everything with your cluster's name (docker.swarm.cluster.name) and sends it to Status200 over OTLP.
2. Inventory poller: a small curl and jq sidecar that every 5 minutes walks the swarm manager's API (/nodes, /services, /tasks, /networks, /secrets, /configs, /volumes) and works out the stacks from the com.docker.stack.namespace service label. It writes one JSON line per object for the collector to forward.
3. AI agent: status200-docker-swarm-ai-agent, which runs the docker commands Status200 AI asks for while it investigates an incident or alert on this cluster. It is read-only unless you allow fixes (see AI agent).

The cluster registers itself with its first data, under the cluster name you configure.

Before you begin#

• A swarm manager node to run the agent on: the inventory poller and the AI agent call API endpoints only managers answer.
• Docker Engine 20.10 or later with the Docker Compose v2 plugin.
• An ingestion key. Create one under Project Settings → Telemetry & APM → Ingestion Keys and copy its value.
• Outbound HTTPS from the node to your Status200 URL.

Where to run the agent#

Run it on a manager node. The collector reports the containers of the node it runs on, so for every node's container metrics, run the collector on every node, all with the same DOCKER_SWARM_CLUSTER_NAME; the inventory poller and the AI agent need to run on one manager only. install.sh leaves the AI agent out on a node that is not a manager (and removes one it installed there before). An AI agent started on a worker anyway never registers for the cluster: it waits until its node is a manager, so it cannot keep the manager's AI agent out. Deploying the agent as a swarm stack instead of with Compose? Keep the inventory poller and the AI agent on managers with the constraint node.role == manager.

Install the agent#

1. Install and start the agent#

   install.shDocker Compose

   install.sh

   Download the install script and run it on a manager node:

   Bash

   curl -sSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/install.sh -o install.sh
   sh install.sh

   The script asks for your Status200 URL, the ingestion key and a name for the cluster. It installs to /opt/status200-docker-swarm-agent, writes a .env file only its owner can read, and starts all three containers. Add --no-ai-agent to leave the AI agent out.

   Docker Compose

   Download docker-compose.yml, otel-collector-config.yaml and inventory-snapshot.sh from the [DockerSwarmAgent directory](<YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent) into a new folder on a manager node:

   Bash

   curl -fsSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/docker-compose.yml -o docker-compose.yml
   curl -fsSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/otel-collector-config.yaml -o otel-collector-config.yaml
   curl -fsSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/inventory-snapshot.sh -o inventory-snapshot.sh
   chmod +x inventory-snapshot.sh

   Create a .env file next to them. It holds the ingestion key, so keep it private with chmod 600 .env:

   .envBash

   STATUS200_URL=https://status200.ru
   STATUS200_SERVICE_TOKEN=your-telemetry-ingestion-key
   DOCKER_SWARM_CLUSTER_NAME=my-swarm

   Start the agent:

   Bash

   docker compose up -d

2. Check the agent is running#

   Bash

   docker compose ps
   docker compose logs --tail 50 status200-docker-swarm-agent status200-docker-swarm-inventory

   docker compose ps lists three containers: the collector (status200-docker-swarm-agent), the inventory poller (status200-docker-swarm-inventory) and the AI agent (status200-docker-swarm-ai-agent).

3. Find the cluster in Status200#

   Open Products → Infrastructure → Docker Swarm → All Clusters. The cluster appears within a few minutes, and its Nodes, Services, Tasks, Stacks, Networks, Secrets, Configs and Volumes pages fill in after the first inventory snapshot, within 5 minutes. The cluster's Topology page draws its stacks, services, tasks and nodes as one map: see Topology.

Environment variables#

The install script writes the first three to /opt/status200-docker-swarm-agent/.env; a Docker Compose install reads them from the .env next to docker-compose.yml. Add or change any of them there, then run docker compose up -d in that folder.

The AI agent's own settings are under AI agent.

What the agent collects#

How it differs from the Docker agent#

The Docker agent models one host, and stamps host.name and container.runtime=docker. The Docker Swarm agent stamps only docker.swarm.cluster.name, on purpose: Status200 then files the data under the swarm cluster, instead of registering each node as a host of its own.

Tag the cluster with project labels#

A resource attribute that starts with status200.label. becomes a project label on the cluster: status200.label.<dimension>=<value> is the label <dimension>:<value>. Add the attributes to the resource processor in otel-collector-config.yaml, next to docker.swarm.cluster.name:

processors:
  resource:
    attributes:
      # ...existing attributes...
      - key: status200.label.team
        value: platform
        action: upsert
      - key: status200.label.env
        value: production
        action: upsert

The collector reads its config only when it starts: run docker compose restart status200-docker-swarm-agent. The cluster then shows the labels team:platform and env:production. Labels are matched without regard to case, so an existing Production label is reused rather than duplicated, and the agent never removes a label someone added in Status200.

Run as a systemd service#

To start the agent with the node and restart it when it fails, install its systemd unit. It assumes the agent lives in /opt/status200-docker-swarm-agent, where the install script puts it:

sudo curl -fsSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/systemd/status200-docker-swarm-agent.service -o /etc/systemd/system/status200-docker-swarm-agent.service
sudo systemctl daemon-reload
sudo systemctl enable --now status200-docker-swarm-agent

Upgrading the agent#

The agent reports the collector version it runs. When that is older than the version this Status200 release pins, a warning sign appears beside the agent's version on the cluster's Overview. Select it to see these commands.

Run the install script again on the manager node, and answer its questions with the same URL, key and cluster name. It downloads the latest files, pulls the images and restarts the agent:

curl -sSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/install.sh -o install.sh
sh install.sh

The script writes .env again from your answers. It keeps only the AI agent's target lists and what AI may do (STATUS200_AI_INVESTIGATION, STATUS200_AI_FIXES): add back any variable you set yourself, such as DOCKER_API_VERSION. Turned AI fixes on? Run it as STATUS200_AI_ALLOW_WRITES=true sh install.sh to keep them on. Left the AI agent out? Pass --no-ai-agent again.

With Docker Compose, download the three files again in the agent's folder (then make again any change you made to docker-compose.yml), and pull and restart:

docker compose pull
docker compose up -d --force-recreate

The collector version is pinned in the files, so pulling alone does not move the agent forward.

Uninstalling the agent#

cd /opt/status200-docker-swarm-agent
docker compose down

AI agent#

The swarm's docker-compose.yml (and install.sh) also runs the Docker Swarm AI agent: a third container, status200-docker-swarm-ai-agent (image status200/resource-ai-agent:release), that runs the read-only docker commands Status200 AI asks for while it investigates an incident or alert on this cluster — docker node ls, docker service ps web --no-trunc, docker service logs --tail 200 web — and, only if you allow it, applies fixes such as a rolling restart of a named service. It reads the same STATUS200_URL, STATUS200_SERVICE_TOKEN and DOCKER_SWARM_CLUSTER_NAME as the collector, so it serves exactly this cluster, and it shows up on the cluster's AI → AI agent page in Status200.

• AI investigations are on by default. Once the agent connects, Status200 AI investigates incidents and alerts on this cluster with it, and the cluster's Overview shows the agent's status. To stop, set STATUS200_AI_INVESTIGATION=false where the agent runs (Change under What AI may do on the AI agent page shows how), or leave the agent out.
• Like the inventory poller it must run on a manager node: node, service and task commands are manager-only. On a worker it runs nothing and does not register for the cluster until the node is a manager (its /status says why). Run one per cluster.
• It is read-only unless you set STATUS200_AI_ALLOW_WRITES=true in .env and run docker compose up -d (or run install.sh with it; the installer writes STATUS200_AI_ALLOW_WRITES, STATUS200_AI_INVESTIGATION, STATUS200_AI_FIXES, STATUS200_AI_WRITE_TARGETS and STATUS200_AI_PROTECTED_TARGETS from its environment into .env, and a re-run keeps what .env already has for the last four unless you pass that variable, even empty); STATUS200_AI_WRITE_TARGETS (service and node name globs, for example web_*,api_*) limits what a fix may touch. STATUS200_AI_FIXES in the same .env says how fixes run — ask-for-approval (a person approves each one), automatic or bypass-approval — and the AI agent page shows it read-only (What AI may do, set by the agent). Draining a node always needs a person's approval, whatever the mode.
• It runs as root with the Docker socket mounted, because the socket is root-owned — and on a manager, whoever can use the socket controls the whole swarm, :ro or not. The agent's command policy is the limit: it never creates or removes services or stacks, never touches secrets or configs, and never changes itself, the collector, the inventory poller or anything in STATUS200_AI_PROTECTED_TARGETS (service and node name globs).
• install.sh --no-ai-agent leaves it out, or delete the status200-docker-swarm-ai-agent service from docker-compose.yml.

What it may run, how fixes work and how to troubleshoot it: Infrastructure AI Agents.

Troubleshooting#

curl -sSL <YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerSwarmAgent/troubleshoot.sh -o troubleshoot.sh
bash troubleshoot.sh    # add -d <dir> if you installed outside /opt/status200-docker-swarm-agent

Error: cannot start pipelines: failed to start "docker_stats" receiver:
Error response from daemon: client version 1.44 is too new.
Maximum supported API version is 1.41

docker version --format '{{ .Server.APIVersion }}'

Next steps#