Docker Agent

The Status200 Docker agent is one container you run on a Docker host. It reads the stats and log files of every container on the host and sends them to Status200, where the host appears with its containers, metrics and logs. A second container, the Docker AI agent, lets Status200 AI investigate the host. Docker monitors alert on what the agent sends.

How it works#

The agent is an OpenTelemetry Collector with Status200's configuration built in. Through the Docker socket it reads each container's stats every 30 seconds and lists every container, image, network and volume every 5 minutes. It tails the containers' log files and sends everything to your Status200 URL over OTLP, with the ingestion key you give it.

The host registers itself with its first data, under the name you give it in DOCKER_HOST_NAME. The AI agent is optional: leave it out and the collector works the same.

Before you begin#

• Docker Engine 20.10 or later, and root access on the host: the agent reads /var/run/docker.sock.
• Docker Compose v2 (the docker compose command), if you install with Docker Compose.
• An ingestion key. Create one under Project Settings → Telemetry & APM → Ingestion Keys and copy its value.
• Outbound HTTPS from the host to your Status200 URL.
• For container logs, containers that use Docker's json-file log driver, which is Docker's default. See Container logs need the json-file log driver.

Install the agent#

1. Start the agent#

   Replace YOUR_STATUS200_URL with your Status200 URL, YOUR_TELEMETRY_INGESTION_TOKEN with the ingestion key, and my-docker-host with a name for this host, such as prod-docker-01. The name is how the host appears in Status200, so keep it stable: a new name registers a new host.

   Docker CLIDocker Compose

   Docker CLI

   Bash

   docker run -d \
     --name status200-docker-agent \
     --user 0:0 \
     --restart unless-stopped \
     -v /var/run/docker.sock:/var/run/docker.sock:ro \
     -v /var/lib/docker/containers:/var/lib/docker/containers:ro \
     -e STATUS200_URL="YOUR_STATUS200_URL" \
     -e STATUS200_SERVICE_TOKEN="YOUR_TELEMETRY_INGESTION_TOKEN" \
     -e DOCKER_HOST_NAME="my-docker-host" \
     status200/docker-agent:release

   Then start the AI agent beside it, with the same values. AI investigations are on by default: while Status200 AI investigates an incident or alert on this host, it runs read-only docker commands through the AI agent, and it changes nothing unless you allow fixes. To install without AI investigations, skip this command.

   Bash

   docker run -d \
     --name status200-docker-ai-agent \
     --user 0:0 \
     --restart unless-stopped \
     --read-only --tmpfs /tmp \
     -v /var/run/docker.sock:/var/run/docker.sock:ro \
     -e STATUS200_URL="YOUR_STATUS200_URL" \
     -e STATUS200_SERVICE_TOKEN="YOUR_TELEMETRY_INGESTION_TOKEN" \
     -e STATUS200_AI_AGENT_RESOURCE_TYPE=docker \
     -e DOCKER_HOST_NAME="my-docker-host" \
     status200/resource-ai-agent:release

   Docker Compose

   Save this as docker-compose.yml in a folder on the host. The second service is the AI agent: leave it out to run the collector without AI investigations.

   docker-compose.ymlYAML

   services:
     status200-docker-agent:
       image: status200/docker-agent:release
       container_name: status200-docker-agent
       user: "0:0"
       restart: unless-stopped
       volumes:
         - /var/run/docker.sock:/var/run/docker.sock:ro
         - /var/lib/docker/containers:/var/lib/docker/containers:ro
       environment:
         - STATUS200_URL=YOUR_STATUS200_URL
         - STATUS200_SERVICE_TOKEN=YOUR_TELEMETRY_INGESTION_TOKEN
         - DOCKER_HOST_NAME=my-docker-host
       logging:
         driver: json-file
         options:
           max-size: "10m"
           max-file: "3"
     status200-docker-ai-agent:
       image: status200/resource-ai-agent:release
       container_name: status200-docker-ai-agent
       user: "0:0"
       restart: unless-stopped
       read_only: true
       tmpfs:
         - /tmp
       volumes:
         - /var/run/docker.sock:/var/run/docker.sock:ro
       environment:
         - STATUS200_URL=YOUR_STATUS200_URL
         - STATUS200_SERVICE_TOKEN=YOUR_TELEMETRY_INGESTION_TOKEN
         - STATUS200_AI_AGENT_RESOURCE_TYPE=docker
         - DOCKER_HOST_NAME=my-docker-host

   Start it in that folder:

   Bash

   docker compose up -d

2. Check the agent is running#

   Bash

   docker ps --filter name=status200-docker-agent
   docker logs -f status200-docker-agent

   The logs should show this line once the collector has started:

   Everything is ready. Begin running and processing data.

3. Find the host in Status200#

   Open Products → Infrastructure → Docker → All Hosts. The host appears under its DOCKER_HOST_NAME within a minute or so, with its containers, metrics and logs. The AI agent shows as connected on the host's AI agent page, and at the bottom of its Overview. The host's Topology page draws its Compose projects, containers, images, networks and volumes as one map: see Topology.

Environment variables#

To change a value, remove the agent and run its command again (Docker CLI), or edit docker-compose.yml and run docker compose up -d (Docker Compose).

What the agent collects#

Each container's metrics and logs are filed under a service named after the container. Each log line gets a severity from the level it names, such as [ERROR], level=warn or {"level":"info"}; a line that names none takes its stream's: stderr is ERROR, stdout is INFO. Lines that start with whitespace or a closing bracket join the line before, so a stack trace arrives as one record. The agent's own logs are left out.

Docker Monitor lists every metric and what it means.

Container logs need the json-file log driver#

The agent tails /var/lib/docker/containers/*/*-json.log, the files Docker's json-file log driver writes. A container that uses another driver leaves no file to read: local writes binary files, and journald, syslog, fluentd and gelf send the logs elsewhere. Its metrics still arrive; its logs do not.

Check a container's log driver, and the daemon's default:

docker inspect <container> --format '{{.HostConfig.LogConfig.Type}}'
docker info --format '{{.LoggingDriver}}'

Switch a Compose service to json-file, with rotation:

services:
  my-app:
    image: my-app:latest
    logging:
      driver: "json-file"
      options:
        max-size: "100m"
        max-file: "5"

Or make it the default for every container created afterwards, in /etc/docker/daemon.json:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "5"
  }
}

Then restart Docker and recreate the containers. A container keeps the log driver it was created with, so restarting it is not enough.

Upgrading the agent#

When the agent is older than your Status200, a warning sign appears beside Agent Version on the host's Overview. Select it to see these commands.

docker pull status200/docker-agent:release
docker pull status200/resource-ai-agent:release
docker rm -f status200-docker-agent status200-docker-ai-agent

docker compose pull
docker compose up -d

The release tag always points at the newest release. To pin a version, use status200/docker-agent:<version>, such as status200/docker-agent:10.0.31. The same image is on the GitHub Container Registry as ghcr.io/status200/docker-agent:release.

Uninstalling the agent#

docker rm -f status200-docker-agent
docker rm -f status200-docker-ai-agent

docker compose down

The host stays in Status200 with the data it sent. If you no longer need it, delete it with Delete Host in its menu.

AI agent#

The commands and the Compose file above, the Docker agent's install.sh and the docker-compose.yml in its [DockerAgent directory](<YOUR_STATUS200_URL>/docs/static/scripts/agents/DockerAgent) all run the Docker AI agent: a second container, status200-docker-ai-agent (image status200/resource-ai-agent:release), that runs the read-only docker commands Status200 AI asks for while it investigates an incident or alert on this host — docker ps -a, docker logs --tail 200 web, docker container inspect web, docker stats --no-stream — and, only if you allow it, applies fixes such as restarting a named container. It reads the same STATUS200_URL, STATUS200_SERVICE_TOKEN and DOCKER_HOST_NAME as the collector, so it serves exactly this host, and it shows up on the host's AI agent page in Status200.

• AI investigations are on by default. Once the agent connects, Status200 AI investigates incidents and alerts on this host with it, and the host's Overview shows the agent's status. To stop, set STATUS200_AI_INVESTIGATION=false where the agent runs (Change under What AI may do on the AI agent page shows how), or leave the agent out.
• It is read-only unless you start it with STATUS200_AI_ALLOW_WRITES=true; STATUS200_AI_WRITE_TARGETS (for example web-*,api-*) limits which containers a fix may touch. Start it with STATUS200_AI_FIXES too, to say how fixes run — ask-for-approval (a person approves each one), automatic or bypass-approval — and the AI agent page shows it read-only (What AI may do, set by the agent).
• It runs as root with the Docker socket mounted, because the socket is root-owned — and whoever can use the socket is root on this host, :ro or not. The agent's command policy is the limit: it never runs exec, run, rm or prune, and it never changes itself, the collector or anything in STATUS200_AI_PROTECTED_TARGETS (container name globs). Environment values in docker inspect output are masked before anything leaves the host.
• install.sh --no-ai-agent leaves it out; docker rm -f status200-docker-ai-agent removes it. Upgrade it like the collector: docker pull status200/resource-ai-agent:release, remove it, and start it again.

What it may run, how fixes work and how to troubleshoot it: Infrastructure AI Agents.

Troubleshooting#

Error: cannot start pipelines: failed to start "docker_stats" receiver:
Error response from daemon: client version 1.44 is too new.
Maximum supported API version is 1.41

docker version --format '{{ .Server.APIVersion }}'

docker run -d ... -e DOCKER_API_VERSION= ...

docker logs status200-docker-agent 2>&1 | grep -E "Started watching file|no files match"
docker inspect <container> --format '{{.HostConfig.LogConfig.Type}}'

docker inspect status200-docker-agent --format '{{range .Config.Env}}{{println .}}{{end}}' | grep DOCKER_HOST_NAME

Next steps#