Why Portainer did not see my running container: troubleshooting Docker API and unix socket communication

The container ran, but Portainer could not see it: hunting down the Docker API communication#

Web management dashboards (such as Portainer) can occasionally display unexpected behavior. I recently ran into an interesting scenario: a container was active and running cleanly from the server terminal, but when opening the Portainer web UI on the same host, the container was missing from the container list.

If you have ever asked "how do I add an already running container to Portainer if both reside on the same machine?", this article breaks down the internal Docker architecture, how Portainer interacts with the engine, and how to troubleshoot visibility issues.

1. Initial scenario#

In the server terminal, I checked running processes with standard commands:

docker ps

CONTAINER ID   IMAGE              COMMAND                  CREATED          STATUS          PORTS                                      NAMES
045839afd846   app-custom-image   "docker-entrypoint.s…"   13 minutes ago   Up 13 minutes   0.0.0.0:8889->3000/tcp, [::]:8889->3000/tcp   my-web-app

The container my-web-app (ID 045839afd846) was up, healthy, and listening on port 8889. Yet in the Portainer UI, the question arose: should Portainer discover this automatically? Is there an import button? Why is it not listed?

2. How Portainer communicates with Docker#

To understand the cause, the communication architecture must be inspected:

1. Portainer does not maintain an independent container state database. It does not keep a separate table of running containers.
2. Portainer is an HTTP REST client for the Docker Engine API. Every time you view the Containers tab, Portainer makes REST calls to the Docker daemon (dockerd).
3. On Linux, this communication happens over the Unix Domain Socket (/var/run/docker.sock).

Under normal conditions, the flow works as follows:

• Portainer makes a connect() syscall to /var/run/docker.sock.
• It issues GET /v1.41/containers/json?all=1.
• dockerd returns a JSON payload containing all active containers, including my-web-app.

Therefore, there is no "Import Container" button. Any container recognized by dockerd must appear automatically unless there is a broken socket connection, permission error, or interface filter.

3. Common causes of container invisibility#

When a container running on the same server fails to appear in Portainer, check these common failure points:

A. Missing or incorrect unix socket volume mount#

If Portainer runs as a Docker container, it requires direct access to the host socket:

-v /var/run/docker.sock:/var/run/docker.sock

If this mount is missing, or if the environment in Portainer (Settings > Environments) was configured to TCP (localhost:2375) without the daemon listening on that network port, the dashboard receives no data.

B. File permissions on /var/run/docker.sock#

On the host, the Docker socket is owned by root:docker with srw-rw---- permissions:

ls -la /var/run/docker.sock
# srw-rw---- 1 root docker 0 Aug 30 10:00 /var/run/docker.sock

If the Portainer process runs under a non-root UID lacking membership in the mapped docker group, the open() syscall returns EACCES (Permission denied).

C. RBAC filters and "external" container tagging#

Containers started directly via CLI (docker run or shell scripts) lack Portainer ownership tags. The dashboard categorizes them as "External". If user filters are active in the UI, these containers may be hidden from non-admin accounts.

D. Kernel security modules (AppArmor / SELinux)#

Host security policies can restrict container access to mounted host sockets, producing silent kernel denials.

4. Step-by-step diagnostic workflow#

To isolate communication failures:

Step 1: Check active socket listeners#

lsof /var/run/docker.sock

Verifies which PIDs are listening and connected to the Docker socket.

Step 2: Test the Docker API from inside Portainer#

Execute a query against the socket directly from inside the Portainer container:

docker exec -it portainer curl --unix-socket /var/run/docker.sock http://localhost/containers/json

• If it returns JSON containing my-web-app: Portainer communicates with Docker successfully. The issue is purely browser cache or UI filters.
• If it returns a permission or connection error: The -v /var/run/docker.sock:... mount or group permissions are blocking access.

Step 3: Inspect Portainer logs#

docker logs --tail 100 portainer

Reveals synchronization errors or HTTP 403 authorization failures.

Step 4: Check kernel denials#

dmesg | grep -i deny

Identifies if AppArmor or SELinux blocked socket descriptor access.

5. Converting external containers into managed stacks#

When a container runs as External, you can start or stop it, but you cannot edit environment variables or redeploy declaratively. The standard solution is converting it into a Stack (Docker Compose):

1. Inspect current container settings:

   docker inspect my-web-app

Note port bindings (8889:3000), volumes, and variables.

1. Create the Stack in Portainer:

Navigate to Stacks > Add Stack and define the Compose structure:

   version: '3.8'

   services:
     my-web-app:
       image: app-custom-image:latest
       container_name: my-web-app
       ports:
         - "8889:3000"
       restart: unless-stopped

1. Perform a clean takeover:

Stop and remove the CLI-created container:

   docker stop my-web-app && docker rm my-web-app

Click Deploy the stack in Portainer. Portainer now holds full declarative control over the container lifecycle.

Key takeaways and practical summary#

1. Containers do not require manual importing: Portainer reflects the real-time state of dockerd over the Unix socket.
2. If containers are missing: Inspect the /var/run/docker.sock volume mount and test API connectivity with curl --unix-socket.
3. CLI containers are labeled "External": To enable declarative editing and redeployments, convert individual containers into managed Stacks.

See also

DevOps Understanding coolify: self-hosted paas architecture, reverse proxy conflicts, and container management

DevOps The odyssey of running cPanel and systemd in Docker: from Cgroups v1 vs v2 battles to persistent volume traps

Linux Understanding tmpfs in Linux: does reducing temporary partitions really free up RAM?

This post is licensed under CC BY-NC.