Skip to content

Container with volume type: image and subpath fails to start after dockerd restart #50999

@flyingflo

Description

@flyingflo

Description

Here is a simple docker compose with an image volume with a subpath. After a restart of dockerd (e.g. due to a system reboot) these containers fail to start.

In the journal we see
level=error msg="failed to start container" container=4a85b4a7dd1024f27ce2f6b1784fd0580d914ed6a9f09ae15032a21f8d5cacb8 error="cannot access path /var/lib/docker/overlay2/1c3e9bc58352ab72e892f188f5c7701e62093e2e8b67bd975cdcb6574d69d5b6/merged/usr: lstat /var/lib/docker/overlay2/1c3e9bc58352ab72e892f188f5c7701e62093e2e8b67bd975cdcb6574d69d5b6/merged/usr: no such file or directory"

docker compose up fails with

Error response from daemon: cannot access path /var/lib/docker/overlay2/1c3e9bc58352ab72e892f188f5c7701e62093e2e8b67bd975cdcb6574d69d5b6/merged/usr: lstat /var/lib/docker/overlay2/1c3e9bc58352ab72e892f188f5c7701e62093e2e8b67bd975cdcb6574d69d5b6/merged/usr: no such file or directory

The issue can be resolved with a new container (down and up).

Reproduce

Note: the image.subpath is required to trigger this bug.

docker-compose.yml

services:
  nix:
    restart: unless-stopped
    init: true
    volumes:
      - type: image
        source: alpine
        target: /image/usr
        image:
          subpath: usr
    image: alpine 
    command: "sh -c 'while : ; do sleep 3; done'"

up it and then restart docker systemctl restart docker.

Expected behavior

Container starts normally after docker restart.

docker version

Client: Docker Engine - Community
 Version:           28.4.0
 API version:       1.51
 Go version:        go1.24.7
 Git commit:        d8eb465
 Built:             Wed Sep  3 21:00:35 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.4.0
  API version:      1.51 (minimum version 1.24)
  Go version:       go1.24.7
  Git commit:       249d679
  Built:            Wed Sep  3 20:57:19 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.27
  GitCommit:        05044ec0a9a75232cad458027ca83437aae3f4da
 runc:
  Version:          1.2.5
  GitCommit:        v1.2.5-0-g59923ef
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    28.4.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.27.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.39.2
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 17
  Running: 15
  Paused: 0
  Stopped: 2
 Images: 208
 Server Version: 28.4.0
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
 runc version: v1.2.5-0-g59923ef
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.14.0-570.23.1.el9_6.x86_64
 Operating System: Rocky Linux 9.6 (Blue Onyx)
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 7.645GiB
 Name: ***
 ID: c49db0fa-5dd5-4e74-8d9e-659c43e20bce
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Registry Mirrors:
  https://***/
 Live Restore Enabled: false
 Default Address Pools:
   Base: 172.17.0.0/16, Size: 24

Additional Info

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/daemonCore Enginearea/volumesVolumeskind/bugBugs are bugs. The cause may or may not be known at triage time so debugging may be needed.

    Type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions