[BUG] Compose v2 is recreating ALL containers where there is a change to one of the "depends_on" in the hierarchy (bring back compose v1 behaviour)

Description

Separating out an issue described on #9600

Basically, this comment (#9600 (comment)) encompasses this issue. Sometimes containers are completely recreated when we do not expect them to, and when using "depends_on", and a dependant container changes in some way, all of the children are restarted. In compose v1.X, this is NOT the case, and ONLY the changed container is recreated and all others are untouched.

Either this is a bug, or a feature request to bring back compose v1 behaviour (at least with a flag).

Steps To Reproduce

Refer #9600 (comment)

Compose Version

Docker Compose version v2.16.0

Docker Environment

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.9.1-docker)
  compose: Docker Compose (Docker Inc., v2.16.0)
  scan: Docker Scan (Docker Inc., v0.21.0)

Server:
 Containers: 4
  Running: 4
  Paused: 0
  Stopped: 0
 Images: 6
 Server Version: 20.10.21
 Storage Driver: overlay2
  Backing Filesystem: xfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 78f51771157abb6c9ed224c22013cdf09962315d
 runc version: v1.1.4-0-g5fd4c4d
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.80.1.el7.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.06GiB
 Name: XXX
 ID: GR4X:Q35F:5E4H:TGFY:O34K:JEYN:WYBW:3QY5:4G7N:TXP6:4GLE:QOG2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

Refer to #9600 for alot of history.

With #10284 you can now explicitly opt-in/out for dependent container to be restarted when a service is recreated.

Can you please share how to configure this in compose file? Or link to the docs :) I couldn't see a reference in the MR.

see https://github.com/compose-spec/compose-spec/blob/master/spec.md#long-syntax-1

@ndeloof

Hi @ndeloof ,
I finally got some time to test this today and it didn't work.

These are the steps I followed:

# compose ver:
Docker Compose version v2.17.0-rc.1

docker compose down
docker system prune --all
# clean slate for testing

# update docker-compose.yml as per example below
docker compose up -d
# pulls all 4 images and creates all 4 containers as expected

# Run "docker compose up -d" multiple times, no containers recreated needlessly - good!
docker compose up -d
docker compose up -d
docker compose up -d

# Then, I changed the image tag of service-4 in docker-compose.yml
docker compose up -d
# ALL the dependent containers got recreated :( 
[+] Running 4/4
 ✔ Container dro-dns-fe-pdns-publisher-1  Running                                                                                           0.0s
 ✔ Container dro-dns-fe-syslog-ng-1       Started                                                                                          21.8s
 ✔ Container dro-dns-fe-pdns-recursor-1   Started                                                                                          21.2s
 ✔ Container dro-dns-fe-dnsdist-1         Started                                                                                          10.8s

# Run "docker compose up -d" multiple times, no containers recreated needlessly - good!
docker compose up -d
docker compose up -d
docker compose up -d
docker compose up -d

# Again, modified the image tag of service-4, ALL the dependent containers got recreated :(

# Run "docker compose up -d" multiple times, no containers recreated needlessly - good!
docker compose up -d
docker compose up -d
docker compose up -d
docker compose up -d

Sample docker-compose.yml file:

version: '3.7'
services:
 service-1:
  image: "XXXXXX.dkr.ecr.ap-southeast-2.amazonaws.com/image1:1.7.0-pdns-1026"
  restart: always
  env_file:
    - .env_X
  network_mode: host
  depends_on:
   service-2:
     condition: service_started
     restart: false
   service-3:
     condition: service_started
     restart: false
 service-2:
  image: "XXXXXX.dkr.ecr.ap-southeast-2.amazonaws.com/image2:4.5.10-pdns-1023"
  restart: always
  env_file:
    - .env_X
  network_mode: host
  volumes:
    - /mnt/data/service-4/dev:/service-4
    - /mnt/data/service-2:/var/service-2
  depends_on:
    service-4:
      condition: service_started
      restart: false
    service-3:
      condition: service_started
      restart: false
 service-3:
  image: "XXXXXX.dkr.ecr.ap-southeast-2.amazonaws.com/image3:1.1.0-15-pdns-1024"
  restart: always
  env_file:
    - .env_X
  network_mode: host
  volumes:
    - /mnt/data/a:/a
 service-4:
  image: "XXXXXX.dkr.ecr.ap-southeast-2.amazonaws.com/image4:3.24-pdns-1022"
  restart: always
  env_file:
    - .env_X
  network_mode: host
  volumes:
    - /mnt/data/a/dev:/a
    - /mnt/data/b:/b
    - /mnt/data/c:/c

Have I made a mistake in the config, or is it just not working as expected?

Having same issue as decribed in #10308 (comment) with Compose v2.24.6 (latest from Ubuntu 24.04 feed)
Is this a new by design bahaviour or how to avoid this?

@andryushchenko

@andryushchenko see #10308 (comment)

@ndeloof

@ndeloof Please can you provide the example of correct usage this option?
I have the same situation as @captainfalcon23 - restart:false is set but conainers still being restarted

I have now spent some time looking into this issue, and as far as I can tell, the restart option of depends_on only applies to the docker compose restart command. At least, that's the only place that the implementation of #10284 touched. However, that does not come into effect when docker compose applies a new convergence on the entire dependency tree.

There's a pair of flags for docker compose up that are supposed to affect whether dependencies are recreated: --always-recreate-deps and --no-recreate; without the presence of either of these two, the recreation strategy should default to RecreateDiverged defined here, as selected by createOptions.dependenciesRecreateStrategy.

Unfortunately, the observed behavior seems to be as if --always-recreate-deps was on by default, but I'm not familiar enough with the internals of the convergence implementation to figure out why RecreateDiverged does not appear to apply.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

There's no indication that this has been addressed in any way. (I hope we don't have to play the stale / bump game every few months for years from here on out.)

This issue has been automatically marked as not stale anymore due to the recent activity.

@koniiiik

@koniiiik using this example compose file:

services:
 service-1:
  image: nginx
  depends_on:
   service-2:
     condition: service_started
     restart: false
 service-2:
  image: nginx

$ docker compose up -d
[+] Running 2/2
 ✔ Container truc-service-2-1  Started                                     0.3s 
 ✔ Container truc-service-1-1  Started                                     0.3s 

$ docker compose restart service-2
[+] Restarting 1/1
 ✔ Container truc-service-2-1  Started                                     0.2s

As you can see, with restart: false set, service-1 which depends on restarted service is not impacted.
I'm closing this issue as "fixed".

@ndeloof

@ndeloof That is not what this issue is about, though. Yes, the restart: false option applies correctly when you restart a dependency. However, it has no effect if the dependency is recreated (for example, if you change the runtime configuration, or upgrade the image). As far as I can tell, there is currently no way to configure docker compose to avoid recreating all dependent services in this situation.

Here's an example that involves upgrading the tag in between docker compose up -d runs:

koniiiik@parahippus /tmp/docker-compose-example $  cat docker-compose.yml 
services:
 service-1:
  image: nginx
  depends_on:
   service-2:
     condition: service_started
     restart: false
 service-2:
   image: nginx:1.27.3
koniiiik@parahippus /tmp/docker-compose-example $  docker compose up -d
[+] Running 3/3
 ✔ Network docker-compose-example_default        Created                                                                                                                                                                                                                    0.1s 
 ✔ Container docker-compose-example-service-2-1  Started                                                                                                                                                                                                                    0.3s 
 ✔ Container docker-compose-example-service-1-1  Started                                                                                                                                                                                                                    0.4s 
koniiiik@parahippus /tmp/docker-compose-example $  cat docker-compose.yml 
services:
 service-1:
  image: nginx
  depends_on:
   service-2:
     condition: service_started
     restart: false
 service-2:
   image: nginx:1.27.4
koniiiik@parahippus /tmp/docker-compose-example $  docker compose up -d
[+] Running 1/1
 ✔ service-2 Pulled                                                                                                                                                                                                                                                         2.7s 
[+] Running 2/2
 ✔ Container docker-compose-example-service-2-1  Started                                                                                                                                                                                                                    0.5s 
 ✔ Container docker-compose-example-service-1-1  Started                                                                                                                                                                                                                    0.5s 
koniiiik@parahippus /tmp/docker-compose-example $  docker compose ps -a
NAME                                 IMAGE          COMMAND                  SERVICE     CREATED          STATUS          PORTS
docker-compose-example-service-1-1   nginx          "/docker-entrypoint.…"   service-1   17 seconds ago   Up 16 seconds   80/tcp
docker-compose-example-service-2-1   nginx:1.27.4   "/docker-entrypoint.…"   service-2   17 seconds ago   Up 16 seconds   80/tcp

I hope you can see that the issue has not been addressed, and that you reopen it.

indeed, thanks for clarification (it's hard to get a clear understanding on the many issues we have to address within my limited brain :P)

No worries, we've all been there. And at least now we have a truly minimal example to reproduce the issue.

Navigation Menu

Repository navigation

[BUG] Compose v2 is recreating ALL containers where there is a change to one of the "depends_on" in the hierarchy (bring back compose v1 behaviour) #10308

Description

Steps To Reproduce

Compose Version

Docker Environment

Anything else?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[BUG] Compose v2 is recreating ALL containers where there is a change to one of the "depends_on" in the hierarchy (bring back compose v1 behaviour) #10308

Description

Description

Steps To Reproduce

Compose Version

Docker Environment

Anything else?

Activity

ndeloof commented on Feb 28, 2023

captainfalcon23 commented on Feb 28, 2023

ndeloof commented on Feb 28, 2023

captainfalcon23 commented on Mar 9, 2023

andryushchenko commented on Jul 3, 2024

ndeloof commented on Jul 3, 2024

andryushchenko commented on Jul 4, 2024

koniiiik commented on Jul 18, 2024

github-actions commented on Mar 16, 2025

koniiiik commented on Mar 17, 2025

stale commented on Mar 17, 2025

ndeloof commented on Mar 17, 2025

koniiiik commented on Mar 17, 2025

ndeloof commented on Mar 17, 2025

koniiiik commented on Mar 17, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions