docker compose up recreates running container that does not have their configs changed in docker-compose.yml

Description

docker compose up stops and recreates running containers that do not have their config changed.

Steps to reproduce the issue:

Run docker compose up
Change config for one service in docker-compose.yml
Run docker compose up again

Describe the results you received:
A lot of containers that are not related to the service that has its config changed are recreated and restarted.

Describe the results you expected:
Only the container of the service that has new config should be recreated and those that depend on it should be restarted.

Additional information you deem important (e.g. issue happens only occasionally):
After switching to v2 it happens almost all of the time. Previously on v1 there was no such problem.

Output of docker compose version:

Docker Compose version v2.6.0

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
  compose: Docker Compose (Docker Inc., v2.6.0)
  scan: Docker Scan (Docker Inc., v0.17.0)

Server:
 Containers: 45
  Running: 39
  Paused: 0
  Stopped: 6
 Images: 36
 Server Version: 20.10.17
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc version: v1.1.2-0-ga916309
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: default
  cgroupns
 Kernel Version: 5.15.0-40-generic
 Operating System: Ubuntu 22.04 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 11.66GiB
 Name: aardvark-vm
 ID: CKOC:QAHC:ZJ6M:5INA:JRIE:BFCW:EUIA:OCHV:BRZ2:MF3K:C4PO:EGDQ
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details:

I can't reproduce locally. Can you provide a sample compose file?
I tried with this,

version: '3.7'

networks:
  none: {}

services:
  one:
    image: 'ubuntu:focal'
    command: 
      - sleep
      - 1d
    networks:
      - none

  two:
    image: 'ubuntu:focal'
    command: 
      - sleep
      - 1d

Once without network then with for the one service.

@debdutdeb

@debdutdeb I tried with your compose file, and ran docker compose up -d twice (with around one minute in between) without modifying anything.

The first run, both of the containers came up. This is expected. The second run, both of them are recreated, but I did not modify anything.

Afterwards, I tried it again, it does not happen. Even with modified compose file, it doesn't always occur.

Noticed i was using 2.0.something - old fork of mine. Checking again with 2.6.x.

Looks like you're right 👀

But can't reproduce reliably though.

Yea, it does not always occur, yet I don't see a pattern as to why it occurs when it occurs

I think the mechanism for determining config changes is by hashing the config and storing that hash in the label com.docker.compose.config-hash.

You can check if the label changes between invocations to further verify what's going on.

Different compose executable versions will sometimes produce different hash outputs on the same config file, so make sure tests are done with a consistent binary version.

Is this a duplicate of #9357?

I'm facing this issue in docker-compose 2.11.2. To reliably reproduce :

Compose file

version: '3.2'
services:
  router:
    image: httpd:2.4
    depends_on:
      - one
  database:
    image: docker.io/centos/postgresql-12-centos7
  one:
    image: alpine:latest
    depends_on:
      - database
  two:
    image: alpine:latest
    depends_on:
      - router

Pre-condition:

images in composer file is not available locally (alpine:latest, httpd:2.4, centos/postgresql-12-centos7). If they already exist, remove them.

Step:

docker-compose up -d.
sleep 90 (optional and doesn't affect the result)
docker-compose run --rm two echo 123. All running containers will be recreated at this step

Note

It occurs if the images are not available locally
it doesn't occur if the service is started with --no-deps or service has no linked services

I'm getting the same issue and all images are available locally.

In my case, it's triggered by a docker-compose run, and a linked running container gets recreated even though nothing has changed.

I checked the hash, it does not change across the unexpected recreates.

EDIT: as pointed out in #10068, in my case the result of docker-compose config --hash <service> does not change, but the result of docker inspect <container> -f '{{json .Config.Labels}}' | jq -r '."com.docker.compose.config-hash"' does change

v2.12.0

Seeing this issue on v2.14.0 too - any word on a fix? We have a use cae where we need to gradually add some services to the compose file and start them only after confirming the previous service is happy and healthy.

After I commented on this, I discovered the flag “-- no-recreate” which seems to solve the problem for me. But I don’t understand why this behaviour isn’t the default. Note we just moved from compose 1.x to latest 2.x so still figuring out a few changes.

After I commented on this, I discovered the flag “-- no-recreate” which seems to solve the problem for me. But I don’t understand why this behaviour isn’t the default. Note we just moved from compose 1.x to latest 2.x so still figuring out a few changes.

That flag shouldn't be the default, you're just working around the bug.

Hi guys, this is a pretty bad regression on well established behaviour of docker compose. We will likey need to return to version 1.x until this is addressed - what needs to happen to have the core developers take a serious look at this?

@sillidev

I'm facing this issue in docker-compose 2.11.2. To reliably reproduce :

Compose file
version: '3.2'
services:
  router:
    image: httpd:2.4
    depends_on:
      - one
  database:
    image: docker.io/centos/postgresql-12-centos7
  one:
    image: alpine:latest
    depends_on:
      - database
  two:
    image: alpine:latest
    depends_on:
      - router
Pre-condition:

images in composer file is not available locally (alpine:latest, httpd:2.4, centos/postgresql-12-centos7). If they already exist, remove them.

Step:

docker-compose up -d.

sleep 90 (optional and doesn't affect the result)

docker-compose run --rm two echo 123. All running containers will be recreated at this step

Note

It occurs if the images are not available locally

it doesn't occur if the service is started with --no-deps or service has no linked services

@sillidev does it not make sense to recreate those containers if the one you're running (run means a new container than what up -d created) depends on those directly or indirectly? ~~You lost me there. This doesn't seem like a regression.~~
rethinking - if run creates a new container to "run" a command, maybe it should run the dependencies individually as well?

So after the test above, I changed the binary back from v2.6.0 to v2.17.0-rc.1, and ran docker compose up -d again, and both container got recreated, although the images did not change, nor did the compose file content. So the trigger is possibly how the problematic compose binary created the containers?

@imjuzcy

@imjuzcy please check the com.docker.compose.config-hash label set on containers, and compare with docker config --hash "*". If compose file has no change, those should match (and container won't be recreated)
Please notice hash computation had some bugs in the past, so switching between compose releases might trigger containers to be recreated, but this is expected then.

Ah, got it.

But back to v2.17.0-rc.1, it's still solid for me, no unnecessary recreates so far.

I also updated to the rc, and working very nicely with nicer coloured output 😆

I've got auto-updates for compose, so it seems this got broken in v2.16.0...

Closing this issue then, feel free to comment / open a follow-up if this strikes back with 2.17+

Sorry,but how can i install the v2.17.0 version in debian?

@leranp

@leranp download binary from https://github.com/docker/compose/releases and install under ~/.docker/cli-plugins

@ndeloof

@ndeloof You missed the s at the end ~/.docker/cli-plugins (I wasted some time figuring out why it doesn't work)

Any tip for installing on Ubuntu? Or even just how to downgrade on Ubuntu to avoid this regression

@aehlke

@aehlke https://docs.docker.com/compose/install/linux/#install-the-plugin-manually

Basically the instructions provided just above your reply. Did you try them before asking? Personally I'll just wait for the next release.

Yes I had tried it without luck. Looks like my system has something wrong with it if those are also the ubuntu intructions. Thank you

thankfully just came across this issue and I managed to use apt (on Ubuntu 22.10) to update from v2.16 to v2.17.2 and this solved my problem.
Thank you!

Upgrading the package on my Debian 11 server also fixed this issue.
Unpacking docker-compose-plugin (2.17.2-1~debian.11~bullseye) over (2.16.0-1~debian.11~bullseye)

Not sure if it is a similar issue, but today I face the problem running docker-compose up recreates containers from another project . Two different project in different folders if I call up in project A it recreates project B and viceversa. First I try rename container_name (which was obviously different) but no success. I add COMPOSE_PROJECT_NAME to .env file to project A and now it works fine.
https://docs.docker.com/compose/environment-variables/envvars/#compose_project_name

Navigation Menu

Repository navigation

docker compose up recreates running container that does not have their configs changed in docker-compose.yml #9600

Additional environment details:

37 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

docker compose up recreates running container that does not have their configs changed in docker-compose.yml #9600

Description

Additional environment details:

Activity

debdutdeb commented on Jun 30, 2022

imjuzcy commented on Jun 30, 2022

debdutdeb commented on Jun 30, 2022

debdutdeb commented on Jun 30, 2022

debdutdeb commented on Jun 30, 2022

imjuzcy commented on Jun 30, 2022

RaphMad commented on Jul 1, 2022

benlangfeld commented on Sep 8, 2022

sillidev commented on Oct 3, 2022

swingbit commented on Dec 2, 2022

captainfalcon23 commented on Dec 20, 2022

captainfalcon23 commented on Dec 20, 2022

benlangfeld commented on Dec 20, 2022

zephyia commented on Jan 24, 2023

debdutdeb commented on Jan 24, 2023

37 remaining items

imjuzcy commented on Mar 9, 2023

ndeloof commented on Mar 9, 2023

imjuzcy commented on Mar 9, 2023

itsthejb commented on Mar 9, 2023

ndeloof commented on Mar 10, 2023

leranp commented on Mar 12, 2023

ndeloof commented on Mar 12, 2023

oleg-erom commented on Mar 16, 2023

aehlke commented on Mar 21, 2023

louis-lau commented on Mar 21, 2023

aehlke commented on Mar 22, 2023

theonlysinjin commented on Mar 29, 2023

FingerlessGlov3s commented on Mar 30, 2023

za-ek commented on Mar 5, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions