Skip to content

[BUG] docker compose up --watch with sync+restart does not show logs after restart #11773

Description

@scooper91

Description

When running docker compose up --watch, the logs for the service appear in the output.

When specifying sync+restart as the action on a file change, when a file is changed, the container restarts, but there are no more logs outputted. When running docker compose logs, you can see the app was restarted correctly and is still logging.

This behaviour does not happen when using rebuild as the action (although the logs aren't picked up straight away so the first logs are potentially missed).

Expected behaviour is that when using sync+restart, when a file is changed, the logs from the container continue to be shown in the output of docker compose up --watch.

Steps To Reproduce

Dockerfile:

FROM node:slim
COPY src src

Contents of src can be anything.

Docker Compose:

services:
  app:
    build: .
    command: >-
      bash -ceuo pipefail "
        echo restart
        i=0
        while true; do echo boop $$i && ((i++)) && sleep 1; done
      "
    stop_signal: SIGKILL
    develop:
      watch:
        - action: sync+restart
          path: ./src
          target: /src

Running docker compose up --watch, then saving a file in src outputs:

➜ docker compose up --watch
[+] Running 1/2
 ✔ Network docker-compose-watch_default  Created                                                                                                                                                              0.1s 
 ⠇ Container docker-compose-watch-app-1  Created                                                                                                                                                              0.8s 
        ⦿ Watch enabled
Attaching to app-1
app-1   | restart
app-1   | boop 0
app-1   | boop 1
app-1   | boop 2
app-1   | boop 3
app-1   | boop 4
app-1   | boop 5
app-1   | boop 6
        ⦿ Syncing "app" after changes were detected
[+] Restarting 1/1
 ✔ Container docker-compose-watch-app-1  Started                                                    0.6s 
app-1 exited with code 0

When running docker compose logs, it shows all the logs correctly.

➜ docker compose logs                                                                                                                                                                                              
app-1  | restart                                                                                                                                                                                                   
app-1  | boop 0                                                                                                                                                                                                    
app-1  | boop 1                                                                                                                                                                                                    
app-1  | boop 2                                                                                                                                                                                                    
app-1  | boop 3                                                                                                                                                                                                    
app-1  | boop 4                                                                                                                                                                                                    
app-1  | boop 5                                                                                                                                                                                                    
app-1  | boop 6                                                                                                                                                                                                    
app-1  | restart                                                                                                                                                                                                   
app-1  | boop 0                                                                                                                                                                                                    
app-1  | boop 1                                                                                                                                                                                                    
app-1  | boop 2                                                                                                                                                                                                    
app-1  | boop 3
app-1  | boop 4
app-1  | boop 5

If you change sync+restart to rebuild in the docker-compose file, and run docker compose up --watch, and save a file, it now prints (most of) the logs correctly:

➜ docker compose up --watch
[+] Running 1/2
 ✔ Network docker-compose-watch_default  Created                                                    0.1s 
 ⠧ Container docker-compose-watch-app-1  Created                                                    0.7s 
        ⦿ Watch enabled
Attaching to app-1
app-1   | restart
app-1   | boop 0
app-1   | boop 1
app-1   | boop 2
app-1   | boop 3
app-1   | boop 4
app-1   | boop 5
        ⦿ Rebuilding service "app" after changes were detected...
app-1   | boop 6
        ⦿ service "app" successfully built
app-1 exited with code 0
app-1 has been recreated
app-1   | boop 2
app-1   | boop 3
app-1   | boop 4
app-1   | boop 5
app-1   | boop 6

Note that in the output, it misses the first few logs.

Doing a docker compose logs only shows the logs for the latest container.

Compose Version

➜ docker compose version
Docker Compose version v2.26.1


```sh
➜ docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.7.10
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019


### Docker Environment

```Text
➜ docker info                                                                                                                                                                                                      
Client: Docker Engine - Community                                                                                                                                                                                  
 Version:    26.1.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.14.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.26.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 125
 Server Version: 26.1.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: e377cd56a71523140ca6ae87e30244719194a521
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-27-generic
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.31GiB
 Name: --
 ID: QBED:UL4T:4VZM:D6KX:NMXE:WYFE:6DDN:PK4Y:CEGA:C5IZ:WBPG:JZES
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

Feature introduction and demo

Activity

  1. jbne commented on May 3, 2024

    @jbne

    Similar behavior happens when sending a docker compose up --watch job to the background and watching the output with docker compose logs --follow in another command window. The service logs show up until the service restart occurs, after which the logs command exits with code 0. One might expect docker compose logs --follow to show the sync/restart messages along with the rest of the output from the restarted (or terminated and started again) service.

  2. dkaste commented on May 26, 2024

    @dkaste

    I think this happens when the number of running services drops to zero, even if temporarily for a restart. For example, if you start two services but one exits due to an error, this issue will occur when the other one is restarted.

    I was able to work around this issue for now by keeping a dummy service alive at all times:

    dummy-watch-workaround:
      image: alpine:latest
      init: true
      command: ["sh", "-c", "while true; do sleep 2; done"]
    
  3. Hellzed commented on May 27, 2024

    @Hellzed

    @dkaste I confirm. Let's say I have a db and a web service, the web service being the one I want to watch in sync+restart mode.

    Just running docker compose up --watch web ends up with the terminal detaching from the service with web-1 exited with code 0.

    But running docker compose up --watch db web behaves correctly.

  4. self-assigned this
    on Jul 24, 2024
  5. torarnv commented on Nov 23, 2024

    @torarnv

    I'm still seeing this with 2.30.3

    ❯ docker compose up --watch
    [+] Running 1/0
     ✔ Container caddy-caddy-1  Running                                                        0.0s
            ⦿ Watch enabled
    Attaching to caddy-1
             ⦿ Syncing and restarting service "caddy" after 1 changes were detected
    caddy-1  | 2024/11/23 00:42:30.609	INFO	shutting down apps, then terminating	{"signal": "SIGTERM"}
    caddy-1  | 2024/11/23 00:42:30.609	WARN	exiting; byeee!! 👋	{"signal": "SIGTERM"}
    caddy-1  | 2024/11/23 00:42:30.609	INFO	http	servers shutting down with eternal grace period
    caddy-1  | 2024/11/23 00:42:30.609	INFO	admin	stopped previous server	{"address": "localhost:2019"}
    caddy-1  | 2024/11/23 00:42:30.609	INFO	shutdown complete	{"signal": "SIGTERM", "exit_code": 0}
             ⦿ service "caddy" restarted
    caddy-1 exited with code 0
             ⦿ Syncing and restarting service "caddy" after 1 changes were detected
    caddy-1  | 2024/11/23 00:42:39.434	INFO	shutting down apps, then terminating	{"signal": "SIGTERM"}
    caddy-1  | 2024/11/23 00:42:39.434	WARN	exiting; byeee!! 👋	{"signal": "SIGTERM"}
    caddy-1  | 2024/11/23 00:42:39.434	INFO	http	servers shutting down with eternal grace period
    caddy-1  | 2024/11/23 00:42:39.434	INFO	admin	stopped previous server	{"address": "localhost:2019"}
    caddy-1  | 2024/11/23 00:42:39.434	INFO	shutdown complete	{"signal": "SIGTERM", "exit_code": 0}
    caddy-1 exited with code 0
             ⦿ Watch disabled
    
    ❯
    

    After one or two changes to the Caddyfile, the service is restarted, and keeps running, but the watch disconnects.

    services:
      caddy:
        image: torarnv/caddy
        build: .
        restart: unless-stopped
        ports:
          - "8080:80"
    
        develop:
          watch:
            - action: sync+restart
              path: ./Caddyfile
              target: /etc/caddy/Caddyfile
    
    
  6. torarnv commented on Nov 23, 2024

    @torarnv

    Can someone please re-open this?

  7. Tarasovych commented on May 9, 2025

    @Tarasovych

    Same issue, occurrs from time to time after container recreation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions