Repository navigation
docker-compose V2 fails for port range and replicas>1 where V1 would succeed #8530
Description
Activity
When running the example from above the V1 docker-compose will issue a warning but proceed without an error:
WARNING: The "dummy" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash.Is it possible that V2 is more restricted and hence just fails instead of reporting a warning like V1 ?
Additionally the docker-compose ports documentation is not very clear about this apparently working port-range feature in V1, i.e. automatically assign a different port from the given range of ports to multi-replica-services.
V2 indeed doesn't reproduce this V1 warning, this is something we should consider.
documentation is not very clear about this apparently working port-range feature in V1
Indeed, I didn't know about it :DCompose v1 creates containers sequentially, as an equivalent to
docker run. Then the port specification is passed to engine. According to https://docs.docker.com/engine/reference/run/#expose-incoming-ports:
"When specifying a range for hostPort only, the
containerPort must not be a range. In this case the
container port is published somewhere within the
specified hostPort range. (e.g.,-p 1234-1236:1234/tcp)".Compose v2 creates containers in parallel, so this engine feature doesn't apply and this results in a port conflict. Seems we will have to drop this (premature?) optimization to restore this feature.
Hi
I have same problem on V2. The host ports are assigned to one container instead of distributing them among all replicas.
d0ac8e9625f3 bitnami/redis-sentinel:latest "/opt/bitnami/script…" About a minute ago Up 53 seconds 0.0.0.0:26379->26379/tcp, :::26379->26379/tcp, 0.0.0.0:26380->26379/tcp, 0.0.0.0:26381->26379/tcp, :::26380->26379/tcp, :::26381->26379/tcp os24_support_redis-sentinel_3On V1 replicas works just fine:
1b5d5d7fbe67 bitnami/redis-sentinel:latest "/opt/bitnami/script…" 7 seconds ago Up 3 seconds 0.0.0.0:26379->26379/tcp, :::26379->26379/tcp os24_support_redis-sentinel_2 4fd28e9d6e21 bitnami/redis-sentinel:latest "/opt/bitnami/script…" 7 seconds ago Up 3 seconds 0.0.0.0:26381->26379/tcp, :::26381->26379/tcp os24_support_redis-sentinel_3 2c0a7016ba04 bitnami/redis-sentinel:latest "/opt/bitnami/script…" 7 seconds ago Up 3 seconds 0.0.0.0:26380->26379/tcp, :::26380->26379/tcp os24_support_redis-sentinel_1This is still problem
Linux dc165-test 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux Docker version 20.10.9, build c2ea9bc Docker Compose version v2.0.1ran a quick test, and this actually isn't an issue with replicas. Even using
replicas: 1container is bound to the whole range, not "somrwhere within the specified range`0.0.0.0:6005->5000/tcp, 0.0.0.0:6006->5000/tcp, 0.0.0.0:6007->5000/tcp, 0.0.0.0:6008->5000/tcp, 0.0.0.0:6009->5000/tcp, 0.0.0.0:6010->5000/tcp, 0.0.0.0:6011->5000/tcp, 0.0.0.0:6012->5000/tcp, 0.0.0.0:6013->5000/tcp, 0.0.0.0:6014->5000/tcp, 0.0.0.0:6015->5000/tcp.. so there's something wrong with compose v2 passing port specification to the engine.
confirmed.
compose-v1 creates container as:"PortBindings": { "5000/tcp": [ { "HostIp": "", "HostPort": "6005-6015" } ] },
compose-v2 creates container as:
"PortBindings": { "5000/tcp": [ { "HostIp": "", "HostPort": "6005" }, { "HostIp": "", "HostPort": "6006" }, { "HostIp": "", "HostPort": "6007" }, ...
the issue is in compose-go, which "extends"
6005-6015:5000into a list of bindings for each and every values in the rangeI'm not sure if this is the same problem, but researching how to solve my own problem took me here. I was having a similar problem on a Mac,
christa % docker compose up Recreating recorder ... error ERROR: for recorder Cannot start service recorder: Ports are not available: listen tcp 0.0.0.0:5000: bind: address already in useand I was able to fix it by checking:
christa % sudo lsof -i -P -n | grep 5000 ControlCe 56829 christa 22u IPv4 0xce0b919697e39c8f 0t0 TCP *:5000 (LISTEN) ControlCe 56829 christa 23u IPv6 0xce0b919684ffd2b7 0t0 TCP *:5000 (LISTEN)and then turning off the
Airplay ReceiverService inSystem Preferences -> SharingReacted by Gerrit Edzards, Robin C Samuel, Mountagha, abdulnayeem2002, AHMAD SHADID $O™ & $IO™, anelisynagano, gaurav107, Sebastian Connan, gautamgadipudi-hpe and reda-drchronoReacted by Robin C Samuel and Sebastian Connancompose-go both extends port range to a list of ports, but also assumes this is expected
and types are designed to only accept an actual port (uint32) not range.I guess we are stuck here :'(
This will also require some clarification on https://github.com/compose-spec/compose-spec/blob/master/spec.md#ports if we want this to be addressed
- the compose spec schema does not allow use of a string as
publishedport value (https://github.com/compose-spec/compose-spec/blob/master/schema/compose-spec.json#L321)
- the compose spec schema does not allow use of a string as
Sorry for the bump, any updates?
the same issue here, I force myself to downgrade to v1
Is there a known workaround for this?
1 remaining item
Docker version 20.10.13, build a224086
Docker Compose version v2.3.3I'm seeing this issue also.
@hacdias @Hazmi35 If you run the
docker compose upseveral times (one for each replica) you can eventually get the system running with the replicas and port mappings.Currently having to remove the replica config an repeat the container instance config for each replica I need.
This issue needs looking at...its been open for a very long time with a significant bug in the use of docker-compose when testing a scaling setup. I've moved to Kubernetes/skaffold.
Reacted by Henrique Dias, Erwann Lagouche and sam$ docker --version
Docker version 20.10.14, build a224086$ docker-compose --version
Docker Compose version v2.4.1Also running into this issues. Sometimes it will bring everything up correctly, and other times I will get the bind error and one of the replicas will not start. I use Docker for Mac so I don't know if that is part of the reason.
If I am not mistaken, the "bug" is still here.
Reacted by Erwann Lagouche, Seonghyeon Cho and Uwe SchmittReacted by Aitor Sobera Ortiz de ZárateSame issue as #7188.
- added a commit that references this issue
on Jul 27, 2022 I have a similar problem as well.
services: api: build: . ports: - 8000-8010:8000-8010
I am trying to create at least 2 instances of the same image and it fails miserably:
$ docker compose up --scale api=2 [+] Running 2/2 ⠿ Container files-api-2 Created ⠿ Container files-api-1 Recreated Attaching to files-api-1, files-api-2 Error response from daemon: driver failed programming external connectivity on endpoint files-api-2 (container id): Bind for 0.0.0.0:8010 failed: port is already allocatedI referenced this in an issue - #9727
I think this issue was resolved in the new version.
This configuration worked for me.
Docker Compose Version.
$ docker-compose -v Docker Compose version v2.5.0Configuration file
docker-compose.yml.version: '3.8' services: video: image: video:latest restart: always ports: - "4060-4062:4060" deploy: mode: replicated replicas: 3
And the replicas it created:
d283b9969098 video:latest "node /usr/src/app/i…" 15 seconds ago Up 14 seconds 0.0.0.0:4062->4060/tcp, :::4062->4060/tcp video-test-video-1 e7b5075fa6c1 video:latest "node /usr/src/app/i…" 15 seconds ago Up 14 seconds 0.0.0.0:4061->4060/tcp, :::4061->4060/tcp video-test-video-3 c225ca9d71ac video:latest "node /usr/src/app/i…" 15 seconds ago Up 14 seconds 0.0.0.0:4060->4060/tcp, :::4060->4060/tcp video-test-video-2All the replicas correctly mapped to
4060,4061, and4062.It's still happening. (m1 mac, macOS 12.1 monterey)
$ docker -v Docker version 20.10.20, build 9fdeb9c $ docker compose version Docker Compose version v2.12.1
services: redis: image: bitnami/redis:latest environment: - ALLOW_EMPTY_PASSWORD=yes deploy: mode: replicated replicas: 3 ports: - 6300-6400:6379
$ docker compose up -d [+] Running 1/4 ⠿ Network composetest_default Created 0.0s ⠿ Container composetest-redis-3 Starting 0.1s ⠿ Container composetest-redis-1 Starting 0.1s ⠿ Container composetest-redis-2 Starting 0.1s Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:6300 -> 0.0.0.0:0: listen tcp 0.0.0.0:6300: bind: address already in use $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 8cb371485fba bitnami/redis:latest "/opt/bitnami/script…" 8 seconds ago Up 7 seconds 0.0.0.0:6300->6379/tcp composetest-redis-3
I'd like to fix this, but don't know where to begin. Any advice?
Reacted by Ritchie Zhu and Gilney Nathanael MathiasReacted by Aitor Sobera Ortiz de ZárateIt seems to be more of a timing issue afaik. I have seen the exact same docker-compose work if there is a delay (in my case due to high cpu load) so that the first container starts and it takes a second or two before the second container starts. When there is almost no delay (just some milliseconds) I get the bind-error.
Reacted by Esteban Canepa and Vedran R.@sh-cho Having the same issue, docker-compose v 2.6.0 on Mac M1.
Only way I could run multiple instances is by removing the port range
(e.g. I had:
ports: - "11001-11004:7777"
). I changed this to:
ports: - "7777"
and then my 4 instances could be started.
Unfortunately with random external ports, but at least they could be started.
Had exactly the same error message:
Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:11001 -> 0.0.0.0:0: listen tcp 0.0.0.0:11001: bind: address already in use
Looks like it tries to assign to "internal port 0" which is quite odd.
CheersAFAICT we need to make the
ContainerCreateAPI calls sequential to enforce a distinct port is assignedGreat work!
Hi, I keep facing this issue with docker compose 2.15.1 with following setting
ports: - "8081-8082:8080" - "8444-8445:8443" - "9991-9992:9990" - "10000-10001:9999" deploy: replicas: 2running
docker compose upstarts the first instance but shows the error message (Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:8081 -> 0.0.0.0:0: listen tcp 0.0.0.0:8081: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.), running it a second time make my 2 replicas working as expected (confirmed withdocker ps)CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 74be71e9b594 ********************************** Up 3 minutes 0.0.0.0:8081->8080/tcp, 0.0.0.0:8444->8443/tcp, 0.0.0.0:9991->9990/tcp, 0.0.0.0:10000->9999/tcp ***************** a07e1560ee5c ********************************** Up 3 minutes 0.0.0.0:8082->8080/tcp, 0.0.0.0:8445->8443/tcp, 0.0.0.0:9992->9990/tcp, 0.0.0.0:10001->9999/tcp *****************Reacted by GPla
Description
Using multiple replicas of a container where each replica should use one of the ports selected by a port-range:
When starting up these containers with docker-compose V1 the container instances use port-mapping where each container picks one host-port from the selected port range to map to internal container port.
When starting up these containers with docker-compose V2 the containers use all ports from the selected port range to map to internal container port, resulting in error like the following for the second instance of that service:
Error response from daemon: driver failed programming external connectivity on endpoint docker-compose-issue_dummy_1 (6d9cff4937b3b6946e0bb949a36362cf91477f841be34ac9bd2693453b6e8837): Bind for 0.0.0.0:6009 failed: port is already allocatedSteps to reproduce the issue:
docker-compose disable-v2docker-compose up -ddocker-compose psdocker-compose downdocker-compose enable-v2docker-compose up -dDescribe the results you received:
docker-compose.yaml only seems to work for V1 and not V2.
Describe the results you expected:
I would expect that docker-compose.yaml can be run by V1 and V2 alike.
Output of
docker version:Output of
docker info:Additional environment details (AWS ECS, Azure ACI, local, etc.):
local, MacOS BigSur 11.4