[BUG] Bind mounts no longer work in Bitbucket Pipelines in v2.29.3

Description

First of all, apologies for the framing here - I don't know enough about the Docker API to understand what the real problem is here, and calling this a bug is probably wrong.

Since upgrading our pipelines runner image to use compose v2.29.3, attempting to run a compose configuration that uses bind mounts in Bitbucket Pipelines is producing this error:

Error response from daemon: authorization denied by plugin pipelines: --mounts is not allowed

I'm guessing the relevant change here is #12078, and the problem is that compose has switched to using a new API for mounts that Bitbucket does not support.

Is there a way to revert to using the old API for bind mounts? Or do we just need to pin v2.29.2 for the foreseeable future?

Steps To Reproduce

services:
    blah:
        image: alpine
        command: ["echo", "hi"]
        volumes:
            - .:/dot

pipelines:
  default:
    - step:
      name: Test compose mount
      services:
        - docker
      script:
        - docker info
        - docker compose run blah

Push to a Bitbucket Cloud repository.

Compose Version

Docker Compose version v2.29.3

Docker Environment

+ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc., v2.29.3)
Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 25.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
WARNING: API is accessible on http://0.0.0.0:2375 without encryption.
         Access to the remote API is equivalent to root access on the host. Refer
         to the 'Docker daemon attack surface' section in the documentation for
         more information: https://docs.docker.com/go/attack-surface/
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Authorization: pipelines
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7c3aca7a610df76212171d200ca3811ff6096eb8
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  userns
 Kernel Version: 5.15.0-1068-aws
 Operating System: Alpine Linux v3.19 (containerized)
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 30.89GiB
 Name: 7d637b9f-a5ae-4e16-8a1f-c3f5e524f487-h8n8b
 ID: 1551bc4e-86a3-4330-8c96-4171cf1c4220
 Docker Root Dir: /var/lib/docker/165536.165536
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  10.2.215.48:5000
  127.0.0.0/8
 Registry Mirrors:
  http://10.2.215.48:5000/
 Live Restore Enabled: false
 Product License: Community Engine

Anything else?

No response

I meet the same issue
I have no /var/lib/mysql before.

My with docker-compose.yml:

services:
  db:
    image: mysql
    volumes:
      - /var/lib/mysql:/var/lib/mysql

(.venv) [root@phuongl 1.9.6]# docker-compose -f docker-compose.yml up
[+] Running 11/11
✔ db Pulled
[+] Running 0/0
[+] Running 1/1default  Creating
✔ Network 196_default  Created
⠋ Container 196-db-1   Creating
Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /var/lib/mysql

but with v2.29.2 directory "/var/lib/mysql" will auto create.

@phuongl-ampere

mount API exists in docker engine for a while, I wonder it is not supported by bitbucket. The bind API is legacy which comes with many limitations. We rely on mount API as compose file syntax offers fine grained support of bind mounts which is only available with this newer API.

@phuongl-ampere can you please open a dedicated issue ?

We had to pin the image in the runner to docker:27.1.2 wich runs the older compose.

Until bitbucket sorts their end in 2027 (or circa) hahahaha

Can you test with the latest v2.29.7 release please?

Confirmed fixed in v2.29.7 :)

It appears this issue was reintroduced with v2.35.0.

I can confirm a regression, caused by confusion between volume mount and actual bind mount.

Maybe we could detect API doesn't support and disable mount support when running on bitbucket
Makes me wonder: is there bitbucket issue to report lack of support for mount API ?

@BVollmerhaus

@BVollmerhaus Can you please run a quick test and execute docker version command within your bitbucket pipeline ?
I'd like to know the reported "API version"

mount API was introduced by https://docs.docker.com/reference/api/engine/version-history/#v125-api-changes (Docker v1.13). I guess bitbucket isn't running such an old release ...

@ndeloof

@ndeloof

I guess bitbucket isn't running such an old release ...

It wouldn't have surprised me. 😬 I yearn for the day we'll be able to move to a different platform.

Anyway, here's the output of docker version in a failing pipeline:

Client:
 Version:           20.10.24
 API version:       1.41
 Go version:        go1.19.7
 Git commit:        297e128
 Built:             Tue Apr  4 18:17:06 2023
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          25.0.5
  API version:      1.44 (minimum version 1.24)
  Go version:       go1.21.8
  Git commit:       e63daec
  Built:            Tue Mar 19 15:05:39 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.7.13
  GitCommit:        7c3aca7a610df76212171d200ca3811ff6096eb8
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Per the docs, it seems quite intentional by Atlassian not to allow mount (assuming their restriction of the --mount option extends to the API used by Compose).

such a restriction on --mount "for security reasons" is weird. Anyway, #12734 makes preference for bind API and adds a test case to cover diverse combinations so we prevent a future regression with this madness

Navigation Menu

Repository navigation

[BUG] Bind mounts no longer work in Bitbucket Pipelines in v2.29.3 #12121

Description

Steps To Reproduce

Compose Version

Docker Environment

Anything else?

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[BUG] Bind mounts no longer work in Bitbucket Pipelines in v2.29.3 #12121

Description

Description

Steps To Reproduce

Compose Version

Docker Environment

Anything else?

Activity

phuongl-ampere commented on Sep 13, 2024

ndeloof commented on Sep 16, 2024

sbe-arg commented on Sep 16, 2024

glours commented on Sep 23, 2024

acarapetis commented on Oct 1, 2024

BVollmerhaus commented on Apr 10, 2025

ndeloof commented on Apr 10, 2025

ndeloof commented on Apr 10, 2025

ndeloof commented on Apr 10, 2025

ndeloof commented on Apr 10, 2025

BVollmerhaus commented on Apr 10, 2025

ndeloof commented on Apr 11, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions