Skip to content

Support Time-Based pull_policy in Docker Compose (e.g., once_per_day) #12563

Closed

Description

@victorwads

Description

Description

In large companies, where hundreds of developers work with company-managed images that receive updates multiple times per week, ensuring that everyone is using an up-to-date image is a real challenge.

Currently, developers need to manually remember to run docker compose pull from time to time. Often, they only realize their image is outdated after hours of debugging unexpected issues—just to discover that the problem was already fixed in a newer image.

🚀 Why Not Just Use pull_policy: always?

  • Setting pull_policy: always forces a docker pull on every docker compose up, even when there’s nothing new to update.
  • This significantly slows down the development workflow, especially in environments with large images.
  • Developers end up either:
    • Forgetting to pull images (causing issues).
    • Suffering slow startups every time they run docker compose up.
    • when offline or poor connection, this could take several time until docker give up to pull

🛠 Proposed Solution

Introduce a time-based pull_policy, such as:

services:
  my-service:
    image: company-registry.com/dev-image:latest
    pull_policy: once_per_day

Other possible values:

  • daily
  • weekly
  • custom: (e.g., custom:86400 for every 24 hours)

This would allow docker compose to automatically check if a pull has already happened today. If it has not, it performs the pull. If it has, it proceeds without pulling, ensuring a balance between up-to-date images and fast startup times.

📌 Feature Summary

  • Introduce pull_policy: once_per_day, once_per_week, etc.
  • Allow docker compose to track the last pull time and only update when necessary.
  • Improve developer workflow efficiency while ensuring images stay updated.
  • Reduce unnecessary network usage and startup delays.

This feature would make a real difference in large-scale development environments! 🚀 Would love to hear feedback from the Compose team. 😃

✍️ Tags:
enhancement, docker-compose, feature-request, pull-policy, developer-experience

Activity

  1. ndeloof commented on Feb 20, 2025

    @ndeloof
    Contributor

    That would be an interesting feature, but docker engine doesn't store the time image has been pulled, so we could compute "need to be checked for update after xx" condition. So this is technically not feasible until engine offers such a feature

  2. marlon-sousa commented on Feb 20, 2025

    @marlon-sousa

    Ok.

    I suppose then we should open a pr on docker to make, some how, a registry of when the images have been last cached on disk.

    Docker-compose would use this information to decide if a pull trial should be performed after pull policy specified time.

    Does that make sense?

  3. victorwads commented on Feb 20, 2025

    @victorwads
    Author

    @ndeloof @marlon-sousa compose could easyly just save this meta data on any userData file, just to control last time compose did pull something. We don't need to validade with docker the last pull.

  4. ndeloof commented on Feb 20, 2025

    @ndeloof
    Contributor

    @victorwads it could but Compose (without r) used to be stateless and this would be a design change for a corner case. Also to be considered actual docker engine it connect to depends on active context. Better keep docker engine as the source of truth.

  5. marlon-sousa commented on Feb 20, 2025

    @marlon-sousa

    @ndeloof agreed.

    Then first we need to make this info available to docker, right?

    Also to be considered actual docker engine it connect to depends on active context.

    Does compose connect to other engines?

    I thought docker would be the only engine. If otherwise, then things start to get complicated.

    Is there any ref documentation on thius matter?

  6. ndeloof commented on Feb 20, 2025

    @ndeloof
    Contributor

    Does compose connect to other engines?

    Compose (and any docker xx command) can communicate to any remote docker engine, depending on the active context. You may use docker commands to control multiple machines, each of those with a distinct engine version and available images (and architectures)

    By the way, Compose only relies on Moby API, so can also access an alternative implementation (typically, some use compose with podman) as long as the API is well implemented.

  7. ndeloof commented on Feb 21, 2025

    @ndeloof
    Contributor

    Actually, moby/moby#31497 introduced image metadata which can be used to know last time image was pulled.

  8. self-assigned this
    on Feb 21, 2025
  9. thaJeztah commented on Feb 21, 2025

    @thaJeztah
    Member

    Setting pull_policy: always forces a docker pull on every docker compose up, even when there’s nothing new to update.

    For this part, we should look where most time is taken; effectively, a docker pull should only have to do a HEAD to check if the digest it has locally is still current; from some quick tests, it still takes close to a second to do so; we should look what's taking up that much;

    time docker pull --quiet ubuntu
    docker.io/library/ubuntu:latest
    
    real	0m5.163s
    user	0m0.015s
    sys	0m0.015s
    
    time docker pull --quiet alpine
    docker.io/library/ubuntu:latest
    
    real	0m0.906s
    user	0m0.017s
    sys	0m0.013s
  10. thaJeztah commented on Feb 21, 2025

    @thaJeztah
    Member

    Actually, moby/moby#31497 introduced image metadata which can be used to know last time image was pulled.

    Reading the ticket here; I guess the request is not when a last pull actually happened, but when it was last checked (which could be "checked, but nothing to do"), so possibly slightly different. That's what's made me slightly wonder if there's things to do to make always more performant, so that it can guarantee the image to be fresh, without that coming with a real penalty (~equivalent to compose having to do a docker image inspect to learn the last time it was checked).

  11. ndeloof commented on Feb 21, 2025

    @ndeloof
    Contributor

    but when it was last checked

    this is actually what metadata reports. Attempt to pull, even image is up to date, will change LastTagTime in metadata:

    $ docker image inspect alpine -f '{{ .Metadata.LastTagTime }}'
    2025-02-21 09:45:10.235550419 +0000 UTC
    $ docker pull alpine
    Using default tag: latest
    latest: Pulling from library/alpine
    Digest: sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c
    Status: Image is up to date for alpine:latest
    docker.io/library/alpine:latest
    $ docker image inspect alpine -f '{{ .Metadata.LastTagTime }}'
    2025-02-21 10:07:22.382362425 +0000 UTC
    
  12. victorwads commented on Feb 28, 2025

    @victorwads
    Author

    @ndeloof Sorry to bother you, I have one more question.

    Could you clarify the publication flow for Docker Compose across repositories and how it aligns with Docker Desktop updates? Specifically, how does the process work for updating documentation like this link:
    🔗 Compose File Services - pull_policy

    Thanks in advance!

  13. ndeloof commented on Feb 28, 2025

    @ndeloof
    Contributor

    Docker Desktop is released monthly and we use to release Compose ~one week before to get the last fixes/and features in
    About docs, those are updated after a release based on recent changes taking place on the compose-spec side

  14. victorwads commented on Feb 28, 2025

    @victorwads
    Author

    Thank Youu @ndeloof

  15. BHSPitMonkey commented on Jul 23, 2025

    @BHSPitMonkey

    This is great! Is there a way (or a plan) to be able to do something similar with build.pull? (i.e. for the images referenced in a Dockerfile from a service's build: spec)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions