Skip to content

remove clamp_keypoints from transforms - #9236

Merged
AntoineSimoulin merged 3 commits into
pytorch:mainfrom
AntoineSimoulin:disable-clamp-kp
Oct 8, 2025
Merged

AntoineSimoulin merged 3 commits into
pytorch:mainfrom
AntoineSimoulin:disable-clamp-kp

Conversation

@AntoineSimoulin

@AntoineSimoulin AntoineSimoulin commented Oct 7, 2025

Copy link
Copy Markdown
Member

Context

This PR proposes an alternative to #9234. We disable the clamping for KeyPoints after the following transformations vertical_flip_keypoints, horizontal_flip_keypoints, _affine_keypoints_with_expand, pad_keypoints, crop_keypoints, perspective_keypoints, and elastic_keypoints. We are fixing this to avoid the issue observed in #9223, where clamping modifies the position of KeyPoints and create a misalignment with the actual locations in the transformed image. This PR will need to be supplemented with #9235 to implement a class similar to SanitizeBoundingBoxes transform to remove non valid KeyPoints, e.g. outside of the image canvas.

Implementation details

In the current proposition, we simply do not apply the clamp_keypoints by default after any transform. For KeyPoints, the clamping must therefore be explicitly set with the ClampKeyPoints transform.

# we manually set bounding boxes and key points for the demo; in a real pipeline, these would come from the data or a CV model’s output.
orig_pts = tv_tensors.KeyPoints(
    [
        [
            [445, 700],  # nose
            [320, 660],
            [370, 660],
            [420, 660],  # left eye
            [300, 620],
            [420, 620],  # left eyebrow
            [475, 665],
            [515, 665],
            [555, 655],  # right eye
            [460, 625],
            [560, 600],  # right eyebrow
            [370, 780],
            [450, 760],
            [540, 780],
            [450, 820],  # mouth
        ],
    ],
    canvas_size=(orig_img.size[1], orig_img.size[0]),
    clamping_mode="hard"
)
cropper = v2.Compose(
    [
        v2.RandomCrop(size=(128, 128)),
        # v2.ClampKeyPoints(),  # must set to enable clamping
    ]
)
crops = [cropper((orig_img, orig_pts)) for _ in range(4)]
plot([(orig_img, orig_pts)] + crops)

Illustration of the changes

Without explicit clamping

image

With explicit clamping (default before)

image

Testing

Please run the following unit tests

pytest test/test_transforms_v2.py -vvv -k "keypoints"
...
661 passed, 9057 deselected in 4.16s

@pytorch-bot

pytorch-bot Bot commented Oct 7, 2025

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/9236

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

✅ No Failures

As of commit 859622b with merge base 7a13ad0 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@NicolasHug NicolasHug left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @AntoineSimoulin !

Comment thread test/test_transforms_v2.py Outdated
@AntoineSimoulin
AntoineSimoulin merged commit b851874 into pytorch:main Oct 8, 2025
62 checks passed
@github-actions

github-actions Bot commented Oct 8, 2025

Copy link
Copy Markdown

Hey @AntoineSimoulin!

You merged this PR, but no labels were added.
The list of valid labels is available at https://github.com/pytorch/vision/blob/main/.github/process_commit.py

AntoineSimoulin added a commit to AntoineSimoulin/vision that referenced this pull request Oct 9, 2025
@NicolasHug NicolasHug added the bug label Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants