Skip to content

[MRG] FIX fix hinge_loss when labels are provided and not inferred from y_true - #17935

Merged
glemaitre merged 9 commits into
scikit-learn:masterfrom
Ultramann:hinge_loss_ignores_labels
Aug 24, 2020
Merged

glemaitre merged 9 commits into
scikit-learn:masterfrom
Ultramann:hinge_loss_ignores_labels

Conversation

@Ultramann

@Ultramann Ultramann commented Jul 16, 2020 •

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Fixes #17630

What does this implement/fix? Explain your changes.

From issue: Multilabel hinge_loss fails if not all labels are present in y_true despite all the labels being explicitly provided using external labels argument (as defined in the function description).

This causes a slightly misleading error message, and stack trace because the function is in the branch assuming that it's handling the binary target case.

This happens because the labels parameter is not directly considered in the logic for determining if we're in the multiclass case. This branch is made by checking of the variable y_true_unique has more than two values. The fix directly addresses this by including labels in the computation of y_true_unique.

Any other comments?

  • Do I need to write a regression test?

@Ultramann Ultramann changed the title Fix hinge_loss bug ignoring labels argument [WIP] Fix hinge_loss bug ignoring labels argument Jul 16, 2020
@Ultramann
Ultramann marked this pull request as draft July 16, 2020 14:46
If labels is a ndarray then the evaluating it with or doesn't work.
@adrinjalali

Copy link
Copy Markdown
Member

This also needs a test

@Ultramann

Copy link
Copy Markdown
Contributor Author

I agree, I'll write one up tonight.

@Ultramann Ultramann changed the title [WIP] Fix hinge_loss bug ignoring labels argument [MRG] Fix hinge_loss bug ignoring labels argument Jul 17, 2020
@Ultramann
Ultramann marked this pull request as ready for review July 17, 2020 14:43
@Ultramann

Copy link
Copy Markdown
Contributor Author

@adrinjalali could you take a look at this with the added test? Many thanks.

Comment on lines +2127 to +2128
assert (hinge_loss(y_true, pred_decision, labels=labels) ==
dummy_hinge_loss)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when comparing floats, we should almost never compare exact values. You could use numpy.testing.assert_array_almost_equal instead.

Comment thread sklearn/metrics/tests/test_classification.py

@glemaitre glemaitre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add an entry to the change log at doc/whats_new/v*.rst. Like the other entries there, please reference this pull request with :pr: and credit yourself (and other contributors if applicable) with :user:.

Comment thread sklearn/metrics/tests/test_classification.py
@glemaitre glemaitre changed the title [MRG] Fix hinge_loss bug ignoring labels argument [MRG] FIX fix hinge_loss when labels are provided and not inferred from y_true Aug 21, 2020

@adrinjalali adrinjalali left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @Ultramann

@glemaitre
glemaitre merged commit b229a66 into scikit-learn:master Aug 24, 2020
@glemaitre

Copy link
Copy Markdown
Member

Merging then. THanks @Ultramann

@Ultramann
Ultramann deleted the hinge_loss_ignores_labels branch August 24, 2020 13:06
jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Multilabel hinge_loss bug

3 participants