FEA Add staged_predict to HistGradientBoosting by haochunchang · Pull Request #16985 · scikit-learn/scikit-learn

haochunchang · 2020-04-21T15:53:19Z

Reference Issues/PRs

Try Fixes #16063

What does this implement/fix? Explain your changes.

Add staged_predict by emulating BaseGradientBoosting. Including private functions.

Add associated tests
Update documentation (Currently, it was copied from BaseGradientBoosting.)
Also add to HistGradientBoostingClassifier

Any other comments?

This is my first PR for a feature request.
I would be very appreciated for any advices or collaborations.

NicolasHug · 2020-04-24T15:34:41Z

thanks for the PR @haochunchang , please ping me when you are ready to get reviews or if you need any help

haochunchang · 2020-04-25T10:54:14Z

I have also checked with an example: test_staged_predict_example.txt (adopted from here).
Although I did not convert it into a jupyter notebook with binder, the plot seems reasonable.

In short, I moved part of the original _raw_predict to _predict_stages.
And staged_predict calls _staged_raw_predict that uses _predict_stages by supplying part of predictors_.

I wasn't sure whether the change is what staged_predict want, but it seems it can incrementally predict the target.

I would be very appreciated for @NicolasHug or anyone who reviews this code and gives any directions for checking :)

NicolasHug

Thanks @haochunchang , it looks in good shape!

I made a few comments, mostly on the style.

Do you also want to add support for the classifier?

NicolasHug · 2020-04-25T14:26:02Z

About the tests, should we add separated tests like

hmm, use your best judgement. It'd be fine eitherway. Also note that we don't need the first one if you do the second

haochunchang · 2020-04-25T16:08:39Z

For simplicity, I added only one test for testing staged_predict.
It tests both Classifier and Regressor by parametrize:

Predictions at each stage equals newly-trained predictions (also test max_iter = 1).
The number of iterations in staged_predict is equal to gb.n_iter_.

haochunchang · 2020-05-08T13:56:04Z

I would be very appreciated for @NicolasHug or anyone who reviews this code and gives any directions for checking :)

NicolasHug

thanks for your patience @haochunchang , we've been busy with the release

This mostly looks good, I'd just suggest to also add staged_predict_proba and staged_decision_function which should be pretty easy now that you have it all laid out.

haochunchang · 2020-05-11T06:05:17Z

Good effort for the release! @NicolasHug
I have also added staged_predict_proba and staged_decision_function and applied your valuable suggestions.
Thanks for your time :)

NicolasHug

thanks @haochunchang , almost there

NicolasHug

thanks @haochunchang

NicolasHug · 2020-05-12T13:02:01Z

maybe @amueller @thomasjpfan can review this one?

NicolasHug · 2020-05-17T12:06:18Z

Thanks @haochunchang !

Sorry we forgot to mention earlier: we'll need a what's new entry:

Please add an entry to the change log at doc/whats_new/v024.rst. Like the other entries there, please reference this pull request with :issue: and credit yourself with :user:. Also since it's new you'll need to create a :mod:sklearn.ensemble section.

You can take inspiration from the 0.23 what'snew if you need

Co-authored-by: Nicolas Hug <[email protected]>

NicolasHug · 2020-05-17T14:04:38Z

Thanks for the work and for your patience @haochunchang !

Just a side note for you next contributions: you don't need to force-push changes, we're always going to squash and merge anyway. When you force push it makes things a little bit harder for us if we have already checked-out your branch ;)

haochunchang · 2020-05-17T14:09:12Z

Got it!
Thanks @NicolasHug and @thomasjpfan for the review and all the suggestions :)

github-actions Bot added the module:ensemble label Apr 21, 2020

haochunchang changed the title ~~[WIP]Add staged_predict() function to HistGradientBoostingRegressor~~ [MRG]Add staged_predict() function to HistGradientBoostingRegressor Apr 25, 2020

NicolasHug reviewed Apr 25, 2020

View reviewed changes

haochunchang changed the title ~~[MRG]Add staged_predict() function to HistGradientBoostingRegressor~~ [WIP]Add staged_predict() function to HistGradientBoostingRegressor Apr 25, 2020

haochunchang force-pushed the add_staged_predict branch from 11c3232 to b921b3a Compare April 25, 2020 15:22

haochunchang changed the title ~~[WIP]Add staged_predict() function to HistGradientBoostingRegressor~~ [WIP]Add staged_predict() function to HistGradientBoosting Apr 25, 2020

haochunchang force-pushed the add_staged_predict branch from b921b3a to 05cd4fc Compare April 25, 2020 15:45

haochunchang changed the title ~~[WIP]Add staged_predict() function to HistGradientBoosting~~ [MRG]Add staged_predict() function to HistGradientBoosting Apr 25, 2020

haochunchang requested a review from NicolasHug April 27, 2020 03:39

NicolasHug reviewed May 8, 2020

View reviewed changes

haochunchang changed the title ~~[MRG]Add staged_predict() function to HistGradientBoosting~~ [WIP]Add staged_predict() function to HistGradientBoosting May 10, 2020

haochunchang force-pushed the add_staged_predict branch 3 times, most recently from 2bfe890 to 700ba6e Compare May 11, 2020 01:24

haochunchang changed the title ~~[WIP]Add staged_predict() function to HistGradientBoosting~~ [MRG]Add staged_predict() function to HistGradientBoosting May 11, 2020

NicolasHug approved these changes May 11, 2020

View reviewed changes

NicolasHug reviewed May 11, 2020

View reviewed changes

Comment thread sklearn/ensemble/_hist_gradient_boosting/tests/test_gradient_boosting.py Outdated

haochunchang force-pushed the add_staged_predict branch from 700ba6e to a20a7c8 Compare May 11, 2020 15:18

NicolasHug reviewed May 11, 2020

View reviewed changes

Comment thread sklearn/ensemble/_hist_gradient_boosting/tests/test_gradient_boosting.py Outdated

Comment thread sklearn/ensemble/_hist_gradient_boosting/tests/test_gradient_boosting.py Outdated

haochunchang force-pushed the add_staged_predict branch from a20a7c8 to ae55f9b Compare May 11, 2020 23:39

NicolasHug reviewed May 12, 2020

View reviewed changes

haochunchang force-pushed the add_staged_predict branch from ae55f9b to 3bddc63 Compare May 12, 2020 00:14

NicolasHug approved these changes May 12, 2020

View reviewed changes

thomasjpfan reviewed May 17, 2020

View reviewed changes

Comment thread sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py Outdated

Comment thread sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py Outdated

Comment thread sklearn/ensemble/_hist_gradient_boosting/tests/test_gradient_boosting.py Outdated

haochunchang force-pushed the add_staged_predict branch from 3bddc63 to b76daa3 Compare May 17, 2020 02:10

thomasjpfan approved these changes May 17, 2020

View reviewed changes

Comment thread sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py Outdated

Comment thread sklearn/ensemble/_hist_gradient_boosting/gradient_boosting.py Outdated

haochunchang force-pushed the add_staged_predict branch from b76daa3 to 9d66d76 Compare May 17, 2020 04:44

ENH Add staged_predict to HistGradientBoosting (scikit-learn#16985)

e9931e5

Co-authored-by: Nicolas Hug <[email protected]>

haochunchang force-pushed the add_staged_predict branch from 9d66d76 to e9931e5 Compare May 17, 2020 12:48

NicolasHug reviewed May 17, 2020

View reviewed changes

Comment thread doc/whats_new/v0.24.rst Outdated

NicolasHug added 2 commits May 17, 2020 09:04

Update doc/whats_new/v0.24.rst

347f57f

hopefully fixed doc issue

d82bb9b

NicolasHug changed the title ~~[MRG]Add staged_predict() function to HistGradientBoosting~~ FEA Add staged_predict to HistGradientBoosting May 17, 2020

NicolasHug merged commit f58c2bc into scikit-learn:master May 17, 2020

haochunchang deleted the add_staged_predict branch May 17, 2020 14:56

viclafargue pushed a commit to viclafargue/scikit-learn that referenced this pull request Jun 26, 2020

FEA Add staged_predict to HistGradientBoosting (scikit-learn#16985)

82815ab

jayzed82 pushed a commit to jayzed82/scikit-learn that referenced this pull request Oct 22, 2020

FEA Add staged_predict to HistGradientBoosting (scikit-learn#16985)

66fd3a8

Conversation

haochunchang commented Apr 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

Uh oh!

NicolasHug commented Apr 24, 2020

Uh oh!

haochunchang commented Apr 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug commented Apr 25, 2020

Uh oh!

haochunchang commented Apr 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

haochunchang commented May 8, 2020

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

haochunchang commented May 11, 2020

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

NicolasHug commented May 12, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

NicolasHug commented May 17, 2020

Uh oh!

Uh oh!

NicolasHug commented May 17, 2020

Uh oh!

haochunchang commented May 17, 2020

haochunchang commented Apr 21, 2020 •

edited

Loading

haochunchang commented Apr 25, 2020 •

edited

Loading

haochunchang commented Apr 25, 2020 •

edited

Loading