Skip to content

Add self-check to pytests - #5087

Merged
msullivan merged 3 commits into
python:masterfrom
elazarg:pytest-selfcheck
May 25, 2018
Merged

Add self-check to pytests#5087
msullivan merged 3 commits into
python:masterfrom
elazarg:pytest-selfcheck

Conversation

@elazarg

@elazarg elazarg commented May 20, 2018

Copy link
Copy Markdown
Contributor

Yet another item for #1673.

This PR uses the api for the self-check.

Example of a failed-run output:

~/workspace/mypy$ pytest -n0 -k test_mypy_package
================================ test session starts ================================
platform darwin -- Python 3.6.5, pytest-3.2.5, py-1.5.2, pluggy-0.4.0
rootdir: ~/workspace/mypy, inifile: pytest.ini
plugins: xdist-1.20.1, forked-0.2, cov-2.5.1
collected 6240 items                                                                 

mypy/test/testselfcheck.py F

===================================== FAILURES ======================================
_________________________ SelfCheckSuite.test_mypy_package __________________________
Self check failed
------------------------------- Captured stdout call --------------------------------
mypy/binder.py:254: error: No return value expected
=============================== 6239 tests deselected ===============================
==================== 1 failed, 6239 deselected in 23.75 seconds =====================

$ ./runtests.py self-check results in a similar output, though slightly more verbose, as usual.

@elazarg
elazarg force-pushed the pytest-selfcheck branch from b914a4d to 5d82cb3 Compare May 20, 2018 21:50
@elazarg elazarg mentioned this pull request May 20, 2018
Merged
@elazarg

elazarg commented May 20, 2018

Copy link
Copy Markdown
Contributor Author

3.4 fails the type check with this error:

mypy/test/data.py:232: error: "TemporaryDirectory" expects no type arguments, but 1 given

I have no idea why did it pass before. (See short discussion here)

@JelleZijlstra

JelleZijlstra commented May 20, 2018

Copy link
Copy Markdown
Member

Maybe we just didn't run a self-check with 3.4 before? That error is consistent with the stubs (https://github.com/python/typeshed/blob/master/stdlib/3/tempfile.pyi): TemporaryDirectory is generic only in 3.5+.

@ilevkivskyi

ilevkivskyi commented May 21, 2018

Copy link
Copy Markdown
Member

I think self check is only run on 3.6. Btw @elazarg I added a new label topic-tests for test related issues.

@elazarg

elazarg commented May 23, 2018 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

I see. I've changed the name of the self testing from package to self-check so the -x flag doesn't work. Otherwise the checks pass.

emmatyping
emmatyping requested changes May 24, 2018
View reviewed changes

@emmatyping emmatyping left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure to update the appveyor/travis configs to replace package with self-check if we are going ahead with that.

@elazarg

elazarg commented May 25, 2018

Copy link
Copy Markdown
Contributor Author

Where's the appveyor check? I've only touched .travis.yml.

@emmatyping

emmatyping commented May 25, 2018

Copy link
Copy Markdown
Member

@elazarg it is excluded on this line https://github.com/python/mypy/blob/master/appveyor.yml#L24

@elazarg

elazarg commented May 25, 2018

Copy link
Copy Markdown
Contributor Author

@ethanhs is it OK now?

emmatyping
emmatyping approved these changes May 25, 2018
View reviewed changes

@emmatyping emmatyping left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI configs look good!

@msullivan
msullivan merged commit f639e1b into python:master May 25, 2018
@elazarg elazarg mentioned this pull request Jun 4, 2018
Merged
gvanrossum pushed a commit that referenced this pull request Jun 23, 2018
@elazarg @gvanrossum
move test_stubs and test_samples to pytest (#5142)
88a6a4e 
Using the api, similarly to #5087.

stdlibsamples is not included in this PR since it requires special handling of working directory and paths.

This is another item for #1673.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@emmatyping emmatyping emmatyping approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@elazarg @JelleZijlstra @ilevkivskyi @emmatyping @msullivan