Non-deterministic test failures

I've been seeing a lot of this semi-randomly lately:

__multicall__ = <MultiCall 0 results, 0 meths, kwargs={'pyfuncitem': <Function 'test_read_through_table_interface'>}>
pyfuncitem = <Function 'test_read_through_table_interface'>
    def pytest_pyfunc_call(__multicall__, pyfuncitem):
        if not __multicall__.execute():
            testfunction = pyfuncitem.obj
            if pyfuncitem._isyieldedfunction():
                testfunction(*pyfuncitem._args)
            else:
                funcargs = pyfuncitem.funcargs
                testargs = {}
                for arg in pyfuncitem._fixtureinfo.argnames:
>                   testargs[arg] = funcargs[arg]
E                   KeyError: 'tmpdir'

I think this has to do with the unicode literals testing functionality running the same test twice and "getting lucky" somehow.

I've seen this too - is it related to the parallel option?

I think it's most likely a bad interaction between the unicode literal double-testing and parallel testing, since the same test is run twice and (I think) there's a race condition between when the fixture arguments are setup and when the test is run.

Ditto on having seen this.

Have you seen it since #2060 was merged? I'm hoping that addresses this, but I was never able to reproduce locally, so I can't be certain.

I haven't seen this error since then. There is another error that comes from time to time related to some kind of race condition in importing astropy on the parallel builds, but this is different. I'll close this for now since the particular error mentioned above hasn't come up since.

I am seeing a similar error. It is due to parallel execution:
pyfuncitem = <Function 'test_coxph'>

@hookimpl(trylast=True)
def pytest_pyfunc_call(pyfuncitem):
    testfunction = pyfuncitem.obj
    if pyfuncitem._isyieldedfunction():
        testfunction(*pyfuncitem._args)
    else:
        funcargs = pyfuncitem.funcargs
        testargs = {}
        for arg in pyfuncitem._fixtureinfo.argnames:

          testargs[arg] = funcargs[arg]

E KeyError: 'flchain'

../../mypythonenv/lib/python3.6/site-packages/_pytest/python.py:154: KeyError
It works if I run the same test sequentially

@umasenthil

@umasenthil thanks for the report. What version of Python and astropy are you using?

@drdavella

@drdavella I am using Python 3.6.3. I am not using astropy. Will it help if I upgrade python version?
Thanks

@umasenthil

@umasenthil if this issue is unrelated to astropy, it would probably be best for you to report it directly to pytest at https://github.com/pytest-dev/pytest/.

@drdavella

@drdavella Will do. Thanks for the pointers

Navigation Menu

Repository navigation

Non-deterministic test failures #2065

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Non-deterministic test failures #2065

Description

Activity

astrofrog commented on Feb 9, 2014

mdboom commented on Feb 10, 2014

embray commented on Feb 14, 2014

mdboom commented on Feb 14, 2014

astrofrog commented on Mar 16, 2014

umasenthil commented on Apr 12, 2018

drdavella commented on Apr 12, 2018

umasenthil commented on Apr 12, 2018

drdavella commented on Apr 13, 2018

umasenthil commented on Apr 13, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions