Skip to content

pytest importlib doesn't work with pickle #7859

Description

@yoav-orca

I'm transitioning from pytest regular imports to --import-mode=importlib imports due to namespace collisions in tests. And I'm getting the following crash, no plugins installed, pytest 6.1.1 CPython 3.8.6, the issue reproduces on Mac & Linux.

Create a file named test_xx.py:

import pickle

def do_action():
    pass

def test_x():
    pickle.dumps(do_action)
$ pytest -v --import-mode=importlib test_xx.py
==================================================== test session starts ====================================================
platform darwin -- Python 3.8.5, pytest-6.1.1, py-1.9.0, pluggy-0.13.1 -- /Users/yoav/.local/share/virtualenvs/qwer-HyU7-4uu/bin/python
cachedir: .pytest_cache
rootdir: /private/tmp/qwer
collected 1 item

test_xx.py::test_x FAILED                                                                                             [100%]

========================================================= FAILURES ==========================================================
__________________________________________________________ test_x ___________________________________________________________

    def test_x():
>       pickle.dumps(do_action)
E       _pickle.PicklingError: Can't pickle <function do_action at 0x10f8aa3a0>: import of module 'test_xx' failed

test_xx.py:7: PicklingError
================================================== short test summary info ==================================================
FAILED test_xx.py::test_x - _pickle.PicklingError: Can't pickle <function do_action at 0x10f8aa3a0>: import of module 'tes...
===================================================== 1 failed in 0.05s =====================================================

Running the same code without pytest or with pytest without importlib work as expected.

pip list:

Package    Version
---------- -------
attrs      20.2.0
iniconfig  1.0.1
packaging  20.4
pip        20.1.1
pluggy     0.13.1
py         1.9.0
pyparsing  2.4.7
pytest     6.1.1
setuptools 50.3.0
six        1.15.0
toml       0.10.1
wheel      0.35.1

I encountered this issue because we have tests using multiprocessing.Pool, and they fail on pickling the target function.

Activity

  1. aklajnert commented on Oct 6, 2020

    @aklajnert
    Contributor

    I'd say that it is closely related to #7856. I didn't dig into that yet, but it seems like the problem is with importing, not pickling.

  2. added 3 commits that reference this issue on Oct 7, 2020
    c08f4d1
    ac2cc44
    6dad144
  3. added a commit that references this issue on Dec 19, 2020
    bb14257
  4. added a commit that references this issue on Apr 2, 2021
    1aebd9a
  5. added a commit that references this issue on Apr 5, 2021
    b706a2c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: bugproblem that needs to be addressed

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions