pickling and repr of exceptions with kwargs

@gpshead

BPO	27015
Nosy	@gpshead, @ncoghlan, @serhiy-storchaka, @Taywee, @remilapeyre
PRs	bpo-27015: Save kwargs given to exceptions constructor #11580

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2016-05-13.16:59:09.479>
labels = ['type-bug', 'library', '3.9', '3.10', '3.11']
title = 'pickling and repr of exceptions with kwargs'
updated_at = <Date 2022-01-06.13:25:55.215>
user = 'https://github.com/Taywee'

bugs.python.org fields:

activity = <Date 2022-01-06.13:25:55.215>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2016-05-13.16:59:09.479>
creator = 'Taywee'
dependencies = []
files = []
hgrepos = []
issue_num = 27015
keywords = ['patch', 'patch', 'patch']
message_count = 4.0
messages = ['265482', '265486', '333769', '334626']
nosy_count = 6.0
nosy_names = ['gregory.p.smith', 'astrand', 'ncoghlan', 'serhiy.storchaka', 'Taywee', 'remi.lapeyre']
pr_nums = ['11580']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue27015'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

When using kwargs to construct a CalledProcessError, the repr doesn't show those args, and using kwargs also breaks pickling:

>>> import pickle; from subprocess import CalledProcessError
>>> CalledProcessError(2, 'foo')
CalledProcessError(2, 'foo')
>>> CalledProcessError(2, 'foo').returncode
2
>>> CalledProcessError(2, 'foo').cmd
'foo'
>>> CalledProcessError(returncode=2, cmd='foo')
CalledProcessError()
>>> CalledProcessError(returncode=2, cmd='foo').returncode
2
>>> CalledProcessError(returncode=2, cmd='foo').cmd
'foo'
>>> pickle.loads(pickle.dumps(CalledProcessError(2, 'foo')))
CalledProcessError(2, 'foo')
>>> pickle.loads(pickle.dumps(CalledProcessError(returncode=2, cmd='foo')))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() missing 2 required positional arguments: 'returncode' and 'cmd'
>>>

This is a problem not only with CalledProcessError, but with all custom exceptions with overridden __init__. BaseException.__new__ saves positional arguments as the "args" attribute, but ignores keyword arguments. repr() and pickle use "args".

I tried to fix the issue, the attached PR solves the issue of saving the kwargs and unpickling the exception but I was not able to fix a regression I caused in test_memory_error_in_PyErr_PrintEx.

Reviewing Rémi's page made me realise that a big part of the root cause here is pickle support in exceptions predating the introduction of __getnewargs__ and __getnewargs_ex__.

Navigation Menu

Repository navigation

pickling and repr of exceptions with kwargs #71202

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

pickling and repr of exceptions with kwargs #71202

Description

Activity

Taywee commented on May 13, 2016

serhiy-storchaka commented on May 13, 2016

remilapeyre commented on Jan 16, 2019

ncoghlan commented on Jan 31, 2019

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions