Repository navigation
simple callback system for Py_FatalError #41948
Description
Activity
mutkuk commented
on May 4, 2005 mutkukmannequinMannequinAuthorIncluded implementation of a simple callback system for
Py_FatalError.Everything is done in "pythonrun.c" and "pydebug.h"
jwpye commented
on May 5, 2005 jwpyemannequinMannequinLogged In: YES
user_id=1044177Again, I think this should be explicitly exclusive to an
embedding application, as I cannot imagine a situation where
an extension module should even have access to this facility.I have done rather limited tests. My application doesn't
seem to like Python 2.5alpha, so I'm going to test the patch
against 2.4.Sorry to butt into your patch, mutkuk, but here's my
recommendation with docs(ugh, cant attach files):Index: Doc/api/init.tex
===================================================================
RCS file: /cvsroot/python/python/dist/src/Doc/api/init.tex,v
retrieving revision 1.23
diff -r1.23 init.tex
39a40,53\begin{cfuncdesc}{void}{Py_InitializeFatalHook}{PyFatalHook fp}
Force \cfunction{Py_FatalError()} to invoke the given
function
instead of printing to standard error and aborting out
of the
process. This is intended for use in embedded
applications that
want to define their own route for handling fatal
errors. Giving
the application the ability to log the error to a special
destination, and to do any appropriate cleanups before
exiting.If used, this \emph{must} be called prior to
\cfunction{Py_Initialize()}. Otherwise,
\cfunction{Py_Initialize()}
will set the hook to the default, and future attempts to
\cfunction{Py_InitializeFatalHook()} will be ignored.
\end{cfuncdesc}Index: Include/pythonrun.h
===================================================================
RCS file: /cvsroot/python/python/dist/src/Include/pythonrun.h,v
retrieving revision 2.65
diff -r2.65 pythonrun.h
24a25,27typedef void (*PyFatalHook)(const char *);
PyAPI_FUNC(void) Py_InitializeFatalHook(PyFatalHook);Index: Python/pythonrun.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/pythonrun.c,v
retrieving revision 2.213
diff -r2.213 pythonrun.c
99a100,109static PyFatalHook _fatalhook = NULL;
void
Py_InitializeFatalHook(PyFatalHook f)
{
/* Only set it if it is uninitialized. */
if (_fatalhook == NULL)
_fatalhook = f;
}150a161,162
Py_InitializeFatalHook(Py_FatalError);
1507a1520,1523
if (_fatalhook != Py_FatalError && _fatalhook != NULL)
_fatalhook(msg);
/* If the hook fell through, finish the process anyways. */mutkuk commented
on May 5, 2005 mutkukmannequinMannequinAuthorLogged In: YES
user_id=1272056Extension case:
Extension developers may mess up too, why leave them orphaned?Doc case:
UGGGHHHYour patch is just slicker, why dont I delete this patch and
you submit. What do you say?I have just 2 suggestions:
-
Declaration of register func would better be in pydebug.h
near Py_FatalError. -
Last one to call register func owns the hook, this is
just expected case so IMHO remove NULL check from register func.
-
jwpye commented
on May 5, 2005 jwpyemannequinMannequinLogged In: YES
user_id=1044177Extension case:
Really, if an extension requires special cleanup, chances
are the application will take notice of it and probably
handle that cleanup itself. If extension developers thought
that it was okay to overload the hook, then they might make
the mistake of overloading the hook in an embedded
application where fatal cleanup was a necessity. If a stack
of hooks were provided, who or what determines which hook
should cause the exit? I don't actually have control over
what PostgreSQL does with its FATAL ereport, so if it hit my
hook first, the application will go bye bye, without
touching the others. Also, it might be possible for
extensions to tap into some signal to allow cleanup. For
instance abort(3) sends the process a SIGABRT. All in all, I
still think it's bad mojo.1: I see your reasoning, but I disagree as when the
developer pokes around the headers it will be more useful
for it, the typedef, to be around the *only* call that the
typedef is used for. Thus showing the complete expectation
for the function's argument, without having to find it in
another file. Matter of fact, I think the typedef isn't
really all that useful except to declare the static function
pointer in pythonrun.c, so I'd rather do without it... =\- Oh, yeah the NULL check is there just in case someone
gets smart and tries to start running Python calls before
Python is initialized(ala Py_InitializeEx()). Besides that
it, Py_FatalError should only be called once during the
process lifetime, so the extra condition isn't really much
of a performance issue.
I'll e-mail you the patch.
- Oh, yeah the NULL check is there just in case someone
mutkuk commented
on May 5, 2005 mutkukmannequinMannequinAuthorLogged In: YES
user_id=1272056Uploaded new patch unchanged.
Extension case:
This func categorizes with PyThreadSatate_*. For debugger
implementors(generally implmented as extension) it is
invaluable. Indeed that's the reason I needed it in the
first place.1: I see your reasoning, but I disagree as when the
developer pokes around the headers it will be more useful
for it, the typedef, to be around the *only* call that the....Not worth to remove it IMHO just feels OK
- Oh, yeah the NULL check is there just in case someone
gets smart and tries to start running Python calls before
Python is initialized(ala Py_InitializeEx()). Besides that
it, Py_FatalError should only be called once during the
process lifetime, so the extra condition isn't really much
of a performance issue.
Got it, agreed.
- Oh, yeah the NULL check is there just in case someone
- changed the title
[-]simple callback system for Py_FatalError [/-][+]simple callback system for Py_FatalError[/+]on Feb 16, 2009 - changed the title
[-]simple callback system for Py_FatalError [/-][+]simple callback system for Py_FatalError[/+]on Feb 16, 2009 jwp commented
on Feb 16, 2009 jwpmannequinMannequinI had actually forgotten that this was still open. Anything I can do to
help speed this along?In case nobody remembers, the purpose of this was to provide a facility
to embedded applications that allows for a more graceful shutdown in
fatal error situations. My original use-case was for a PostgreSQL PL
extension where an abort() is quite undesirable as it would take down
the entire cluster(yeah, all the postmaster processes).IIRC, the only thoughts "against" it were rather for trying to remove
FatalErrors entirely.hepl?
Here is a refreshed patch against trunk.
Since the previous one did not provide any context, I hope I copied the
lines in the right locations.Amaury, any interest in getting this committed for 3.2?
jwp commented
on Aug 2, 2010 jwpmannequinMannequinWould it be possible to require the embedding application to define the Py_FatalError symbol?
Admittedly, it would be nice to not have the callback installation code. =\
4 remaining items
fatalhook-2.patch: I don't understand the documentation. It says "Cause :cfunc:`Py_FatalError` to invoke the given function instead of printing to standard error and aborting out of the process.", but if the callback does nothing, the message+traceback is printed.
If the "fatal hook" is supposed to replace the function displaying the message, you should move the code displaying message+traceback in a subfunction and use it as the default hook function.
Pseudo code:
---------------static fatal_error(const char *msg) { printf("Fatal error: %s\n", msg); ... print traceback ... }
static PyFatalHook fatalhook_func = fatal_error;
void Py_FatalError(const char *msg) { if (fatalhook_func != NULL) fatalhook_func(msg); ... windows debugger code ... abort(); }
NULL is a special hook value: don't print message+traceback, but exit (call abort()).
The hook can exit the process using something else than abort(). But if the hook doesn't exit, the default exit code is executed (call abort()), but the "Fatal error..."+traceback is not printed.
fatalhook_func != Py_FatalError
I think that this test is just overkill, or it should be moved to Py_SetFatalHook (e.g. set the hook to NULL if Py_FatalError is passed).
I also made Py_SetFatalHook() return the previous hook;
it could be useful to set a function temporarily,
even if this is not thread safe.The previous hook can also be used to chain hooks. For example, if you would like to display the traceback but also do a special thing before exit, you can do something like:
------------------void init() { previous = Py_SetFatalHook(my_hook) } void my_hook(const char *msg) { ... cleanup ... previous(msg); }
About thread safety: because Py_FatalError() is called in the worst case (when something really bad happens), I think that it is better to not use something related to thread to avoid issues like deadlocks, and keep the code simple.
Sorry, the documentation in the patch is wrong. It should be: "Cause :cfunc:`Py_FatalError` to invoke the given function before printing to standard error and aborting out of the process."
I don't think it's worth making it more complex. If the application does not want the default behavior (which is already quite minimal anyway), it can always install a hook that never returns.
Sorry, the documentation in the patch is wrong
Can you update your patch please?
jonathanmcdougall commented
on Dec 22, 2012 jonathanmcdougallmannequinMannequinThe latest patch does not allow changing the default behaviour of
aborting the process, which is for me a problem. I am both embedding
and extending python using Boost.Python, but scripting is optional. In
case python fails to initialize, I want to disable scripting but keep
the program running.Libraries shouldn't abort; they should report errors to the caller and
let it figure out what to do.jonathanmcdougall commented
on Dec 23, 2012 jonathanmcdougallmannequinMannequinWhile trying to come up with a patch, I'm starting to realize the number
of modifications needed to make this work. Py_FatalError() is called all
over the place assuming that it never returns. The scope of this feature
is enormous.I'm wondering what would happen if I threw an exception from there :)
JamesPye commented
on Apr 24, 2013 JamesPyemannequinMannequinThinking about this again.. perhaps a better approach would be to force the embedder to define the symbol in their binary.
That is, libpython.x doesn't define Py_FatalError. The binary that links in libpython defines it.
(and a "me too" on Jonathan's comments.. for pg-python, abort() is not desirable.)
i obviously didn't add this to 3.3, unassigning to reflect the attention it (isn't) currently getting. sorry!
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields: