Repository navigation
Reset threads after fork #2885
Description
Activity
To be on the safe side of things, the default behavior for
data.tablecould also be set to switch back to single-threaded mode after a fork, but to allow overriding that behavior with an option.Once that proves stable, that logic could be reversed.
Perhaps an additional argument reset_after_fork could also be added to
setDTthreads()to set this behavior after the package has been loaded (like infst::threads_fst()), so:data.table::setDTthreads(threads = 4, reset_after_fork = FALSE)
to set the threads and disable switching back to single-threaded mode at the same time.
thanks for a great package by the way ! :-)
Reacted by Matt Dowle
I understand that using OpenMP multi-threading in a fork of R session may end up in a deadlock as noted in this code. And threads are down limit to 1 in a fork. For safety concerns, the limit continues after fork.
This behavior makes RStudio Server users harder to leverage the multi-threading capabilities data.table has to offer since rstudio/rstudio#1827 notes that a refresh of RStudio server web page could trigger the behavior and since then data.table is limited to be single-threaded.
The recent release of fst package adopts the reset-after-fork behavior unless an option is toggled, as discussed at fstpackage/fst#110, fstpackage/fst#112. I'm not sure if it makes sense that data.table adopts this behavior in future if this is thoroughly tested?