Repository navigation
Error in data.table::set(): not a resizable vector #7583
Description
Activity
we had tracked the {irace} issue in #7501, thought it was solved. will check again. cc @ben-schwen who might remember more details.
- We'll need to investigate this ourselves, but judging by the reports, I suspect that Prof. Ripley's system may be running an earlier submission of version 1.18.0 that we had to redo due to the the many reverse dependency failures that we had initially missed. In particular, we should have fixed this in #7501.
Thank you! I can wait. He give me until 31 of January.
If you want me to change the code in the package, please let me know. I can also submit a new version to force a rebuild in CRAN. Unfortunately, I don't have access to this M1mac target for testing.
I cannot reproduce the error on my Ubuntu system.
The shown error is probably due to serialization. You probably serialize and deserialize
self$experiment_logat some point. You can check this either viatruelength(self$experiment_log)` # will show 0 .Internal(inspect(self$experiment_log)) # which shows also the truelength as tl=0 data.table:::selfrefok(self$experiment_log) # which will show -1
All three of them should work. As pointed out by Ivan an intermediate submission of
1.18.0did not catch this and tried to useseton such malformed data.table.However, the current version makees the selfrefok check and otherwise calls
setalloccol(self$experiment_log)which fixes this.Hope that helps!
- There's also two Mac-specific failures unrelated to data.table. It seems that your code either doesn't normalizePath() enough or does that too much:[FAILED] 31: path_rel2abs("..//a", ".//") -> /private/a but expected: /a [FAILED] 1: path_rel2abs("c/b", ".") -> /private/var/folders/pk/n4bndnt1287ctrd_ftthnnnr0000gp/T/RtmpQsOiNh/working_dir/RtmpSWvJc5/filee8228499dae/c/b but expected: /var/folders/pk/n4bndnt1287ctrd_ftthnnnr0000gp/T/RtmpQsOiNh/working_dir/RtmpSWvJc5/filee8228499dae/c/bOn macOS, /tmp is symlinked to /private/tmp and, similarly, the path to the session temporary directory is absolute but not canonical (it's not /var/folders/whatever, but /private/var/folders/whatever). I had hit a similar problem myself (also on M1mac) when I fed possibly non-existent paths under the session temporary directory to basename() and dirname(). The solution was to treat the potentially non-existent paths as strings and don't normalize them at all.
If I have a serialised a data.table with a previous version, do I need to regenerate and resave with 1.18.0?
If I have a serialised a data.table with a previous version, do I need to regenerate and resave with 1.18.0?
We should have fixed this by checking if the truelength and selfref are ok before resizing (both in adding as well as deleting columns). However, there might be some strange corner case where a data.table with a new R version > 4.5.0 and data.table > 1.18.0 is serialized and cannot be unserialized without
setalloccolon an older data.table or R version (we need to check these!).If I have a serialised a data.table with a previous version, do I need to regenerate and resave with 1.18.0?
Ideally, any code that unserializes
data.tables should then callsetDT()on them. This should be enough for anydata.table, whether originally created by 1.18.0 or an earlier version. Saving with 1.18.0 shouldn't be necessary. An older/newersetDT()will re-create the list of column pointers anyway; the newer function will make the list growable, and the older one will instead install a finalizer.The automatic fixes in
setand the[method only replace thedata.tablein the environment of the immediate caller ofset(or the[method). Any references to thatdata.tableabove in the function call stack stay invalid, and some or all by-reference changes may not propagate.Reacted by Jan GoreckiIt seems the error has disappeared in CRAN: https://cran.r-project.org/web/checks/check_results_irace.html
I'll contact Prof Ripley to make sure everything is OK.
- I'll contact Prof Ripley to make sure everything is OK.He's very particular about following the e-mail etiquette. You should probably send your message to [email protected] if you want to contact him. I highly recommend to double-check the code and tests that work with file paths. Try setting $TMPDIR to a path to a symbolic link and you might be able to find a bug in your tests. That one failure on M1mac was not completely spurious.
I highly recommend to double-check the code and tests that work with
file paths. Try setting $TMPDIR to a path to a symbolic link and you
might be able to find a bug in your tests. That one failure on M1mac
was not completely spurious.Those are not failures in CRAN. They are printed as [FAILED] for me to look at in the future when I find time but they don't produce any errors/warnings/notes in CRAN.
I'll hold sending any emails anyway since the failures have disappeared. Thanks for the advice!
So I guess we can close this here.
Please feel free to reopen if the issue resurfaces.
I am the maintainer of irace. CRAN has notified me of an error that started with data.table 1.18.0. See https://www.stats.ox.ac.uk/pub/bdr/M1mac/irace.out You can see one of the errors below.
This seems M1mac-specific. The same tests work in other targets.
I have no clue how to fix this problem. The error may be on my usage of data.table. In any case, any hints on how to fix or work-around the error would be welcome.
Thank you!