Repository navigation
Try to get the fast_reader options to give useful errors when they have no effect #5574
Description
Activity
One reason is that by default
fast_readeris set with a value ofTrue. So my take was to only enforce it if explicitly set as a dict, or with'force'. Initially this caused at least half of the entire test suite to fail. I found one major problem was that even withoutguessset,ui.read()would always first callget_readerfor a non-fast version, apparently just to get an instance of the Reader, or save 1 or 2 lines of code just after theif fast_reader_param and format is not None....
Changing that, and checking for thefast_readeroption inget_readeras well for selecting a default Reader finally even allowed the auto-format detection to work with the fast readers!
There were a few more quirks with the guessing, since dictionary-type kwargs are apparently generally passed by reference, getting in particular thefast_readerreset to an empty dict after the first call of thereaderinstance. And I changed thecparserexception type for column no. mismatches, since a test intest_readwas expectingInconsistentTableErrorhere (probably meaning that it had never gotten through to the actual fast reader up to now); that in turn required a whole bunch of the C reader tests to be modified which did expect aCParserErrorin similar cases. Might be simpler to revert that and just havetest_wrong_quotechecking for two different error types; OTOH I think it's preferable to have the slow and fast readers throw consistent errors where possible...
Pushed the updates to the old branch for now.@eteq - looks like #5578 addresses the issues so far, but it does include some nontrivial changes to the guessing logistics, so @plim probably rightly labelled this as experimental. I also think @taldcroft might have a better insight on that part of the code.
This came out of discussions in #5552 (#5552 (comment), #5552 (comment), #5552 (comment)).
The crux of the issue is this: the
fast_readeroptions that get passed intoascii.io.readare silently ignored if the reader is not a "fast" reader. And that's confusing because the options actually affect the output. So it would be much clearer to have the reader raise an exception if the user givesfast_readeroptions but the reader is not a fast reader.Or is there some specific reason why that doesn't happen?
I took a shot at this in https://github.com/eteq/astropy/tree/fortranexp2-fast-error but that approach doesn't seem to have worked (although maybe @dhomeier tried it and had better results?)
cc @taldcroft