Repository navigation
Latex tables with units don't read properly #14853
Description
Description
A table with units, written to a tex file, doesn't read properly back to a table with units.
Expected behavior
Round trips.
How to Reproduce
Make a file with
\begin{table}
\begin{tabular}{cccccccc}
cosmology & name & $H_0$ & $\Omega_{m,0}$ & $T_{0}$ & $N_{eff}$ & $m_{nu}$ & $\Omega_{b,0}$ \\
& & $\mathrm{km\,Mpc^{-1}\,s^{-1}}$ & & $\mathrm{K}$ & & $\mathrm{eV}$ & \\
FlatLambdaCDM & Planck18 & 67.66 & 0.30966 & 2.7255 & 3.046 & 0.0 .. 0.06 & 0.04897 \\
\end{tabular}
\end{table}
Read file with
QTable.read(<file>)Versions
macOS-10.16-x86_64-i386-64bit
Python 3.10.11 (main, Apr 20 2023, 13:59:00) [Clang 14.0.6 ]
astropy 6.0.dev44+ga0261d10da
Numpy 1.24.3
pyerfa 2.0.0
Scipy 1.10.0
Matplotlib 3.7.1
Activity
[-]Latex tables with units don'[/-][+]Latex tables with units don't read properly[/+]nstarman commentedon May 21, 2023
@taldcroft would you be able to take a look at this? We have a new contributor who had a nice PR for writing Cosmology to LaTeX, and we'd like to complete the round trip, but if Table can't read LaTeX then we're stuck!
nstarman commentedon May 23, 2023
@taldcroft, it looks like the writer adds a units row (as strings), but the reader doesn't parse that row as separate from the rest and consequently thinks every column is a string. Does I/O have a robust means to detect and parse a units row in formats that do not have special metadata, like ECSV?
taldcroft commentedon May 23, 2023
@nstarman - Not in general. Detection and parsing of units is specific to the format. CDS and DAOphot have units. See e.g.
astropy/astropy/io/ascii/daophot.py
Line 176 in 08dba88
nstarman commentedon May 24, 2023
Darn. I suppose it's too format-specific to generalize well. Any chance you're interested in resolving this issue? 🤞😆
taldcroft commentedon May 24, 2023
It seems like writing is if you are writing a paper and want a starting point for a table. But if someone wants to publish a cosmology as a machine-readable table then they really should do so in a reasonable format, e.g. ApJ MRT which supports units. In other words, this wouldn't be high on my priority list. 😄
nstarman commentedon May 25, 2023
Good points all for Cosmology, but stepping back to the more general case of an Astropy Table, it's unfortunate that our LaTeX I/O is not symmetric, that it can write units but not read them back. I think regardless of the specific use case, this is something we should fix.
hamogu commentedon Jun 5, 2023
I'm not opposed to making them round-trip if someone wants to, but I don't see a use case for it. We'll never be able to read all LaTeX tables (just think
\multicol), and if you already have an astropy table, don't save it as LeTeX for storage. I think of the LaTeX tables more likeplt.plot: You want something that represents our data from the table in a paper, but nobody would expect that astropy can round-trip a table to png and back.The reading of LaTeX tables was added to parse specifically ApJ articles on astro-ph, because sometimes that's the only pure-text format that you can get - it's bad, but still much better than parsing a pdf. (A&A is less of a problem, because most tables in A&A that are longer than a few rows get transferred to simbad). In that sense, header data (units) is actually the least important part to me - LaTeX tables rarely have more than a dozen columns and you can fix the units by hand by hand if you have to. Also, there are so many different ways of setting a unit in a table (depending on journal style any or all of the following): After 0 ,1, or 2
\hlinein enclosed in[]or not, in a footnote, as\multicolumnover two columns (data and error), askm/sorkm s$^{-1}$or$\mathrm{km}\;\mathrm{s}^{-1}$,$\mathrm{km}\ \mathrm{s}^{-1}$,$\textnormal{km}\;\textnormal{s}^{-1}$- and that's even before you get into arcmin vs.\arcminvs'etc. Thus, that unit reading almost certainly would only work for tables that are written by Astropy, and not for anything else. That's not a very good use case to me, because the tables written by astropy are exactly those that you do not need to read - they could just be conserved in some other format.I would judge things like those listed in #1525 (e.g. use the
\\as line break instead of the line break in the file) as more important, because that's about that data.Again, if someone has a convincing use case and write a PR, that does not mean that we'll reject it, but in my view it's a waste of time to attempt that.
nstarman commentedon Jun 6, 2023
I definitely agree with many of these points, but for a point-by-point
Agreed on png, especially because it's a lossy format, however I don't think it would be entirely unreasonable to want non-rasterized svg and pdf to be able to round trip as those formats can losslessly encode data.
For building I/O on top of the Table I/O, as we've been doing over at Cosmology, having to fix a table by hand is a bit of a show-stopper.
For sure, this is definitely a problem. Arguably a case for a flexible reader, but that's a task and a half. Even just tables that Astropy writes would be a good place to start.
Yes and no. Tools like ShowYourWork (CI/CD for writing papers) will write a table to LaTeX to be embedded in a paper.
Having only the O in I/O is less useful: for unit-test verification, parsing a person's paper (you know the exact format they used to write b/c script files are included with the tex files), etc.
Point being, there are domains in which LaTeX is the file format you want to conserve data in and want to be able to read.
For sure. Higher priority.
hamogu commentedon Jan 10, 2025
Coming back to this on a review of open
io.asciitickets. I think we agree that there is no way read even a reasonable fraction of tables with units in LaTeX in the wild. There are just too many formats. It would technically be possible to read tabes that have units in exactly the form that astropy writes, to round-trip LaTeX tables. However, personally, I think that would be more confusing for users not less. Why does this table read and not that? Can you add also parsing this spelling of that unit that I need right here?At least "reading units in LaTeX tables is out of scope" is a well-defined line.
The use case presented in
is very, very narrow. If you run Showyourwork, you have all the scripts and input data - you can regenerate the table before writing it out if you want to check that you get the same answer, you don't have to read the LaTeX table. If you write a showyourwork and want the table as an intermediate step that showyourwork backs up to zenodo (because it's expensive to generate), pick a proper storage format and use LaTeX just as an output. If you want to write unit tests that involve your LaTeX table, compare the strong outputs without reading it back in.
I don't think that a general purpose package like astropy should cater to such special cases, when doing so impacts the clarity and simplicity of use for normal users (who will be confused what can and cannot be read).
If we feel that absolute round-tripping ability is important, we can remove the ability to write units - that should work today. However, I also think that's not a good idea: The typical use case is to turn your astropy table into LaTeX for publishing and then you want the units.
So, I suggest we declare "reading units in LaTeX tables" to be out of scope, add a line in the docs that says so, and close this issue.
nstarman commentedon Jan 10, 2025
SGTM. I would modify this slightly to point users to our page on universal IO and registering custom readers. We say it's out of scope for Astropy but we've also built a powerful and flexible system and users are free to do what they need.