Repository navigation
read_csvy doesn't read the primary example on csvy.org #13
Description
Not sure if the schema is canonical and has changed, but read_csvy assumes there's a headline field fields, which is now nested in resources[[1]]$schema$fields.
C/P of file currently on the front page of csvy.org
---
name: my-dataset
resources:
- order: 1
schema:
fields:
- name: var1
type: string
- name: var2
type: integer
- name: var3
type: number
dialect:
csvddfVersion: 1.0
delimiter: ","
doubleQuote: false
lineTerminator: "\r\n"
quoteChar: "\""
skipInitialSpace: true
header: true
---
var1,var2,var3
A,1,2.0
B,3,4.3
Current output of read_csvy (CRAN):
read_csvy('~/data.table/inst/tests/test.csvy')
# var1 var2 var3
# 1 A 1 2.0
# 2 B 3 4.3
Warning message:
In check_metadata(y, out) :
Metadata is missing for variables listed in data: var1, var2, var3
Activity
a bunch of changes (#10, #13)
leeper commentedon Mar 4, 2018
The spec changed. I've updated this. Thanks for reporting it!
MichaelChirico commentedon Mar 4, 2018
@leeper thanks. could you point me to the spec? I tried finding it but ended up in a maze of circular links...
leeper commentedon Mar 4, 2018
File level: https://frictionlessdata.io/specs/tabular-data-package/
Dataset and variable level: https://frictionlessdata.io/specs/table-schema/
jrovegno commentedon Mar 5, 2018
@leeper Please do not update the package with the new specs. I am not clear if for the community of users of csvy, it is worthwhile that in the format several dataset can be implemented in a single file.
For simplicity originally csvy was thought to contain only one dataset with its metada.
Then it occurred to me that we could be "CSVY is a simple container of a Tabular Data Package", but that may make the format even more incompatible with the existing CSV.
I would like to know your opinion about it.
leeper commentedon Mar 5, 2018
@jrovegno I don't see a problem with following the spec except that I don't think I would ever encourage creating a single file with multiple tables in it. So, I'd say the new spec looks good except I would limit it to one table per file. As such, I think you could use the spec as currently stated on the webpage AND a simplified version that just contains the
dialectelement and thefieldssub-element ofresources.jrovegno commentedon Mar 5, 2018
@leeper Great idea, can you make a PR with the changes in the specs of CSVY?
MichaelChirico commentedon Mar 6, 2018
Also FYI that we're trying to improve
fread(and later,fwrite) to handlecsvythrough theyamlargument; progress here:Rdatatable/data.table#2656