Skip to content

revdeps fail after setDT shallow copy earlier  #6575

Description

@tdhock

revdep check machine https://rcdata.nau.edu/genomic-ml/data.table-revdeps/analyze/2024-10-14/ says #6551 breaks several revdeps:

https://rcdata.nau.edu/genomic-ml/data.table-revdeps/analyze/2024-10-14/DTwrappers.txt

* checking tests ...
  Running 'testthat.R'
 ERROR
Running the tests in 'tests/testthat.R' failed.
Complete output:
  > library(testthat)
  > library(DTwrappers)
  > 
  > test_check("DTwrappers")
  [ FAIL 4 | WARN 3 | SKIP 0 | PASS 0 ]
  
  == Failed tests ================================================================
  -- Failure ('test-dtwrappers.R:15:3'): select first 3 rows of dataset ----------
  `firstthree_dtwrappers` not equal to `chosefirstthree`.
  Attributes: < Names: 2 string mismatches >
  Attributes: < Length mismatch: comparison on first 2 components >
  Attributes: < Component 1: Modes: character, externalptr >
  Attributes: < Component 1: target is character, current is externalptr >
  Attributes: < Component 2: Modes: numeric, character >
  Attributes: < Component 2: Lengths: 3, 2 >
  Attributes: < Component 2: target is numeric, current is character >
  -- Failure ('test-dtwrappers.R:25:3'): select first 5 entries of dataset with all columns --
  `chose_first_5_allcolumns` not equal to `chose_first5_dtwrappers`.
  target is data.table, current is data.frame
  -- Failure ('test-dtwrappers.R:34:3'): count rows in dataset -------------------
  `countrows` not equal to `countrows_dtwrappers`.
  Types not compatible: integer is not list
  -- Error ('test-dtwrappers.R:41:1'): (code run outside of `test_that()`) -------
  Error in `.(variable = "Sepal.Length", mean = mean(Sepal.Length))`: could not find function "."
  Backtrace:
      x
   1. +-DTwrappers::dt.calculate(...) at test-dtwrappers.R:41:1
   2. | \-DTwrappers:::eval.dt.statement(...)
   3. |   \-base::eval(...)
   4. |     \-base::eval(...)
   5. +-iris[, .(variable = "Sepal.Length", mean = mean(Sepal.Length))]
   6. \-base::`[.data.frame`(iris, , .(variable = "Sepal.Length", mean = mean(Sepal.Length)))
  
  [ FAIL 4 | WARN 3 | SKIP 0 | PASS 0 ]

https://rcdata.nau.edu/genomic-ml/data.table-revdeps/analyze/2024-10-14/DTwrappers2.txt

* checking re-building of vignette outputs ... ERROR
Error(s) in re-building vignettes:
  ...
--- re-building 'Introduction_to_DTwrappers2.Rmd' using knitr

Quitting from lines 81-82 [unnamed-chunk-4] (Introduction_to_DTwrappers2.Rmd)
Error: processing vignette 'Introduction_to_DTwrappers2.Rmd' failed with diagnostics:
could not find function "."
--- failed re-building 'Introduction_to_DTwrappers2.Rmd'

SUMMARY: processing the following file failed:
  'Introduction_to_DTwrappers2.Rmd'

Error: Vignette re-building failed.
Execution halted

* checking PDF version of manual ... OK
* DONE

Status: 1 ERROR

https://rcdata.nau.edu/genomic-ml/data.table-revdeps/analyze/2024-10-14/marginaleffects.txt

* checking examples ... ERROR
Running examples in 'marginaleffects-Ex.R' failed
The error most likely occurred in:

> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
> ### Name: comparisons
> ### Title: Comparisons Between Predictions Made With Different Regressor
> ###   Values
> ### Aliases: comparisons avg_comparisons
> 
> ### ** Examples
> 
> ## Don't show: 
> if (interactive() || isTRUE(Sys.getenv("R_DOC_BUILD") == "true")) (if (getRversion() >= "3.4") withAutoprint else force)({ # examplesIf
+ ## End(Don't show)
+ ## Don't show: 
+ }) # examplesIf
> ## End(Don't show)
> library(marginaleffects)
> 
> # Linear model
> tmp <- mtcars
> tmp$am <- as.logical(tmp$am)
> mod <- lm(mpg ~ am + factor(cyl), tmp)
> avg_comparisons(mod, variables = list(cyl = "reference"))
Warning: The `cyl` variable is treated as a categorical (factor) variable, but
  the original data is of class numeric. It is safer and faster to convert
  such variables to factor before fitting the model and calling a
  `marginaleffects` function.
  
  This warning appears once per session.
  FALSE

          Contrast Estimate Std. Error     z Pr(>|z|)    S  2.5 % 97.5 %
 mean(6) - mean(4)    -6.16       1.54 -4.01   <0.001 14.0  -9.17  -3.15
 mean(8) - mean(4)   -10.07       1.45 -6.93   <0.001 37.8 -12.91  -7.22

Term: cyl
Type:  response 
Columns: term, contrast, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 

> avg_comparisons(mod, variables = list(cyl = "sequential"))

          Contrast Estimate Std. Error     z Pr(>|z|)    S 2.5 % 97.5 %
 mean(6) - mean(4)    -6.16       1.54 -4.01  < 0.001 14.0 -9.17  -3.15
 mean(8) - mean(6)    -3.91       1.47 -2.66  0.00781  7.0 -6.79  -1.03

Term: cyl
Type:  response 
Columns: term, contrast, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 

> avg_comparisons(mod, variables = list(cyl = "pairwise"))

          Contrast Estimate Std. Error     z Pr(>|z|)    S  2.5 % 97.5 %
 mean(6) - mean(4)    -6.16       1.54 -4.01  < 0.001 14.0  -9.17  -3.15
 mean(8) - mean(4)   -10.07       1.45 -6.93  < 0.001 37.8 -12.91  -7.22
 mean(8) - mean(6)    -3.91       1.47 -2.66  0.00781  7.0  -6.79  -1.03

Term: cyl
Type:  response 
Columns: term, contrast, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 

> 
> # GLM with different scale types
> mod <- glm(am ~ factor(gear), data = mtcars)
> avg_comparisons(mod, type = "response")

          Contrast Estimate Std. Error    z Pr(>|z|)    S 2.5 % 97.5 %
 mean(4) - mean(3)    0.667      0.117 5.68   <0.001 26.1 0.436  0.897
 mean(5) - mean(3)    1.000      0.157 6.39   <0.001 32.5 0.693  1.307

Term: gear
Type:  response 
Columns: term, contrast, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 

> avg_comparisons(mod, type = "link")

          Contrast Estimate Std. Error    z Pr(>|z|)    S 2.5 % 97.5 %
 mean(4) - mean(3)    0.667      0.117 5.68   <0.001 26.1 0.436  0.897
 mean(5) - mean(3)    1.000      0.157 6.39   <0.001 32.5 0.693  1.307

Term: gear
Type:  link 
Columns: term, contrast, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 

> 
> # Contrasts at the mean
> comparisons(mod, newdata = "mean")
Error in comparisons(mod, newdata = "mean") : object '..idx' not found
Calls: comparisons -> [ -> [.data.frame

https://rcdata.nau.edu/genomic-ml/data.table-revdeps/analyze/2024-10-14/snpsettest.txt

-* checking examples ... OK
+* checking examples ... ERROR
+Running examples in 'snpsettest-Ex.R' failed
+The error most likely occurred in:
+
+> base::assign(".ptime", proc.time(), pos = "CheckExEnv")
+> ### Name: harmonize_sumstats
+> ### Title: Harmonizing GWAS summary to reference data
+> ### Aliases: harmonize_sumstats
+> 
+> ### ** Examples
+> 
+> ## Don't show: 
+> data.table::setDTthreads(1)
+> ## End(Don't show)
+> ## GWAS summary statistics
+> head(exGWAS)
+     id chr   pos A1 A2      pvalue
+1 SNP_0   1 50215  G  C 0.196935302
+2 SNP_2   1 50768  A  G 0.662046499
+3 SNP_3   1 50833  T  G 0.582259626
+4 SNP_4   1 50848  C  T 0.232650995
+5 SNP_5   1 51330  A  G 0.081534282
+6 SNP_6   1 51696  T  C 0.002349804
+> 
+> ## Load reference genotype data
+> bfile <- system.file("extdata", "example.bed", package = "snpsettest")
+> x <- read_reference_bed(path = bfile)
+Reading /tmp/th798/13300208/R-devel/1303/snpsettest.Rcheck/snpsettest/extdata/example.fam 
+Reading /tmp/th798/13300208/R-devel/1303/snpsettest.Rcheck/snpsettest/extdata/example.bim 
+Reading /tmp/th798/13300208/R-devel/1303/snpsettest.Rcheck/snpsettest/extdata/example.bed 
+ped stats and snps stats have been set. 
+'p' has been set. 
+'mu' and 'sigma' have been set.
+Created a bed.matrix with 300 individuals and 2,942 markers.
+> 
+> ## Harmonize by SNP IDs
+> hsumstats1 <- harmonize_sumstats(exGWAS, x)
+-----
+Checking the reference data for harmonization...
+Found 0 monomoprhic SNPs in the reference data.
+Found 0 duplicate SNP IDs in the reference data.
+Excluded 0 SNPs from the harmonization.
+-----
+Checking the GWAS summary statistics...
+2,753 variants to be matched.
+Error in `[.data.frame`(x@snps[ref_keep_ind, ], sumstats[, .(id, pvalue)],  : 
+  unused arguments (on = .(id), nomatch = NULL)
+Calls: harmonize_sumstats -> [
+Execution halted
 * checking for unstated dependencies in vignettes ... OK
 * checking package vignettes ... OK
-* checking re-building of vignette outputs ... OK
+* checking re-building of vignette outputs ... ERROR
+Error(s) in re-building vignettes:
+  ...
+--- re-building 'reference_1000Genomes.Rmd' using rmarkdown
+--- finished re-building 'reference_1000Genomes.Rmd'
+
+--- re-building 'snpsettest_intro.Rmd' using rmarkdown
+
+Quitting from lines 68-78 [harmonization] (snpsettest_intro.Rmd)
+Error: processing vignette 'snpsettest_intro.Rmd' failed with diagnostics:
+unused arguments (on = .(id), nomatch = NULL)
+--- failed re-building 'snpsettest_intro.Rmd'
+
+--- re-building 'snpsettest_model.Rmd' using rmarkdown
+--- finished re-building 'snpsettest_model.Rmd'
+
+SUMMARY: processing the following file failed:
+  'snpsettest_intro.Rmd'
+
+Error: Vignette re-building failed.

@MichaelChirico @OfekShilon

Activity

  1. added this to the 1.17.0 milestone on Nov 15, 2024
  2. MichaelChirico commented on Dec 12, 2024

    @MichaelChirico
    Member

    @Rdatatable/revdep-managers I don't see these issues anymore -- in fact the logs are suspiciously free of errors. Is everything WAI?

  3. tdhock commented on Dec 13, 2024

    @tdhock
    MemberAuthor

    no, not WAI. if you look at the logs, the issues are still there, but there is some error from git bisect code which prevents results from getting saved and appearing on the report web page, tdhock/data.table-revdeps#32

  4. tdhock commented on Dec 17, 2024

    @tdhock
    Author
  5. tdhock commented on Jan 10, 2025

    @tdhock
    MemberAuthor

    snpsettest is fixed using 6ad0524
    others are still there

  6. MichaelChirico commented on Jan 18, 2025

    @MichaelChirico
    Member

    Issue is solved for {DTwrappers} and {DTwrappers2} after #6725, but not for {marginaleffects} 😿

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    revdepReverse dependencies

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions