Skip to content

froll compiled using -O0 fails 2 tests #3353

Closed
Closed
froll compiled using -O0 fails 2 tests#3353
Assignees
jangorecki
Labels
High
Milestone
1.12.2

Description

@mattdowle
mattdowle
opened on Feb 4, 2019

We've lost coverage stats due to this :

Still running (2 of 20): Rscript -e library(covr);codecov()
The command Rscript -e library(covr);codecov() exited with 1.
Log:
Error: Failure in `/private/var/folders/nz/vv4_9tw56nv9k3tkvyszvwg80000gn/T/Rtmp1nDGQA/R_LIBS154b3111ce992/data.table/data.table-tests/froll.Rout.fail`
UE)
> afun_compare(x, n)
> x = c(rep(NA, 60), rnorm(1e3), rep(NA, 60)); n = sample(51, length(x), TRUE)
> afun_compare(x, n)
> x = c(rep(NA, 60), rnorm(1e3+1), rep(NA, 60)); n = sample(51, length(x), TRUE)
> afun_compare(x, n)
> #### random NA
> x = makeNA(rnorm(1e3)); n = sample(50, length(x), TRUE)
> afun_compare(x, n)
> x = makeNA(rnorm(1e3+1)); n = sample(50, length(x), TRUE)
> afun_compare(x, n)
> x = makeNA(rnorm(1e3)); n = sample(51, length(x), TRUE)
> afun_compare(x, n)
> x = makeNA(rnorm(1e3+1)); n = sample(51, length(x), TRUE)
> afun_compare(x, n)
> rm(num)
> 
> setDTthreads(oldDTthreads)
> if (length(.test.froll.failed)) stop(sprintf("froll unit tests failed: %s.", paste(.test.froll.failed, collapse=", "))) else cat("froll unit tests successfully passed\n")
Error: froll unit tests failed: 9999.123, 9999.162.
Execution halted
Execution halted
/Users/travis/.travis/job_stages: line 523: 87220 Terminated: 15          travis_jigger "${!}" "${timeout}" "${cmd[@]}"

Activity

  1. mattdowle
    added this to the 1.12.2 milestone on Feb 4, 2019
  2. mattdowle
    changed the title [-]Covergage failing on 2 froll tests[/-] [+]Coverage failing on 2 froll tests[/+] on Feb 4, 2019
  3. jangorecki
    self-assigned this
    on Feb 5, 2019

  4. jangorecki commented on Feb 5, 2019

    @jangorecki
    jangorecki
    on Feb 5, 2019
    Member

    https://travis-ci.org/Rdatatable/data.table/jobs/486208986
    it looks like the issue related to rounding error and error correction. Which so far was observed only on windows with 2+ threads #3346, here is Linux and OSX with 1 thread.

  5. jangorecki commented on Feb 5, 2019

    @jangorecki
    jangorecki
    on Feb 5, 2019
    Member

    Related from covr/README.md

    Because covr modifies the package code it is possible there are unknown edge cases where that modification affects the output. In addition when tracking coverage for compiled code covr compiles the package without optimization, which can modify behavior (usually due to package bugs which are masked with higher optimization levels).

    to reproduce this issue it is enough to set -O0 in Makevars, reinstall and run Rscript tests/froll.R. This flag change the way how NaN are handled by compiler.
    There might be another problem that matching base R output in those test is likely to depends on -O level that has been used during base R installation.

    ma = function(x, n, na.rm=FALSE, nf.rm=FALSE) {
  if (!is.double(x)) x = as.double(x)
  if (!is.integer(n)) n = as.integer(n)
  ans = rep(NA_real_, nx<-length(x))
  if (nf.rm) x[!is.finite(x)] = NA_real_ # exact=F consistency due to https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17441
  for (i in n:nx) ans[i]=mean(x[(i-n+1):i], na.rm=na.rm)
  ans
}
n = 4
x = 1:16
x[5] = NaN
x[6] = NA
cc(test=FALSE, debug=TRUE, clean=TRUE) # -O0
data.table(x, frollmean(x, n, algo="exact"), ma(x, n))
#         x    V2    V3
#...
#  4:     4   2.5   2.5
#  5:   NaN   NaN   NaN
#  6:    NA    NA   NaN
#  7:     7    NA   NaN
#  8:     8    NA   NaN
#  9:     9    NA    NA
# 10:    10   8.5   8.5
#...
cc(test=FALSE, debug=FALSE, clean=TRUE) # -O3
data.table(x, frollmean(x, n, algo="exact"), ma(x, n))
#          x    V2    V3
#...
#  4:     4   2.5   2.5
#  5:   NaN   NaN   NaN
#  6:    NA   NaN   NaN
#  7:     7   NaN   NaN
#  8:     8   NaN   NaN
#  9:     9    NA    NA
# 10:    10   8.5   8.5
#...

  6. jangorecki commented on Feb 6, 2019

    @jangorecki
    jangorecki
    on Feb 6, 2019
    Member

    Since #3362 is merge issues like this are now properly failing our GL pipeline.

  7. jangorecki
    added
    High
    on Feb 6, 2019
  8. jangorecki
    changed the title [-]Coverage failing on 2 froll tests[/-] [+]compilation using -O0 failing on 2 froll tests[/+] on Feb 6, 2019
  9. jangorecki
    changed the title [-]compilation using -O0 failing on 2 froll tests[/-] [+]froll compiled using -O0 fails 2 tests[/+] on Feb 6, 2019

  10. jangorecki commented on Feb 6, 2019

    @jangorecki
    jangorecki
    on Feb 6, 2019
    Member

    The issues is more broad, it also affects gmean, it has been discovered in froll because it is being comprehensively tested.

    dd(omp=TRUE) ## compile with -O0, still with omp
d = data.table(g=c(1L,2L,2L,2L), y=c(3,NaN,NA,7))
d[, mean(y), g, verbose=TRUE]
rm(d)
cc(test=FALSE, clean=TRUE) ## compile with -O3
d = data.table(g=c(1L,2L,2L,2L), y=c(3,NaN,NA,7))
d[, mean(y), g, verbose=TRUE]
rm(d)

    output is

    GForce optimized j to 'gmean(y)'
       g    V1
   <int> <num>
1:     1     3
2:     2    NA # -O0

GForce optimized j to 'gmean(y)'
       g    V1
   <int> <num>
1:     1     3
2:     2   NaN # -O3

    Worth to note that base R is not consistent on NaN/NA handling in mean, as discussed before in https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17441

  11. jangorecki
    added a commit that references this issue on Feb 6, 2019

    escape tests if NaN-NA behaviour does not match, closes #3353

    fdabca5
  12. mattdowle
    closed this as completedin #3364on Feb 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

High

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions