Skip to content

do not use valgrind for initial pass /return values from fail/pass/crash parsing #62

Description

@tdhock

hi @akhikolla there is a problem in your current implementation.

Right now deepstate_compile_fun runs the test harness with multiple inputs under valgrind. This is problematic for several reasons

  1. sometimes valgrind aborts without saving fun_output/.pass and fail binary files. e.g. rcpp_read_out_of_bound: "new/new[] failed and should throw an exception, but cannot throw exceptions and so is aborting instead. Sorry." So valgrind aborts before deepstate has a chance to save the pass/fail binary files. In contrast if you run the test harness without valgrind then these pass/fail binary files are saved to fun_output/
  2. it is very slow so there are not many random inputs that can be tested.
  3. the valgrind log file is useless because it contains messages from all of the different random inputs.

To fix I propose that you change the code to

  1. deepstate_compile_fun: run initial test harness NOT under valgrind, don't save any logs, just save fun_output/*.pass or fail files.
  2. deepstate_analyze_fun: run deepstate_analyze_fun_inputs (new function, see below) on all pass/fail binary files.
  3. deepstate_analyze_fun_inputs("path/to/pkg/inst/testfiles/fun_output/HASH.fail") run test harness with this set of inputs under valgrind, save and return valgrind messages and R inputs.

what do you think? does that make sense? can you code that for next week please?

Activity

  1. akhikolla commented on Oct 7, 2020

    @akhikolla
    Owner

    From what I understand we should not use Valgrind in the compile, analyze steps, and use the inputs from the analyze step and run it with Valgrind.
    This is a good approach as we don't really use the log generated in the first step of the compilation.
    I can implement this for next time.

  2. changed the title [-]do not use valgrind for initial pass[/-] [+]do not use valgrind for initial pass /return values from fail/pass/crash parsing[/+] on Oct 14, 2020
  3. tdhock commented on Oct 14, 2020

    @tdhock
    Author
    > data.table(input_file="~/R/RcppExports/packages/BNSL/inst/testfiles/mi/mi_output/00f6ebd2f636a3f88055f63e4ea26fc6fbce6906.pass", input_list=list(list(x=c(1, 5 , NA), y=1:10)), trace=list(data.table(file.line="mi.cpp:55", msg="stack overflow")))
                                                                                                    input_file
    1: ~/R/RcppExports/packages/BNSL/inst/testfiles/mi/mi_output/00f6ebd2f636a3f88055f63e4ea26fc6fbce6906.pass
       input_list             trace
    1:  <list[2]> <data.table[1x2]>
    > 

    suggested data type return value for one pass/crash/fail file = 1 row data table
    (several pass/fail/crash files should result in a data table with that many rows)

  4. tdhock commented on Oct 23, 2020

    @tdhock
    Author

    is this resolved?

  5. akhikolla commented on Oct 23, 2020

    @akhikolla
    Owner

    Yes, the issue is resolved.

    > rd <- deepstate_analyze_fun("~/extdata/packages/BNSL/inst/testfiles/mi")
    Read 1 item
    Read 1 item
    Read 1 item
    > rd
                                                                                                          binaryfile
    1: /home/akolla/extdata/packages/BNSL/inst/testfiles/mi/mi_output/009e5ab9b0b69e3f96b34a1e8ef26f27fb5451d0.crash
    2: /home/akolla/extdata/packages/BNSL/inst/testfiles/mi/mi_output/00b86d0e250dd13dcf8410799279a38b33390382.crash
    3: /home/akolla/extdata/packages/BNSL/inst/testfiles/mi/mi_output/00d4c92d20520ff4d766d3cd53fa2f72f46bd98f.crash
          inputs              logtable
    1: <list[3]>     <data.table[1x5]>
    2: <list[3]>     <data.table[1x5]>
    3: <list[3]> No source trace found
    > rd$logtable
    [[1]]
              kind                    msg          errortrace
    1: InvalidRead Invalid read of size 8 src/mi_cmi.cpp : 57
                                                              address trace
    1: Address 0xaa90cc0 is 0 bytes after a block of size 288 alloc'd    NA
    
    [[2]]
              kind                    msg          errortrace
    1: InvalidRead Invalid read of size 8 src/mi_cmi.cpp : 57
                                                              address trace
    1: Address 0x96e23b0 is 0 bytes after a block of size 752 alloc'd    NA
    
    [[3]]
    [1] "No source trace found"
    
    > 
    

    Still working on compressing the Read 1 item message on the console. Used sink(/dev/null), invisible they didn't work.

  6. tdhock commented on Oct 23, 2020

    @tdhock
    Author

    usually the "read 1 item" comes from some function that takes an argument you can use to suppress it, e.g.

    > cat("2 3", file=f <- tempfile())
    > scan(f)
    Read 2 items
    [1] 2 3
    > scan(f, quiet = TRUE)
    [1] 2 3
    > 
  7. akhikolla commented on Oct 24, 2020

    @akhikolla
    Owner

    Yes, got it. I updated it.

  8. akhikolla commented on Nov 4, 2020

    @akhikolla
    Owner

    Issue resolved. Using Valgrind in the second pass and able to parse the log successfully.

  9. tdhock commented on Nov 4, 2020

    @tdhock
    Author

    great

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions