Skip to content

Failed 'tests/misc/sys_settrace_features.py' #6689

Description

@gvanem

I've built MicroPython on Windows and notice some failures due to mixed expectation of / vs \ in filenames.
The diff between:

  • tests\results\misc_sys_settrace_features.py.exp and
  • tests\results\misc_sys_settrace_features.py.out

contains a lot of lines like these:

--- tests/results/misc_sys_settrace_features.py.exp 2020-12-10 13:21:13
+++ tests/results/misc_sys_settrace_features.py.out 2020-12-10 13:21:13
@@ -101,730 +101,730 @@
  0:   @__main__:do_tests => miscmisc/sys_settrace_features.py:97
  1:   @__main__:<module> => miscmisc/sys_settrace_features.py:104
 ### trace_handler::main event: call
- 0:   @sys_settrace_subdir.trace_generic:<module> => miscF:\MingW32\src\Languages\MicroPython\tests\misc\sys_settrace_subdir\trace_generic.py:1
+ 0:   @sys_settrace_subdir.trace_generic:<module> => misc/sys_settrace_subdir/trace_generic.py:1
  1:   @__main__:do_tests => miscmisc/sys_settrace_features.py:97
  2:   @__main__:<module> => miscmisc/sys_settrace_features.py:104

The line responsible for this difference is AFAICS in tests\misc\sys_settrace_features.py:
"misc" + "".join(frame.f_code.co_filename.split("tests/misc")[-1:]),

Where is the cause of these diffs? And how is the .exp file generated?

Activity

  1. stinos commented on Dec 10, 2020

    @stinos
    Contributor

    The .exp for this test is generated by running the file with CPython so the line which has frame.f_code.co_filename.split is the culprit indeed. This should do the trick (replace the slashes so the split() works again correctly):

    "misc" + "".join(frame.f_code.co_filename.replace('\\', '/').split("tests/misc")[-1:]),
    
  2. gvanem commented on Dec 10, 2020

    @gvanem
    Author

    I patched tests/misc/sys_settrace_features.py and did:

    cd tests
    py -3 run-tests misc\sys_settrace_features.py
    

    Still fails:

    FAIL  misc/sys_settrace_features.py
    1 tests performed (838 individual testcases)
    0 tests passed
    1 tests failed: sys_settrace_features
    

    Wrong syntax or wrong patch?

  3. stinos commented on Dec 10, 2020

    @stinos
    Contributor

    Can you show the diff (i.e. run-tests --print-failures)?

  4. gvanem commented on Dec 10, 2020

    @gvanem
    Author

    With a py -3 run-tests --print-failures misc\sys_settrace_features.py, there is no files in results. So one would assume there are no errors?

    But a py -3 run-tests misc\sys_settrace_features.py, gives this huge diff misc_sys_settrace_features.diff.txt

    How is this obfuscated script supposed d to work?

  5. stinos commented on Dec 10, 2020

    @stinos
    Contributor

    Both commandlines shown are the same so not really sure what you mean. The diff still has the exact same problem, which is strange because e.g.
    r'miscF:\MingW32\src\Languages\MicroPython\tests\misc\sys_settrace_subdir\trace_generic.py'.replace('\\', '/').split("tests/misc")[-1:] is ['/sys_settrace_subdir/trace_generic.py'] so it looks as if the patch has no effect; was it applied to the wrong file?

    How is this obfuscated script supposed d to work?

    Which one?

  6. gvanem commented on Dec 10, 2020

    @gvanem
    Author

    I meant py -3 run-tests misc\sys_settrace_features.py, give huge diffs.

  7. stinos commented on Dec 10, 2020

    @stinos
    Contributor

    I'm testing it here and the patch makes the path problem go away, and tests pass, so I'm not sure what the problem is on your machine. See #6690 which works for me.

  8. gvanem commented on Dec 11, 2020

    @gvanem
    Author

    I've applied you diff and reran cd tests & py -3 ./run-tests. The same diffs on:

    • tests/results/misc_sys_settrace_features.py.exp and
    • tests/results/misc_sys_settrace_features.py.out

    as before.

    And I noted on AppVeyor all these relevant tests are marked skip (why?):

    [00:01:01] skip  misc/sys_settrace_features.py
    [00:01:01] skip  misc/sys_settrace_generator.py
    [00:01:01] skip  misc/sys_settrace_loop.py
    

    E.g. this one.

    So how can you prove your diff works?

  9. stinos commented on Dec 11, 2020

    @stinos
    Contributor

    And I noted on AppVeyor all these relevant tests are marked skip (why?):

    Because settrace isn't enabled by default and the CI builds d not change that - you also had to change code or pass different flags to make to have it enabled, right?

    So how can you prove your diff works?

    Here's a build for which I enabled it and where you can see all 3 tests pass both for the msvc and mingw ports: https://ci.appveyor.com/project/stinos/micropython/build/job/4jaq3kg21j8yrsba?fullLog=true (I don't know how long this stays available though)

    So obviously something is different for you, but I don't know what and can't really help you. You could start by adding a print(frame.f_code.co_filename) into sys_settrace_features.py and then work from there to figure out why the path you get there doesn't get the expected treatment. Note you seem to be using py -3, not sure if it matters but run-tests defaults to using python3.exe unless you explcicitly set the the MICROPY_CPYTHON3 environment variable.

  10. gvanem commented on Dec 11, 2020

    @gvanem
    Author

    Does MICROPY_PY_SYS_SETTRACE 0 or 1 has anything to do with this? I fail to see where / how prelude->qstr_source_file gets
    set.

  11. stinos commented on Dec 11, 2020

    @stinos
    Contributor

    Sorry now I'm confused: I always assumed you were running a build where you explicitly enabled MICROPY_PY_SYS_SETTRACE, which for the windows ports means e.g. putting this in ports/windows/mpconfigport.h:

    #define MICROPY_PERSISTENT_CODE_SAVE (1)
    #define MICROPY_COMP_CONST (0)
    #define MICROPY_PY_SYS_SETTRACE (1)
    

    If that is not enabled, all related tests should be skipped automatically because the test files start with

    try:
        sys.settrace
    except AttributeError:
        print("SKIP")
        raise SystemExit
    

    So when run with micropython without settrace functionality, this prints SKIP and run-tests picks that up to skip the test.

    However from your explanation I now get the impression you didn't enable it explicitly, yet are gettings test failures. No idea how that is even possible.

    To figure out what is wrong you'll have to supply enough information so we can get an idea of what you are doing exactly:

    • what version fo the code you are using (master branch? which commit?)
    • compiler version (I see you have a directory MingW32, note that only ming-w64 is supported)?
    • how you build micropython (from which shell, which commands etc)
    • if you run micropython and type import sys.settrace on the command line, what happens?
  12. gvanem commented on Dec 11, 2020

    @gvanem
    Author

    Off course I have MICROPY_PY_SYS_SETTRACE=1.

  13. stinos commented on Dec 11, 2020

    @stinos
    Contributor

    Well, the previous comment asked whether MICROPY_PY_SYS_SETTRACE has anything to do with this; to be on the safe side I did not interpret that as 'off course' :)
    Anyway: without further info from your side it's going to be hard to fix anything, unless you can figure it out yourself.

  14. gvanem commented on Dec 11, 2020

    @gvanem
    Author

    if you run micropython and type import sys.settrace on the command line, what happens?

     MicroPython.exe -c "import sys; sys.settrace(1)" & echo %?
     0
    

    Trying to simplify this nerdy tool:

    • cd tests & python3 misc\sys_settrace_features.py > temp.1 compared to
    • cd tests & ..\MicroPython.exe misc\sys_settrace_features.py > temp.2

    Isn't that how run-tests basically does it? With your patch, temp.1 and temp.2 are different.

  15. stinos commented on Dec 11, 2020

    @stinos
    Contributor

    Isn't that how run-tests basically does it?

    Basically, yes, but perhaps not exactly, and that might matter here. Though if I run those commands (only difference for me being I invoke ..\ports\windows\micropython.exe) I get 2 files which are exactly the same.

  16. stinos commented on Dec 17, 2020

    @stinos
    Contributor

    @gvanem any news here? In the meantime I confirmed the fixes in aforementioned PR on multiple systems. Did you already figure out why simple replacing the slashes doesn't work on your setup?

  17. dpgeorge commented on Dec 17, 2020

    @dpgeorge
    Member

    I made a minor change to this settrace test in e0bb7a5 to make it work with more recent Python 3 versions. Not sure if that changes anything here.

  18. stinos commented on Dec 17, 2020

    @stinos
    Contributor

    I had that one in my PR already, and the diff shown earlier only shows path differences, no zipimport calls, so shouldn't matter here I think.

  19. gvanem commented on Dec 17, 2020

    @gvanem
    Author

    @stinos I did a git pull 10 min ago and rebuilt everything (for Win32 with lwIP sockets) and re-ran this simple case:

    cd tests  &
    py -3 misc\sys_settrace_features.py > temp.1
     ..\MicroPython.exe misc\sys_settrace_features.py > temp.2
    diff -u3 temp.1 temp.2
    

    temp.1 contains strings like:

    ### trace_handler::main event: call
     0:   @sys_settrace_subdir.trace_generic:<module> => miscF:\MingW32\src\Languages\MicroPython\tests\misc\sys_settrace_subdir\trace_generic.py:1
     1:   @__main__:do_tests => misctests\misc\sys_settrace_features.py:106
     2:   @__main__:<module> => misctests\misc\sys_settrace_features.py:113
    

    Seems the replace() ass-u-mes wrongly. And adding a print (" co_filename: %s" % frame.f_code.co_filename), I see many:
    f:\ProgramFiler\Python36\lib\encodings\cp1252.py

    But I'm utterly confused about what happens here.

  20. stinos commented on Dec 17, 2020

    @stinos
    Contributor

    I did a git pull 10 min ago

    Seeing the differences you have, is it possible you just pulled master? That won't cut it, my PR isn't merged there yet. The code from the PR can be fetched like

    git fetch upstream pull/6690/head:pr/6690
    git checkout pr/6690
    

    where 'upstream' is the name of the MicroPython remote you use.

    temp.1 contains strings like:

    As mentioned earlier: the equivalent of this code is in the PR

    filename = r"F:\MingW32\src\Languages\MicroPython\tests\misc\sys_settrace_subdir\trace_generic.py"
    print("misc" + "".join(filename.replace("\\", "/").split("tests/misc")[-1:])) # prints misc/sys_settrace_subdir/trace_generic.py
    

    so the most logical explanation would be that this code is not what you are using? How else could you still get the complete filename printed?

    I see many: f:\ProgramFiler\Python36\lib\encodings\cp1252.py

    Which should also be solved by the PR since it explicitly skips any stack frame with "encodings" in it's name.

  21. dpgeorge commented on Dec 17, 2020

    @dpgeorge
    Member

    I had that one in my PR already,

    Ah, indeed, sorry, I missed it!

  22. added
    testsRelates to tests/ directory in source
    on Dec 17, 2020
  23. gvanem commented on Dec 28, 2020

    @gvanem
    Author

    I'm closing this as this test-command (in my Makefile) works:

    cd ./tests ; py -3 ./run-tests misc/sys_settrace_features.py
    pass  misc/sys_settrace_features.py
    1 tests performed (838 individual testcases)
    1 tests passed
    

    But adding option --print-failures as advised gives either no output or indicates a crash:

    cd ./tests ; py -3 ./run-tests --print-failures misc/sys_settrace_features.py
    
    FAILURE F:/MingW32/src/Languages/MicroPython/tests/results\extmod_uselect_poll_basic.py
    --- F:/MingW32/src/Languages/MicroPython/tests/results/extmod_uselect_poll_basic.py.exp 2020-12-28 11:13:05
    +++ F:/MingW32/src/Languages/MicroPython/tests/results/extmod_uselect_poll_basic.py.out 2020-12-28 11:13:05
    @@ -1 +1,5 @@
    -SKIP
    +modify:TypeError
    +Traceback (most recent call last):
    +  File "extmod/uselect_poll_basic.py", line 41, in <module>
    +OSError: 10038
    +CRASH
    \ No newline at end of file
    
    FAILURE F:/MingW32/src/Languages/MicroPython/tests/results\extmod_uselect_poll_udp.py
    --- F:/MingW32/src/Languages/MicroPython/tests/results/extmod_uselect_poll_udp.py.exp 2020-12-28 11:02:50
    +++ F:/MingW32/src/Languages/MicroPython/tests/results/extmod_uselect_poll_udp.py.out 2020-12-28 11:02:50
    @@ -1 +1,2 @@
    -CPYTHON3 CRASH
    \ No newline at end of file
    +0
    +True
    

    Note it runs extmod/uselect*.py when not told to. I fail to understand this junk tool.

  24. stinos commented on Dec 28, 2020

    @stinos
    Contributor

    I fail to understand this junk tool.

    Have you considered your failure to understanding is the problem here, not the tool? I politely ignored all your other snarky comments in this thread, but this kind of language is not appreciated here. We want to maintain a nice and friendly environment. And there's plenty of room for criticism and reporting problems, I mean after all it's an issue tracker, but there's no reason to do that in such a disrespectful manner.

  25. added a commit that references this issue on Aug 10, 2022
    0b098f5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsRelates to tests/ directory in source

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions