Skip to content

unix: can't import frozen modules from a script that was executed from a file #2322

Description

@dpgeorge

The unix port can import frozen modules (str and mpy) from the REPL but it can't import them when executing a file (eg from ./micropython test.py). This is because when a file is executed from the command line the first entry in sys.path (which is originally the empty string '') is replaced by the base path of the executing file. This is the correct (CPython) behaviour.

But it means that frozen modules can no longer be found in the path -- they require the empty string in the path in order for them to be found in the import search (note that bare-metal ports always have '' in the path so they are ok).

Some ways to fix:

  1. Add a hack to the import machinery to always search frozen modules.
  2. Add a special directory to sys.path (eg :frozen:/) and prefix all the filenames in the list of frozen files with this special directory. The benefit of this is that it would allow the user to control the order of search when looking for frozen modules.

Activity

  1. pfalcon commented on Aug 15, 2016

    @pfalcon
    Contributor

    I don't think I like # 2. Too much adhoc magic, which likely will interfere with resource (non-executable) file access.

  2. dpgeorge commented on Aug 15, 2016

    @dpgeorge
    MemberAuthor

    Note that frozen bytecode (frozen mpy files) can't really be exposed as a read-only file (unlike frozen strings), so they'd anyway need to live in a separate "virtual" filesystem to any frozen resource files.

  3. pfalcon commented on Feb 6, 2017

    @pfalcon
    Contributor

    I'm now being hit by this and eager to fix this ;-).

  4. pfalcon commented on Feb 6, 2017

    @pfalcon
    Contributor

    Add a hack to the import machinery to always search frozen modules.

    So, I can see following way: if sys.path[0] != "", search before it only frozen modules. That at least should give the same search order as for REPL. But what did we decide on overridability of frozen modules? Actually, need to check what CPython does wrt to frozen modules search order and overridability.

  5. pfalcon commented on Feb 6, 2017

    @pfalcon
    Contributor

    The current workaround is to specify empty path component via MICROPYPATH (de48a27 fixes a case when there're multiple path components, first empty).

  6. dpgeorge commented on Feb 7, 2017

    @dpgeorge
    MemberAuthor

    But what did we decide on overridability of frozen modules?

    I think it's useful and important to be able to control the order (and whether they are even searched) of import of frozen modules. Controlling that via sys.path seems the most obvious way.

  7. pfalcon commented on Apr 30, 2017

    @pfalcon
    Contributor

    It now seems that behavior is different from str vs mpy frozen modules (which is probably rooted in the fact that we have inconsistent, duplicated code to look up each type).

  8. pfalcon commented on Apr 30, 2017

    @pfalcon
    Contributor

    Ok, now frozen mpy's don't seem to work even from repl.

  9. pfalcon commented on Apr 30, 2017

    @pfalcon
    Contributor

    Ok, that's because frozen mpy is disable for unix port.

  10. pfalcon commented on Apr 30, 2017

    @pfalcon
    Contributor

    ...or not

  11. pfalcon commented on Apr 30, 2017

    @pfalcon
    Contributor

    Ok, something just glitches in make apparently.

  12. belyalov commented on Mar 21, 2018

    @belyalov

    Any updates on this?
    It looks to be still broken:

    Having a build of micropython with unittests module frozen:

    root@e1ec92421b56:/build# micropython
    MicroPython v1.9.3-477-g7b0a020 on 2018-03-21; linux version
    Use Ctrl-D to exit, Ctrl-E for paste mode
    >>> import unittest
    >>>

    Simple script

    root@e1ec92421b56:/build# cat z.py
    import unittest

    Doesn't work:

    root@e1ec92421b56:/build# micropython z.py
    Traceback (most recent call last):
      File "z.py", line 1, in <module>
    ImportError: no module named 'unittest'
  13. dpgeorge commented on Dec 17, 2021

    @dpgeorge
    MemberAuthor

    Fixed by e0bf461

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions