Repository navigation
Towards module library support #258
Description
We're currently at that situation when we have basic module loading support, but don't have support for a loading modules from a "module library". Due to this, there's intuitive tendency to put all stuff into C code, even if it would be more easily managed in Python code (for example, constant assignments).
So, I propose that we start defining where MicroPython module library is located and how it is used.
Here's rough proposal, based of course on how CPython does it:
- Let's start with Unix port which requires most flexibility.
- Let there be environment variable which contains module library search path. That would be MICROPYTHONPATH if we follow the convention (OMG, that's really TOO LONG, that's why I proposed to stick with shorter "short" name for project, while "descriptive" name of course can be MicroPython).
- Let environment variable list of search directories, separated by ":".
- If the environment variable is not defined, let default path be "~/.micropython/lib:/usr/lib/micropython"
- Now let's map that to bare metal ports.
Next steps:
6. If having a self-contained binary is consider a big boon, let's work towards having frozen modules a-la CPython, which are embedded into interpreter (first attempt in that direction was here: #147).
Activity
dpgeorge commentedon Feb 3, 2014
I'm fully supportive of this. But I must stay focused on the intial reason for uPy, namely to get it running on the pyboard. Points 5, 6 and 7 encompass pyboard, so that's great, and it shouldn't be much work to implement them. But I will have to leave the unix side of things to others to implement.
I'm starting to agree that
micropythonas a binary name is long, but I see no alternative (and it's dead obvious to newcomers why it's called that). For env variables, how about MICROPYPATH, following theMICROPY_xxxconfiguration variables?pfalcon commentedon Feb 3, 2014
I'm only happy to implement it for unix, just would like to know that basic idea is good and stm port can follow (just to have a designated location for library, to separate "module library" and "user app modules", the latter living in "current directory", configurability can be left for later).
MICROPYPATH sounds good.
dpgeorge commentedon Feb 3, 2014
Agreed then. I'd probably have the STM library directory as /lib:/sd/lib, where /sd is the sd card.
Regarding point 6, what exactly do you mean? There is already sys, os, io, platform in CPython. Shouldn't these be enough?
pfalcon commentedon Feb 4, 2014
Re: point 6 - you're of course right, module search path is kept in sys.path, and that of course should be enough to let users customize it. We'll just init it from environment on platforms which have it.
And I see that stm actually already looks for modules in 0:/src/ and 0:/ , so it would be easy change, by I already started rewrite to use sys.path directly in import.
KeithJRome commentedon Feb 4, 2014
Perhaps I am misunderstanding the discussion here - but isn't this type of problem addressed best by PEP302? You don't really want to monkey around with sys.path for injecting platform-level modules... the better approach is to leave sys.path clean and tidy and use a custom meta path hook (added to sys.meta_path), which is consulted prior to any path processing.
I implemented this feature for IronPython a couple of years ago (to allow embedding a stable /lib as a zip resource instead of requiring external installation of python). It worked pretty well there, was a nice and clean solution.
http://www.python.org/dev/peps/pep-0302/
dpgeorge commentedon Feb 10, 2014
The discussion here is (partly) about the environment variable for the unix port, eg the equivalent of PYTHONPATH but for uPy. On the STM, this would be set to something pointing to the flash and the SD card. This is still inline with the way Python does it.
I agree with @KeithJRome that we should use the standard Python hooks for import if we want to allow complex customisation.
pfalcon commentedon Feb 14, 2014
Ok, so this was essentially implemented (for unix). Left open because:
pfalcon commentedon Feb 14, 2014
Edited description to remove superfluous content re: bare-metal ports (as we agreed, letting user set sys.path in some startup file is good way to support its configurability).
pfalcon commentedon Feb 22, 2014
Ok, so STM port now known to work as expected too. This ticket brings some other topics, like support for frozen modules, but there's no pressing need for this so far (there's still no python stdlib modules in codebase), so let's handle that separately.
Move to ASF4 and introduce SAMD51 support. (micropython#258)