Repository navigation
Discussion of Python 3.5 support #1329
Description
Activity
We "by definition" have "%-formatting for binary strings", I'm not sure however what details PEP 461 may have.
retrying system calls that fail with EINTR
is potentially interesting, because it's boring to handle it in each app, and actually nobody handles it in each app, unless faces issues, and then it's likely users, and not an author. So, doing that PEP kinda saves from such situations. But then again it may be just adding more bloat than necessary. Need to read PEP, and actually, all PEPs (in detail, again, or at all).
I'm not sure however what details PEP 461 may have.
We are missing: "%b" for interpolating bytes objects, "%a" for repr of bytes, "%c" in uPy truncates to 7-bit char but should be full 8-bit char, and we don't support % operator on bytearray object.
Type hints sound like they could be useful.
Personally I'd like to see unpacking generalizations and async/await support.
Reacted by Patrick ArminioI'd like to see unpacking generalizations
I just had a read of that PEP and it looks like it might be some effort, and a decent increase in code size, to implement that. It allows things like
lst = [*a, b, *c]andfoo(*a, b, *c).Running uPy testsuite against CPython3.5b3 produces few warnings/errors.
Running uPy testsuite against CPython3.5b3 produces few warnings/errors.
A few too many, or few enough to easily fix them? Do you have any examples of failures?
I built 3.5b3, and that managed to install itself as system-wide python3, so I scratched ny head for a bit while testsuite fails. As soon as I figured that out, I switched it back and didn't look back, and don't think it's a priority before 3.5 release. So, just FYI ;-).
3.5.0 was released today.
CPy 3.5 is now distributed with Arch Linux so that's what I'll be using. There are only a few errors (mostly related to new , * syntax).
Evaluation order of dictionary key/value pairs in CPy 3.5 has changed. Key is now evaluated first in a dict literal.
Hi,
Is Python 3.5 coroutine syntax (await / async) supported be uPy ?19 remaining items
If it's not too problematic PEP448 would be useful. There is evident demand: I've already had to explain to users that it's not yet supported.
I appreciate that PEP572 is Python 3.8. My interest is on the theory that it might improve performance and/or code size e.g. in comprehensions and lambdas.
I appreciate that PEP572 is Python 3.8. My interest is on the theory that it might improve performance and/or code size e.g. in comprehensions and lambdas.
Same here! It'll come anyway so it might be as well available in time with the release of 3.8
Notwithstanding the fact that
numpyis very unlikely to get ported to MicroPython, it might we worth implementing PEP 465 anyway since (a) it would probably be easy and small, (b) it would provide compatibility and (c) having a spare infix operator lying around is often handy.FWIW, I've have ended up implementing a tiny linear algebra system on uPy (they are handy for optimising some dynamic systems and the ESP32 with SPIRAM has the space and horsepower to cope) and having the
@and@=operators would have been quite helpful for improving readability and compactness.(I also note that MicroPython already supports
Ellipsisand it never gets used in any of the standard libraries, so there is a precedent for implementing language features that have no built-in use cases.)Reacted by Bram WastiNotwithstanding the fact that
numpyis very unlikely to get ported to MicroPython, it might we worth implementing PEP 465 anyway since (a) it would probably be easy and small, (b) it would provide compatibility and (c) having a spare infix operator lying around is often handy.@nickovs I have already implemented this in #4740
(I am working on porting a useful subset of numpy, based on ublas/lapack like regular numpy).
Reacted by Nicko van SomerenReacted by Bram WastiPEP 465 can be crossed off of the list now
Reacted by Juan Luis Cano RodríguezShould PEP 485 be crossed off of the list?
Yes, list is now updated for PEP 475 and 485.
- unpinned this issue
on Feb 2, 2022 Now that #5807 has been merged, we have partial support for PEP 448, namely:
Function calls are proposed to support an arbitrary number of unpackings rather than just one:
>>> print(*[1], *[2], 3) 1 2 3 >>> dict(**{'x': 1}, y=2, **{'z': 3}) {'x': 1, 'y': 2, 'z': 3}
The remaining bit that is not implemented is:
Unpacking is proposed to be allowed inside tuple, list, set, and dictionary displays:
>>> *range(4), 4 (0, 1, 2, 3, 4) >>> [*range(4), 4] [0, 1, 2, 3, 4] >>> {*range(4), 4} {0, 1, 2, 3, 4} >>> {'x': 1, **{'y': 2}} {'x': 1, 'y': 2}
I think we could leverage the partial implementation to easily implement the rest. It could work like this:
-
We need two intrinsic functions that basically look like this:
def capture_args(*args): return args def capture_kwargs(**kwargs): return kwargs
-
If the compiler encounters
*in a tuple, list or set display or**in a dictionary display, then it "rewrites" the code like this:*range(4), 4 # becomes tuple(capture_args(*range(4), 4)) [*range(4), 4] # becomes list(capture_args(*range(4), 4)) {*range(4), 4} # becomes set(capture_args(*range(4), 4)) {'x': 1, **{'y': 2}} # becomes dict(capture_kwargs(x=1, **{'y': 2}))
Reacted by Pavlo Bielousov-
I think we could leverage the partial implementation to easily implement the rest.
That's interesting, and definitely simple... but I'm not sure how to implement intrinsic functions (efficiently).
I wonder how CPython does it, do they create a comprehension function to build the tuple/list/set/dict? If so that would alter the semantics.
I wonder how CPython does it
I looked at the patch that implements it and they just added new opcodes for each case:
+#define BUILD_LIST_UNPACK 149 +#define BUILD_MAP_UNPACK 150 +#define BUILD_MAP_UNPACK_WITH_CALL 151 +#define BUILD_TUPLE_UNPACK 152 +#define BUILD_SET_UNPACK 153
I looked at the patch that implements it and they just added new opcodes for each case:
I guess we could do that as well.
I think this feature is rarely used (??) so if MicroPython implements it I would say it should be done as "cheaply" as possible, ie with a minimum of firmware size. Maybe adding new opcodes is the smallest way to implement it, or maybe it's some other way.
It looks like the hardest thing (biggest increase in code size) would be to change the compiler to detect these cases. So investigations would be best to start there.
MicroPython aims to implement the Python 3.x "standard". At the moment x is currently 4, ie we try to be compatible with CPython 3.4. It's new territory as to what to do when CPython evolves to larger version numbers. I would say we should try to follow the changes and implement them when possible/sensible.
Python 3.5 had a feature freeze on 24 May 2015 and is scheduled for final release on 13 September 2015. This ticket here is about discussing if, what and how we should upgrade uPy to version 3.5 of the language. It may be that some issues need to break off into separate tickets and that's fine but we should link to them from this one.
The PEP discussing the new features in 3.5: https://www.python.org/dev/peps/pep-0478/
A more friendly overview in the docs: https://docs.python.org/3.5/whatsnew/3.5.html
Below is a list of finalised/accepted PEPs for 3.5 grouped into their impact to MicroPython.
Extensions to the syntax:
Extensions and changes to the runtime:
Standard library changes:
Miscellaneous changes that are not relevant to MicroPython:
Other Language Changes
"namereplace"error handlers. The"backslashreplace"error handlers now work with decoding and translating.New modules
Changes to MicroPython built-in modules
isclose()provides a way to test for approximate equality.OrderedDictclass is now implemented in C, which makes it 4 to 100 times faster.OrderedDict.items(),OrderedDict.keys(),OrderedDict.values()views now supportreversed()iteration.index(),insert(), andcopy(), and supports the + and * operators.namedtuple()can now be updated.UserStringclass now implements the__getnewargs__(),__rmod__(),casefold(),format_map(),isprintable(), andmaketrans()methods to match the corresponding methods ofstr.merge()can now be customized by passing a key function in a new optional key keyword argument, and a new optionalreversekeyword argument can be used to reverse element comparisonBufferedIOBase.readinto1()method, that uses at most one call to the underlying raw stream'sRawIOBase.read()orRawIOBase.readinto()methods.JSONDecodeErrorinstead ofValueErrorto provide better context information about the error.infandnan.isclose()provides a way to test for approximate equality.gcd()function has been added. Thefractions.gcd()function is now deprecated.scandir()function returning an iterator ofDirEntryobjects has been added.urandom()function now uses thegetrandom()syscall on Linux 3.17 or newer, and getentropy() on OpenBSD 5.6 and newer, removing the need to use /dev/urandom and avoiding failures due to potential file descriptor exhaustion.get_blocking()andset_blocking()functions allow getting and setting a file descriptor's blocking mode (O_NONBLOCK.)os.path.commonpath()function returning the longest common sub-path of each passed pathname.sub()andsubn()functions now replace unmatched groups with empty strings instead of raising an exception.re.errorexceptions have new attributes,msg,pattern,pos,lineno, andcolno, that provide better context information about the errorsocket.sendfile()method allows sending a file over a socket by using the high-performanceos.sendfile()function on UNIX, resulting in uploads being from 2 to 3 times faster than when using plainsocket.send().socket.sendall()method no longer resets the socket timeout every time bytes are received or sent. The socket timeout is now the maximum total duration to send all data.socket.listen()method is now optional. By default it is set to SOMAXCONN or to 128, whichever is less.SSLSocket.version()method to query the actual protocol version in use.SSLSocketclass now implements aSSLSocket.sendfile()method.SSLSocket.send()method now raises either thessl.SSLWantReadErrororssl.SSLWantWriteErrorexception on a non-blocking socket if the operation would block. Previously, it would return 0.cert_time_to_seconds()function now interprets the input time as UTC and not as local time, per RFC 5280. Additionally, the return value is always an int.SSLObject.shared_ciphers()andSSLSocket.shared_ciphers()methods return the list of ciphers sent by the client during the handshake.SSLSocket.do_handshake(),SSLSocket.read(),SSLSocket.shutdown(), andSSLSocket.write()methods of theSSLSocketclass no longer reset the socket timeout every time bytes are received or sent.match_hostname()function now supports matching of IP addresses.set_coroutine_wrapper()function allows setting a global hook that will be called whenever a coroutine object is created by an async def function. A correspondingget_coroutine_wrapper()can be used to obtain a currently set wrapper.is_finalizing()function can be used to check if the Python interpreter is shutting down.monotonic()function is now always available.(Changes to non-built-in modules will need to be documented elsewhere.)
The above list should be edited if/when progress is made on a given feature.