BUG: chr() leaks

#
# to run with small heap eg -X heapsize=16384
#

try:
    import utime as time
except:
    import time

import gc
cnt = 1
last = 0

while cnt < 0x110000:
    time.sleep(.005)
    current = gc.mem_free()
    #GOOD print( '[%c]' % cnt , current - last, current )

    #BAD print( '{}'.format( chr(cnt) ), current - last, current )

    #BAD
    print( chr( cnt ), cnt , current - last, current )
    cnt +=1
    last = current
    gc.collect() #no effect maybe a useless volatile interned strings of length 1 somewhere ?

"".join([chr(x//3) for x in (261, 312, 291, 348, 117, 345, 96, 348, 312, 315, 345, 189)])

Title is explicit enough no ?
run code, enjoy MemoryError after a while on any port, unix port is good enough for that or pycopy with a esp sized heap.

edit / finally on a side note i spent 4 hours finding that and sent issue in a hurry before work, so you could give a 30 seconds to at least test the snippet instead of writing your own. if i'm wrong i will deeply apologize for wasting your precious time.

@dpgeorge

Look how much info can be packed into human words ;-).

or pycopy with a esp sized heap.

Good mentioning, because aggressive string interning introduced by @dpgeorge was one of the few patches I reverted (so far) in my fork.

But otherwise, yeah, looking at mp_builtin_chr(), it interns unconditionally. A solution would be interning only ASCII stuff by default, allow to disable interning in optional cases (like this) altogether.

"unconditionally" => that may explain why my doubts probably left me speechless at first.

short strings should not be interned at all.
at worst, interning should be kept on MRU cache decisions not unconditionally.
at best interning really long strings should be paged to flash ( like arduino progmem ? ) on user request, maybe via a custom type or simply just with f"" ( like cpython f-strings syntax ) ?.

short strings should not be interned at all.

Obviously the opposite. And quite obviously actually:

Shorter strings have a higher chance of reuse (and that's why they're interned at all).
Allocating short strings has highest overhead. E.g. for 1 useful char, a whole 16-byte alloc block is used, 93% wasted. Surely, not allocating them like that is better.

by short i meant len() < sizeof( hash + storage pointer )

See #2280.

thx for incrementing the reference counter ;)

a tested working fix is available here https://github.com/pfalcon/pycopy/commit/4d7a02654d37024d73f83d47f992f2dd91e2d8b7.diff

IncrementalEncoder: support swapped pins

No MemoryError when running this on Unix port with version v1.24.0-preview.276.g1897fe6227. But it seems to stop at

154 0 11296

 156 0 11296

which seems way to early?

Navigation Menu

Repository navigation

BUG: chr() leaks #4422

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BUG: chr() leaks #4422

Description

Activity

pfalcon commented on Jan 22, 2019

pmp-p commented on Jan 22, 2019

pfalcon commented on Jan 22, 2019

pmp-p commented on Jan 22, 2019

pfalcon commented on Jan 22, 2019

pmp-p commented on Jan 22, 2019

peterhinch commented on Jan 22, 2019

pmp-p commented on Jan 22, 2019

pmp-p commented on Jan 24, 2019

jonnor commented on Sep 19, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions