json.dumps super-linear slowdown on large lists (100× slower than N individual calls)

@ntoll

Port, board and/or hardware

webassembly port, via PyScript 2026.2.1 in browser (no physical board)

MicroPython version

MicroPython v1.27.0 (2025-12-10), via PyScript 2026.2.1 in browser

Reported originally in the PyScript Discord. @ntoll asked me to open this here; Damien George has reportedly reproduced the issue and has a candidate fix in hand.

Reproduction

MicroPython json.dumps performance issue — minimal reproducer

index.html

=============================================================

Summary

json.dumps(lst) where lst is a single list of 50,000 ~50-character
strings takes roughly 15-25 seconds in MicroPython (WASM build,
as used by PyScript).

The equivalent work split across 50,000 separate json.dumps() calls
on individual strings (then joined manually) takes well under a
second on the same runtime. Same inputs, same total output bytes,
~100× speed difference.

This suggests the slowdown is inside a single json.dumps call and
grows superlinearly with the output size produced by that call,
rather than with the number of items encoded.

Environment

PyScript 2026.2.1 (MicroPython build)
Load this file via any static HTTP server and open in a browser.
Results print both on-page and to the browser console.

Notes

CPython 3.12 completes TEST A in ~50 ms. The discrepancy is not
inherent to Python-level JSON encoding; it appears specific to
MicroPython's json.dumps implementation.
No special characters (quotes, backslashes, control chars) appear
in the input strings. The slowdown is NOT driven by escape handling.
The output size is ~2.5 MB in every test below that uses the full
dataset, so results across tests are directly comparable.

Expected behaviour

CPython 3.12 completes TEST A in ~50 ms. The discrepancy is not inherent to Python-level JSON encoding; it appears specific to MicroPython's json.dumps implementation.

Observed behaviour

Building input data...
_io.js:31 50000 strings, avg length 48 chars
_io.js:31
_io.js:31 TEST A: json.dumps(list_of_50000_strings) [single call]
_io.js:31 13633.6 ms json.dumps(lst) output_bytes=2600000
_io.js:31
_io.js:31 TEST B: 50000x json.dumps(one_string), joined manually
_io.js:31 167.9 ms 50000x json.dumps(str) output_bytes=2550001
_io.js:31
_io.js:31 TEST C: json.dumps on a small slice (10% of the data)
_io.js:31 141.7 ms json.dumps(5000 strings) output_bytes=260000
_io.js:31
_io.js:31 TEST D: json.dumps on a tiny slice (1% of the data)
_io.js:31 1.8 ms json.dumps(500 strings) output_bytes=26000
_io.js:31
_io.js:31 correctness: TEST A and TEST B produce equal output? False
_io.js:31 output size: 2600000 bytes
_io.js:31
_io.js:31 Expected pattern if the issue reproduces:
_io.js:31 - TEST A is orders of magnitude slower than TEST B,
_io.js:31 despite producing identical output from identical inputs.
_io.js:31 - TEST C / TEST D scale faster than linearly with input size,
_io.js:31 suggesting super-linear cost inside a single json.dumps call.
_io.js:31
_io.js:31 Done.

Small correction to the reproducer output above: the correctness: TEST A and TEST B produce equal output? False line and the ~50KB difference in output_bytes (2,600,000 vs 2,550,001) are due to a bug in my manual-join code in TEST B, not in json.dumps. Likely a separator-handling detail — I haven't tracked it down.

Additional Information

No, I've provided everything above.

Code of Conduct

Yes, I agree

Thanks for the clear bug report. I can reproduce the behaviour on the unix port of MicroPython (which makes debugging and fixing much easier). The following takes a long time:

import json
lst = ["a" * 50] * 50000
s = json.dumps(lst)
print(len(s))

The root cause is the slow growth of the variable string vstr_t when extending it as the json output is created. Each time more memory is needed to store the next bit of the json output, only 16 bytes are added to the output string. So to create an output of 2.5Mb requires about 150k memory allocations, which is what slows things down substantially.

The fix... would be to adjust the memory allocation scheme used by vstr_ensure_extra(). But that requires a bit of thought on the best approach.

Thank you for the informative post! This explains a lot of things.

I was running this in the browser (PyScript 2026.2.1) so for this specific environment, the way I am currently working around it for very large objects is to use to_js + stringify, in case somebody runs into this.

   from pyscript import window
   from pyscript.ffi import to_js
   
   _stringify = window.eval("(function(a) { return JSON.stringify(a); })")
   
   def fast_dumps(obj):
       return _stringify(to_js(obj))

This is reminiscent of #16316

Navigation Menu

Repository navigation

json.dumps super-linear slowdown on large lists (100× slower than N individual calls) #19147

Port, board and/or hardware

MicroPython version

Reproduction

Summary

Environment

Notes

Expected behaviour

Observed behaviour

Additional Information

Code of Conduct

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

json.dumps super-linear slowdown on large lists (100× slower than N individual calls) #19147

Description

Port, board and/or hardware

MicroPython version

Reproduction

Summary

Environment

Notes

Expected behaviour

Observed behaviour

Additional Information

Code of Conduct

Activity

dpgeorge commented on Apr 23, 2026

nitau7 commented on Apr 24, 2026

jburgy commented on May 9, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions