Reduce memory usage by `Dependant` by ~50% by ipeluffo · Pull Request #15336 · fastapi/fastapi

ipeluffo · 2026-04-14T10:45:29Z

Context

In #14742, I reported a significant spike in memory usage when upgrading FastAPI from v0.120.4 to v0.121.x (and later versions).

My analysis traced the regression to this refactor, which replaced computed fields in __post_init__ with functools.cached_property.

Root cause

The Python docs for functools.cached_property note:

This decorator interferes with the operation of [PEP 412](https://peps.python.org/pep-0412/) key-sharing dictionaries. This means that instance dictionaries can take more space than usual.

While the worst of this was addressed in Python 3.12+ python/cpython#101815, there is still an overhead: accessing __dict__ on an instance forces creation of a full dictionary object rather than a compact PyDictValues, adding a PyDictObject (three fields) per instance. At scale, this adds up.

The exact quote from the cpython issue:

It remains true that cached_property can cause instances to use slightly more memory, but the reason for this has changed (and the extra memory used will be significantly less.) The new reason is that any access of dict on an instance will force creation of a real dictionary object for that instance, rather than just a PyDictValues. The keys remain shared in the created dict, though. So the added memory use is no longer "all the keys" but rather just "a PyDictObject" (which consists of only three fields.)

Benchmarks

I wrote a script to measure memory usage across varying numbers of Dependant instances:

Benchmark script

import gc
import tracemalloc
from fastapi.dependencies.models import Dependant

samples_tests = (
    200_000,
    100_000,
    50_000,
    1_000,
    500,
)

def measure(samples: int):
    gc.collect()
    tracemalloc.start()
    objs = [Dependant() for i in range(samples)]

    # Trigger all cached properties
    for o in objs:
        o.oauth_scopes
        o.cache_key
        o._uses_scopes
        o._is_security_scheme
        # o._security_scheme  # skipped: asserts a SecurityBase, not None
        o._security_dependencies
        o.is_gen_callable
        o.is_async_gen_callable
        o.is_coroutine_callable
        o.computed_scope

    current, peak = tracemalloc.get_traced_memory()
    tracemalloc.stop()
    return current, peak, objs  # keep objs alive until after measurement

for samples in samples_tests:
    cur, peak, _ = measure(samples)
    print(f"Samples: {samples} - current={cur/1e6:.1f} MB  peak={peak/1e6:.1f} MB")

Before (current master, Python 3.13.12):

Samples: 200000 - current=318.4 MB  peak=318.4 MB
Samples: 100000 - current=159.2 MB  peak=159.2 MB
Samples: 50000 - current=79.6 MB  peak=79.6 MB
Samples: 1000 - current=1.6 MB  peak=1.6 MB
Samples: 500 - current=0.8 MB  peak=0.8 MB

After (this PR):

Samples: 200000 - current=156.8 MB  peak=156.8 MB
Samples: 100000 - current=78.4 MB  peak=78.4 MB
Samples: 50000 - current=39.2 MB  peak=39.2 MB
Samples: 1000 - current=0.8 MB  peak=0.8 MB
Samples: 500 - current=0.4 MB  peak=0.4 MB

Memory usage is reduced by approximately 50% across all cases.

Approach

The key enabler for the memory reduction is the use of __slots__. However, __slots__ is not compatible with functools.cached_property, so the caching strategy had to change.

I considered reverting to the previous __post_init__ approach but decided against it for two reasons:

No lazy initialization — all computed fields would be evaluated eagerly on every Dependant instantiation, even if never accessed.
Awkward dependency ordering — several computed fields depend on each other, making __post_init__ harder to maintain.

Instead, this PR introduces a different caching mechanism that preserves lazy evaluation while being compatible with __slots__.

``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

… usage ``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

…ory usage ``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

…ry usage ``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

``` Samples: 200000 - current=318.4 MB peak=318.4 MB Samples: 100000 - current=159.2 MB peak=159.2 MB Samples: 50000 - current=79.6 MB peak=79.6 MB Samples: 1000 - current=1.6 MB peak=1.6 MB Samples: 500 - current=0.8 MB peak=0.8 MB ```

``` Samples: 200000 - current=156.8 MB peak=156.8 MB Samples: 100000 - current=78.4 MB peak=78.4 MB Samples: 50000 - current=39.2 MB peak=39.2 MB Samples: 1000 - current=0.8 MB peak=0.8 MB Samples: 500 - current=0.4 MB peak=0.4 MB ```

codspeed · 2026-04-14T10:47:16Z

Merging this PR will not alter performance

✅ 20 untouched benchmarks

_{Comparing ipeluffo:perf/improve-memory-dependencies (0719e69) with master (0832139)¹}

No successful run was found on master (c61384e) during the generation of this report, so 0832139 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

ipeluffo · 2026-04-14T11:08:45Z

+    unwrapped = inspect.unwrap(cast(Callable[..., Any], _impartial(call)))
    return unwrapped


-def _impartial(func: Callable[..., Any]) -> Callable[..., Any]:
+def _impartial(func: Callable[..., Any] | None) -> Callable[..., Any] | None:


[info] This is unrelated to my PR but mypy was complaining in the pre-commit check. The change looks safe and sensible

gryevns · 2026-04-28T14:37:41Z

@YuriiMotov is there anything needed to move this one forward and get some feedback/review? Thanks

ipeluffo · 2026-05-13T09:44:29Z

@tiangolo , I'd be great if we can some feedback on this before the PR gets lost. Thanks

gryevns · 2026-06-08T08:34:53Z

@tiangolo could you please advise how to move this PR forwards, it's becoming more and more important as we fall behind FastAPI updates. Thanks!

--------- Co-authored-by: Federico Jasson <[email protected]>

github-actions · 2026-07-24T21:12:13Z

This pull request has a merge conflict that needs to be resolved.

tiangolo · 2026-07-24T21:18:00Z

Thank you for the investigation in the original discussion and the work on this PR! 🙌

I used it plus some additional work and reports to base the work in this one #16049

It is now released in FastAPI 0.140.0. 🚀

I'll now close this one. Thank you!

ipeluffo and others added 14 commits April 14, 2026 10:57

Refactor Dependant.oauth_scopes property for improved memory usage

a92e6a3

Fix wrong default

381ffe4

Enable slots option

2daccb0

``` Samples: 200000 - current=156.8 MB peak=156.8 MB Samples: 100000 - current=78.4 MB peak=78.4 MB Samples: 50000 - current=39.2 MB peak=39.2 MB Samples: 1000 - current=0.8 MB peak=0.8 MB Samples: 500 - current=0.4 MB peak=0.4 MB ```

Make cache field typing more accurate

f44a232

🎨 Auto format

b433966

ipeluffo added 2 commits April 14, 2026 12:07

Fix ruff linting issue

17fd86b

Fix mypy linting issue

95cd016

ipeluffo commented Apr 14, 2026

View reviewed changes

ipeluffo marked this pull request as draft April 14, 2026 11:10

ipeluffo marked this pull request as ready for review April 14, 2026 11:12

ipeluffo and others added 2 commits June 18, 2026 17:52

Keep original logic to calculate fields

cb18282

--------- Co-authored-by: Federico Jasson <[email protected]>

🎨 Auto format

0719e69

tiangolo mentioned this pull request Jul 24, 2026

⚡️ Reduce memory usage in dependencies #16049

Merged

3 of 5 tasks

github-actions Bot added the conflicts Automatically generated when a PR has a merge conflict label Jul 24, 2026

tiangolo closed this Jul 24, 2026

Navigation Menu

Repository navigation

Reduce memory usage by `Dependant` by ~50% - #15336

Reduce memory usage by `Dependant` by ~50%#15336
ipeluffo wants to merge 18 commits into
fastapi:masterfrom
ipeluffo:perf/improve-memory-dependencies

ipeluffo commented Apr 14, 2026

Uh oh!

codspeed Bot commented Apr 14, 2026 •

edited

Loading

Uh oh!

ipeluffo Apr 14, 2026 •

edited

Loading

Uh oh!

gryevns commented Apr 28, 2026

Uh oh!

ipeluffo commented May 13, 2026

Uh oh!

gryevns commented Jun 8, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

tiangolo commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ipeluffo commented Apr 14, 2026

Context

Root cause

Benchmarks

Approach

Uh oh!

codspeed Bot commented Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Footnotes

Uh oh!

ipeluffo Apr 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gryevns commented Apr 28, 2026

Uh oh!

ipeluffo commented May 13, 2026

Uh oh!

gryevns commented Jun 8, 2026

Uh oh!

github-actions Bot commented Jul 24, 2026

Uh oh!

tiangolo commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codspeed Bot commented Apr 14, 2026 •

edited

Loading

ipeluffo Apr 14, 2026 •

edited

Loading