Detect blocking calls in coroutines using BlockBuster by cbornet · Pull Request #2858 · Kludex/starlette

cbornet · 2025-02-01T10:55:52Z

Summary

This PR uses the blockbuster library to detect blocking calls made in the asyncio event loop during unit tests.
Avoiding blocking calls is hard as these can be deeply buried in the code or made in 3rd party libraries.
Blockbuster makes it easier to detect them by raising an exception when a call is made to a known blocking function (eg: time.sleep).

Checklist

I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
I've updated the documentation accordingly.

cbornet · 2025-02-01T10:58:34Z

            self.size += len(data)

-        if self._in_memory:
+        if self._in_memory and self.file.tell() + len(data) <= self.file._max_size:


The SpooledTemporaryFile rollover is blocking

Should come soon: agronholm/anyio#867

cbornet · 2025-02-01T11:03:12Z

                return {"type": "http.disconnect"}

-            body = request.read()
+            body = await anyio.to_thread.run_sync(request.read)


Httpx request.read can be blocking (eg: reading multipart file).
await request.aread() can't be used as-is since the request stream is a SyncByteStream not an AsyncByteStream.

IMO, this is not a problem in unit tests and we can neglect it.

cbornet · 2025-02-01T11:03:34Z

        file.write("<html>Hello, <a href='{{ url_for('homepage') }}'>world</a></html>")

-    async def homepage(request: Request) -> Response:
+    def homepage(request: Request) -> Response:


Is it OK to change ?
templates.TemplateResponse is blocking.

Why is templates.TemplateResponse blocking?

It loads the index.html from the FS using jinja2.FileSystemLoader.get_source() which does blocking calls.
The path is:

test_templates.py:27: in homepage return templates.TemplateResponse(request, "index.html") ../starlette/templating.py:208: in TemplateResponse template = self.get_template(name) ../starlette/templating.py:131: in get_template return self.env.get_template(name) ../venv/lib/python3.11/site-packages/jinja2/environment.py:1016: in get_template return self._load_template(name, globals) ../venv/lib/python3.11/site-packages/jinja2/environment.py:975: in _load_template template = self.loader.load(self, name, self.make_globals(globals)) ../venv/lib/python3.11/site-packages/jinja2/loaders.py:126: in load source, filename, uptodate = self.get_source(environment, name) ../venv/lib/python3.11/site-packages/jinja2/loaders.py:204: in get_source if os.path.isfile(filename):

path.isfile is a blocking call as it does an os.stat.
The get_source code also does a file.read() blocking call later.

Jinja loads it only once, and then it is served from the in-memory cache.

Yes. For production code, it’s up to the user to see if it’s a perf issue or not.
Here I’m modifying the test code so that the test passes with BlockBuster on.
I think it doesn’t change the test itself and what it is actually testing. Is it OK?

cbornet · 2025-02-01T11:07:54Z

+@pytest.fixture(autouse=True)
+def blockbuster(request):
+    with blockbuster_ctx("starlette") as bb:
+        bb.functions["os.stat"].can_block_in("/mimetypes.py", "init")


FileResponse's constructor calls mimetypes .guess_type which is blocking the first time it is called.

Kludex

I'm not keen to add a dependency, but seems a cool library.

Kludex · 2025-02-01T13:32:23Z

            self.size += len(data)

-        if self._in_memory:
+        if self._in_memory and self.file.tell() + len(data) <= getattr(self.file, "_max_size", sys.maxsize):


what's this about?

If the data to write makes the file exceed the SpooledTemporaryFile _max_size, self.file.write will do a blocking rollover operation.

Other than accessing the private attribute this makes sense to me. Insofar as it's an optimization that fails gracefully I think it would be okay to do this.

Kludex · 2025-02-01T13:34:02Z

            # A timeout is set for 0.1 second in order to ensure that
            # we never deadlock the test run in an infinite loop
-            with anyio.move_on_after(0.1):
+            with anyio.move_on_after(0.2):


BlockBuster has a small toll on performance.
In CI, the test machines are already slow, the cancellation sometimes arrives too late and the test becomes flaky (I had no issue on my computer, only in CI).

Kludex · 2025-02-01T13:34:20Z

        file.write("<html>Hello, <a href='{{ url_for('homepage') }}'>world</a></html>")

-    async def homepage(request: Request) -> Response:
+    def homepage(request: Request) -> Response:


Why is templates.TemplateResponse blocking?

adriangb · 2025-02-01T15:48:55Z

@cbornet I'm curious about the choice to call out file IO as blocking. In my experience file IO is tricky:

The issue with blocking calls is not if they are theoretically blocking or not, the issue is how long they block for. E.g. sorting a list is blocking work but as long as it's <1ms it's probably fine. When you start getting into multi ms blocks is when things get ugly since IO protocols that expect a regular tick and such may start timing out, causing cascading failures.
File IO can of course block for a long time (e.g. reading GBs of data in one go) but it can also be very fast (small operation on an SSD disk). And it's hard to know ahead of time which one it will be.
The overhead of starting up threads is non negligible, and anyio's to_thread has a relatively low semaphore (40 last time I checked) which is easy to accidentally exhaust, possibly leading to deadlocks.

Because of this my general approach has been to be conservative with chucking things into threads, especially file IO since it's often not a problem in practice.

cbornet · 2025-02-01T19:37:49Z

@adriangb thanks for your feedback.
I agree with you that the impact of blocking calls depends a lot on the time it blocks. But how to know in advance ? Things can run perfectly well on your laptop or CI with SSDs and fall apart when you deploy to AWS with EFS slow network disks (bad recent personal experience).
I’ve seen that in general file ops are deferred to threads (or if using Linux, you can use aiofile which has true async support).
And I see that starlette already does it in a bunch of places (eg: UploadFile.write)
Starting a thread has a cost but I think anyio uses a thread pool ?
Anyway, if you don’t want to use threads, it’s possible to:

set exemptions in the places detected by blockbuster. This way, you still have a kind of warning that something could be better.
or completely disable file IO blockings detection by blockbuster.

cbornet · 2025-02-24T17:20:09Z

Hi. Could you provide guidance on what I should do for this PR ?

Co-authored-by: Marcelo Trylesinski <[email protected]>

cbornet · 2025-04-06T16:29:35Z

Hi. Could you provide guidance on what I should do for this PR ?

Hi. I rebased. Is there anything I can do to make progress on this PR ?

codspeed · 2026-09-06T14:34:10Z

Merging this PR will degrade performance by 10.81%

⚠️

Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 241 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
❌	Simulation	`test_multipart[boundary-like-file]`	8.4 ms	9.4 ms	-10.81%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing cbornet:blockbuster (cc34d6d) with main (0d9c288)}

cubic-dev-ai

All reported issues were addressed across 6 files

_{Reply with feedback, questions, or to request a fix.

Re-trigger cubic}

Kludex · 2026-09-06T15:04:40Z

Sorry for taking so long here @cbornet .

Thanks for your work on BlockBuster. I've also added it to https://github.com/pydantic/pydantic-ai. 🙏

cbornet commented Feb 1, 2025

View reviewed changes

cbornet force-pushed the blockbuster branch from 4628539 to 5465945 Compare February 1, 2025 11:16

cbornet mentioned this pull request Feb 1, 2025

✅ Detect blocking calls in coroutines using BlockBuster fastapi/fastapi#13290

Closed

cbornet force-pushed the blockbuster branch 4 times, most recently from 09e27f0 to 8a12d86 Compare February 1, 2025 12:01

cbornet closed this Feb 1, 2025

cbornet reopened this Feb 1, 2025

cbornet closed this Feb 1, 2025

cbornet reopened this Feb 1, 2025

cbornet closed this Feb 1, 2025

cbornet reopened this Feb 1, 2025

cbornet closed this Feb 1, 2025

cbornet reopened this Feb 1, 2025

cbornet force-pushed the blockbuster branch 7 times, most recently from d9da912 to 34fc433 Compare February 1, 2025 13:34

Kludex reviewed Feb 1, 2025

View reviewed changes

cbornet force-pushed the blockbuster branch 2 times, most recently from 5ca95ab to b139473 Compare February 24, 2025 17:17

cbornet force-pushed the blockbuster branch from b139473 to e1c88ad Compare February 24, 2025 17:17

cbornet and others added 3 commits April 6, 2025 18:26

Detect blocking calls in coroutines using BlockBuster

b936193

Update .gitignore

b6572b3

Co-authored-by: Marcelo Trylesinski <[email protected]>

Bump blockbuster to 1.5.23

241c7e6

cbornet force-pushed the blockbuster branch from e1c88ad to 241c7e6 Compare April 6, 2025 16:27

Merge origin/main and support BlockBuster on Python 3.15

6d0bbd8

Allow blocking request reads in TestClient

b2b8370

cubic-dev-ai Bot reviewed Sep 6, 2026

View reviewed changes

Comment thread tests/conftest.py

Reuse run_in_threadpool for debug responses

cc34d6d

Kludex enabled auto-merge (squash) September 6, 2026 15:04

Kludex merged commit 4ac5302 into Kludex:main Sep 6, 2026
12 checks passed

cbornet deleted the blockbuster branch September 6, 2026 16:35

Conversation

cbornet commented Feb 1, 2025

Summary

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cbornet Feb 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Kludex left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cbornet Feb 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adriangb commented Feb 1, 2025

Uh oh!

cbornet commented Feb 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cbornet commented Feb 24, 2025

Uh oh!

cbornet commented Apr 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed Bot commented Sep 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will degrade performance by 10.81%

Performance Changes

Uh oh!

cubic-dev-ai Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Kludex commented Sep 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cbornet Feb 1, 2025 •

edited

Loading

cbornet Feb 1, 2025 •

edited

Loading

cbornet commented Feb 1, 2025 •

edited

Loading

cbornet commented Apr 6, 2025 •

edited

Loading

codspeed Bot commented Sep 6, 2026 •

edited

Loading

cubic-dev-ai Bot left a comment •

edited

Loading