Skip to content

Explicitly using fsspec and zarr FsspecStore causes RuntimeError "Task attached to a different loop" #3487

Description

@dmitriyrepin

Summary

Mixing explicit use of fsspec and zarr FsspecStore produces the following error:
'RuntimeError: Task <Task pending ... > attached to a different loop. Task was destroyed but it is pending!'

Reproducers:

  1. When one explicitly creates fsspec.AbstractFileSystem and then uses it in zarr.create_array(), he/she will observe error mentioned above
    gs_file = "gs://my_data/array.tmp"

    fs, url = fsspec.url_to_fs(gs_file)
    store1: fsspec.AbstractFileSystem = fs.get_mapper(url)
    arr1 = zarr.create_array(store1, shape=(2, 2), dtype='int32', overwrite=True)
    arr1[:] = np.array([[1, 2], [3, 4]], dtype='int32')
  1. When one explicitly uses fsspec functionality and then uses zarr.create_array(), he/she will observe error mentioned above:
    gs_file = "gs://my_data/array.tmp"

    fs: fsspec.AbstractFileSystem = fsspec.filesystem('gs')
    info = fs.info(gs_file)

    # Create a zarr array in the cloud
    store1: fsspec.AbstractFileSystem = zarr.storage.FsspecStore.from_url(gs_file, read_only=False)
    arr1 = zarr.create_array(store1, shape=(2, 2), dtype='int32', overwrite=True)
    arr1[:] = np.array([[1, 2], [3, 4]], dtype='int32')

Suspected cause:

When one explicitly uses fsspec, it will create asyncio thread with an event loop (the fsspecIO thread will also be created).
If one later would try to use the zarr FsspecStore, which under the hood also uses fsspec, it will create the second asyncio thread with the second event loop (the zarr_io thread will also be created), which likely causes the error down the road

Image

Versions

  • zarr: 3.1.2
  • Python: 3.13.5
  • fsspec: 2025.9.0
  • gcsfs: 2025.9.0

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions