This repository was archived by the owner on Mar 6, 2026. It is now read-only.
Dependency version mismatch on pyarrow #1912
Description
version 3.22.0 uses ArrowDtype which is only available in pyarrow>=10.0.1, but dependency requirement specifies pyarrow >= 3.0.0.
Environment details
- OS type and version: Darwin mac-mini.lan 20.6.0 Darwin Kernel Version 20.6.0 (also on GCP AppEngine)
- Python version: Python 3.10.11
- pip version: pip 23.3.1
google-cloud-bigqueryversion: 3.22.0
Steps to reproduce
note: google-cloud-aiplatform requires pyarrow 7.0.0 for python 3.10
- install google-cloud-bigquery and google-cloud-aiplatform db-dtypes
- call to_dataframe on RowIterator
Code example
# example
client = bigquery.Client()
df = bqclient.query(sql_statement).result().to_dataframe()Stack trace
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/google/cloud/bigquery/table.py", line 2283, in to_dataframe
range_date_dtype = pandas.ArrowDtype(
File "/layers/google.python.pip/pip/lib/python3.10/site-packages/pandas/core/dtypes/dtypes.py", line 2096, in __init__
raise ImportError("pyarrow>=10.0.1 is required for ArrowDtype")
ImportError: pyarrow>=10.0.1 is required for ArrowDtype"
Activity
tswast commentedon May 7, 2024
What version of pandas do you have?
I suspect older versions of pandas may be compatible with older pyarrow.
naoki-jarvisml commentedon May 8, 2024
I have pandas 2.2.2. I've reported this issue since pip's dependency resolution didn't prevent this issue from happening. I'd expect google-cloud-bigquery 3.22.0 to require pyarrow>=10.0.1, or older version of pandas which maybe compatible with this change assuming you're right about it.
tswast commentedon May 8, 2024
We test with minimum versions of all of our dependencies to make sure they are compatible and the latest versions of all of our dependencies.
For the in-betweens we have some version detection logic throughout the code, but it sounds like we need to extend it for this case.
Linchin commentedon May 8, 2024
Thanks for raising the issue! Indeed I only accounted for when pandas is an older version. I will add a min version check for pyarrow too.