Support Traversable for json/toml/yaml config file sources (#299) by hramezani · Pull Request #902 · pydantic/pydantic-settings

hramezani · 2026-07-07T10:02:40Z

Summary

Closes #299.

Adds typing and runtime support for passing a Traversable (e.g. importlib.resources.files(...).joinpath(...)) as json_file, toml_file, or yaml_file. This enables loading a configuration file that is packaged inside a distribution — including resources that don't live on the filesystem, such as a file inside a zip/wheel — without a cast to Path.

Background

Issue #299 raised two points:

File options don't type-check against lists — already resolved: PathType includes Sequence[...].
PathType should include Traversable — still a real typing gap. The issue also noted that a Traversable "currently works" at runtime. That is now only partially true: a genuine non-Path Traversable (a zip/wheel-packaged resource) still works for JSON/YAML, but regressed for TOML — TomlConfigSettingsSource._any_file_exists wraps every path in Path(path), which raises TypeError for an object that isn't os.PathLike.

Changes

Introduce a dedicated ConfigFileSourceType alias (Path | str | Traversable | Sequence[Path | str | Traversable]) and apply it to the config-file fields of SettingsConfigDict and to the Json/Toml/Yaml *ConfigSettingsSource constructors (and the shared _read_files/_read_file). PathType — used by secrets_dir — is intentionally left unchanged so directory handling keeps its Path-only semantics.
Fix TomlConfigSettingsSource._any_file_exists to query a non-Path Traversable directly via is_file() instead of coercing through Path(...). This also fixes a latent bug where a single str path was iterated character-by-character.
Version-guard the Traversable import (importlib.resources.abc on 3.11+, importlib.abc on 3.10), since importlib.abc.Traversable was removed in 3.12+.
Export ConfigFileSourceType from the package.
Add a docs note under the config-file sources section.

Tests

New zip-packaged (genuine non-Path) Traversable regression tests for the TOML source (plain load and toml_table_header) and the YAML source.
The JSON source already had TestTraversableSupport; updated its now-stale "not added in annotation" comment.

Verified locally: mypy clean (no new errors over baseline), ruff clean, full test suite green (excluding pre-existing Windows-only symlink test failures unrelated to this change).

🤖 Generated with Claude Code

Add typing and runtime support for passing a `Traversable` (e.g. `importlib.resources.files(...).joinpath(...)`) as `json_file`, `toml_file`, or `yaml_file`, so packaged configuration resources no longer need a `cast` to `Path`. - Introduce a dedicated `ConfigFileSourceType` alias (`Path | str | Traversable | Sequence[...]`) and apply it to the config-file fields of `SettingsConfigDict` and the `*ConfigSettingsSource` constructors. `PathType` (used by `secrets_dir`) is left unchanged. - Fix `TomlConfigSettingsSource._any_file_exists`, which coerced every path via `Path(path)` and raised `TypeError` for a non-`Path` `Traversable` such as a resource inside a zip/wheel. It now queries such objects directly via `is_file()`, and also correctly handles a single `str` path. - Guard the `Traversable` import by version (`importlib.resources.abc` on 3.11+, `importlib.abc` on 3.10). - Add zip-packaged (genuine non-`Path`) Traversable regression tests for the toml and yaml sources, and a docs note. Co-Authored-By: Claude Opus 4.8 <[email protected]>

Copilot

Pull request overview

This PR extends pydantic-settings’ config-file sources (JSON/TOML/YAML) to accept importlib.resources Traversable objects (including non-filesystem resources like files inside a zip/wheel), aligning both typing and runtime behavior and adding regression coverage for TOML/YAML.

Changes:

Introduces ConfigFileSourceType (includes Traversable) and applies it to SettingsConfigDict and the JSON/TOML/YAML config sources.
Updates shared config-file reading logic to operate on Path | Traversable and fixes TOML existence checks for non-Path Traversable.
Adds docs note and new regression tests for zip-packaged Traversable TOML/YAML loading.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/test_source_yaml.py	Adds regression test for loading YAML from a zip-packaged non-`Path` `Traversable`.
tests/test_source_toml.py	Adds `Traversable` fixture + TOML regression tests; note: new skip conditions need adjustment.
tests/test_source_json.py	Updates comment to reflect `Traversable` is now in the annotation.
pydantic_settings/sources/types.py	Adds `ConfigFileSourceType` and version-guarded `Traversable` import; exports the alias.
pydantic_settings/sources/providers/yaml.py	Switches `yaml_file` typing to `ConfigFileSourceType` and supports `Traversable` in `_read_file`.
pydantic_settings/sources/providers/toml.py	Switches `toml_file` typing and fixes `_any_file_exists` for `Traversable` and single-string inputs.
pydantic_settings/sources/providers/json.py	Switches `json_file` typing and supports `Traversable` in `_read_file`.
pydantic_settings/sources/base.py	Updates `ConfigFileSourceMixin` to read `Path
pydantic_settings/sources/init.py	Re-exports `ConfigFileSourceType` from `pydantic_settings.sources`.
pydantic_settings/main.py	Updates `SettingsConfigDict` field types for json/toml/yaml to `ConfigFileSourceType`.
docs/index.md	Documents `Traversable` support for `json_file`/`toml_file`/`yaml_file`.

Comments suppressed due to low confidence (1)

pydantic_settings/sources/init.py:46

ConfigFileSourceType is exported from pydantic_settings.sources, but it’s not currently re-exported from the top-level pydantic_settings package. If the intent is “export from the package” (as stated in the PR description), add it to pydantic_settings/__init__.py (import + __all__) so from pydantic_settings import ConfigFileSourceType works.

from .types import (
    DEFAULT_PATH,
    ENV_FILE_SENTINEL,
    ConfigFileSourceType,
    DotenvFiltering,
    DotenvType,
    EnvPrefixTarget,
    ForceDecode,
    NoDecode,
    PathType,
    PydanticModel,
)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Use `sys.version_info < (3, 11)` (matching `import_toml`, where `tomli` is only needed below 3.11) instead of `<= (3, 11)` for the new Traversable test skip conditions, addressing PR review feedback. Co-Authored-By: Claude Opus 4.8 <[email protected]>

hramezani · 2026-07-07T10:18:43Z

Thanks for the review! Addressed in 450bf45:

Skip condition off-by-one (both TOML tests): Changed sys.version_info <= (3, 11) → < (3, 11) for the two new Traversable tests, matching the guard in import_toml() (where tomli is only required below 3.11). In practice sys.version_info always carries a micro version, so <= (3, 11) never actually matched a 3.11.x release, but < (3, 11) is the clearer and correct form.
ConfigFileSourceType top-level export (suppressed note): Kept the export at pydantic_settings.sources only, for consistency with the sibling aliases PathType and DotenvType, which are likewise not re-exported from the top-level pydantic_settings package. "Export from the package" in the description referred to pydantic_settings.sources.

hramezani · 2026-07-08T07:43:51Z

@copilot resolve the merge conflicts in this pull request

…onfig-files # Conflicts: # pydantic_settings/sources/providers/json.py # pydantic_settings/sources/providers/toml.py # pydantic_settings/sources/providers/yaml.py

Copilot · 2026-07-08T07:46:38Z

@copilot resolve the merge conflicts in this pull request

Resolved in 74f8bfa. The only conflicts were in the import sections of json.py, toml.py, and yaml.py — main's #901 added from ..utils import InitState while our branch replaced PathType with ConfigFileSourceType. The resolution keeps both imports. All JSON/TOML/YAML tests pass.

hramezani requested a review from Copilot July 7, 2026 10:06

Copilot started reviewing on behalf of hramezani July 7, 2026 10:07

View session

Copilot AI reviewed Jul 7, 2026

View reviewed changes

Comment thread tests/test_source_toml.py Outdated

Comment thread tests/test_source_toml.py Outdated

hramezani mentioned this pull request Jul 7, 2026

Typing for multiple PathTypes (and support Traversable) for SettingsConfigDict json_file, toml_file, yaml_file, etc #299

Closed

Copilot started work on behalf of hramezani July 8, 2026 07:44

View session

Merge remote-tracking branch 'origin/main' into support-traversable-c…

74f8bfa

…onfig-files # Conflicts: # pydantic_settings/sources/providers/json.py # pydantic_settings/sources/providers/toml.py # pydantic_settings/sources/providers/yaml.py

Copilot finished work on behalf of hramezani July 8, 2026 07:47

hramezani merged commit d51a822 into main Jul 8, 2026
19 checks passed

hramezani deleted the support-traversable-config-files branch July 8, 2026 07:54

hramezani mentioned this pull request Aug 7, 2026

Prepare release 2.15.0 #930

Merged

Navigation Menu

Repository navigation

Support Traversable for json/toml/yaml config file sources (#299) - #902

Support Traversable for json/toml/yaml config file sources (#299)#902
hramezani merged 3 commits into
mainfrom
support-traversable-config-files

hramezani commented Jul 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

hramezani commented Jul 7, 2026

Uh oh!

hramezani commented Jul 8, 2026

Uh oh!

Copilot AI commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

hramezani commented Jul 7, 2026

Summary

Background

Changes

Tests

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

hramezani commented Jul 7, 2026

Uh oh!

hramezani commented Jul 8, 2026

Uh oh!

Copilot AI commented Jul 8, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants