Skip to content

--config-file incorrectly expects [tool.pytest] instead of [pytest] in TOML config on pytest 9 #14705

Description

@francis-clairicia

Description

After a migration from a custom config file pytest-benchmark.ini:

[pytest]
python_files = bench_*.py
python_classes = Bench
python_functions = bench_*
addopts =
    --strict-markers
    --benchmark-warmup=on
    --benchmark-warmup-iterations=10000
    --benchmark-timer=time.process_time
    --benchmark-time-unit=us
    --benchmark-min-time=0.000002
    --benchmark-sort=fullname
    --benchmark-disable-gc
minversion = 7.4
testpaths = benchmarks

... to pytest-benchmark.toml with the help of the documentation:

[pytest]
minversion = "9.0"
python_files = ["bench_*.py"]
python_classes = ["Bench"]
python_functions = ["bench_*"]
strict_config = true
strict_markers = true
strict_parametrization_ids = true
addopts = [
    "--benchmark-warmup=on",
    "--benchmark-warmup-iterations=10000",
    "--benchmark-timer=time.process_time",
    "--benchmark-time-unit=us",
    "--benchmark-min-time=0.000002",
    "--benchmark-sort=fullname",
    "--benchmark-disable-gc",
]
testpaths = ["benchmarks"]

... launching pytest -c pytest-benchmark.toml --collect-only ignores the benchmark files defined and uses the default configuration values.

I tried to put the configuration in [tool.pytest] and it worked:

- [pytest]
+ [tool.pytest]
minversion = "9.0"
python_files = ["bench_*.py"]
python_classes = ["Bench"]
python_functions = ["bench_*"]
strict_config = true
strict_markers = true
strict_parametrization_ids = true
addopts = [
    "--benchmark-warmup=on",
    "--benchmark-warmup-iterations=10000",
    "--benchmark-timer=time.process_time",
    "--benchmark-time-unit=us",
    "--benchmark-min-time=0.000002",
    "--benchmark-sort=fullname",
    "--benchmark-disable-gc",
]
testpaths = ["benchmarks"]

Environment

  • pytest: 9.1.1
  • OS: Linux

Activity

  1. The-Compiler commented on Jul 14, 2026

    @The-Compiler
    Member

    Good catch, indeed the code hardcodes the names for [pytest] headers:

    # '.toml' files are considered if they contain a [tool.pytest] table (toml mode)
    # or [tool.pytest.ini_options] table (ini mode) for pyproject.toml,
    # or [pytest] table (toml mode) for pytest.toml/.pytest.toml.
    elif filepath.suffix == ".toml":
    if sys.version_info >= (3, 11):
    import tomllib
    else:
    import tomli as tomllib
    toml_text = filepath.read_text(encoding="utf-8")
    try:
    config = tomllib.loads(toml_text)
    except tomllib.TOMLDecodeError as exc:
    raise UsageError(f"{filepath}: {exc}") from exc
    # pytest.toml and .pytest.toml use [pytest] table directly.
    if filepath.name in ("pytest.toml", ".pytest.toml"):
    if "pytest" in config:
    # TOML mode - preserve native TOML types.
    return {
    k: ConfigValue(v, origin="file", mode="toml")
    for k, v in config["pytest"].items()
    }
    top_level_options = [
    key for key, value in config.items() if not isinstance(value, dict)
    ]
    if top_level_options:
    raise UsageError(
    f"{filepath}: pytest configuration must be under a "
    f"[pytest] table (found top-level options: "
    f"{', '.join(top_level_options)})"
    )
    # "pytest.toml" files are always the source of configuration, even if empty.
    return {}
    # pyproject.toml uses [tool.pytest] or [tool.pytest.ini_options].
    else:
    tool_pytest = config.get("tool", {}).get("pytest", {})
    # Check for toml mode config: [tool.pytest] with content outside of ini_options.
    toml_config = {k: v for k, v in tool_pytest.items() if k != "ini_options"}
    # Check for ini mode config: [tool.pytest.ini_options].
    ini_config = tool_pytest.get("ini_options", None)
    if toml_config and ini_config:
    raise UsageError(
    f"{filepath}: Cannot use both [tool.pytest] (native TOML types) and "
    "[tool.pytest.ini_options] (string-based INI format) simultaneously. "
    "Please use [tool.pytest] with native TOML types (recommended) "
    "or [tool.pytest.ini_options] for backwards compatibility."
    )
    if toml_config:
    # TOML mode - preserve native TOML types.
    return {
    k: ConfigValue(v, origin="file", mode="toml")
    for k, v in toml_config.items()
    }
    elif ini_config is not None:
    # INI mode - TOML supports richer data types than INI files, but we need to
    # convert all scalar values to str for compatibility with the INI system.
    def make_scalar(v: object) -> str | list[str]:
    return v if isinstance(v, list) else str(v)
    return {
    k: ConfigValue(make_scalar(v), origin="file", mode="ini")
    for k, v in ini_config.items()
    }

    The comments sound a bit like this is intentional, but I'm not entirely sure it is. In any case, fixing it in a backwards-compatible way is unfortunately going to make that logic even more complex, most likely it would need to consider both [tool.pytest] and [pytest] in any custom TOML files.

  2. francis-clairicia commented on Jul 14, 2026

    @francis-clairicia
    Author

    I don't think the fix needs to be backward-compatible because:

    1. the behavior has never been documented in this way,
    2. even with the ini format, the section [tool:pytest] is read only with the setup.cfg file.

    Until now, the "tool" section has always been dedicated to the project file.

  3. The-Compiler commented on Jul 14, 2026

    @The-Compiler
    Member

    From a cursory GitHub code search, I suppose it might indeed be fine. Still a corner case though.

  4. molloyzak13 commented on Jul 14, 2026

    @molloyzak13

    reply_target: #14705
    The comment at line 128 names pyproject.toml only (it says the else branch handles "pyproject.toml" and reads from [tool.pytest]) but the condition at line 109 checks for pytest.toml and .pytest.toml. A custom TOML file passed with --config-file falls into that else branch and reads the wrong section. The comments document a three-way split but the branching only implements two.

    PR #14707 has the right approach, keeping the [tool.pytest] lookup for pyproject.toml and routing custom TOML files to [pytest].

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions