Skip to content

fix(data-transfer): match export category codes against the keyword subfield - #698

Merged
navneetkumar-pim-webkul merged 2 commits into
3.xfrom
fix/export-category-keyword-subfield
Sep 11, 2026
Merged

navneetkumar-pim-webkul merged 2 commits into
3.xfrom
fix/export-category-keyword-subfield

Conversation

@navneetkumar-pim-webkul

Copy link
Copy Markdown
Collaborator

Description

Fixes #696.

A product export profile with a category filter matched no products. The job finished as completed with {"created": 0, "skipped": 0, "processed": 0} and produced no file, so it read as "the catalogue has no matching products" rather than as a failure.

values.categories is indexed as analyzed text, so the indexed tokens are lowercased. The export cursor filtered it with a terms query, which is term-level and does not analyze its input, so a category code carrying an uppercase letter was compared against a lowercased token and matched nothing.

The clause now targets the exact-value keyword subfield, values.categories.keyword.

The product grid sidesteps the same mapping by lowercasing its values through QueryString::escapeArrayValue(). The export cursor never got that treatment, and ProductCursorBoolQueryTest had no case covering the category clause, so nothing caught it. This adds that case.

Only the Elasticsearch path is affected. The database cursor uses whereJsonContains and was always correct.

The grid's lowercase approach is also fragile for a second reason worth a separate look: the standard analyzer splits on hyphens, so a category code such as summer-sale indexes as two tokens and matches nothing either way. The keyword subfield is exact for both cases.

How To Test This?

  1. Set ELASTICSEARCH_ENABLED=true and index the catalogue.
  2. Have a category whose code contains an uppercase letter, e.g. Footwear, with a few products assigned to it.
  3. Create a product export profile and set Categories to that category.
  4. Run the export. Every product in the category should be exported; before this change the run completed with zero records and no file.
  5. vendor/bin/pest packages/Webkul/DataTransfer/tests/Unit/Helpers/Sources/Export/Elastic/ProductCursorBoolQueryTest.php

Verification

Gate Result
vendor/bin/pint --test passed
vendor/bin/phpstan analyse (level 2) no errors
vendor/bin/rector --dry-run no changes
vendor/bin/pest DataTransfer + ElasticSearch 9 failed, rest passed

Those 9 failures are pre-existing: the same cases fail on unmodified 3.x at d33fc74c, and none touch the cursor.

Note for deployment: PHP changes do not reach queue workers until they restart, so php artisan queue:restart is needed for this to take effect on a running installation.

https://claude.ai/code/session_01JvM9wYbv339YvEnEuvsKrT

…ubfield

values.categories is indexed as analyzed text, so its tokens are lowercased,
while `terms` is a term-level query that does not analyze its input. A product
export filtered on a category code carrying an uppercase letter therefore
matched nothing and completed with an empty file and a "0 records" summary.
Query the exact-value keyword subfield instead.

The product grid's CategoryFilter sidesteps the same mapping by lowercasing its
values through QueryString::escapeArrayValue(); the export cursor never got
that treatment and had no test covering the category clause, so the regression
went unnoticed. Cover it here.

Fixes #696

Claude-Session: https://claude.ai/code/session_01JvM9wYbv339YvEnEuvsKrT
Copilot AI lite review requested due to automatic review settings September 10, 2026 17:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change is narrowly scoped, aligns with Elasticsearch field semantics, and includes a focused regression test for the corrected clause.

Pull request overview

Fixes Elasticsearch-backed product exports where category filters could match zero products due to terms queries targeting an analyzed text field (values.categories) instead of the exact-match keyword subfield.

Changes:

  • Update the Elasticsearch category filter clause to query values.categories.keyword (exact match) instead of values.categories.
  • Add a unit test ensuring the bool query uses the keyword subfield for category code filters.
File summaries
File Description
packages/Webkul/DataTransfer/src/Helpers/Sources/Export/Elastic/ProductCursor.php Switches category terms clause to values.categories.keyword via a constant to ensure exact-match behavior in Elasticsearch exports.
packages/Webkul/DataTransfer/tests/Unit/Helpers/Sources/Export/Elastic/ProductCursorBoolQueryTest.php Adds coverage asserting category filters target the keyword subfield.
Review details
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread packages/Webkul/DataTransfer/src/Helpers/Sources/Export/Elastic/ProductCursor.php Outdated
@navneetkumar-pim-webkul
navneetkumar-pim-webkul merged commit c8e55b2 into 3.x Sep 11, 2026
23 checks passed
@navneetkumar-pim-webkul
navneetkumar-pim-webkul deleted the fix/export-category-keyword-subfield branch September 11, 2026 06:54
navneetkumar-pim-webkul added a commit that referenced this pull request Sep 17, 2026
Eight merged pull requests since v3.1.0 carried no changelog entry: the
role-escalation and MagicAI SSRF fixes (#689, #690, #691), the export
column selection and category keyword matching fixes (#697, #698), the
media replacement fix (#709), the Gemini endpoint (#699), the channel
deletion fix (#688), the purifier cache race (#675), and the admin UI
and configuration work squashed into #700.

3.1.1 is a patch release, so everything is filed under bug fixes and
improvements; no feature section.

The heading is stamped 3.1.1 to match Core::VERSION, which the branch
already carries.
navneetkumar-pim-webkul added a commit that referenced this pull request Sep 17, 2026
Eight merged pull requests since v3.1.0 carried no changelog entry: the
role-escalation and MagicAI SSRF fixes (#689, #690, #691), the export
column selection and category keyword matching fixes (#697, #698), the
media replacement fix (#709), the Gemini endpoint (#699), the channel
deletion fix (#688), the purifier cache race (#675), and the admin UI
and configuration work squashed into #700.

3.1.1 is a patch release, so everything is filed under bug fixes and
improvements; no feature section.

The heading is stamped 3.1.1 to match Core::VERSION, which the branch
already carries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Product export with a category filter returns zero products (Elasticsearch)

2 participants