feat(magic-ai): output ceiling, platform-aware discovery, scoped translation by navneetkumar-pim-webkul · Pull Request #719 · unopim/unopim

navneetkumar-pim-webkul · 2026-09-22T05:56:57Z

Issue Reference

N/A — no tracking issue.

Description

Magic AI generation, model discovery and product translation work, plus two
small admin UI fixes and one database-portability fix.

Generation ceiling. Adds a "Maximum Output Tokens" setting to the Magic AI
text generation configuration and raises the default from 1024 to 4096. HTML
output spends tokens on markup, so the old ceiling cut a table-heavy
description mid-tag. A migration raises the seeded system prompts still
holding the old value and leaves a ceiling an administrator tuned themselves
alone. A generation that stops on the ceiling is now reported as such instead
of silently returning half a sentence, and a fragment cut mid-tag is dropped
rather than rendered as stray text.

Model discovery. Discovery now runs against the platform's own base URL,
so a proxy or a regional endpoint lists the models it will actually serve. The
OpenAI, Anthropic and Gemini fetches are pinned to the validated address with
redirects barred, matching every other provider — without that they followed
redirects and re-resolved DNS against a caller-supplied host. A custom base
typed without a version segment is retried with one, and a failed fetch
reports the host and status rather than rendering the upstream error page
verbatim. An allow list and an auto-selection limit are configurable through
MAGIC_AI_ALLOWED_MODELS and MAGIC_AI_AUTO_SELECT_LIMIT, and
recommendations are ranked so the handful pre-selected after a fetch are the
cheap, widely used tiers. Adds Concentrate AI as a provider, and an Azure
platform now addresses a model by its configured deployment name.

Translation. AI translation wrote every value into the channel-and-locale
bucket regardless of the attribute's own scope, so a locale-only attribute
landed where nothing would read it. Each value now goes to the bucket its
scope dictates, an attribute that no longer exists is skipped instead of
failing the job, and the translate dialog offers a field only when it is
locale-scoped and holds a value in the source scope. The dialog also names
source channel and locale separately from the targets, shows the value each
attribute currently holds, and warns before replacing existing content.

Other fixes. A database failure in the AI chat no longer surfaces the
connection host, schema and full SQL in the chat window. Image generation runs
on the platform and model configured for it rather than the conversation's
text platform, and restores the provider credentials it set. A non-identifier
attribute code — one starting with a digit, or containing a hyphen or a space
— no longer breaks every MySQL JSON path query with error 3143. Clearing a
file picker is no longer treated as a new file to scan, and the confirmation
dialog no longer renders behind the element that opened it.

How To Test This?

Ceiling. Configuration > Magic AI > Text Generation now shows "Maximum
Output Tokens". Generate a long HTML description against a low ceiling; a
truncated generation raises a warning flash and no dangling < remains.
Discovery. Add a platform with a custom base URL (with and without a
/v1 segment) and fetch models; both discover, and the stored api_url is
the base that answered. Point one at an unreachable host — the message
names the host and status instead of an HTML page.
Allow list. Set MAGIC_AI_ALLOWED_MODELS and MAGIC_AI_AUTO_SELECT_LIMIT,
fetch again, and confirm only the allowed models are offered and only the
limit is pre-selected.
Translation. On a product with a locale-only (not channel-scoped)
AI-translatable attribute, run Translate; the value lands in the locale
bucket and is visible after a locale switch. A field with no source value is
not offered.
Chat. Force a database error during a chat turn; the window shows the
generic message, not SQL.
Admin. Open a file picker and cancel it — no scan error. Open a
confirmation dialog from inside a dropdown — it renders above.

Screenshots

N/A in this draft — UI screenshots for the translate dialog and the new
setting to be attached before review.

Checklist

vendor/bin/pest passes locally
vendor/bin/pint --test reports no style issues
Tailwind classes are reordered
New user-facing strings use translation keys (all 33 locales)
Target branch is 3.x, the current release line (master targets the next major)

Documentation

My pull request requires an update on the documentation repository.

The new "Maximum Output Tokens" setting, the MAGIC_AI_ALLOWED_MODELS and
MAGIC_AI_AUTO_SELECT_LIMIT environment variables, and Concentrate AI as a
supported provider all need documenting.

MySQL only accepts a bare JSON path member that looks like an identifier, so an attribute code starting with a digit or containing a hyphen or a space made the server reject the whole path with error 3143.

…form A failure originating in the database carries the connection host, schema and full SQL, which reached the chat window verbatim; an infrastructure failure now returns the generic message instead. Image generation ran on the conversation's text platform, ignoring the platform and model configured for image generation, and left the provider credentials it set behind in the process configuration.

…slation Adds a "Maximum Output Tokens" setting and raises the default ceiling from 1024 to 4096; HTML output spends tokens on markup, so the old ceiling cut a table-heavy description mid-tag. A migration raises the seeded prompts still holding the old value and leaves a tuned one alone. A generation that stops on the ceiling is now reported instead of silently returning half a sentence, and a fragment cut mid-tag is dropped. Model discovery now runs against the platform's own base URL, so a proxy or a regional endpoint lists the models it will actually serve, and the OpenAI, Anthropic and Gemini fetches are pinned to the validated address with redirects barred, as every other provider already was. A custom base typed without a version segment is retried with one, a failed fetch reports the host and status rather than the upstream error page, and an allow list and auto-selection limit are configurable through MAGIC_AI_ALLOWED_MODELS and MAGIC_AI_AUTO_SELECT_LIMIT. Adds Concentrate AI, and addresses an Azure model by its configured deployment name. AI translation wrote every value into the channel-and-locale bucket regardless of the attribute's own scope, so a locale-only attribute landed where nothing would read it. Each value now goes to the bucket its scope dictates, an attribute that no longer exists is skipped, and the translate dialog offers a field only when it is locale-scoped and holds a value in the source scope.

…e its opener Clearing a file picker handed nothing to the pre-submit scan, which was treated as a new file to check. The confirmation dialog also rendered behind the element that opened it, so it is now teleported to the body. Rebuilds the admin assets.

Teleporting the dialog to the body moved it out of `#app`, where the admin end-to-end specs look for it. The z-index raise alone keeps it above the element that opens it.

Content generation now resolves through `askResult()`, so the ACL test mock returns a GeneratedContent. The translate modal and the session list each added a second element matching an existing page-wide locator, so both are scoped to the element under test.

Both template saves raise the same "updated successfully" flash, so the second assertion could match the first save's flash and let the test finish while the section and field write was still in flight — leaving the next test reading a passport that never gained its section. The save now waits for the update response and, when it carries a redirect_url, for the SPA navigation that replaces #app and fires no load event.

ramsey/composer-install@v3 pins actions/cache v4.2.4 internally, and setup-node@v4 and login-action@v3 are node20 builds; all three raise the runner's deprecation annotation. The majors bumped here differ only in requiring the node 24 runtime, which every ubuntu-24.04 runner provides.

The workflow only existed locally, so no run has published a pest-tia-baseline artifact since the 3.0 branch was renamed; the last successful run is past the 30-day retention window and its artifact has been collected, which leaves `pest --tia --filtered` with no graph to download and no filtering to apply. Track the workflow, trigger it on the 3.x line as well as master, and hold the artifact for 90 days so a quiet month no longer costs the baseline.

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved translation-scope, token-validation, model-selection, provider-cache, and migration-rollback findings remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 · 2

Open (3)

Use scoped provider configuration for image generation · New
Filter bulk translation options by locale scope and valid source values · New
Align generation request token limit with configured maximum · New

What changed in this PR

This PR expands Magic AI with configurable output limits, platform-aware model discovery, scoped translation, image-provider selection, admin fixes, and expanded tests. Approval is blocked by unresolved scope, validation, provider-cache, and migration rollback issues.

Changes:

Adds output-token ceilings and truncation reporting.
Improves model discovery, filtering, ranking, and provider support.
Updates translation workflows, database portability, UI behavior, localization, and CI coverage.

File	Summary
`tests/Pest.php`	Registers Magic AI tests.
`tests/e2e-pw/tests/02-configuration/magicAI.spec.js`	Scopes translation modal assertions.
`tests/e2e-pw/tests/02-configuration/agentingPIM.spec.js`	Updates chat badge assertions.
`tests/e2e-pw/tests/01-catalog/passport-battery-dpp.spec.js`	Stabilizes template-save waits.
`public/themes/admin/default/build/manifest.json`	Updates built asset hashes.
`phpunit.xml`	Adds Magic AI test suites.
`packages/Webkul/Product/tests/Unit/Services/ProductValueMapperTest.php`	Tests scoped value mapping.
`packages/Webkul/Product/src/Services/ProductValueMapper.php`	Adds scoped reads and writes.
`packages/Webkul/MagicAI/tests/Unit/ScopedProviderConfigTest.php`	Tests provider configuration isolation.
`packages/Webkul/MagicAI/tests/Unit/SafeProviderExtrasTest.php`	Tests extras validation.
`packages/Webkul/MagicAI/tests/Unit/ProviderOverridesTest.php`	Tests provider override filtering.
`packages/Webkul/MagicAI/tests/Unit/AiProviderTest.php`	Tests provider discovery.
`packages/Webkul/MagicAI/tests/Feature/TranslationWorkflowTest.php`	Tests translation authorization and queuing.
`packages/Webkul/MagicAI/tests/Feature/SaveTranslatedDataJobTest.php`	Tests scoped translation persistence.
`packages/Webkul/MagicAI/tests/Feature/PlatformConnectionTest.php`	Tests platform connection validation.
`packages/Webkul/MagicAI/tests/Feature/MagicPromptManagementTest.php`	Tests prompt administration.
`packages/Webkul/MagicAI/tests/Feature/MagicAIPlatformModelTest.php`	Tests platform models and defaults.
`packages/Webkul/MagicAI/src/Support/ModelRecommender.php`	Adds model filtering and ranking.
`packages/Webkul/MagicAI/src/Services/LaravelAiAdapter.php`	Reports truncation and Azure deployment names.
`packages/Webkul/MagicAI/src/Responses/GeneratedContent.php`	Adds generation result metadata.
`packages/Webkul/MagicAI/src/Providers/MagicAIServiceProvider.php`	Registers Magic AI configuration.
`packages/Webkul/MagicAI/src/Models/MagicAIPlatform.php`	Adds factory support.
`packages/Webkul/MagicAI/src/MagicAI.php`	Adds output-token ceilings and result APIs.
`packages/Webkul/MagicAI/src/Jobs/SaveTranslatedDataJob.php`	Saves values by attribute scope.
`packages/Webkul/MagicAI/src/Jobs/SaveTranslatedAllAttributesJob.php`	Handles scoped bulk translations.
`packages/Webkul/MagicAI/src/Database/Seeders/MagicAISystemPromptSeeder.php`	Raises seeded token defaults.
`packages/Webkul/MagicAI/src/Database/Migrations/2026_09_21_000001_raise_seeded_system_prompt_max_tokens.php`	Migrates prompt ceilings.
`packages/Webkul/MagicAI/src/Database/Factories/MagicAIPlatformFactory.php`	Adds platform factory states.
`packages/Webkul/MagicAI/src/Contracts/ReportsTruncation.php`	Defines truncation reporting.
`packages/Webkul/MagicAI/src/Config/magic_ai.php`	Adds model discovery configuration.
`packages/Webkul/MagicAI/src/Agents/MagicContentAgent.php`	Raises agent token defaults.
`packages/Webkul/Core/tests/Unit/GrammarJsonPathQuotingTest.php`	Tests JSON path quoting.
`packages/Webkul/Core/tests/Unit/GrammarJsonPathEngineTest.php`	Tests JSON paths across engines.
`packages/Webkul/Core/src/Helpers/Database/Grammars/MySQLGrammar.php`	Quotes non-identifier JSON keys.
`packages/Webkul/AiAgent/tests/Unit/ModelSelectionConfigTest.php`	Tests model limits and allow lists.
`packages/Webkul/AiAgent/tests/Unit/ModelRecommenderTest.php`	Updates recommender tests.
`packages/Webkul/AiAgent/tests/Unit/GenerateImagePlatformTest.php`	Tests image platform resolution.
`packages/Webkul/AiAgent/tests/Unit/ChatWidgetTranslationKeysTest.php`	Tests chat translation keys.
`packages/Webkul/AiAgent/tests/Unit/ChatWidgetAutoScrollTest.php`	Tests chat auto-scrolling.
`packages/Webkul/AiAgent/tests/Unit/AiErrorResolverTest.php`	Tests database error sanitization.
`packages/Webkul/AiAgent/src/Chat/Tools/GenerateImage.php`	Selects configured image platforms.
`packages/Webkul/AiAgent/src/Chat/AiErrorResolver.php`	Hides infrastructure error details.
`packages/Webkul/Admin/tests/Feature/MagicAi/TranslationTest.php`	Tests locale-only translation.
`packages/Webkul/Admin/tests/Feature/MagicAi/OrphanPlatformFallbackTest.php`	Updates generation mocks.
`packages/Webkul/Admin/tests/Feature/MagicAi/MagicAiContentImageTest.php`	Updates content-generation mocks.
`packages/Webkul/Admin/tests/Feature/MagicAi/GenerationTokenCeilingTest.php`	Tests truncation handling.
`packages/Webkul/Admin/tests/Feature/MagicAi/CustomModelDiscoveryTest.php`	Tests custom discovery fallback.
`packages/Webkul/Admin/tests/Feature/MagicAi/ConcentrateProviderTest.php`	Tests Concentrate AI support.
`packages/Webkul/Admin/tests/Feature/MagicAi/AzureModelDiscoveryTest.php`	Tests Azure model discovery.
`packages/Webkul/Admin/tests/Feature/Catalog/ProductTranslateSourceValuesTest.php`	Tests translation source values.
`packages/Webkul/Admin/tests/Feature/Acl/MagicAi/MagicAiContentGenerationAclTest.php`	Updates ACL generation mocks.
`packages/Webkul/Admin/src/Resources/views/configuration/magic-ai/platform/index.blade.php`	Adds platform discovery behavior.
`packages/Webkul/Admin/src/Resources/views/components/tinymce/index.blade.php`	Adds token ceiling UI and warnings.
`packages/Webkul/Admin/src/Resources/views/components/products/dynamic-attribute-fields.blade.php`	Hides translation controls for locked fields.
`packages/Webkul/Admin/src/Resources/views/components/modal/confirm.blade.php`	Raises confirmation modal z-index.
`packages/Webkul/Admin/src/Resources/views/components/media/image.blade.php`	Handles empty image selections.
`packages/Webkul/Admin/src/Resources/views/components/media/gallery.blade.php`	Handles empty gallery selections.
`packages/Webkul/Admin/src/Resources/views/components/media/files.blade.php`	Handles empty file selections.
`packages/Webkul/Admin/src/Resources/lang/zh_TW/app.php`	Adds Traditional Chinese strings.
`packages/Webkul/Admin/src/Resources/lang/zh_CN/app.php`	Adds Simplified Chinese strings.
`packages/Webkul/Admin/src/Resources/lang/pt_PT/app.php`	Adds Portuguese strings.
`packages/Webkul/Admin/src/Resources/lang/pt_BR/app.php`	Adds Brazilian Portuguese strings.
`packages/Webkul/Admin/src/Resources/lang/nl_NL/app.php`	Adds Dutch strings.
`packages/Webkul/Admin/src/Resources/lang/ko_KR/app.php`	Adds Korean strings.
`packages/Webkul/Admin/src/Resources/lang/id_ID/app.php`	Adds Indonesian strings.
`packages/Webkul/Admin/src/Resources/lang/es_VE/app.php`	Adds Venezuelan Spanish strings.
`packages/Webkul/Admin/src/Resources/lang/es_ES/app.php`	Adds Spanish strings.
`packages/Webkul/Admin/src/Resources/lang/en_US/app.php`	Adds English strings.
`packages/Webkul/Admin/src/Resources/lang/en_NZ/app.php`	Adds New Zealand English strings.
`packages/Webkul/Admin/src/Resources/lang/en_GB/app.php`	Adds British English strings.
`packages/Webkul/Admin/src/Resources/lang/en_AU/app.php`	Adds Australian English strings.
`packages/Webkul/Admin/src/Resources/lang/de_DE/app.php`	Adds German strings.
`packages/Webkul/Admin/src/Resources/lang/da_DK/app.php`	Adds Danish strings.
`packages/Webkul/Admin/src/Resources/assets/js/app.js`	Skips empty media scans.
`packages/Webkul/Admin/src/Resources/assets/css/app.css`	Adds focus and compact-select styling.
`packages/Webkul/Admin/src/Http/Requests/ProductAttributeForm.php`	Validates product scope inputs.
`packages/Webkul/Admin/src/Http/Requests/MagicAI/ContentGenerationRequest.php`	Validates generation inputs.
`packages/Webkul/Admin/src/Http/Controllers/MagicAI/MagicAIPlatformController.php`	Uses platform-based discovery.
`packages/Webkul/Admin/src/Http/Controllers/MagicAI/MagicAIController.php`	Adds scoped translation and truncation responses.
`packages/Webkul/Admin/src/Http/Controllers/Catalog/ProductController.php`	Returns scoped translation values.
`packages/Webkul/Admin/src/Config/system.php`	Adds maximum-token configuration.
`CHANGELOG.md`	Documents the feature and fixes.
`.github/workflows/translation_tests.yml`	Updates Composer actions.
`.github/workflows/static_analysis.yml`	Updates Composer actions.
`.github/workflows/playwright_test.yml`	Updates Composer actions.
`.github/workflows/pest_tests.yml`	Updates Composer and Node actions.
`.github/workflows/pest_tests_pgsql.yml`	Updates Composer and Node actions.
`.github/workflows/pest_tests_mariadb.yml`	Updates Composer and Node actions.
`.github/workflows/linting_tests.yml`	Updates Composer actions.
`.github/workflows/docker-publish.yml`	Updates Docker login action.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

…eiling Image generation now runs through ScopedProviderConfig so the cached provider instance is evicted around the call. The bulk translate dialog only offers locale-scoped attributes with a scalar source value, and the generation request accepts the same MAX_TOKENS_CEILING as the settings.

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues remain in validation, token handling, migrations, provider behavior, and test coverage.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 4

Open (4)

Whitespace-only source values bypass empty-content guard · New
Image generation drops provider-specific platform extras · New
Azure connection tests ignore configured deployment overrides · New
Mocked 401 response is exhausted before error assertion · New

Resolved since last review (3)

Use scoped provider configuration for image generation
Align generation request token limit with configured maximum
Filter bulk translation options by locale scope and valid source values

+        if ($sourceData === null || $sourceData === '' || is_array($sourceData)) {
+            return false;
+        }


+                $overrides = ['key' => $platform->api_key];

-                    if ($this->context->platform->api_url) {
-                        config(["ai.providers.{$configKey}.url" => $this->context->platform->api_url]);
-                    }
+                if ($platform->api_url) {
+                    $overrides['url'] = $platform->api_url;
+                }


+    protected function modelName(): string
+    {
+        if ($this->aiProvider !== AiProvider::Azure) {
+            return $this->model;
+        }
+
+        $deployment = trim((string) (ProviderOverrides::decode($this->platform->extras)['deployment'] ?? ''));
+
+        return $deployment !== '' ? $deployment : $this->model;


+    $client = fakeProviderClient([
+        new Response(401, [], '<html><body>Unauthorized</body></html>'),
+    ]);


navneetkumar-pim-webkul added 11 commits September 21, 2026 21:40

fix(core): quote non-identifier segments in MySQL JSON paths

71e11b2

MySQL only accepts a bare JSON path member that looks like an identifier, so an attribute code starting with a digit or containing a hyphen or a space made the server reject the whole path with error 3143.

docs(changelog): record the unreleased magic ai and admin changes

1f96b85

fix(admin): keep the confirm modal inside the app root

4de737a

Teleporting the dialog to the body moved it out of `#app`, where the admin end-to-end specs look for it. The z-index raise alone keeps it above the element that opens it.

docs(changelog): fold the unreleased feature entries into improvements

30000eb

navneetkumar-pim-webkul marked this pull request as ready for review September 22, 2026 15:17

Copilot AI lite review requested due to automatic review settings September 22, 2026 15:17

Copilot started reviewing on behalf of navneetkumar-pim-webkul September 22, 2026 15:18 View session

Copilot AI reviewed Sep 22, 2026

View reviewed changes

Comment thread packages/Webkul/AiAgent/src/Chat/Tools/GenerateImage.php Outdated

Comment thread packages/Webkul/Admin/src/Http/Controllers/Catalog/ProductController.php

Comment thread packages/Webkul/Admin/src/Http/Requests/MagicAI/ContentGenerationRequest.php Outdated

navneetkumar-pim-webkul and others added 2 commits September 23, 2026 15:40

Merge branch '3.x' into feat/magic-ai-generation-ceiling-and-discovery

c471175

navneetkumar-pim-webkul requested a lite review from Copilot September 23, 2026 10:29

Copilot started reviewing on behalf of navneetkumar-pim-webkul September 23, 2026 10:30 View session

navneetkumar-pim-webkul merged commit 9777321 into 3.x Sep 23, 2026
24 checks passed

navneetkumar-pim-webkul deleted the feat/magic-ai-generation-ceiling-and-discovery branch September 23, 2026 10:36

Copilot AI reviewed Sep 23, 2026

View reviewed changes

navneetkumar-pim-webkul mentioned this pull request Sep 24, 2026

fix(admin): rebuild admin assets so the remove-model button turns red on hover #733

Merged

2 of 6 tasks

Navigation Menu

Repository navigation

feat(magic-ai): output ceiling, platform-aware discovery, scoped translation - #719

feat(magic-ai): output ceiling, platform-aware discovery, scoped translation#719
navneetkumar-pim-webkul merged 13 commits into
3.xfrom
feat/magic-ai-generation-ceiling-and-discovery

navneetkumar-pim-webkul commented Sep 22, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

navneetkumar-pim-webkul commented Sep 22, 2026

Issue Reference

Description

How To Test This?

Screenshots

Checklist

Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants