Repository navigation
Main chunk points to the wrong runtime chunk when using Module Federation #19439
Description
Activity
I've done some attempts and I've figured out something more:
-
I've tried working with previous versions of our packages, but the issue was already there, only I hadn't found out
-
I've tried pointing to remotes that are reachable and not reachable, but the issue is always there, so I think that the content of the remote app doesn't impact the issue
-
I wanted to try working without the runtime file, but the app stops working. With
runtimeset tofalse, I have to setexperiments.outputModuletofalseto make the rest of the app work, but with those settings the import of the remote app appears without single quotes:module.exports = remote@http://localhost:3001/assets/remoteEntry.js;
-
This happens even if I change the library settings in
ModuleFederationPlugintolibrary: { type: 'var', name: 'host' }`
-
Setting
runtimetosingledoesn't solve the issue, because Webpack still outputs tworuntime-appfiles -
Passing a
runtimestring to theModuleFederationPluginsettings, or setting the field tofalsedoesn't solve the issue
-
@gtempesta-pixartprinting This is expected, because with module federation you have two runtimes - one internal (for loading packages inside federation), the second is for main application (splitting and dynamic chunks)
I will look deeply soon at this
Reacted by Giorgio Tempesta@alexander-akait Thanks for taking the time to look into the issue.
I've done some further investigation and I'd like to share what I've found.
Why I don't think it's expected
I don't think the behavior I'm experiencing is expected, because:
- The two generated files have the same name, the only thing that changes is the chunk hash
- The content of the files is also very similar, sometimes the only thing that changes is the chunk hash of the referenced file(s)
- The first output of the watch command and the output of the build command don't have this issue.
To make it easier to reason about it, I've added a new folder to my repository generated-output-example.
In the folder there are two
runtime-appfiles: runtime-app.d0415bae809a613ce32c.js and runtime-app.2f49b02885c5fb16b004.js. The first one is referenced under entrypoints in the asset-manifest.json file, while the second one is referenced by the app.51cd46ce3502605c797c.js file, which is also referenced in the asset-manifest.json file.What is happening when running the code in the browser
First thing, I've discovered that this doesn't only happen when changing code in the file that loads the micro frontend, it happens with every change to any file that's dynamically loaded by the main file (
app.js). Anyway this is what happens:- After we save a change in a file that's dynamically loaded, the changes are saved in a chunk
- The correct chunk is referenced by one of the two runtimes that are generated
- The browser loads the two runtimes (one is requested by the manifest and the other by the
app.jsfile) - Only the runtime referenced in the manifest holds the reference to the chunk with the changes
- The runtime referenced in
app.jsreferences a chunk that doesn’t have the changes - The browser only loads the chunk that’s referenced in the runtime that’s loaded by the app file and not the chunk that's references in the runtime that's listed in the manifest
- If I look for the changes in the browser (Open the Developers Tools and Search) the change is never loaded by the browser
How to check this in my repository
- Checkout the project
- Clear the public folder
- Run watch
- Add a log to
design-area.js(for exampleconsole.log(’test’);) - Look for the log inside the public folder and mark down the chunk hash. If there are more than one, mark all the chunk hashes of the files you find
- Now open
asset-manifest.jsonand open theruntime-appthat’s referenced there - Open that file and look for the chunk hash: it should be there
- Now go to the app file referenced in
asset-manifest.json - Look for the
runtime-appreferenced there - Open the
runtime-appfile and look for the chunk hash(es) of the files with the initial log: it’s not there
Let's do the same check inside my generated-output-example folder.
- I've added this log to the
design-area.jsfile:console.log('second test log');(the log is not committed to the project) - The output code is only present in this chunk: resources_js_chunks_design-area_js.6572e66ecc4cdcf33f12.js, so the chunk hash I should look for is
6572e66ecc4cdcf33f12 - If I open asset-manifest, I will find this runtime-app under entrypoints: runtime-app.d0415bae809a613ce32c.js
- If I look for the chunk hash
6572e66ecc4cdcf33f12in this runtime, I can find it - Now I open the app that's referenced in the manifest, which is: app.51cd46ce3502605c797c.js
- The app references a different runtime: runtime-app.2f49b02885c5fb16b004.js
- If I open the file and look for the chunk hash
6572e66ecc4cdcf33f12, it is not there!
I hope this helps debugging.
Looks like a bug in https://github.com/gtempesta/webpack-issue-repro/blob/main/webpack/webpack.dev.config.js#L62 or your should use other logic for generations, because in our files we have the valid value and code is working
Looks like a bug in https://github.com/gtempesta/webpack-issue-repro/blob/main/webpack/webpack.dev.config.js#L62 or your should use other logic for generations, because in our files we have the valid value and code is working
But the chunk referenced in app is the one without the changes, so in my opinion there is an issue even without considering the asset-manifest generation.
So
app.51cd46ce3502605c797c.js
-> runtime-app.2f49b02885c5fb16b004.js
-> resources_js_chunks_design-area_js.2876b31bdeb0b3a137d5.js
-> no log hereThe changes are in
resources_js_chunks_design-area_js.6572e66ecc4cdcf33f12.js
which is only referenced by runtime-app.d0415bae809a613ce32c.js which in turn is referenced in the manifest file.If we will have wrong values for runtimes we will generate wrong runtime code and it will be broken, but your code works...
If we will have wrong values for runtimes we will generate wrong runtime code and it will be broken, but your code works...
I wouldn't say that my code works, because the
app.jschunk references the wrong runtime, so I'm not able to see the correct changes in the browser when runningwatch.I've done a couple more attempts to prove my point.
1. Overwrite the generated
appfileWith the current setup, I've looked for the linked runtime in the generated
appfile, let's say the line:import __webpack_require__ from "./runtime-app.2f49b02885c5fb16b004.js";
And replaced it with the correct runtime, let's say:
import __webpack_require__ from "./runtime-app.d0415bae809a613ce32c.js";
In this case the code works.
2. Remove the Module Federation plugin
I've removed the Module Federation plugin from the Webpack configuration and replaced the import of the remote application with an empty object.
const { default: remoteMountMethod } = {};
Instead of:
const { default: remoteMountMethod } = await import( 'remote_bucket/remoteMountMethod' );
Now the app chunk doesn't even reference a runtime, and everything works as expected.
3. Configure a different plugin
I've configured
webpack-assets-manifestinstead ofwebpack-manifest-plugin: the two manifests generate files with the same issues.Update
I've done some more attempts that can point to a possible solution.
1.
CleanWebpackPluginI've added
CleanWebpackPluginto the Webpack configuration. I thought that this would force the main chunk to link an existing asset, but it wasn't the case: it kept linking an old chunk which was already deleted, making the issue even more apparent.(This setting is not present in the main branch, as it doesn't add any value).
2.
realContentHashAnother attempt:
- changed
chunkhashtocontenthashin theoutputsettings - set
realContentHashtotruein theoptimizationsettings
At this point making a change in the
design-area.jsfile finally triggered an error from Webpack which made everything more clear:ERROR in RealContentHashPlugin Some kind of unexpected caching problem occurred. An asset was cached with a reference to another asset (952ed40e29224276644e) that's not in the compilation anymore. Either the asset was incorrectly cached, or the referenced asset should also be restored from cache.3. Cache settings
At this point I went to the Cache documentation and tried different configurations, until I've found one that fixed the issue:
cache: false,
I tried to apply this change to the original project, but this is not sustainable: every change now takes about 30s to build.
Any other change from the documentation (for example changing the type to
filesystem) doesn't solve the issue.But at this point it's clear that the issue is on the way that Webpack manages the cache: can anyone of the maintainers take a deep look at this topic?
You can already play with those settings in the linked repository, just comment / uncomment
realContentHash: trueandcache: false.To trigger the issue you just have to start the
watchprocess and add or edit logs in thedesign-area.jsfile.- changed
I've found a workaround to this issue, which involves some code changes but at least we're able to work using the watch command.
The workaround consists in exporting two Webpack configurations instead of one: one configuration uses module federation, while the other uses code splitting and the runtime.
The two configurations export two different manifests, so the application has to read the two manifests to link the correct files.
At this point we have two separate applications running on the same page, and to make them communicate I've attached the export of the micro frontend to the window.
const mainAppConfig = { // [...] }; const microFrontendConfig = { // [...] }; module.exports = [mainAppConfig, microFrontendConfig];
Then in the code, on one side:
const { default: designAreaModule } = await import( 'remote/mountMyDesignArea' ); window.designArea = designAreaModule;
And on a separate file:
const { default: remoteMountMethod } = window.designArea;
What I've found is that at this point the main app doesn't even link a runtime file, and everything works as expected because the runtime is already listed with the entrypoints of the manifest.
You can find the complete implementation here: https://github.com/gtempesta/webpack-issue-repro/tree/workaround.
Sorry for delay, look at deeply on this problem, yeah, it is a bug
Shorty, because we generate this code in the chunk code to load runtime logic:
import __webpack_require__ from "./runtime-app.cd925ca1545fa8a9af92.js";
we don't consider runtime chunk hash and including chunks (and modules) (due to this content hash doesn't work too), ideally we need to get runtime chunk of current chunk, then get all chunks from runtime (and nested) and consider them as a part of chunk hash, fix should be here - https://github.com/webpack/webpack/blob/main/lib/esm/ModuleChunkFormatPlugin.js#L206
Reacted by XiaoCurrently, we process otherChunks（async-chunk & app-chunk） first and then runtimeChunks（runtime-app）, which works fine in most cases.
However, the issue with this ESM case is that one of the app-chunk in otherChunks depends on runtimeChunks.
If we determined the processing order based on a dependency graph, this wouldn't be a problem.// webpack/lib/Compilation.js for (const chunk of otherChunks) processChunk(chunk); for (const chunk of runtimeChunks) processChunk(chunk);
fix should be here - https://github.com/webpack/webpack/blob/main/lib/esm/ModuleChunkFormatPlugin.js#L206
Does it mean I just have to update to 5.99.9?
fix should be here - https://github.com/webpack/webpack/blob/main/lib/esm/ModuleChunkFormatPlugin.js#L206
Does it mean I just have to update to 5.99.9?
@gtempesta-pixartprinting Sorry, we're still working on the fix. We'll notify you once the patched version is released.
Reacted by Giorgio TempestaSorry to ask again, but this is the first time that I open an issue in an Open Source project.
I've seen that the issue is closed and the PR got merged into the main repository, however the latest release (v5.99.9) happened one week earlier, so I suppose that I'm not getting the updates if I install the latest version, is that correct?
So, what's gonna happen next? All the changes will be available after the next release? Is there an easy way to know when this is going to happen?
Thanks for the patience.
I've seen that the issue is closed and the PR got merged into the main repository, however the latest release (v5.99.9) happened one week earlier, so I suppose that I'm not getting the updates if I install the latest version, is that correct?
Yes, it is a part of
5.100.0releaseSo, what's gonna happen next? All the changes will be available after the next release? Is there an easy way to know when this is going to happen?
Hard to say, we have roadmap for
5.100.0and we need to finish it before release, but it will be this month, anyway you can use version of Github in your package.json and get the latest version with fixesReacted by Giorgio Tempesta
Bug report
What is the current behavior?
When running the
watchcommand, at some point theruntime-appchunk referenced in theappchunk is different from the one referenced in theasset-manifest.json.Since we load this code in a PHP app, we use the manifest to load the chunks dynamically.
Having the manifest reference a different chunk than the one referenced in the
appchunk means that the browser loads two differentruntime-appchunks, one requested by the manifest, and the other by theappchunk.Since the second one is the oldest, the end result is that the
watchcommand is not doing its job, because from a certain moment, I'm not able to see the changes I make to the code reflected in the browser.If the current behavior is a bug, please provide the steps to reproduce.
Here is a GitHub repository with the code needed to reproduce the issue: https://github.com/gtempesta/webpack-issue-repro/.
How to reproduce:
npm installto install the dependenciesnpm run watchto start watching for changesasset-manifest.jsonfile in the public folder, and open theappchunk that is referenced in theentry-pointsfield. Now look for theruntime-appchunk in theappchunk and check that it's the same that is referenced in theasset-manifest.jsonfile: it should be the same (with the same hash)app.jsand save itdesign-area.jsfile and save itruntime-appchunk in theasset-manifest.jsonfile and in theappchunk should have different hashesThe issue happens only when using the
watchcommand, and not when using thebuildcommand (which is not implemented in this repo).What is the expected behavior?
I would expect that the
runtime-appchunk referenced in theasset-manifest.jsonfile and in theappchunk are always the same, so that I can use the manifest to load the chunks dynamically without having to worry about which one is loaded.Other relevant information:
webpack version: 5.99.5
Node.js version: 18.17.1
Operating System: macOS Sequoia 15.3.2
Additional tools: