Repository navigation
Moving forward with Dynamic Modules? #252
Description
Activity
Do we have outstanding objections?
Yes. Dynamic Modules as written today is something I, as a TC39 delegate, would not feel comfortable agreeing to because of its syntax poisoning and the inconsistent/confusing developer experience it creates. While I understand the drive to push something through, I'm not okay pushing through a solution that further confuses/complicates the developer story.
A possible path forward could be to:
- Merge the cyclical module record into ecma262
- Ensure namespace objects can be in a non-finalized state
(similar to what is in Dynamic Modules now) - Treat all modules in Node as a Dynamic Module
Treating all modules as dynamic reduces ecma262 complexity and gives Node the freedom to create modules that interop as it needs (even supporting
export * from "cjs"). It is possible for Node to then create dynamic modules that are 💯 compliant when interacting with other ES modules. For example, Node could pass the test262 test suite, while also supporting CJS, WASM, or other module formats as they come.Reacted by Michael Ciniawsky and Evan Plaice@jdalton If that is the case we need an alternative, otherwise I feel like we have no choice but to not ship any support for named exports from CJS by our April deadline. This feels like a regrettable, but inevitable choice. To be clear, I feel we must reach consensus on something, and if not we are going to have to remove any sort of named export support if we want to ship something (whatever it is). I feel your complaints are able to be handled in a follow on but there is a lack of alternative proposal being presented by your repeated comments. As such, would you be willing take over championship as we are lacking consensus and it seems you are seeking to prevent forward progress on the current proposal? As it stands, if we lack consensus, we should do as we have done to other features and remove that which is unable to reach consensus. However, if we reach quorum in the group to go forward with Dynamic Modules, I feel that would be preferable to not having anything.
If that is the case we need an alternative, otherwise I feel like we have no choice but to not ship any support for named exports from CJS by our April deadline. This feels like a regrettable, but inevitable choice.
I understand. That may be.
I feel your complaints are able to be handled in a follow on but there is a lack of alternative proposal being presented by your repeated comments
I believe it's more a fundamental design difference that isn't suited as a follow-on.
As such, would you be willing take over championship as we are lacking consensus and it seems you are seeking to prevent forward progress on the current proposal?
Take on championship of dynamic modules? Yes (in whatever capacity is needed)
As it stands, if we lack consensus, we should do as we have done to other features and remove that which is unable to reach consensus.
Okay.
However, if we reach quorum in the group to go forward with Dynamic Modules, I feel that would be preferable to not having anything.
I prefer interop too, but not at this cost. I feel the proposal, as is, will not pass the TC39.
Reacted by Evan Plaice- Given that you could block it at TC39 that seems certain, not likely.…On Mon, Jan 21, 2019, 11:09 AM John-David Dalton ***@***.*** wrote: If that is the case we need an alternative, otherwise I feel like we have no choice but to not ship any support for named exports from CJS by our April deadline. This feels like a regrettable, but inevitable choice. I understand. That may be. I feel your complaints are able to be handled in a follow on but there is a lack of alternative proposal being presented by your repeated comments I believe it's more a fundamental design difference that isn't suited as a follow-on. As such, would you be willing take over championship as we are lacking consensus and it seems you are seeking to prevent forward progress on the current proposal? Take on championship of dynamic modules? Yes *(in whatever capacity is needed)* As it stands, if we lack consensus, we should do as we have done to other features and remove that which is unable to reach consensus. Okay. However, if we reach quorum in the group to go forward with Dynamic Modules, I feel that would be preferable to not having anything. I prefer interop too, but not at this cost. I feel the proposal, as is, will not pass the TC39. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#252 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAOUowqCRiaMcKGv_pZ_zD5xP7VOySe1ks5vFfRFgaJpZM4aLA3I> .
It’s worth noting that it already was considered acceptable by the subset of delegates present in November; but certainly with both your delegate objection, and lacking consensus in the modules group, it proceeding is unlikely.
Do we have outstanding objections?
Yes.
I have the following concerns with dynamic modules:
- As @jdalton points out, the current dynamic modules proposal creates an "uncanny valley" situation where interop is almost-but-not-quite transparent. This, along with other slight differences from Babel-style interop, will likely lead to user confusion that cannot be addressed with simple answers.
- I'm concerned that dynamic modules (and import CJS support) binds us long term to either or both of the following stragegies for JS format disambiguation:
.mjsvs..cjs- boilerplate configuration in package.json
There are two alternatives:
- The one that @jdalton mentioned, where essentially everything becomes a dynamic module. The only changes required to ecma262 are the ones mentioned above, although to be honest I'm not sure what the implementation would look like in Node. It would likely be quite different from
--experimental-modules. - Non-transparent interop.
Ultimately, I believe that this isn't so much about the 262 spec as it is about answering the question of what kind of interop we want. TC39 and V8 will want to know that we have consensus on a strategy before making changes. Perhaps we should shift the conversation in that direction.
Let me ask a question: it seems to me that non-transparent interop, where we cannot import from CJS but must use
import.meta.requireis similar, can be a viable solution for our users. Do you agree or disagree?Reacted by Mauve SignweaverIf we are unwilling to move forward with named exports from CJS currently, and due to our LTS cutoff, would people be ok completely tabling this and aim for next year's release to discuss named exports from CJS?
Let me ask a question: it seems to me that non-transparent interop, where we cannot import from CJS but must use import.meta.require is similar, can be a viable solution for our users. Do you agree or disagree?
I think it isn't a yes/no answer so your questions isn't really something I can reply to. However, I will point out that it doesn't serve any users which expect to be able to use named imports from CJS and the alternatives that will eventually allow such remain unclear if the demands in this thread are held over time. It appears that there is a desire to frame Dynamic Modules as not suitable for 1 case, and therefore it is acceptable to not serve any cases that it might fulfill.
If we are unwilling to move forward with named exports from CJS currently, and due to our LTS cutoff, would people be ok completely tabling this and aim for next year's release to discuss named exports from CJS?
I think our group has already decided that is the procedure to ensure forward progress. Controversial items get punted to later phases.
@robpalme I agree, just want to be clear about tabling all discussions of named exports from CJS seems like what we are agreeing to since it seems we will not be able to move forward with Dynamic Modules. Being explicit about that should help us focus on making our April deadline and avoid discussing named exports from CJS; thus allowing us to more readily reach consensus on other issues that are still likely to be resolved before then.
We are already bound forever to explicit parse goal disambiguation, whether by extension or package.json, due to the ambiguous parsing between the Module and Script goals in the spec. Dynamic Modules don’t change this.
As for this “April cutoff”, I’m pretty sure there was never consensus for any kind of cutoff - i, for one, am not on board with any sort of time-based deadline that might lead to shipping something incomplete.
@ljharb this has been discussed in the past, the LTS branch is cut off in April, regardless of what we do in this group. If we do not upstream something by then, we will wait until the next April for the next LTS. It has been lightly discussed that we should upstream the minimal kernel at least.
Reacted by Jordan Harband- Fwiw we will still be able to make changes after April, experimental apis can receive breaking changes, even during LTS The biggest challenge is that if we need to make semver major changes outside of the scope of esm then we will be stuck (e.g. package.exports)…On Mon, Jan 21, 2019, 1:17 PM Bradley Meck ***@***.***> wrote: @ljharb <https://github.com/ljharb> this has been discussed in the past, the LTS branch is cut off in April, regardless of what we do in this group. If we do not upstream *something* by then, we will wait until the next April for the next LTS. It has been lightly discussed that we should upstream the minimal kernel at least. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#252 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAecVxdAcfTniiylpPmOB1ARaTyx9a6dks5vFgRKgaJpZM4aLA3I> .Reacted by Jordan Harband
If it remains behind a flag, i wouldn’t block that.
For the record, i do want this to advance, i do think that even with this restriction, it’s a massively better situation than without dynamic modules, and i strongly disagree that no interop is better than partial interop, especially when the excluded portion is the very rare edge case of a star export from a CJS module combined with a circular dependency.
I would not like to see us take yet another year to land ESM support unflagged. We do have some minimal agreement on a kernel and it seems unlikely to change in a breaking way. If we can focus on what would be unable to be seen as stable enough to potentially unflag over the LTS I would rather prioritize those as there is a chance we could unflag during LTS over the next year.
I don’t find it acceptable to ever unflag with only the minimal kernel - my concern is about ecosystem adoption, and those behavior patterns will become set once we unflag, and as such, i think we must have a complete implementation first.
31 remaining items
HTML is creating “Synthetic Modules” whatwg/webidl#722
Is that something we could build off of, or emulate, to make dynamic modules possible?
it has no difference in actual semantics to our current dynamic module wrapper used in translation.
Right - but if the web is doing it regardless, then any TC39 concerns would not really apply, and node would be able to do it too?
@ljharb it (Synthetic Modules) doesn't provide any new capabilities over what we already have.
The set of exported names is static, and determined at creation time (as an argument to [$CreateSyntheticModule$]), while the set of exported values can be changed over time using [$SetSyntheticModuleExport$]. It has no imports or dependencies.Our current synthetic system (
createDynamicModule) is a bit like:The set of exported names is static, and determined at creation time (as an argument to createDynamicModule), while the set of exported values can be changed over time using reflect.exports.$name.set(). It has no imports or dependencies.Reacted by Jordan HarbandRight - but if the web is doing it regardless, then any TC39 concerns would not really apply, and node would be able to do it too?
@ljharb It seems that there is wiggle room for Synthetic Modules moving to the spec
Sure, but apparently it doesn’t affect us.
@devsnek brilliant breakdown
@ljharb I think what is worth looking at is considering more closely the TDZ issue:
- If the specs are not substantially different — logically speaking TDZ issue would still not be solved
- If the specs are substantially different — is the TDZ issue solved (at least for our needs
export *)
Do we have an "okay now it conforms" break here?
Note: Going through the details, I don't see that we do… but I might be wrong.
I'm intrigued by @jdalton's suggestion to "Treat all modules in Node as a Dynamic Module," effectively abandoning STMR in Node. It's not clear whether that proposal is truly dead.
I see @bmeck's objection that "It is tantamount to just using a different module system from the ECMAScript spec by never using ECMAScript's standard and instead making one of our own."
Point taken, but there seems to be good reason to think that there would be no user-visible results of this, except that named exports from CJS would magically start working. If the dynamic modules implementation could pass all of the same tests that STMR would, then that seems like an acceptable tradeoff to me.
I see that @jdalton replied "It looks like we're at draw here," and I see no further discussion since then, (except slightly later, "it'd be rad if you could work with me and @zenparsing to poke at treating all modules as dynamic"). Did anything happen? Are we just waiting around for somebody to write up a PR and see if it passes the tests? If it does pass all of the compatibility tests, would that be unacceptable to people? To @bmeck?
In light of the alternatives (out-of-order execution, unreliable CJS exports detection, surrendering CJS named exports), I think this approach is worth another look.
And, to clarify, would "treating all modules as dynamic" even require TC39 approval? We'd literally be doing our own thing instead of ESM, right?
dynamic modules are a form of out-of-order execution.
I may be misunderstanding dynamic modules, but I thought that they did not in fact execute the dynamic modules early.
According to this comment, under dynamic modules, "Node assumes that the exports are as the imports defined them, leaves the execution to the execution phase, and if during the execution it finds that the imports were not the same as the exports, throws an exception."
right, it moves validation to the execution phase, which means your program may be terminated because of invalid imports after parts of it already started running.
I think I'd call that "late validation" rather than "out-of-order execution."
In particular, the "out-of-order execution" problem seems to be where
import 'a'; import 'b';would executebbeforeaifbis CJS andais ESM, which is what people are (quite understandably) objecting to in the OOO thread.But if everything is a dynamic module, then all scripts will execute in the intended lexical order.
It is true that invalid imports would be caught later that would have been caught earlier under STMR, but IMO that seems like an acceptable tradeoff to support CJS named exports.
Reacted by Wesley WighamClosing due to lack of movement
I would like to propose that we move forward with Dynamic Modules. I would just like to see if we have consensus moving forward. We don't have an alternative, and it satisfies a large category of usage. I would like to reach consensus if we can move forward using Dynamic Modules and allow us to plan on integrating it into our implementation. Do we have outstanding objections? I would like to add this (albiet late) to the TC39 agenda so it can at least be discussed.