Repository navigation
Non-async source map. #331
Description
Activity
Fetching and compiling WebAssembly is inherently async.
We could add a sync API to create a
SourceMapConsumergiven that you've already fetched and compiled the wasm. Would that work?If so, I can help you craft a PR that implements this.
I'm not sure. Does node require an async interface for wasm? For reference, I'd like to use source maps with screeps (0.6.x works). Their docs for wasm is here: http://docs.screeps.com/modules.html It doesn't seem to include anything async, but I've also not tested it yet.
A sync API is also needed for e.g. stack traces, as they are collected synchronously (evanw/node-source-map-support#206).
If we can initialize whatever needs to be async ahead of time, and have a sync way of constructing and using
SourceMapConsumers, that would unblock Jest (and probably Babel) at leastReacted by cimak, Franck Freiburger, snek, Corey Farrell, Chanon S., Elliot Nelson, Joe Lencioni, Simon Siefke, shitpoet, Linus Unnebäck and 3 moreSketch of what would need to happen:
-
Eagerly fetch / read the wasm in
SourceMapConsumer.initializeand save it in some global withinlib/source-map-consumer.js -
Return a promise from
SourceMapConsumer.initializethat resolves after the wasm is instantiated, and therefore we can synchronously create new consumers -
Add a
SourceMapConsumer.synchronouseNew(open to bike shedding on the name) function that throws if the wasm has not been initialized, and otherwise creates a new consumer synchronously
-
What if
SourceMapConsumer.initializeorSourceMapConsumer.withor any other function that is async accepts the already loaded wasm file? In that scenario, you could just not return a promise and it would just work, no globals required.I would prefer not to expose the wasm directly, since it is an implementation detail, and its interface may change. If we could wrap it up and hide the internals, then that could work.
@loganfsmyth and I discussed this topic a couple weeks ago because a sync API is important for tools like babel.
Reacted by lsw2013, Gajus Kuizinas and Joe LencioniI think the upfront-delay approach would be potentially reasonable in my mind, but it is painful in some cases. I've wanted to explore doing that for
babel-registeranyway. We'd definitely need a new major version ofnode-source-map-supportthat returned a promise that resolved once the WASM was finished loading, then other tooling could do a similar thing.The main issue for
babel-registeris that a lot of people use it vianode -r babel-register app.js, and I don't believe Node exposes a way for-ritems to delay execution, so it would probably require some hackery to make that work by overwriting Node-private stuff :(PostCSS needs sync source map support as well. We use source map in throwing errors.
I think the best option would be to load the wasm into the library source directly instead of distributing it separately. This would be a fairly trivial build step and fix this entire issue. You would need to use the sync interface of WASM (
new WebAssembly.Module(buffer)) but it would definitely be worth it.POC sync SourceMapConsumer https://github.com/devsnek/node-source-map-support/blob/master/source_map.js
@devsnek The difficulty there is that is that the sync interface isn't guaranteed to work. Chrome for instance throws on anything beyond 4k: webpack/webpack#6475 (comment) While your approach will work on Node, it does not offer a solution for general usage. You're not wrong though, this would likely be the way to got on the Node side of things. On the browser side, I'm curious about compiling the WASM to ASMjs-style code, but I haven't had time to explore it.
The
@webassemblyjspackages could be of interest.
It looks like there might be synchronous APIs. \cc @xtuc@jdalton can you elaborate?
@loganfsmyth what do you think the current size is?
CC @fitzgen I'm curious what you think?
@jasonLaster
mappings.wasmis 48k at the moment.I'm also not 100% clear on what @jdalton is suggesting.
5 remaining items
- added a commit that references this issue
on May 1, 2019 I noticed a couple other problems with the WebAssembly approach:
-
When debugging code that uses the newer
source-map, the calls likethis._wasm.exports.original_location_for()are a black box. You cannot step into them using the debugger, or inspect the their implementation easily. Even if we could get that working, I'm not familiar with Rust, and it seems like an odd requirement when the rest of the entire stack is JavaScript. -
From a security standpoint, the
mappings.wasmfile contains 48kb of opaque binary data. There is no easy way to decompile it or scan for suspicious code. We simply have to trust that it is the output of building the fitzgen/source-map-mappings project, and that nothing was tampered with along the way.
It totally makes sense to include a WebAssembly binary for people who need these speed optimizations. But it's difficult to understand why the JavaScript reference implementation was completely eliminated.
@fitzgen Would the maintainers consider bringing back the JavaScript implementation and maintaining it alongside the Rust version? If not, maybe it's time to fork this project. This issue has been open for over a year now without resolution. Thanks!
-
Even if we could get that working, I'm not familiar with Rust, and it seems like an odd requirement when the rest of the entire stack is JavaScript.
I can appreciate that it's an unexpected speed bump, but I do question how often users would realistically do that. WASM debugging in devtools is something that will improve over time I'm sure.
There is no easy way to decompile it or scan for suspicious code. We simply have to trust that it is the output of building ...
Since WASM runs in a sandbox, it doesn't really have access to anything to do anything anyway AFAIK. A worst all it could do is return the incorrect result.
Would the maintainers consider bringing back the JavaScript implementation and maintaining it alongside the Rust version?
I think maintaining parallel implementations would be a recipe for inconsistent behavior across the module. I have thought about setting up a build step that would compile the WASM into JS code in order to address the sync case.
I think maintaining parallel implementations would be a recipe for inconsistent behavior across the module.
Is
source-mapan implementation of a specification? If so, there is inherent value in providing a reference implementation written in the language that everybody knows. [Here "everybody" means everybody who installs NPM packages.]Or is the specification merely "whatever the
source-mapprogram code does"? In that case, providing two versions of the code would have some downsides.In my opinion, the current API, where constructors return a Promise, is a code-smell. It is idiomatic for a constructor to return an instance of the class, such that
a = new Foo(); a instanceof Foo.A better API is for the
SourceMapConsumerconstructor to be synchronous, and for it to throw an error if the user has not first initialized the wasm module in environments that require async initialization.Usage in the browser and node can be async like this:
async function createSourceMapConsumer(...) { await SourceMapConsumer.initializeModule(); // does not need to be a static method of the class; can be any static (non-instance) function return new SourceMapConsumer(...); }Usage in node can optionally be synchronous:
function createSourceMapConsumer(...) { return new SourceMapConsumer(...); }Constructor semantics will be simpler because they will no longer return promises.
async function createSourceMapConsumer(), however, can return a promise. I includedcreateSourceMapConsumerfor demonstration purposes but it doesn't not necessarily need to be included in source-map's API.Additionally, in ECMAScript module environments -- browser, node, or otherwise -- the async initialization step can be performed via top-level await in the ESM entrypoint file. The entire module will
importasync, and the API will be synchronous after that.Reacted by Sagnik Pradhan, TeamworkGuy2, Benjamin Hodgson, Ferdinand Prantl, Konstantin Vyatkin, DokkanWiki, Marat and Hiroki Osame@cspotcode/source-mapand@cspotcode/source-map-consumerimplement a sync API for node using the WASM/rust component while preserving an async API for when it is necessary. For anyone who finds this issue in the future, it may work for you.Reacted by Sampo KivistöReacted by Sagnik Pradhan and Lance PollardWe could add a sync API to create a SourceMapConsumer given that you've already fetched and compiled the wasm. Would that work? If so, I can help you craft a PR that implements this.
Hello, was there anything done regarding this solution specifically for browsers ? I like the idea of pre-fetching the wasm and maps + caching, then carrying on from there synchronously.
Edit:
I looked into this and I achieved this by caching the consumer like so:let basicConsumer; //.... window.sourceMap.SourceMapConsumer.fromSourceMap(cachedSourceMap).then( (consumer: any) => { basicConsumer = consumer; }, ); //later on basicConsumer.originalPositionFor({ line: parsed.lineNumber, column: parsed.columnNumber, }); //....not sure if this was obvious but works for me.
@cspotcode can you elaborate? I only see async APIs in those two links.
Actually it just works, though the docs all say async stuff.
import smc from '@cspotcode/source-map' const json = JSON.parse(mapContent) as smc.RawSourceMap const sm = new smc.SourceMapConsumer(json)
Reacted by Philip Healy and lin onetwo- added a commit that references this issue
on Sep 29, 2024 Any news about how to get sourcemap results asynchronously ?
- added a commit that references this issue
on Oct 12, 2025 - added a commit that references this issue
on Oct 26, 2025
Is it possible to use the new Rust wasm lib-mapping without requiring support for async/await or futures, in general?
For context, I'm writing code in a context where all async operations are disallowed, which means that I'm stuck on 0.6.x of source-map. I was hoping to get the performance benefit of the wasm implementation but requiring async is a deal breaker.