Repository navigation
TypeScript Language Service is slow to load projects over network file systems #16426
Description
Activity
So the idea is to always assume a file exists if it's explicitly listed? IIRC this used to be the old behavior but we stopped that because it was actually slower with Node's module resolution on local file systems.
So the underlying problem is that number of fstat-s in the node module resolution is too high and it is very visible on a network drive. Our current solution is by overriding file/directoryExists - but maybe you can suggest some other way to solve it also.
And we need a clean way to integrate with the language service so that we don't have to maintain a separate entrypoint/binary. The LS plugin mechanism (#11976) was insufficient because the base LSHost is already created with the sys ServerHost and there is no way to override it.
Do you need the node module resolution? can you share a sample
--traceResolutionoutput, i would like to understand what modules are we looking for and where.path mapping is not tied to node module resolution.
Mohamed Hegazy (@mhegazy) the problem, IIRC, is that if you use
"classic", you also get inappropriate file-statting in a different way when walking up the directory spine. I think a mix ofmoduleResolution: "none"andmoduleResolution: "node"is needed, but vikerman and the team can correct me if I'm wrong.Basically:
- Loading from
package.jsonand the like is potentially needed from path-mapped directories. - Walking up each directory's
node_modulesis not needed.
- Loading from
- We actually don't need package.json logic also. We do depend on some minimal parts of the node module lookup like looking for /index (Not sure if classic has that one)
- Walking up for node_modules and Types (@types) are definitely not needed.
Looking through the trace from vikerman, we are doing too many lookups for imports in .d.ts file, e.g. rxjs importing other parts of rxjs. we should optimize that. that will not remove all lookups but should reduce them by a big factor.
If that does not work either, we can try to add a flag to limit file lookups, but i would rather do that as a last resort.
I think we're going to have to push this to the 2.6 milestone until we can discuss any solutions with Mohamed Hegazy (@mhegazy) when he comes back.
- ghost added this to the milestone
on Oct 5, 2017 Just to keep the two linked together, there's also an issue tracking pluggable module resolution at #18896.
This is a duplicate of #11979. Both reports request a way for the language service to avoid Node module-resolution filesystem traversal for an explicitly enumerated project while retaining path mappings, including an integration point for a manifest-backed host.
The broader custom module-resolution hook is tracked by #18896.
TypeScript Version: 2.3.4
We have our code on a networked/FUSE file system with different root directories for generated files and such. We use the node module resolution. To load a fairly simple Angular+Angular material+RxJS project(transitively ~1.7K .ts and .d.ts files) the language service takes around 24 seconds to resolve all modules and return diagnostics. We can see about ~50K file stats being made to resolve everything.
In our case we actually generate tsconfig.json for our editors through a build step and we actually create the list of all the files the project needs in the "files" section of tsconfig.json.
We built a custom solution that uses this files list to proxy the ServerHost in the Language service - to respond to fileExists and directoryExists by using the "files" list instead of hitting the file system a whole lot of times. This reduced the initial project load time to 4 seconds (and is mostly just the overhead of reading all the files) which is very much an
Here is a PR of the fix we have currently - https://github.com/vikerman/TypeScript/pull/1/files
It would be nice to have a cleaner solution for this in the language service itself (maybe as an option?). This would help any team who have a similar network/FUSE file system as their source repository.
Chuck Jazdzewski (@chuckjaz) Alex Eagle (@alexeagle)