Repository navigation
Building a TypeScript Syntax Tree with source from another language (F#) #1514
Description
Activity
Daniel Rosenwasser (@DanielRosenwasser) is putting together a walk through, it would be great to get your feedback about it early.
Will do! Thanks a lot for your reply. If Daniel Rosenwasser (@DanielRosenwasser) can give me a link to the draft, I'll start checking it right away :)
I am also interested in the draft. I spent time in June trying to access the TypeScript AST from TypeScript itself. I put the effort aside while the new language service was being developed. With access to the TypeScript AST, we can figure out how consume it from .NET as I mentioned here.
Hey guys, we're working on it and looking at the current constraints on such a task. I will keep you all posted.
No hurry! Please take your time and tell us if we can help with anything. Thanks a lot for all your hard work 👍
I got the ball rolling with TypeScript 1.4 AST from Node.js.
Now I'm trying to generate TypeScript files and actively working on this right now for work. I posted this as a question on StackOverflow:
How do I use TypeScript 1.4 API to write a .ts file?
Am I on the right track? What do I need to do to have it populate the
.textproperties?///<reference path='../TypeScript/built/local/typescript.d.ts' /> import ts = require('typescript'); export function main() { var m = <ts.ModuleDeclaration>ts.createNode(ts.SyntaxKind.ModuleDeclaration); var nm = <ts.Identifier>ts.createNode(ts.SyntaxKind.Identifier); nm.text = "Blah" m.name = nm; var sf = <ts.SourceFile>ts.createNode(ts.SyntaxKind.SourceFile); sf.statements = <ts.NodeArray<ts.ModuleElement>>[m]; console.log("file: "+sf.text); // sf.text is undefined :( } main();
Hey guys, we have some documentation in flux here.
I've spoken with others on the team; there are difficulties associated with dynamically creating an AST and using our emitter. For one thing, there are times in which we look back at the source text for certain operations, so you need to have a valid TypeScript corpus to base some checking/emit off of. I don't think an AST-to-AST transformation is going to work as very ideally.
It honestly may be better to do a straight TS emit if what you're looking for is definition file emit, if you're looking to use it as a verification pass for an intermediate language, or if you're looking to take advantage of our downlevel emit functionality.
Cameron Taggart (@ctaggart) to populate
.textproperties, you simply assign them as you are doing. Our approach to reasoning is "write once" throughout different stages of the compiler. So for instance, thetextfield will be set once during parsing and never written again; flags for type checking are only set during type checking and never written to again.suggestion: putting the wiki in a repo of MD files would help with people updating and sending PRs
The basic building block of the Abstract Syntax Tree (AST). In general node represent non-terminals in the language grammar; some terminals are kept in the tree such as identifiers and literals.
Just curious which terminals are excluded? Perhaps
comments. I haven't reviewed the grammer (I know its in the specification).Basarat Ali Syed (@basarat) Wiki suggestion isn't a bad idea, we'll consider it.
Terminals which have useful (i.e. semantically meaningful) textual content are kept. So for instance, anything that has a
textproperty will have that. On the other hand, you don't need something like theEqualsGreaterThan(=>) token when you know you have an arrow function, so we don't hang on to it.Off the top of my head, I can think of
SyntaxKindNumericLiteralStringLiteralTemplateHeadLiteralTemplateMiddleLiteralTemplateTailLiteralIdentifierNoSubstitutionTemplateLiteralIf you do need tokens beyond those on any given
Node, they can be "rehydrated" on demand using the sourcetext by callinggetChildren().- added a commit that references this issue
on Jan 8, 2015 Daniel Rosenwasser (@DanielRosenwasser) Thanks for the documentation!
My current attempt is failing with a
Error: Object #<SourceFileObject> has no method 'getSyntacticDiagnostics'. My guess is that I need to create the SourceFile some other way or do some more processing to it. Any ideas?https://github.com/ctaggart/TsAst/blob/c6db6f6b5d35cae81e2cb037280bb2914ab403ab/app.ts
///<reference path='../TypeScript/built/local/typescript.d.ts' /> import ts = require('typescript'); // https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API function compile(sourceFile: ts.SourceFile) { var outputs = []; var compilerOptions: ts.CompilerOptions = { noImplicitAny: true, noEmitOnError: true, target: ts.ScriptTarget.Latest, module: ts.ModuleKind.CommonJS }; var defaultCompilerHost = ts.createCompilerHost(compilerOptions); var compilerHost: ts.CompilerHost = { getSourceFile: function (filename, languageVersion, onError) { console.log("getSourcefile: " + filename); if (filename === "file.ts") //return ts.createSourceFile(filename, contents, compilerOptions.target); return sourceFile; return defaultCompilerHost.getSourceFile(filename, languageVersion, onError); }, writeFile: function (name, text, writeByteOrderMark) { outputs.push({ name: name, text: text, writeByteOrderMark: writeByteOrderMark }); }, getDefaultLibFilename: defaultCompilerHost.getDefaultLibFilename, useCaseSensitiveFileNames: defaultCompilerHost.useCaseSensitiveFileNames, getCanonicalFileName: defaultCompilerHost.getCanonicalFileName, getCurrentDirectory: defaultCompilerHost.getCurrentDirectory, getNewLine: defaultCompilerHost.getNewLine }; var program = ts.createProgram(["file.ts"], compilerOptions, compilerHost); var errors = program.getDiagnostics(); if (!errors.length) { var checker = program.getTypeChecker(true); errors = checker.getDiagnostics(); program.emitFiles(); } return { outputs: outputs, errors: errors.map( e => e.file.filename + "(" + e.file.getLineAndCharacterFromPosition(e.start).line + "): " + e.messageText ) }; } export function main() { var m = <ts.ModuleDeclaration>ts.createNode(ts.SyntaxKind.ModuleDeclaration); var nm = <ts.Identifier>ts.createNode(ts.SyntaxKind.Identifier); nm.text = "Blah" m.name = nm; var sf = <ts.SourceFile>ts.createNode(ts.SyntaxKind.SourceFile); //sf.statements = <ts.NodeArray<ts.ModuleElement>>[m]; var result = compile(sf); console.log(JSON.stringify(result)); } main();
SourceFiles are a bit more involved than otherNodes - they are created in thecreateSourceFilefunction inparser.tsrather than simply withcreateNode.Thanks a lot for the documentation Daniel Rosenwasser (@DanielRosenwasser) and for the work on the compiler services. They're looking great! Understood about emitting source files instead of building an AST directly. In any case, understanding the AST makes it easier to build an emitter and we can still use the compiler API to parse d.ts files for interacting with JS libraries in a type-safe way from F#.
Thanks again! I'm closing the issue now. Please reopen it if necessary.
Ah, understood - that's a use case I hadn't yet realized. Please feel free to ask us any further questions about it.
For future travelers, Daniel Rosenwasser (@DanielRosenwasser) just added this example on how to build a source string from a synthetic AST: https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API#creating-and-printing-a-typescript-ast
Reacted by Cameron Taggart, Dave and Milan PavlikReacted by Alfonso Garcia-Caro, Alex Rattray and Cameron Taggart- locked and limited conversation to collaborators
on Jun 18, 2018
Hi there! Since TypeScript is getting more and more wonderful features like tuples or union types, I wonder if it makes sense to retarget a project like FunScript (F# to JS compiler) to TypeScript and leverage all its power. Cameron Taggart (@ctaggart) suggested we could use the improved compiler services to create an AST directly instead of generating TS code.
From the repository I can see there's still some work to be done with the compiler services. But if you have time, I'd like to make a couple of questions to know what to expect of the upcoming version.
Thanks a lot in advance!