Feature Suggestion: Static Indexing Format for Compiler

[HerringtonDarkholme]

Hi TypeScript team! Congratulations for the new TypeScript-Go release!

I understand that the new Compiler API is currently under design and development. However, I want to discuss the possibility of introducing a static indexing format from the TypeScript compiler that creates a snapshot of the codebase for external tools to consume.

Intriguing Example

We already have fast TypeScript linters in Rust, but they cannot handle type-aware linting. Even if TypeScript-Go is fast and provides Go API, the community needs to repay the effort to rewrite a new linter in Go. Or JS-based tools need to talk to Go using FFI.

Alternatively, a language agnostic language server indexing can help external tools utilize type information.

Design Consideration

This is not a new idea. LSIF from Microsoft and SCIP from Sourcegraph are two prominent examples for type indexing.

While it is less flexible than the Compiler API, it opens up new opportunities for various use cases. LSP/copmiler API are dynamic and interactive. APIs can even read/write/transform code on the fly (e.g. like Vue/Svelte/Angular language tools).

In contrast language indexing provides an offline and static alternative for code analysis and manipulation.

Of course, if TS-go can provide a compiler API in JS/Go, implementing indexing can be done in userland. But if it is hard to ship a JS-Go FFI, the indexing format can be a good workaround.

Usage

1. Offline code browsing like GitHub/SourceGraph
2. Non-JS/non-Go linter integration, e.g. biome, ast-grep
3. Code auditing for compliance and quality checks
4. Automated code rewriting based on types

Prior Art

• Language Server Index Format (LSIF) Specification
• scip-typescript
• lsif-node
• scip

[Brooooooklyn]

I'm working on related attempts in this repo https://github.com/Brooooooklyn/typescript-go-napi

[0xdevalias]

I'm working on related attempts in this repo

Provide JavaScript API for typescript-go

@Brooooooklyn At the risk of being slightly off topic.. (issues weren't open there, so couldn't ask on the repo itself) would you see that as (eventually) being able to be compiled to web assembly and consumed within a service worker/similar? (eg. with Monaco). Or would that be outside of the scope of what you're looking into there?

[Brooooooklyn]

@0xdevalias Go compilation to WebAssembly has very poor performance: https://x.com/youyuxi/status/1899701468480897443. so compiling to WebAssembly is not the primary goal of ts-go-napi. I will try to port capabilities like transform and typecheck to the JS API, so everyone can easily develop tools like ts-loader.

[0xdevalias]

Go compilation to WebAssembly has very poor performance

@Brooooooklyn Ah true, that sucks :(

I will try to port capabilities like transform and typecheck to the JS API, so everyone can easily develop tools like ts-loader.

@Brooooooklyn Makes sense :)

[jakebailey]

#514 is discussing this topic.

I for one don't understand the perf numbers; if you have your testing methodology then that would be best posted there. We did not intend for people to right out the gate try and run this repo in WASM; concurrency is enabled by default when it should not be in WASM.

But I think this is off-topic for this thread.

[jakebailey]

Also note that we are also working on support for the API, and that NAPI is off the table for Go because you can't load multiple Go libs into the same process safely. We tested that out a good while ago.