kythe.io@v0.0.68-0.20240422202219-7225dbc01741/kythe/typescript/README.md (about)

     1  # Kythe indexer for TypeScript
     2  
     3  ## Development
     4  
     5  ### Building code
     6  
     7  ```shell
     8  cd kythe/typescript
     9  bazel test :all
    10  ```
    11  
    12  
    13  ### Running tests
    14  
    15  To run tests use:
    16  
    17  ```shell
    18  cd kythe/typescript
    19  bazel test :indexer_test
    20  ```
    21  
    22  To run single test from file `testdata/foo.ts`:
    23  
    24  ```shell
    25  bazel test --test_arg=foo :indexer_test
    26  ```
    27  
    28  ### Writing tests
    29  
    30  By default in TypeScript, files are "scripts", where every declaration is in the
    31  global scope. If the file has any `import` or `export` declaration, they become
    32  a "module", where declarations are local. To make tests isolated from one
    33  another, prefix each test with an `export {}` to make them modules. In larger
    34  TypeScript projects this doesn't come up because all files are modules.
    35  
    36  ### Build system and dependencies
    37  
    38  We use [aspect-build/rules_ts](https://github.com/aspect-build/rules_ts) to handle
    39  TypeScript rules. Dependencies are specified in `package.json` file whick you can find
    40  in the top-level directory of this repo and not in the current package.
    41  
    42  We don't use `package-lock.json` and instead use [pnpm](https://pnpm.io/motivation)
    43  (pnpm-lock.yaml in the top-level directory) to pin dependencies. When updating
    44  dependencies in `package.json` you need to update `pnpm-lock.yaml` by
    45  running `npx pnpm update`.
    46  
    47  ## Design notes
    48  
    49  ### Plugin system
    50  
    51  Indexer is based on plugin system. Each plugin takes an `IndexerHost` instance
    52  and emits Kythe nodes, facts and edges. Plugin usually iterates through a set of
    53  srcs files and processes them resulting in Kythe data.
    54  
    55  There is one default plugin that always included - `TypescriptIndexer`. This
    56  plugin does the main indexing work: go through all symbols in file and emit
    57  nodes and edges for them. Other plugins live outside of this repo and are can
    58  be passed as optional array.
    59  
    60  ### Separate compilation
    61  
    62  The Google TypeScript build relies heavily on TypeScript's `--declaration` flag
    63  to enable separate compilation. The way this works is that after compiling
    64  library A, we generate -- using that flag -- the "API shape" of A into `a.d.ts`.
    65  Then when compiling a library B that uses A, we compile `b.ts` and `a.d.ts`
    66  together.  The Kythe process sees the same files as well.
    67  
    68  What this means for indexing design is that a TypeScript compilation may see
    69  only the generated shape of a module, and not its internals.  For example,
    70  given a file like
    71  
    72  ```
    73  class C {
    74    get x(): string { return 'x'; }
    75  }
    76  ```
    77  
    78  The generated `.d.ts` file for it describes this getter as if it was a readonly
    79  property:
    80  
    81  ```
    82  class C {
    83    readonly x: string;
    84  }
    85  ```
    86  
    87  In practice, what this means is that code should not assume it can to "peek into"
    88  another module to determine the VNames of entities. Instead, when looking at
    89  some hypothetical code that accesses the `x` member of an instance of `C`, we
    90  should use a consistent naming scheme to refer to `x`.
    91  
    92  ### Choosing VNames
    93  
    94  In code like:
    95  
    96  ```
    97  let x = 3;
    98  x;
    99  ```
   100  
   101  the TypeScript compiler resolves the `x`s together into a single `Symbol`
   102  object. This concept maps nicely to Kythe's `VName` concept except that
   103  `Symbol`s do not themselves have unique names.
   104  
   105  You might at first think that you could just, at the `let x` line, choose a name
   106  for the `Symbol` there and then reuse it for subsequent references. But it's not
   107  guaranteed that you syntactically encounter a definition before its use, because
   108  code like this is legal:
   109  
   110  ```
   111  x();
   112  function x() {}
   113  ```
   114  
   115  So the current approach instead starts from the `Symbol`, then from there jumps
   116  to the *declarations* of the `Symbol`, which then point to syntactic positions
   117  (like the `function` above), and then from there maps the declaration back to
   118  the containing scopes to choose a unique name.
   119  
   120  This seems to work so far but we might need to revisit it. I'm not yet clear on
   121  whether this approach is correct for symbols with declarations in multiple
   122  modules, for example.
   123  
   124  ### Module name
   125  
   126  A file `foo/bar.ts` has an associated *module name* `foo/bar`. This is distinct
   127  (without the extension) because it's also possible to define that module via
   128  other file names, such as `foo/bar.d.ts`, and all such files all define into the
   129  single extension-less namespace.
   130  
   131  TypeScript's `rootDirs`, which merge directories into a single shared namespace
   132  (e.g. like the way `.` and `bazel-bin` are merged in bazel), also are collapsed
   133  when computing a module name. In the test suite we use a `fake-genfiles`
   134  directory to recreate a `rootDirs` environment.
   135  
   136  Semantic VNames (like an exported value) use the module name as the 'path' field
   137  of the VName. VNames that refer specifically to the file, such as the file text,
   138  use the real path of the file (including the extension).