golang.org/x/tools/gopls@v0.15.3/doc/design/implementation.md (about)

     1  
     2  # Gopls architecture
     3  
     4  Last major update: Jan 16 2024
     5  
     6  This doc presents a high-level overview of the structure of gopls to
     7  help new contributors find their way. It is not intended to be a
     8  complete description of the implementation, nor even of any key
     9  components; for that, the package documentation (linked below) and
    10  other comments within the code are a better guide.
    11  
    12  The diagram below shows selected components of the gopls module and
    13  their relationship to each other according to the Go import graph.
    14  Tests and test infrastructure are not shown, nor are utility packages,
    15  nor packages from the [x/tools] module. For brevity, packages are
    16  referred to by their last segment, which is usually unambiguous.
    17  
    18  The height of each blob corresponds loosely to its technical depth.
    19  Some blocks are wide and shallow, such as [protocol], which declares
    20  Go types for the entire LSP protocol. Others are deep, such as [cache]
    21  and [golang], as they contain a lot of dense logic and algorithms.
    22  
    23  <!-- Source: https://docs.google.com/drawings/d/1CK6YSLt7G3svRoZf7skJI-lxRol2VI90YOxHcYS0DP4 -->
    24  ![Gopls architecture](architecture.svg)
    25  
    26  Starting from the bottom, we'll describe the various components.
    27  
    28  The lowest layer defines the request and response types of the
    29  Language Server Protocol:
    30  
    31  - The [protocol] package defines the standard protocol; it is mostly
    32    generated mechanically from the schema definition provided by
    33    Microsoft.
    34    The most important type is DocumentURI, which represents a `file:`
    35    URL that identifies a client editor document. It also provides
    36    `Mapper`, which maps between the different coordinate systems used
    37    for source positions: UTF-8, UTF-16, and token.Pos.
    38  
    39  - The [command] package defines Gopls's non-standard commands, which
    40    are all invoked through the `workspace/executeCommand` extension
    41    mechanism. These commands are typically returned by the server as
    42    continuations of Code Actions or Code Lenses; most clients do not
    43    construct calls to them directly.
    44  
    45  The next layer defines a number of important and very widely used data structures:
    46  
    47  - The [file] package defines the primary abstractions of a client
    48    file: its `Identity` (URI and content hash), and its `Handle` (which
    49    additionally provides the version and content of a particular
    50    snapshot of the file.
    51  
    52  - The [parsego] package defines `File`, the parsed form of a Go source
    53    file, including its content, syntax tree, and coordinary mappings
    54    (Mapper and token.File). The package performs various kinds of tree
    55    repair to work around error-recovery shortcomings of the Go parser.
    56  
    57  - The [metadata] package defines `Package`, an abstraction of the
    58    metadata of a Go package, similar to the output of `go list -json`.
    59    Metadata is produced from [go/packages], which takes
    60    care of invoking `go list`. (Users report that it works to some extent
    61    with a GOPACKAGESDRIVER for Bazel, though we maintain no tests for this
    62    scenario.)
    63  
    64    The package also provides `Graph`, the complete import graph for a
    65    workspace; each graph node is a `Package`.
    66  
    67  The [settings] layer defines the data structure (effectively a large
    68  tree) for gopls configuration options, along with its JSON encoding.
    69  
    70  The [cache] layer is the largest and most complex component of gopls.
    71  It is concerned with state management, dependency analysis, and invalidation:
    72  the `Session` of communication with the client;
    73  the `Folder`s that the client has opened;
    74  the `View` of a particular workspace tree with particular build
    75  options;
    76  the `Snapshot` of the state of all files in the workspace after a
    77  particular edit operation;
    78  the contents of all files, whether saved to disk (`DiskFile`) or
    79  edited and unsaved (`Overlay`);
    80  the `Cache` of in-memory memoized computations,
    81  such as parsing go.mod files or build the symbol index;
    82  and the `Package`, which holds the results of type checking a package
    83  from Go syntax.
    84  
    85  The cache layer depends on various auxiliary packages, including:
    86  
    87  - The [filecache] package, which manages gopls' persistent, transactional,
    88    file-based key/value store.
    89    
    90  - The [xrefs], [methodsets], and [typerefs] packages define algorithms
    91    for constructing indexes of information derived from type-checking,
    92    and for encoding and decoding these serializable indexes in the file
    93    cache.
    94  
    95    Together these packages enable the fast restart, reduced memory
    96    consumption, and synergy across processes that were delivered by the
    97    v0.12 redesign and described in ["Scaling gopls for the growing Go
    98    ecosystem"](https://go.dev/blog/gopls-scalability).
    99  
   100  The cache also defines gopls's [go/analysis] driver, which runs
   101  modular analysis (similar to `go vet`) across the workspace.
   102  Gopls also includes a number of analysis passes that are not part of vet.
   103  
   104  The next layer defines four packages, each for handling files in a
   105  particular language:
   106  [mod] for go.mod files;
   107  [work] for go.work files;
   108  [template] for files in `text/template` syntax; and
   109  [golang], for files in Go itself.
   110  This package, by far the largest, provides the main features of gopls:
   111  navigation, analysis, and refactoring of Go code.
   112  As most users imagine it, this package _is_ gopls.
   113  
   114  The [server] package defines the LSP service implementation, with one
   115  handler method per LSP request type. Each handler switches on the type
   116  of the file and dispatches to one of the four language-specific
   117  packages.
   118  
   119  The [lsprpc] package connects the service interface to our [JSON RPC](jsonrpc2)
   120  server.
   121  
   122  Bear in mind that the diagram is a dependency graph, a "static"
   123  viewpoint of the program's structure. A more dynamic viewpoint would
   124  order the packages based on the sequence in which they are encountered
   125  during processing of a particular request; in such a view, the bottom
   126  layer would represent the "wire" (protocol and command), the next
   127  layer up would hold the RPC-related packages (lsprpc and server), and
   128  features (e.g. golang, mod, work, template) would be at the top.
   129  
   130  <!--
   131  A dynamic view would be an interesting topic for another article.
   132  This slide deck [requires Google network]
   133  The Life of a (gopls) Query (Oct 2021)
   134  https://docs.google.com/presentation/d/1c8XJaIldzii-F3YvEOPWHK_MQJ_o8ua5Bct1yDa3ZlU
   135  provides useful (if somewhat out of date) information.
   136  -->
   137  
   138  The [cmd] package defines the command-line interface of the `gopls`
   139  command, around which gopls's main package is just a trivial wrapper.
   140  It is usually run without arguments, causing it to start a server and
   141  listen indefinitely.
   142  It also provides a number of subcommands that start a server, make a
   143  single request to it, and exit, providing traditional batch-command
   144  access to server functionality. These subcommands are primarily
   145  provided as a debugging aid (but see
   146  [#63693](https://github.com/golang/go/issues/63693)).
   147  
   148  [cache]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cache
   149  [cmd]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cmd
   150  [command]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/protocol/command
   151  [debug]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/debug
   152  [file]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/file
   153  [filecache]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/filecache
   154  [go/analysis]: https://pkg.go.dev/golang.org/x/tools@master/go/analysis
   155  [go/packages]: https://pkg.go.dev/golang.org/x/tools@master/go/packages
   156  [gopls]: https://pkg.go.dev/golang.org/x/tools/gopls@master
   157  [jsonrpc2]: https://pkg.go.dev/golang.org/x/tools@master/internal/jsonrpc2
   158  [lsprpc]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/lsprpc
   159  [memoize]: https://github.com/golang/tools/tree/master/internal/memoize
   160  [metadata]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cache/metadata
   161  [methodsets]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cache/methodsets
   162  [mod]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/mod
   163  [parsego]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cache/parsego
   164  [protocol]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/protocol
   165  [server]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/server
   166  [settings]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/settings
   167  [golang]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/golang
   168  [template]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/template
   169  [typerefs]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cache/typerefs
   170  [work]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/work
   171  [x/tools]: https://github.com/golang/tools@master
   172  [xrefs]: https://pkg.go.dev/golang.org/x/tools/gopls@master/internal/cache/xrefs