github.com/dolthub/go-mysql-server@v0.18.0/ARCHITECTURE.md (about)

     1  # Architecture overview
     2  
     3  This document provides an overview of all parts and pieces of the
     4  project as well as how they fit together. It is meant to help new
     5  contributors understand where things may be, and how changes in some
     6  components may interact with other components of the system.
     7  
     8  ## Root package (`sqle`)
     9  
    10  This is where the engine lives. The engine is the piece that
    11  coordinates and makes all other pieces work together as well as the
    12  main API users of the system will use to create and configure an
    13  engine and perform queries.
    14  
    15  ## Engine tests
    16  
    17  Engine tests live in the `enginetest` package, and are written in a
    18  harnessed manner to allow integrators to run them on their own
    19  database implementation. The `memory_engine_test.go` runs these tests
    20  on the built-in in-memory database implementation in the `memory`
    21  package.
    22  
    23  ### How to add integration tests
    24  
    25  Most new features / bug fixes should be tested by adding a new test
    26  query and expected results to the list in `queries.go`.
    27  
    28  ## `sql`
    29  
    30  This package is probably the most important of the project. It has
    31  several main roles:
    32  - Defines the main interfaces used in the rest of the packages `Node`,
    33    `Expression`, ...
    34  - Provides implementations of components used in the rest of the
    35    packages `Row`, `Context`, `ProcessList`, `Catalog`, ...
    36  - Defines the `information_schema` database, which is a special
    37    database and contains some information about the schemas of other
    38    tables.
    39  
    40  ### `sql/analyzer`
    41  
    42  The analyzer is the most complex component of the project. The
    43  analyzer takes the parsed query plan and transforms it into an
    44  execution plan by running a series of rules. These rules do many
    45  things, such as resolve tables and columns, removing redundant data,
    46  optimizing things for performance, etc.
    47  
    48  There are several phases on the analyzer, because some rules need to
    49  be run before others, some need to be executed several times, other
    50  just once, etc.  Inside `rules.go` are all the default rules and the
    51  phases in which they're executed.
    52  
    53  On top of that, all available rules are defined in this package. Each
    54  rule has a specific role in the analyzer. Rules should be as small and
    55  atomic as possible and try to do only one job and always produce a
    56  tree that is as resolved as the one it received or more.
    57  
    58  ### `sql/expression`
    59  
    60  This package includes the implementation of all the SQL expressions
    61  available in go-mysql-server, except functions. Arithmetic operators,
    62  logic operators, conversions, etc are implemented here.
    63  
    64  Inside `registry.go` there is a registry of all the default functions,
    65  even if they're not defined here.
    66  
    67  ### `sql/expression/function`
    68  
    69  Implementation of all the functions available in go-mysql-server.
    70  
    71  ### `sql/expression/function/aggregation`
    72  
    73  Implementation of all the aggregation functions available in
    74  go-mysql-server.
    75  
    76  ### `sql/parse`
    77  
    78  This package exposes the `Parse` function, which parses a SQL query
    79  and translates it into a query plan.
    80  
    81  Parsing is done using `vitess` parser with a few custom additions for
    82  non-standard syntax.
    83  
    84  ### `sql/plan`
    85  
    86  All the different nodes of the execution plan (except for very
    87  specific nodes used in some optimisation rules) are defined here.
    88  
    89  For example, `SELECT foo FROM bar` is translated into the following
    90  plan:
    91  
    92  ```
    93  Project(foo)
    94   |- Table(bar)
    95  ```
    96  
    97  Which means, the execution plan is a `Project` node projecting `foo`
    98  and has a `ResolvedTable`, which is `bar` as its children.
    99  
   100  Each node inside this package implements at least the `sql.Node`
   101  interface, but it can implement more. `sql.Expressioner`, for example.
   102  
   103  Along with the nodes, `Inspect` and `Walk` functions are provided as
   104  utilities to inspect an execution tree.
   105  
   106  ## `server`
   107  
   108  Contains all the code to turn an engine into a runnable server that
   109  can communicate using the MySQL wire protocol.
   110  
   111  ## `auth`
   112  
   113  This package contains all the code related to the audit log,
   114  authentication and permission management in go-mysql-server.
   115  
   116  There are two authentication methods:
   117  - **None:** no authentication needed.
   118  - **Native:** authentication performed with user and password. Read,
   119    write or all permissions can be specified for those users. It can
   120    also be configured using a JSON file.
   121  
   122  ## `internal/similartext`
   123  
   124  Contains a function to `Find` the most similar name from an array to a
   125  given one using the Levenshtein distance algorithm. Used for
   126  suggestions on errors.
   127  
   128  ## `_integration`
   129  
   130  To ensure compatibility with some clients, there is a small example
   131  connecting and querying a go-mysql-server server from those
   132  clients. Each folder corresponds to a different client.
   133  
   134  For more info about supported clients see
   135  [SUPPORTED_CLIENTS.md](/SUPPORTED_CLIENTS.md).
   136  
   137  These integrations tests can be run using this command:
   138  
   139  ```
   140  make TEST=${CLIENT FOLDER NAME} integration
   141  ```
   142  
   143  It will take care of setting up the test server and shutting it down.
   144  
   145  ## `_example`
   146  
   147  A small example of how to use go-mysql-server to create a server and
   148  run it.
   149  
   150  # Connecting the dots
   151  
   152  `server` uses the engine defined in `sql`.
   153  
   154  Engine uses audit logs and authentication defined in `auth`, parses
   155  using `sql/parse` to convert a query into a query plan, with nodes
   156  defined in `sql/plan` and expressions defined in `sql/expression`,
   157  `sql/expression/function` and `sql/expression/function/aggregation`.
   158  
   159  After parsing, the obtained query plan is analyzed using the analyzer
   160  defined in `sql/analyzer` and its rules to resolve tables, fields,
   161  databases, apply optimisation rules, etc.
   162  
   163  If indexes can be used, the analyzer will transform the query so it
   164  uses indexes (either supplied natively by tables or provided by an
   165  external driver).
   166  
   167  Once the plan is analyzed, it will be executed recursively from the
   168  top of the tree to the bottom to obtain the results and they will be
   169  sent back to the client using the MySQL wire protocol.