github.com/google/trillian-examples@v0.0.0-20240520080811-0d40d35cef0e/clone/cmd/sumdbverify/README.md (about)

     1  # `sumdbverify`
     2  
     3  This tool clones the log for sum.golang.org and ensures that:
     4   1. The cloned data matches the commitments made in the log checkpoint
     5   2. That no module + version is declared with different hashes in the log
     6  
     7  ## Background
     8  
     9  This is a quick summary of https://blog.golang.org/module-mirror-launch but is not
    10  intended to replace this introduction.
    11  If you have no context on Go SumDB, read that intro first :-)
    12  
    13  Go SumDB is a Verifiable Log based on Trillian, which contains entries of the form:
    14  ```
    15  github.com/google/trillian v1.3.11 h1:pPzJPkK06mvXId1LHEAJxIegGgHzzp/FUnycPYfoCMI=
    16  github.com/google/trillian v1.3.11/go.mod h1:0tPraVHrSDkA3BO6vKX67zgLXs6SsOAbHEivX+9mPgw=
    17  ```
    18  Every module & version used in the Go ecosystem will have such an entry in this log,
    19  and the values are hashes which commit to the state of the repository and its `go.mod`
    20  file at that particular version.
    21  
    22  Clients can be assured that they have downloaded the same version of a module as
    23  everybody else provided all of the following are true:
    24   1. The hash of what they have downloaded matches an entry in the SumDB Log
    25   2. There is only one entry in the Log for the `module@version`
    26   3. Entries in the Log are immutable / the Log is append-only
    27   4. Everyone else sees the same Log as this user
    28  
    29  Verification of these is performed by different parties:
    30   1. The client checks an inclusion proof for their {module, version, hashes} in the Log
    31   2. This `sumdbverify` tool verifies there are no conflicting versions
    32   3. A [witness](https://github.com/transparency-dev/witness) verifies the log is append-only
    33   4. A [distributor](https://github.com/transparency-dev/distributor) aggregates multiple witness confirmations to verify consensus
    34  
    35  ## Running in Docker
    36  
    37  The `docker-compose` scripts in this directory allow for deployment of the `sumdbclone` tool in a single command:
    38  
    39  ```bash
    40  docker compose up -d
    41  ```
    42  
    43  This will bring up the following containers:
    44   * `sumdbverify-db-1`: the database containing a local clone of the log
    45   * `sumdbverify-clone-1`: a service that periodically syncs the log into the DB
    46   * `sumdbverify-verify-1` a service that periodically reads the DB to verify the contents
    47  
    48  The clone tool will initially run in a batch mode to download all of the entries.
    49  To see the status of this, use the following command:
    50  
    51  ```bash
    52  docker logs sumdbverify-clone-1 -f
    53  
    54  I0404 10:23:13.018137       1 clone.go:177] 18636.3 leaves/s, last leaf=4700160 (remaining: 12130037, ETA: 10m50s), time working=97.1%
    55  ```
    56  
    57  Once the initial batch clone is complete, the tool will periodically poll the log and clone any new entries.
    58  
    59  Any errors in verifying the contents of the log will show up as crash looping of `sumdbverify-verify-1`.
    60  Inspecting the `docker logs` for this process will reveal the cause of the errors.