github.com/abayer/test-infra@v0.0.5/velodrome/fetcher/README.md (about)

     1  Overview
     2  ========
     3  
     4  Fetcher retrieves a github repository history and stores it in a MySQL local
     5  database.
     6  
     7  For now, it downloads three types of resources:
     8  - Issues (including pull-requests)
     9  - Events (associated to issues)
    10  - Comments (regular comments and review comments)
    11  
    12  All of these resources will allow us to:
    13  - Compute average time-to-resolution for an issue/pull-request
    14  - Compute time between label creation/removal: lgtm'd, merged
    15  - break-down based on specific flags (size, priority, ...)
    16  
    17  The model is described more precisely in [../sql](../sql/).
    18  
    19  Fetcher only downloads what is not already in the SQL database by trying to find
    20  the latest events it knows about. It will poll from Github on a regular basis,
    21  and this can be configured with the `--frequency` flag.
    22  
    23  There is no set-up required as `fetcher` will create the github database if it
    24  doesn't exist, along with the various required tables.
    25  
    26  Testing locally
    27  ===============
    28  
    29  It is strongly suggested NOT to download Kubernetes/kubernetes from scratch with
    30  the fetcher, as it puts a lot of pressure on the token and takes a significant
    31  amount of time. It has been done already and one should use an export of the
    32  existing database.
    33  
    34  It is fine to download a smaller repository for testing, like
    35  kubernetes/test-infra or kubernetes/contrib.
    36  
    37  The SQL database can be set-up on Google Cloud SQL, and then accessed with the
    38  cloud-sql proxy as described here:
    39  https://github.com/GoogleCloudPlatform/cloudsql-proxy
    40  
    41  
    42  Create a new version
    43  ====================
    44  Push
    45  ----
    46  
    47  Run the push script:
    48  ```
    49  make push IMG=gcr.io/your-repo/fetcher
    50  ```
    51  
    52  Update deployment
    53  -----------------
    54  
    55  Update the `deployment.yaml` with the new generated version.