github.com/auxten/ginkgo@v0.0.0-20220130172820-7d98ad59d232/README.md (about)

     1  # gink-go
     2  Ginkgo is a p2sp file transfer tool designed for ML dataset.
     3  
     4  ## Build
     5  
     6  ```bash
     7  go build cmd/ginkgo.go
     8  ```
     9  
    10  ## Usage
    11  
    12  On server side, just start server with the following command:
    13  
    14  ```bash
    15  ginkgo 
    16  ```
    17  
    18  On client side, just run ginkgo as a scp command, both directory and file are supported.
    19  
    20  ```bash
    21  ginkgo SrcHost:/path/to/src /path/to/dest
    22  ```
    23  
    24  - The `/path/to/src` can be either absolute or relative.
    25  - The src and dest path handling behavior is just compatible with GNU scp command
    26  
    27  ## How
    28  
    29  - Client will get other clients list from server side and broadcast itself to them.
    30  - Client chooses the block to download according to consistent hashing result of its serving `host:port`.
    31  - Other clients also choose which peer to download the block from by the hash.
    32  - The seed.Seed structure records the blocks and files metadata.
    33  
    34  
    35  Golang version of [gingko](https://github.com/auxten/gingko)
    36  
    37  > `ginkgo` is an alternative form of `gingko`. `Gink-go` is the golang version `Gingko`.
    38  
    39  
    40  ## Todo
    41  
    42  - [x] HTTP range downloader
    43  - [x] HTTP file server
    44  - [x] Directory support
    45  - [x] Consistent hashing locator
    46  - [ ] Client rate limit
    47  - [ ] Server rate limit