github.com/auxten/ginkgo@v0.0.0-20220130172820-7d98ad59d232/README.md (about) 1 # gink-go 2 Ginkgo is a p2sp file transfer tool designed for ML dataset. 3 4 ## Build 5 6 ```bash 7 go build cmd/ginkgo.go 8 ``` 9 10 ## Usage 11 12 On server side, just start server with the following command: 13 14 ```bash 15 ginkgo 16 ``` 17 18 On client side, just run ginkgo as a scp command, both directory and file are supported. 19 20 ```bash 21 ginkgo SrcHost:/path/to/src /path/to/dest 22 ``` 23 24 - The `/path/to/src` can be either absolute or relative. 25 - The src and dest path handling behavior is just compatible with GNU scp command 26 27 ## How 28 29 - Client will get other clients list from server side and broadcast itself to them. 30 - Client chooses the block to download according to consistent hashing result of its serving `host:port`. 31 - Other clients also choose which peer to download the block from by the hash. 32 - The seed.Seed structure records the blocks and files metadata. 33 34 35 Golang version of [gingko](https://github.com/auxten/gingko) 36 37 > `ginkgo` is an alternative form of `gingko`. `Gink-go` is the golang version `Gingko`. 38 39 40 ## Todo 41 42 - [x] HTTP range downloader 43 - [x] HTTP file server 44 - [x] Directory support 45 - [x] Consistent hashing locator 46 - [ ] Client rate limit 47 - [ ] Server rate limit