github.com/fholzer/ordered-concurrently/v3@v3.0.0-20221001131746-406a6eece748/README.md (about) 1 <a href="https://github.com/tejzpr/ordered-concurrently/actions/workflows/tests.yml"><img src="https://github.com/tejzpr/ordered-concurrently/actions/workflows/tests.yml/badge.svg" alt="Tests"/></a> 2 [](https://codecov.io/gh/tejzpr/ordered-concurrently) 3 [](https://pkg.go.dev/github.com/tejzpr/ordered-concurrently) 4 [](https://gitpod.io/#https://github.com/tejzpr/ordered-concurrently) 5 [](https://goreportcard.com/report/github.com/tejzpr/ordered-concurrently) 6 7 # Ordered Concurrently 8 A library for parallel processing with ordered output in Go. This module processes work concurrently / in parallel and returns output in a channel in the order of input. It is useful in concurrently / parallelly processing items in a queue, and get output in the order provided by the queue. 9 10 # Usage 11 ## Get Module 12 ```go 13 go get github.com/tejzpr/ordered-concurrently/v3 14 ``` 15 ## Import Module in your source code 16 ```go 17 import concurrently "github.com/tejzpr/ordered-concurrently/v3" 18 ``` 19 ## Create a work function by implementing WorkFunction interface 20 ```go 21 // Create a type based on your input to the work function 22 type loadWorker int 23 24 // The work that needs to be performed 25 // The input type should implement the WorkFunction interface 26 func (w loadWorker) Run(ctx context.Context) interface{} { 27 time.Sleep(time.Millisecond * time.Duration(rand.Intn(100))) 28 return w * 2 29 } 30 ``` 31 ## Demo 32 [Go Playground](https://go.dev/play/p/60b_x0YHzYu) 33 34 ## Run 35 ### Example - 1 36 ```go 37 func main() { 38 max := 10 39 inputChan := make(chan concurrently.WorkFunction) 40 ctx := context.Background() 41 output := concurrently.Process(ctx, inputChan, &concurrently.Options{PoolSize: 10, OutChannelBuffer: 10}) 42 go func() { 43 for work := 0; work < max; work++ { 44 inputChan <- loadWorker(work) 45 } 46 close(inputChan) 47 }() 48 for out := range output { 49 log.Println(out.Value) 50 } 51 } 52 ``` 53 ### Example - 2 - Process unknown number of inputs 54 ```go 55 func main() { 56 inputChan := make(chan concurrently.WorkFunction, 10) 57 ctx := context.Background() 58 output := concurrently.Process(ctx, inputChan, &concurrently.Options{PoolSize: 10, OutChannelBuffer: 10}) 59 60 ticker := time.NewTicker(100 * time.Millisecond) 61 done := make(chan bool) 62 wg := &sync.WaitGroup{} 63 go func() { 64 input := 0 65 for { 66 select { 67 case <-done: 68 return 69 case <-ticker.C: 70 inputChan <- loadWorker(input) 71 wg.Add(1) 72 input++ 73 default: 74 } 75 } 76 }() 77 78 var res []loadWorker 79 go func() { 80 for out := range output { 81 res = append(res, out.Value.(loadWorker)) 82 wg.Done() 83 } 84 }() 85 86 time.Sleep(1600 * time.Millisecond) 87 ticker.Stop() 88 done <- true 89 close(inputChan) 90 wg.Wait() 91 92 // Check if output is sorted 93 isSorted := sort.SliceIsSorted(res, func(i, j int) bool { 94 return res[i] < res[j] 95 }) 96 if !isSorted { 97 log.Println("output is not sorted") 98 } 99 } 100 ``` 101 # Credits 102 1. [u/justinisrael](https://www.reddit.com/user/justinisrael/) for inputs on improving resource usage. 103 2. [mh-cbon](https://github.com/mh-cbon) for identifying potential [deadlocks](https://github.com/tejzpr/ordered-concurrently/issues/2).