github.com/fholzer/ordered-concurrently/v3@v3.0.0-20221001131746-406a6eece748/README.md (about)

     1  <a href="https://github.com/tejzpr/ordered-concurrently/actions/workflows/tests.yml"><img src="https://github.com/tejzpr/ordered-concurrently/actions/workflows/tests.yml/badge.svg" alt="Tests"/></a>
     2  [![codecov](https://codecov.io/gh/tejzpr/ordered-concurrently/branch/master/graph/badge.svg?token=6WIXWRO3EW)](https://codecov.io/gh/tejzpr/ordered-concurrently)
     3  [![Go Reference](https://pkg.go.dev/badge/github.com/tejzpr/ordered-concurrently.svg)](https://pkg.go.dev/github.com/tejzpr/ordered-concurrently)
     4  [![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/tejzpr/ordered-concurrently)
     5  [![Go Report Card](https://goreportcard.com/badge/github.com/tejzpr/ordered-concurrently)](https://goreportcard.com/report/github.com/tejzpr/ordered-concurrently)
     6  
     7  # Ordered Concurrently
     8  A library for parallel processing with ordered output in Go. This module processes work concurrently / in parallel and returns output in a channel in the order of input. It is useful in concurrently / parallelly processing items in a queue, and get output in the order provided by the queue.
     9  
    10  # Usage 
    11  ## Get Module
    12  ```go
    13  go get github.com/tejzpr/ordered-concurrently/v3
    14  ```
    15  ## Import Module in your source code
    16  ```go
    17  import concurrently "github.com/tejzpr/ordered-concurrently/v3" 
    18  ```
    19  ## Create a work function by implementing WorkFunction interface
    20  ```go
    21  // Create a type based on your input to the work function
    22  type loadWorker int
    23  
    24  // The work that needs to be performed
    25  // The input type should implement the WorkFunction interface
    26  func (w loadWorker) Run(ctx context.Context) interface{} {
    27  	time.Sleep(time.Millisecond * time.Duration(rand.Intn(100)))
    28  	return w * 2
    29  }
    30  ```
    31  ## Demo 
    32  [Go Playground](https://go.dev/play/p/60b_x0YHzYu)
    33  
    34  ## Run
    35  ### Example - 1
    36  ```go
    37  func main() {
    38  	max := 10
    39  	inputChan := make(chan concurrently.WorkFunction)
    40  	ctx := context.Background()
    41  	output := concurrently.Process(ctx, inputChan, &concurrently.Options{PoolSize: 10, OutChannelBuffer: 10})
    42  	go func() {
    43  		for work := 0; work < max; work++ {
    44  			inputChan <- loadWorker(work)
    45  		}
    46  		close(inputChan)
    47  	}()
    48  	for out := range output {
    49  		log.Println(out.Value)
    50  	}
    51  }
    52  ```
    53  ### Example - 2 - Process unknown number of inputs
    54  ```go
    55  func main() {
    56  	inputChan := make(chan concurrently.WorkFunction, 10)
    57  	ctx := context.Background()
    58  	output := concurrently.Process(ctx, inputChan, &concurrently.Options{PoolSize: 10, OutChannelBuffer: 10})
    59  
    60  	ticker := time.NewTicker(100 * time.Millisecond)
    61  	done := make(chan bool)
    62  	wg := &sync.WaitGroup{}
    63  	go func() {
    64  		input := 0
    65  		for {
    66  			select {
    67  			case <-done:
    68  				return
    69  			case <-ticker.C:
    70  				inputChan <- loadWorker(input)
    71  				wg.Add(1)
    72  				input++
    73  			default:
    74  			}
    75  		}
    76  	}()
    77  
    78  	var res []loadWorker
    79  	go func() {
    80  		for out := range output {
    81  			res = append(res, out.Value.(loadWorker))
    82  			wg.Done()
    83  		}
    84  	}()
    85  
    86  	time.Sleep(1600 * time.Millisecond)
    87  	ticker.Stop()
    88  	done <- true
    89  	close(inputChan)
    90  	wg.Wait()
    91  
    92  	// Check if output is sorted
    93  	isSorted := sort.SliceIsSorted(res, func(i, j int) bool {
    94  		return res[i] < res[j]
    95  	})
    96  	if !isSorted {
    97  		log.Println("output is not sorted")
    98  	}
    99  }
   100  ```
   101  # Credits
   102  1.  [u/justinisrael](https://www.reddit.com/user/justinisrael/) for inputs on improving resource usage.
   103  2.  [mh-cbon](https://github.com/mh-cbon) for identifying potential [deadlocks](https://github.com/tejzpr/ordered-concurrently/issues/2).