github.com/fluhus/gostuff@v0.4.1-0.20240331134726-be71864f2b5d/ppln/doc.go (about) 1 // Package ppln provides generic parallel processing pipelines. 2 // 3 // NOTE: this API is currently experimental and may change in future releases. 4 // 5 // # General usage 6 // 7 // This package provides two modes of operation: serial and non-serial. 8 // Serial transforms each value of type T1 to a value of type T2. The outputs 9 // are ordered in the same order of the inputs. Non-serial transforms each value 10 // of type T1 to zero or more values of type T2. The order of the outputs is 11 // arbitrary, but correlated with the order of inputs. 12 // 13 // Each of the functions blocks the calling function until either the processing 14 // is done (puller was called on the last value) or until stopped. 15 // 16 // # Stopping 17 // 18 // Each user-function (pusher, mapper, puller) may return an error. 19 // Returning a non-nil error stops the pipeline prematurely, and that 20 // error will be propagated to the caller. 21 // 22 // # Number of goroutines 23 // 24 // Each pipeline creates ngoroutines+2 new goroutines and blocks the calling 25 // one. There is one pusher goroutine, one puller goroutine, and ngoroutines 26 // mapper goroutines. 27 // 28 // A special case is when ngoroutines==0, in which case no new goroutines are 29 // created. Processing is done serially using the calling goroutine. 30 package ppln