github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.11.x/concepts/pipeline-concepts/job.md (about)

     1  # Job
     2  
     3  A Pachyderm job is an execution of a pipeline that triggers
     4  when new data is detected in an input repository. Each
     5  job runs your code against the current commit and
     6  then submits the results to the output repository and creates a single output commit. A pipeline
     7  triggers a new job every time you submit new changes, a commit, into your
     8  input source.
     9  
    10  Each job has the following stages:
    11  
    12  | Stage     | Description  |
    13  | --------- | ------------ |
    14  | Starting  | Pachyderm starts the job when it detects new data in the input repository. <br> The new data appears as a commit in the input repository, and Pachyderm <br> automatically launches the job. Pachyderm spins the number of Pachyderm worker pods <br> specified in the pipeline spec and spreads the workload among them. |
    15  | Running   | Pachyderm runs the transformation code that is specified <br> in the pipeline specification against the data in the input commit. |
    16  | Merging   | Pachyderm concatenates the results of the processed <br> data into one or more files, uploads them to the output repository, completes the final output commits, and creates/persists all the versioning metadata |