github.com/pachyderm/pachyderm@v1.13.4/doc/docs/1.9.x/concepts/pipeline-concepts/index.md (about)

     1  # Pipeline Concepts
     2  
     3  Pachyderm Pipeline System (PPS) is the computational
     4  component of the Pachyderm platform that enables you to
     5  perform various transformations on your data. Pachyderm
     6  pipelines have the following main concepts:
     7  
     8  **Pipeline**
     9  :   A pipeline is a job-spawner that waits for certain
    10      conditions to be met. Most commonly, this means
    11      watching one or more Pachyderm repositories for new
    12      data. When a new data arrives, a pipeline executes
    13      a user-defined piece of code to perform an operation
    14      and process the data. Each of these executions is
    15      called a job.
    16  
    17  **Job**
    18  :   A job is an individual execution of a pipeline. A job
    19      can succeed or fail. Within a job, data and processing
    20      can be broken up into individual units of work called datums.
    21  
    22  **Datum**
    23  :   A datum is the smallest indivisible unit of work within
    24      a job. Different datums can be processed in parallel
    25      within a job.