github.com/pachyderm/pachyderm@v1.13.4/doc/docs/master/how-tos/use-pachyderm-ide/index.md (about)

     1  # Use the Pachyderm IDE with `python-pachyderm`
     2  
     3  !!! note
     4      The Pachyderm IDE is an enterprise feature,
     5      which is also available for testing during
     6      the 14-day free trial.
     7      Contact sales@pachyderm.com for more information.
     8  
     9  This section describes how you can use the `python-pachyderm`
    10  client from within the Pachyderm IDE.
    11  
    12  !!! note
    13      You need to have Pachyderm IDE installed as described in
    14      [Deploy the Pachyderm IDE](../../deploy-manage/deploy/deploy-pachyderm-ide.md).
    15  
    16  ## Overview
    17  
    18  When you deploy the Pachyderm IDE, you get JupyterHub and
    19  a customized JupyterLab UI running next to your Pachyderm
    20  cluster.
    21  
    22  Before we proceed, let's clarify the following terms
    23  as they might be confusing for first-time users:
    24  
    25  * JupyterHub is a popular data science platform that enables users
    26  to quickly spin out multiple single-tenant Jupyter Notebook server instances.
    27  
    28  * Jupyter Notebooks provide a means for conducting experiments with data and
    29  code written in Python which is familiar to many data scientists. Because of
    30  the built-in rich-text support, visualizations, the easy-to-use web interface,
    31  many enterprise users prefer Jupyter Notebooks to the classic Terminal prompt.
    32  JupyterHub brings all the benefits of Jupyter Notebooks without the need
    33  to install or configure anything on user machines except for a web browser.
    34  
    35  * JupyterLab is an alternative UI for the classic Jupyter Notebook IDE
    36  that enables you to author and test notebooks and
    37  code.
    38  
    39  * [python-pachyderm](https://github.com/pachyderm/python-pachyderm) is an
    40  official Python client for Pachyderm. For Python developers who prefer to
    41  communicate with Pachyderm directly through the API, rather than by using
    42  the `pachctl` tool, `python-pachyderm` is the right choice.
    43  The [API Documentation](https://pachyderm.github.io/python-pachyderm/python_pachyderm.m.html)
    44  describes various API operations that you can execute to interact with
    45  Pachyderm.
    46  
    47  `python-pachyderm` is preinstalled in your Pachyderm IDE.
    48  
    49  ### Difference in Pipeline Creation Methods
    50  
    51  `python-pachyderm` supports the standard
    52  [create_pipeline](https://pachyderm.github.io/python-pachyderm/python_pachyderm.m.html#python_pachyderm.Client.create_pipeline)
    53  method that is
    54  also available through the Pachyderm CLI and UI. When you use
    55  `create_pipeline`, you need to build a new Docker image and push
    56  it to an image registry every
    57  time you update the code in your pipeline. Users that are less familiar
    58  with Docker might find this process a bit cumbersome. However, you must
    59  use this method for all non-Python code.
    60  
    61  When you use `python-pachyderm`, in addition to the
    62  `create_pipeline` method,
    63  you can use the [create_python_pipeline](https://pachyderm.github.io/python-pachyderm/python_pachyderm.m.html#python_pachyderm.create_python_pipeline)
    64  function that does not require
    65  you to include your code in a Docker image and rebuild it each time you make
    66  a change. Instead, this function creates a PFS repository
    67  called `<pipeline_name>_source` and puts the source code into it. Also, it
    68  creates a `<pipeline_name>_build` repository to build Python dependencies.
    69  Therefore, when you use `create_python_pipeline`, your DAG includes two
    70  additional repositories for each pipeline.
    71  Because of that, you do not need
    72  to build a new Docker image every time you change something in your
    73  pipeline code. You can run your code instantly. This method is ideal for
    74  users who want to avoid building Docker images.
    75  
    76  While you can mix and match pipeline creation methods in Pachyderm IDE, you
    77  might eventually want to pick one method that works for your use case. It is a
    78  matter of personal preference which method to use. While some users, those
    79  who write code in Python, in particular, might
    80  find it convenient to avoid the Docker build workflow, others might want to
    81  enable Docker in JupyterHub or build Docker images from their local machines.
    82  
    83  In the [OpenCV Example for JupyterHub](https://github.com/pachyderm/jupyterhub-pachyderm),
    84  both methods are used in the same notebook cell.