github.com/pachyderm/pachyderm@v1.13.4/examples/spouts/README.md (about)

     1  >![pach_logo](../img/pach_logo.svg) INFO - Pachyderm 2.0 introduces profound architectural changes to the product. As a result, our examples pre and post 2.0 are kept in two separate branches:
     2  > - Branch Master: Examples using Pachyderm 2.0 and later versions - https://github.com/pachyderm/pachyderm/tree/master/examples
     3  > - Branch 1.13.x: Examples using Pachyderm 1.13 and older versions - https://github.com/pachyderm/pachyderm/tree/1.13.x/examples
     4  
     5  # Pachyderm Spouts Examples
     6  
     7  [Spouts](https://docs.pachyderm.com/1.13.x/concepts/pipeline-concepts/pipeline/spout/) are a way to get streaming data from any source into Pachyderm.
     8  
     9  We have released a new *spouts 2.0* implementation
    10  in Pachyderm 1.12. Please take a look at our examples:
    11  
    12  ## For versions 2.0 and newer
    13  
    14  - [Spout101](https://github.com/pachyderm/pachyderm/tree/master/examples/spouts/spout101)
    15  
    16  - More extensive - Pachyderm's integration of spouts with RabbitMQ: https://github.com/pachyderm/pachyderm/tree/master/examples/spouts/go-rabbitmq-spout 
    17  
    18  ## For versions 1.12.0 to 1.13.2
    19  - [Spout101](https://github.com/pachyderm/pachyderm/tree/1.13.x/examples/spouts/spout101)
    20  
    21  - More extensive - Pachyderm's integration of spouts with RabbitMQ: https://github.com/pachyderm/pachyderm/tree/1.13.x/examples/spouts/go-rabbitmq-spout 
    22  
    23  ## For versions 1.11.x and oldest
    24  
    25  !!! Warning
    26      The following examples are based on our previous version of spout. That implementation is now deprecated. Those examples will be adapted to spout 2.0 shortly.
    27  
    28  ### Email Sentiment Analysis
    29  
    30  [This example](https://github.com/pachyderm/pachyderm/tree/1.13.x/examples/spouts/EmailSentimentAnalyzer) connects to an IMAP mail account, 
    31  collects all the incoming mail and analyzes it for positive or negative sentiment,
    32  sorting the emails into folders in its output repo with scoring information added to a header "X-VADER-Sentiment-Score".
    33  
    34  It is inspired by the [email sentiment analysis bot](https://github.com/shanglun/SentimentAnalyzer) documented in [this article](https://www.toptal.com/java/email-sentiment-analysis-bot) by Shanglung Wang, 
    35  
    36  It uses [Python-based VADER](https://github.com/cjhutto/vaderSentiment) from CJ Hutto at Georgia Tech.
    37  
    38  ```
    39  Hutto, C.J. & Gilbert, E.E. (2014). VADER: A Parsimonious Rule-based Model for
    40  Sentiment Analysis of Social Media Text. Eighth International Conference on
    41  Weblogs and Social Media (ICWSM-14). Ann Arbor, MI, June 2014.
    42  ```
    43  ### Ingress data from an S3 bucket using SQS
    44  
    45  [This example](https://github.com/pachyderm/pachyderm/tree/1.13.x/examples/spouts/SQS-S3) shows how to use spouts to ingress data from an S3 bucket using an SQS queue for notification of new items added to the bucket. 
    46  
    47  ### Commit messages from a Kafka queue
    48  
    49  A [simple example](https://github.com/pachyderm/pachyderm/tree/1.13.x/examples/spouts/go-kafka-spout) of using spouts with Kafka to process messages and write them to files.
    50  
    51  ### Spout Marker
    52  
    53  https://github.com/pachyderm/pachyderm/tree/1.13.x/examples/spouts/spout-marker
    54  
    55  
    56  
    57  
    58