github.com/Jeffail/benthos/v3@v3.65.0/website/docs/components/inputs/twitter_search.md (about) 1 --- 2 title: twitter_search 3 type: input 4 status: experimental 5 categories: ["Services","Social"] 6 --- 7 8 <!-- 9 THIS FILE IS AUTOGENERATED! 10 11 To make changes please edit the contents of: 12 lib/input/twitter_search.go 13 --> 14 15 import Tabs from '@theme/Tabs'; 16 import TabItem from '@theme/TabItem'; 17 18 :::caution EXPERIMENTAL 19 This component is experimental and therefore subject to change or removal outside of major version releases. 20 ::: 21 Consumes tweets matching a given search using the Twitter recent search V2 API. 22 23 24 <Tabs defaultValue="common" values={[ 25 { label: 'Common', value: 'common', }, 26 { label: 'Advanced', value: 'advanced', }, 27 ]}> 28 29 <TabItem value="common"> 30 31 ```yaml 32 # Common config fields, showing default values 33 input: 34 label: "" 35 twitter_search: 36 query: "" 37 tweet_fields: [] 38 poll_period: 1m 39 backfill_period: 5m 40 cache: "" 41 api_key: "" 42 api_secret: "" 43 ``` 44 45 </TabItem> 46 <TabItem value="advanced"> 47 48 ```yaml 49 # All config fields, showing default values 50 input: 51 label: "" 52 twitter_search: 53 query: "" 54 tweet_fields: [] 55 poll_period: 1m 56 backfill_period: 5m 57 cache: "" 58 cache_key: last_tweet_id 59 rate_limit: "" 60 api_key: "" 61 api_secret: "" 62 ``` 63 64 </TabItem> 65 </Tabs> 66 67 Continuously polls the [Twitter recent search V2 API](https://developer.twitter.com/en/docs/twitter-api/tweets/search/api-reference/get-tweets-search-recent) for tweets that match a given search query. 68 69 Each tweet received is emitted as a JSON object message, with a field `id` and `text` by default. Extra fields [can be obtained from the search API](https://developer.twitter.com/en/docs/twitter-api/fields) when listed with the `tweet_fields` field. 70 71 In order to paginate requests that are made the ID of the latest received tweet is stored in a [cache resource](/docs/components/caches/about), which is then used by subsequent requests to ensure only tweets after it are consumed. It is recommended that the cache you use is persistent so that Benthos can resume searches at the correct place on a restart. 72 73 Authentication is done using OAuth 2.0 credentials which can be generated within the [Twitter developer portal](https://developer.twitter.com). 74 75 76 ## Fields 77 78 ### `query` 79 80 A search expression to use. 81 82 83 Type: `string` 84 85 ### `tweet_fields` 86 87 An optional list of additional fields to obtain for each tweet, by default only the fields `id` and `text` are returned. For more info refer to the [twitter API docs.](https://developer.twitter.com/en/docs/twitter-api/fields) 88 89 90 Type: `array` 91 Default: `[]` 92 93 ### `poll_period` 94 95 The length of time (as a duration string) to wait between each search request. This field can be set empty, in which case requests are made at the limit set by the rate limit. This field also supports cron expressions. 96 97 98 Type: `string` 99 Default: `"1m"` 100 101 ### `backfill_period` 102 103 A duration string indicating the maximum age of tweets to acquire when starting a search. 104 105 106 Type: `string` 107 Default: `"5m"` 108 109 ### `cache` 110 111 A cache resource to use for request pagination. 112 113 114 Type: `string` 115 116 ### `cache_key` 117 118 The key identifier used when storing the ID of the last tweet received. 119 120 121 Type: `string` 122 Default: `"last_tweet_id"` 123 124 ### `rate_limit` 125 126 An optional rate limit resource to restrict API requests with. 127 128 129 Type: `string` 130 Default: `""` 131 132 ### `api_key` 133 134 An API key for OAuth 2.0 authentication. It is recommended that you populate this field using [environment variables](/docs/configuration/interpolation). 135 136 137 Type: `string` 138 139 ### `api_secret` 140 141 An API secret for OAuth 2.0 authentication. It is recommended that you populate this field using [environment variables](/docs/configuration/interpolation). 142 143 144 Type: `string` 145 146