go.temporal.io/server@v1.23.0/common/archiver/s3store/README.md (about) 1 # Amazon S3 blobstore 2 ## Configuration 3 See https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials on how to set up authentication against s3 4 5 Enabling archival is done by using the configuration below. `Region` and `bucket URI` are required 6 ``` 7 archival: 8 history: 9 state: "enabled" 10 enableRead: true 11 provider: 12 s3store: 13 region: "us-east-1" 14 logLevel: 0 15 visibility: 16 state: "enabled" 17 enableRead: true 18 provider: 19 s3store: 20 region: "us-east-1" 21 logLevel: 0 22 23 namespaceDefaults: 24 archival: 25 history: 26 state: "enabled" 27 URI: "s3://<bucket-name>" 28 visibility: 29 state: "enabled" 30 URI: "s3://<bucket-name>" 31 ``` 32 33 ## Visibility query syntax 34 You can query the visibility store by using the `tctl workflow listarchived` command 35 36 The syntax for the query is based on SQL 37 38 Supported column names are 39 - WorkflowId *String* 40 - WorkflowTypeName *String* 41 - StartTime *Date* 42 - CloseTime *Date* 43 - SearchPrecision *String - Day, Hour, Minute, Second* 44 45 WorkflowId or WorkflowTypeName is required. If filtering on date use StartTime or CloseTime in combination with SearchPrecision. 46 47 Searching for a record will be done in times in the UTC timezone 48 49 SearchPrecision specifies what range you want to search for records. If you use `SearchPrecision = 'Day'` 50 it will search all records starting from `2020-01-21T00:00:00Z` to `2020-01-21T59:59:59Z` 51 52 ### Limitations 53 54 - The only operator supported is `=` due to how records are stored in s3. 55 56 ### Example 57 58 *Searches for all records done in day 2020-01-21 with the specified workflow id* 59 60 `./tctl --ns samples-namespace workflow listarchived -q "StartTime = '2020-01-21T00:00:00Z' AND WorkflowId='workflow-id' AND SearchPrecision='Day'"` 61 ## Storage in S3 62 Workflow runs are stored in s3 using the following structure 63 ``` 64 s3://<bucket-name>/<namespace-id>/ 65 history/<workflow-id>/<run-id> 66 visibility/ 67 workflowTypeName/<workflow-type-name>/ 68 startTimeout/2020-01-21T16:16:11Z/<run-id> 69 closeTimeout/2020-01-21T16:16:11Z/<run-id> 70 workflowID/<workflow-id>/ 71 startTimeout/2020-01-21T16:16:11Z/<run-id> 72 closeTimeout/2020-01-21T16:16:11Z/<run-id> 73 ``` 74 75 Enable AWS SDK Logging with config parameter `logLevel`. For example enable debug logging with `logLevel: 4096`. Possbile Values: 76 * LogOff = 0 = 0x0 77 * LogDebug = 4096 = 0x1000 78 * LogDebugWithSigning = 4097 = 0x1001 79 * LogDebugWithHTTPBody = 4098 = 0x1002 80 * LogDebugWithRequestRetries = 4100 = 0x1004 81 * LogDebugWithRequestErrors = 4104 = 0x1008 82 * LogDebugWithEventStreamBody = 4112 = 0x1010 83 * LogDebugWithDeprecated = 4128 = 0x1020 84 85 86 ## Using localstack for local development 87 1. Install awscli from [here](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) 88 2. Install localstack from [here](https://github.com/localstack/localstack#installing) 89 3. Launch localstack with `SERVICES=s3 localstack start` 90 4. Create a bucket using `aws --endpoint-url=http://localhost:4566 s3 mb s3://temporal-development` 91 5. Launch the server with the localstack s3 environment config`--env development-cass-s3 start`