go.temporal.io/server@v1.23.0/common/archiver/s3store/README.md (about)

     1  # Amazon S3 blobstore
     2  ## Configuration
     3  See https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials on how to set up authentication against s3
     4  
     5  Enabling archival is done by using the configuration below. `Region` and `bucket URI` are required
     6  ```
     7  archival:
     8    history:
     9      state: "enabled"
    10      enableRead: true
    11      provider:
    12        s3store:
    13          region: "us-east-1"
    14          logLevel: 0
    15    visibility:
    16      state: "enabled"
    17      enableRead: true
    18      provider:
    19        s3store:
    20          region: "us-east-1"
    21          logLevel: 0
    22  
    23  namespaceDefaults:
    24    archival:
    25      history:
    26        state: "enabled"
    27        URI: "s3://<bucket-name>"
    28      visibility:
    29        state: "enabled"
    30        URI: "s3://<bucket-name>"
    31  ```
    32  
    33  ## Visibility query syntax
    34  You can query the visibility store by using the `tctl workflow listarchived` command
    35  
    36  The syntax for the query is based on SQL
    37  
    38  Supported column names are
    39  - WorkflowId *String*
    40  - WorkflowTypeName *String*
    41  - StartTime *Date*
    42  - CloseTime *Date*
    43  - SearchPrecision *String - Day, Hour, Minute, Second*
    44  
    45  WorkflowId or WorkflowTypeName is required. If filtering on date use StartTime or CloseTime in combination with SearchPrecision.
    46  
    47  Searching for a record will be done in times in the UTC timezone
    48  
    49  SearchPrecision specifies what range you want to search for records. If you use `SearchPrecision = 'Day'`
    50  it will search all records starting from `2020-01-21T00:00:00Z` to `2020-01-21T59:59:59Z` 
    51  
    52  ### Limitations
    53  
    54  - The only operator supported is `=` due to how records are stored in s3.
    55  
    56  ### Example
    57  
    58  *Searches for all records done in day 2020-01-21 with the specified workflow id*
    59  
    60  `./tctl --ns samples-namespace workflow listarchived -q "StartTime = '2020-01-21T00:00:00Z' AND WorkflowId='workflow-id' AND SearchPrecision='Day'"`
    61  ## Storage in S3
    62  Workflow runs are stored in s3 using the following structure
    63  ```
    64  s3://<bucket-name>/<namespace-id>/
    65  	history/<workflow-id>/<run-id>
    66  	visibility/
    67              workflowTypeName/<workflow-type-name>/
    68                  startTimeout/2020-01-21T16:16:11Z/<run-id>
    69                  closeTimeout/2020-01-21T16:16:11Z/<run-id>
    70              workflowID/<workflow-id>/
    71                  startTimeout/2020-01-21T16:16:11Z/<run-id>
    72                  closeTimeout/2020-01-21T16:16:11Z/<run-id>
    73  ```
    74  
    75  Enable AWS SDK Logging with config parameter `logLevel`. For example enable debug logging with `logLevel: 4096`. Possbile Values:
    76  * LogOff = 0 = 0x0
    77  * LogDebug = 4096 = 0x1000
    78  * LogDebugWithSigning = 4097 = 0x1001
    79  * LogDebugWithHTTPBody = 4098 = 0x1002
    80  * LogDebugWithRequestRetries = 4100 = 0x1004
    81  * LogDebugWithRequestErrors = 4104 = 0x1008
    82  * LogDebugWithEventStreamBody = 4112 = 0x1010
    83  * LogDebugWithDeprecated = 4128 = 0x1020
    84  
    85  
    86  ## Using localstack for local development
    87  1. Install awscli from [here](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
    88  2. Install localstack from [here](https://github.com/localstack/localstack#installing)
    89  3. Launch localstack with `SERVICES=s3 localstack start`
    90  4. Create a bucket using `aws --endpoint-url=http://localhost:4566 s3 mb s3://temporal-development` 
    91  5. Launch the server with the localstack s3 environment config`--env development-cass-s3 start`