github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/docs/_posts/2022-11-13-relnotes-3.12.md (about)

     1  ---
     2  layout: post
     3  title:  "AIStore 3.12 Release Notes"
     4  date:   Nov 13, 2022
     5  author: Alex Aizman
     6  categories: aistore release
     7  ---
     8  
     9  This AIStore release, version 3.12, has been in development for almost four months. It includes a number of significant changes that can be further detailed and grouped as follows:
    10  
    11  * [v3.12 changelog](https://github.com/NVIDIA/aistore/releases/tag/v1.3.15)
    12  
    13  Some of the more conceptual changes are briefly described below.
    14  
    15  # Remote AIS clusters
    16  
    17  In AIS, _remote_ buckets from different cloud (and non-cloud) providers, on the one hand, and from all inter-connected AIS clusters, on the other, form a _global namespace_ of all buckets.
    18  
    19  To connect one cluster to another, user calls `attach-remote-ais(URL, alias)` - an API where the `URL` is an access point (endpoint) provided by any gateway of the cluster in question, and the `alias` is human-readable alias.
    20  
    21  Henceforth, the cluster remains known (addressable and accessible) by its given alias. In fact, aliases and unique identifiers can be used interchangeably.
    22  
    23  Internally, however, we now solely rely on the cluster UUIDs.
    24  
    25  # Presence versus Existence
    26  
    27  AIS completely hides the fact that a _remote_ bucket may be only partially present in the cluster. Or not present at all.
    28  
    29  Moreover, users can start reading, writing, listing, and otherwise operating on buckets that are totally unknown to AIS at the time of the first read, write, or list etc.
    30  
    31  > When a user references a new bucket, behind-the-scenes AIS looks it up, confirms the bucket's existence and accessibility, and - if confirmed - on-the-fly updates its cluster-wide global metadata. The latter is called BMD and contains bucket definitions, associated management policies, and bucket properties.
    32  
    33  > To show BMD, run `ais show cluster bmd --json`
    34  
    35  There are cases, however, and scenarios when users may not want, or mean, to do any of the above - examples including:
    36  
    37  * list or [summarize](https://github.com/NVIDIA/aistore/blob/main/api/bucket.go#L191) only those buckets that are _present_ in the cluster
    38  * list only those objects that are _present_ in a bucket that is _present_ in the cluster
    39  
    40  And so on.
    41  
    42  To that end, AIS v3.12 introduces [presence](https://github.com/NVIDIA/aistore/blob/main/api/apc/query.go#L70) - an enumerated filter that can be used in API calls to override existing defaults (that hasn't changed).
    43  
    44  All API changes are further referenced in the [changelog](https://github.com/NVIDIA/aistore/releases/tag/v1.3.15).
    45  
    46  # Listing remote objects
    47  
    48  In 3.12, much of the functionality has been substantially revised. In particular, execution of any request to list objects in a _remote_ bucket now further depends on:
    49  
    50  1. the [presence](#presence-versus-existence) specifier, and
    51  2. requested object properties
    52  
    53  > It is, in a sense, easier to satisfy the request for (name, size, and version) than, say, `list-objects(name, atime, custom)`
    54  
    55  One of the associated flows is illustrated by the following sequence diagram:
    56  
    57  ![Listing remote objects](/assets/list-objects-3.12.png)
    58  
    59  The sequence includes 7 (seven) steps, whereby in Step 1 client is asking either for the entire bucket contents or - if the bucket is very large - for the next page. Whatever it is, the client gets it back in Step 7.
    60  
    61  The rest of it must be mostly self-explanatory, with maybe two minor comments:
    62  
    63  * in Step 2, ais proxy selects a target (denoted as K) to call the remote backend
    64  * <span style="color:green;">Steps 2, 5, and 6</span> are executed over an internal network using long-lived TCP connections and streaming intra-cluster transport
    65  
    66  # S3 API
    67  
    68  In addition to the native [REST API](/docs/http_api.md), AIS also provides Amazon S3 with certain [disclosed](/docs/s3compat.md) limitations. In v3.12, the most significant changes include:
    69  
    70  * multipart upload
    71  * ability to operate on all buckets from all providers (previously, only s3 buckets)
    72  * configurable root '/' access
    73  
    74  If configured, AIS will handle S3 API calls at `http(s)://aistore` endpoint (with the default being `http(s)://aistore/s3`)
    75  
    76  In addition, returned errors are now properly XML formatted, and [more](https://github.com/NVIDIA/aistore/releases/tag/v1.3.15).