github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/python/aistore/sdk/README.md (about) 1 ## AIS Python SDK 2 3 AIS Python SDK provides a (growing) set of client-side APIs to access and utilize AIS clusters, buckets, and objects. 4 5 The project is, essentially, a Python port of the [AIS Go APIs](https://aiatscale.org/docs/http-api), with additional objectives that prioritize *utmost convenience for Python developers*. 6 7 Note that only Python 3.x (version 3.6 or later) is currently supported. 8 9 --- 10 11 ## Installation 12 13 14 ### Install as a Package 15 16 The latest AIS release can be easily installed either with Anaconda or `pip`: 17 18 ```console 19 $ conda install aistore 20 ``` 21 22 ```console 23 $ pip install aistore 24 ``` 25 26 ### Install From Source 27 28 If you'd like to work with the current upstream (and don't mind the risk), install the latest master directly from GitHub: 29 30 ```console 31 $ git clone https://github.com/NVIDIA/aistore.git 32 33 $ cd aistore/python/ 34 35 # upgrade pip to latest version 36 $ python -m pip install --upgrade pip 37 38 # install dependencies 39 $ pip install -r aistore/common_requirements 40 41 $ pip install -e . 42 ``` 43 --- 44 45 ## Quick Start 46 47 In order to interact with your running AIS instance, you will need to create a `client` object: 48 49 ```python 50 from aistore.sdk import Client 51 52 client = Client("http://localhost:8080") 53 ``` 54 55 The newly created `client` object can be used to interact with your AIS cluster, buckets, and objects. 56 See the [examples](https://github.com/NVIDIA/aistore/blob/main/python/examples/sdk) and the [reference docs](https://aiatscale.org/docs/python-sdk) for more details 57 58 **External Cloud Storage Buckets** 59 60 AIS supports a number of different [backend providers](https://aiatscale.org/docs/providers) or, simply, backends. 61 62 > For exact definitions and related capabilities, please see [terminology](https://aiatscale.org//docs/overview#terminology). 63 64 Many bucket/object operations support remote cloud buckets (third-party backend-based cloud buckets), including a few of the operations shown above. To interact with remote cloud buckets, you need to *specify the provider* of choice when instantiating your bucket object as follows: 65 66 ```python 67 # Head AWS bucket 68 client.bucket("my-aws-bucket", provider="aws").head() 69 ``` 70 71 ```python 72 # Evict GCP bucket 73 client.bucket("my-gcp-bucket", provider="gcp").evict() 74 ``` 75 76 ```python 77 # Get object from Azure bucket 78 client.bucket("my-azure-bucket", provider="azure").object("filename.ext").get() 79 ``` 80 81 ```python 82 # List objects in AWS bucket 83 client.bucket("my-aws-bucket", provider="aws").list_objects() 84 ``` 85 86 Please note that certain operations do **not** support external cloud storage buckets. Please refer to the [SDK reference documentation](https://aiatscale.org/docs/python_sdk.md) for more information on which bucket/object operations support remote cloud buckets, as well as general information on class and method usage. 87 88 --- 89 ### HTTPS 90 91 The SDK supports HTTPS connectivity if the AIS cluster is configured to use HTTPS. To start using HTTPS: 92 93 1. Set up HTTPS on your cluster: [Guide for K8s cluster](https://github.com/NVIDIA/ais-k8s/blob/master/playbooks/docs/ais_https_configuration.md) 94 2. If using a self-signed certificate with your own CA, copy the CA certificate to your local machine. If using our built-in cert-manager config to generate your certificates, you can use [our playbook](https://github.com/NVIDIA/ais-k8s/blob/master/playbooks/docs/ais_generate_https_cert.md) 95 3. Options to configure the SDK for HTTPS connectivity: 96 - Skip verification (for testing, insecure): 97 - `client = Client(skip_verify=True)` 98 - Point the SDK to use your certificate using one of the below methods: 99 - Pass an argument to the path of the certificate when creating the client: 100 - `client = Client(ca_cert=/path/to/cert)` 101 - Use the environment variable 102 - Set `AIS_SERVER_CRT` to the path of your certificate before initializing the client 103 - If your AIS cluster is using a certificate signed by a trusted CA, the client will default to using verification without needing to provide a CA cert. 104 --- 105 106 ### ETLs 107 108 AIStore also supports [ETLs](https://aiatscale.org/docs/etl), short for Extract-Transform-Load. ETLs with AIS are beneficial given that the transformations occur *locally*, which largely contributes to the linear scalability of AIS. 109 110 > Note: AIS-ETL requires [Kubernetes](https://kubernetes.io/). For more information on deploying AIStore with Kubernetes (or Minikube), refer [here](https://github.com/NVIDIA/aistore/blob/main/deploy/dev/k8s/README.md). 111 112 Check out the [provided examples](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/etl_templates.py) to learn more about working with AIS ETL. 113 114 --- 115 116 ### API Documentation 117 118 |Module|Summary| 119 |--|--| 120 |[api.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/client.py)|Contains `Client` class, which has methods for making HTTP requests to an AIStore server. Includes factory constructors for `Bucket`, `Cluster`, and `Job` classes.| 121 |[cluster.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/cluster.py)|Contains `Cluster` class that represents a cluster bound to a client and contains all cluster-related operations, including checking the cluster's health and retrieving vital cluster information.| 122 |[bucket.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/bucket.py)|Contains `Bucket` class that represents a bucket in an AIS cluster and contains all bucket-related operations, including (but not limited to) creating, deleting, evicting, renaming, copying.| 123 |[object.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/object.py)|Contains class `Object` that represents an object belonging to a bucket in an AIS cluster, and contains all object-related operations, including (but not limited to) retreiving, adding and deleting objects.| 124 |[object_group.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/object_group.py)|Contains class `ObjectGroup`, representing a collection of objects belonging to a bucket in an AIS cluster. Includes all multi-object operations such as deleting, evicting, prefetching, copying, and transforming objects.| 125 |[job.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/job.py)|Contains class `Job` and all job-related operations.| 126 |[dsort.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/dsort.py)|Contains class `Dsort` and all dsort-related operations.| 127 |[etl.py](https://github.com/NVIDIA/aistore/blob/main/python/aistore/sdk/etl.py)|Contains class `Etl` and all ETL-related operations.| 128 129 For more information on SDK usage, refer to the [SDK reference documentation](https://aiatscale.org/docs/python_sdk.md) or see the examples [here](https://github.com/NVIDIA/aistore/blob/main/python/examples/sdk/). 130 131 132 ## References 133 134 * [AIStore GitHub](https://github.com/NVIDIA/aistore) 135 * [Documentation](https://aiatscale.org/docs) 136 * [AIStore pip package](https://pypi.org/project/aistore/) 137 * [Videos and demos](https://github.com/NVIDIA/aistore/blob/main/docs/videos.md)