github.com/NVIDIA/aistore@v1.3.23-0.20240517131212-7df6609be51d/docs/cli/storage.md (about)

     1  ---
     2  layout: post
     3  title: STORAGE
     4  permalink: /docs/cli/storage
     5  redirect_from:
     6   - /cli/storage.md/
     7   - /docs/cli/storage.md/
     8  ---
     9  
    10  `ais storage` command supports the following subcommands:
    11  
    12  ```console
    13  $ ais storage <TAB-TAB>
    14  cleanup     disk        mountpath   summary     validate
    15  ```
    16  
    17  Alternatively (or in addition), run with `--help` to view subcommands and short descriptions, both:
    18  
    19  ```console
    20  $ ais storage --help
    21  NAME:
    22     ais storage - monitor and manage clustered storage
    23  
    24  USAGE:
    25     ais storage command [command options] [arguments...]
    26  
    27  COMMANDS:
    28     show       show storage usage and utilization, disks and mountpaths
    29     summary    show bucket sizes and %% of used capacity on a per-bucket basis
    30     validate   check buckets for misplaced objects and objects that have insufficient numbers of copies or EC slices
    31     mountpath  show and attach/detach target mountpaths
    32     disk       show disk utilization and read/write statistics
    33     cleanup    perform storage cleanup: remove deleted objects and old/obsolete workfiles
    34  
    35  OPTIONS:
    36     --help, -h  show help
    37  ```
    38  
    39  As always, each subcommand (above) will have its own help and usage examples - the latter possibly spread across multiple markdowns.
    40  
    41  > You can easily look up examples and descriptions of any keyword via a simple `find`, for instance:
    42  
    43  ```console
    44  $ find . -type f -name "*.md" | xargs grep "ais.*mountpath"
    45  ```
    46  
    47  ## Table of Contents
    48  - [Storage cleanup](#storage-cleanup)
    49  - [Show capacity usage](#show-capacity-usage)
    50  - [Validate buckets](#validate-buckets)
    51  - [Mountpath (and disk) management](#mountpath-and-disk-management)
    52  - [Show mountpaths](#show-mountpaths)
    53  - [Attach mountpath](#attach-mountpath)
    54  - [Detach mountpath](#detach-mountpath)
    55  
    56  ## Storage cleanup
    57  
    58  As all other supported batch operations (aka `xactions`), cleanup runs asynchronously and can be monitored during its run, e.g.:
    59  
    60  ```console
    61  # ais storage cleanup
    62  Started storage cleanup "BlpmlObF8", use 'ais job show xaction BlpmlObF8' to monitor the progress
    63  ```
    64  
    65  Further references:
    66  
    67  * [Batch operations](/docs/batch.md)
    68  * [`ais show job`](/docs/cli/job.md)
    69  
    70  ## Show capacity usage
    71  
    72  For command line options and usage examples, please refer to:
    73  
    74  * [bucket summary](/docs/cli/bucket.md#show-bucket-summary)
    75  
    76  ## Validate buckets
    77  
    78  `ais storage validate [BUCKET | PROVIDER]`
    79  
    80  Checks all objects of the bucket `BUCKET` and show the number of found issues:
    81  the number of misplaced objects, the number of objects that have insufficient number of copies etc.
    82  Non-zero number of misplaced objects may mean a bucket needs rebalancing.
    83  
    84  If the optional argument is omitted, show information about all buckets.
    85  
    86  Because the command checks every object, it may take a lot of time for big buckets.
    87  It is recommended to set bucket name or provider name to decrease execution time.
    88  
    89  ### Example
    90  
    91  Validate only AIS buckets
    92  
    93  ```
    94  $ ais storage validate  ais://
    95  BUCKET            OBJECTS         MISPLACED       MISSING COPIES
    96  ais://bck1        2               0               0
    97  ais://bck2        3               1               0
    98  ```
    99  
   100  The bucket `ais://bck2` has 3 objects and one of them is misplaced, i.e. it is inaccessible by a client.
   101  It results in `ais ls ais://bck2` returns only 2 objects.
   102  
   103  ## Mountpath (and disk) management
   104  
   105  There are two related commands:
   106  
   107  * `ais storage disk`
   108  * `ais storage mountpath`
   109  
   110  where `mountpath` is a higher-level abstraction that typically "utilizes" a single undivided disk. More exactly:
   111  
   112  > A *mountpath* is a single disk **or** a volume (a RAID) formatted with a local filesystem of choice, **and** a local directory that AIS utilizes to store user data and AIS metadata. A mountpath can be disabled and (re)enabled, automatically or administratively, at any point during runtime. In a given cluster, a total number of mountpaths would normally compute as a direct product of (number of storage targets) x (number of disks in each target).
   113  
   114  You can manage and monitor (i.e., `show`) disks and mountpaths using `ais storage` command.
   115  
   116  > For strictly monitoring purposes, you can universally use `ais show` command, e.g.: `ais show storage disk`, etc.
   117  
   118  ## Show disks
   119  
   120  `ais storage disk show [TARGET_ID]`
   121  
   122  or, same:
   123  
   124  `ais show storage disk [TARGET_ID]`
   125  
   126  ## Show mountpaths
   127  
   128  As the name implies, the syntax:
   129  
   130  `ais show storage mountpath [TARGET_ID]`
   131  
   132  for example:
   133  
   134  ```console
   135  $ ais show storage mountpath t[TqPtghbiRw]
   136  
   137  TqPtghbiRw
   138          Used Capacity (all disks): avg 15% max 18%
   139                                                  /ais/mp1/2 /dev/nvme0n1(xfs)
   140                                                  /ais/mp2/2 /dev/nvme1n1(xfs)
   141                                                  /ais/mp3/2 /dev/nvme2n1(xfs)
   142                                                  /ais/mp4/2 /dev/nvme3n1(xfs)
   143  ```
   144  
   145  As always, `--help` will also list supported options. Note in particular the option to run continuously and periodically:
   146  
   147  ```console
   148  $ ais show storage mountpath --help
   149  NAME:
   150     ais show storage mountpath - show target mountpaths
   151  
   152  USAGE:
   153     ais show storage mountpath [command options] [TARGET_ID]
   154  
   155  OPTIONS:
   156     --refresh value  interval for continuous monitoring;
   157                      valid time units: ns, us (or µs), ms, s (default), m, h
   158     --count value    used together with '--refresh' to limit the number of generated reports (default: 0)
   159     --json, -j       json input/output
   160     --help, -h       show help
   161  ```
   162  
   163  Show mountpaths for a given target or all targets.
   164  
   165  > **Ease of Usage** notice: like all other `ais show` commands, `ais show storage mountpath` is an alias (or a shortcut) - in this specific case - for `ais storage mountpath show`.
   166  
   167  ### Examples (_slightly outdated_)
   168  
   169  ```console
   170  $ ais storage mountpath show 12356t8085
   171  247389t8085
   172          Available:
   173  			/tmp/ais/5/3
   174  			/tmp/ais/5/1
   175          Disabled:
   176  			/tmp/ais/5/2
   177  
   178  $ ais storage mountpath show
   179  12356t8085
   180          Available:
   181  			/tmp/ais/5/3
   182  			/tmp/ais/5/1
   183          Disabled:
   184  			/tmp/ais/5/2
   185  147665t8084
   186          Available:
   187  			/tmp/ais/4/3
   188  			/tmp/ais/4/1
   189  			/tmp/ais/4/2
   190  426988t8086
   191  	No mountpaths
   192  ```
   193  
   194  ## Attach mountpath
   195  
   196  `ais storage mountpath attach TARGET_ID=MOUNTPATH [DAEMONID=MOUNTPATH...]`
   197  
   198  Attach a mountpath on a specified target to AIS storage.
   199  
   200  ### Examples
   201  
   202  ```console
   203  $ ais storage mountpath attach 12367t8080=/data/dir
   204  ```
   205  
   206  ## Detach mountpath
   207  
   208  `ais storage mountpath detach TARGET_ID=MOUNTPATH [DAEMONID=MOUNTPATH...]`
   209  
   210  Detach a mountpath on a specified target from AIS storage.
   211  
   212  ### Examples
   213  
   214  ```console
   215  $ ais storage mountpath detach 12367t8080=/data/dir
   216  ```