github.com/artpar/rclone@v1.67.3/docs/content/netstorage.md (about)

     1  ---
     2  title: "Akamai Netstorage"
     3  description: "Rclone docs for Akamai NetStorage"
     4  versionIntroduced: "v1.58"
     5  ---
     6  
     7  # {{< icon "fas fa-database" >}} Akamai NetStorage
     8  
     9  Paths are specified as `remote:`
    10  You may put subdirectories in too, e.g. `remote:/path/to/dir`.
    11  If you have a CP code you can use that as the folder after the domain such as \<domain>\/\<cpcode>\/\<internal directories within cpcode>.
    12  
    13  For example, this is commonly configured with or without a CP code:
    14  * **With a CP code**. `[your-domain-prefix]-nsu.akamaihd.net/123456/subdirectory/`
    15  * **Without a CP code**. `[your-domain-prefix]-nsu.akamaihd.net`
    16  
    17  
    18  See all buckets
    19     rclone lsd remote:
    20  The initial setup for Netstorage involves getting an account and secret. Use `rclone config` to walk you through the setup process.
    21  
    22  ## Configuration
    23  
    24  Here's an example of how to make a remote called `ns1`.
    25  
    26  1. To begin the interactive configuration process, enter this command:
    27  
    28  ```
    29  rclone config
    30  ```
    31  
    32  2. Type `n` to create a new remote.
    33  
    34  ```
    35  n) New remote
    36  d) Delete remote
    37  q) Quit config
    38  e/n/d/q> n
    39  ```
    40  
    41  3. For this example, enter `ns1` when you reach the name> prompt.
    42  
    43  ```
    44  name> ns1
    45  ```
    46  
    47  4. Enter `netstorage` as the type of storage to configure.
    48  
    49  ```
    50  Type of storage to configure.
    51  Enter a string value. Press Enter for the default ("").
    52  Choose a number from below, or type in your own value
    53  XX / NetStorage
    54     \ "netstorage"
    55  Storage> netstorage
    56  ```
    57  
    58  5. Select between the HTTP or HTTPS protocol. Most users should choose HTTPS, which is the default. HTTP is provided primarily for debugging purposes.
    59  
    60  
    61  ```
    62  Enter a string value. Press Enter for the default ("").
    63  Choose a number from below, or type in your own value
    64   1 / HTTP protocol
    65     \ "http"
    66   2 / HTTPS protocol
    67     \ "https"
    68  protocol> 1
    69  ```
    70  
    71  6. Specify your NetStorage host, CP code, and any necessary content paths using this format: `<domain>/<cpcode>/<content>/`
    72  
    73  ```
    74  Enter a string value. Press Enter for the default ("").
    75  host> baseball-nsu.akamaihd.net/123456/content/
    76  ```
    77  
    78  7. Set the netstorage account name
    79  ```
    80  Enter a string value. Press Enter for the default ("").
    81  account> username
    82  ```
    83  
    84  8. Set the Netstorage account secret/G2O key which will be used for authentication purposes. Select the `y` option to set your own password then enter your secret.
    85  Note: The secret is stored in the `rclone.conf` file with hex-encoded encryption.
    86  
    87  ```
    88  y) Yes type in my own password
    89  g) Generate random password
    90  y/g> y
    91  Enter the password:
    92  password:
    93  Confirm the password:
    94  password:
    95  ```
    96  
    97  9. View the summary and confirm your remote configuration.
    98  
    99  ```
   100  [ns1]
   101  type = netstorage
   102  protocol = http
   103  host = baseball-nsu.akamaihd.net/123456/content/
   104  account = username
   105  secret = *** ENCRYPTED ***
   106  --------------------
   107  y) Yes this is OK (default)
   108  e) Edit this remote
   109  d) Delete this remote
   110  y/e/d> y
   111  ```
   112  
   113  This remote is called `ns1` and can now be used.
   114  
   115  ## Example operations
   116  
   117  Get started with rclone and NetStorage with these examples. For additional rclone commands, visit https://rclone.org/commands/.
   118  
   119  ### See contents of a directory in your project
   120  
   121      rclone lsd ns1:/974012/testing/
   122  
   123  ### Sync the contents local with remote
   124  
   125      rclone sync . ns1:/974012/testing/
   126  
   127  ### Upload local content to remote
   128      rclone copy notes.txt ns1:/974012/testing/
   129  
   130  ### Delete content on remote
   131      rclone delete ns1:/974012/testing/notes.txt
   132  
   133  ### Move or copy content between CP codes.
   134  
   135  Your credentials must have access to two CP codes on the same remote. You can't perform operations between different remotes.
   136  
   137      rclone move ns1:/974012/testing/notes.txt ns1:/974450/testing2/
   138  
   139  ## Features
   140  
   141  ### Symlink Support
   142  
   143  The Netstorage backend changes the rclone `--links, -l` behavior. When uploading, instead of creating the .rclonelink file, use the "symlink" API in order to create the corresponding symlink on the remote. The .rclonelink file will not be created, the upload will be intercepted and only the symlink file that matches the source file name with no suffix will be created on the remote.
   144  
   145  This will effectively allow commands like copy/copyto, move/moveto and sync to upload from local to remote and download from remote to local directories with symlinks. Due to internal rclone limitations, it is not possible to upload an individual symlink file to any remote backend. You can always use the "backend symlink" command to create a symlink on the NetStorage server, refer to "symlink" section below.
   146  
   147  Individual symlink files on the remote can be used with the commands like "cat" to print the destination name, or "delete" to delete symlink, or copy, copy/to and move/moveto to download from the remote to local. Note: individual symlink files on the remote should be specified including the suffix .rclonelink.
   148  
   149  **Note**: No file with the suffix .rclonelink should ever exist on the server since it is not possible to actually upload/create a file with .rclonelink suffix with rclone, it can only exist if it is manually created through a non-rclone method on the remote.
   150  
   151  ### Implicit vs. Explicit Directories
   152  
   153  With NetStorage, directories can exist in one of two forms:
   154  
   155  1. **Explicit Directory**. This is an actual, physical directory that you have created in a storage group.
   156  2. **Implicit Directory**. This refers to a directory within a path that has not been physically created. For example, during upload of a file, nonexistent subdirectories can be specified in the target path. NetStorage creates these as "implicit." While the directories aren't physically created, they exist implicitly and the noted path is connected with the uploaded file.
   157  
   158  Rclone will intercept all file uploads and mkdir commands for the NetStorage remote and will explicitly issue the mkdir command for each directory in the uploading path. This will help with the interoperability with the other Akamai services such as SFTP and the Content Management Shell (CMShell). Rclone will not guarantee correctness of operations with implicit directories which might have been created as a result of using an upload API directly.
   159  
   160  ### `--fast-list` / ListR support
   161  
   162  NetStorage remote supports the ListR feature by using the "list" NetStorage API action to return a lexicographical list of all objects within the specified CP code, recursing into subdirectories as they're encountered.
   163  
   164  * **Rclone will use the ListR method for some commands by default**. Commands such as `lsf -R` will use ListR by default. To disable this, include the `--disable listR` option to use the non-recursive method of listing objects.
   165  
   166  * **Rclone will not use the ListR method for some commands**. Commands such as `sync` don't use ListR by default. To force using the ListR method, include the  `--fast-list` option.
   167  
   168  There are pros and cons of using the ListR method, refer to [rclone documentation](https://rclone.org/docs/#fast-list). In general, the sync command over an existing deep tree on the remote will run faster with the "--fast-list" flag but with extra memory usage as a side effect. It might also result in higher CPU utilization but the whole task can be completed faster.
   169  
   170  **Note**: There is a known limitation that "lsf -R" will display number of files in the directory and directory size as -1 when ListR method is used. The workaround is to pass "--disable listR" flag if these numbers are important in the output.
   171  
   172  ### Purge
   173  
   174  NetStorage remote supports the purge feature by using the "quick-delete" NetStorage API action. The quick-delete action is disabled by default for security reasons and can be enabled for the account through the Akamai portal. Rclone will first try to use quick-delete action for the purge command and if this functionality is disabled then will fall back to a standard delete method.
   175  
   176  **Note**: Read the [NetStorage Usage API](https://learn.akamai.com/en-us/webhelp/netstorage/netstorage-http-api-developer-guide/GUID-15836617-9F50-405A-833C-EA2556756A30.html) for considerations when using "quick-delete". In general, using quick-delete method will not delete the tree immediately and objects targeted for quick-delete may still be accessible.
   177  
   178  {{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/netstorage/netstorage.go then run make backenddocs" >}}
   179  ### Standard options
   180  
   181  Here are the Standard options specific to netstorage (Akamai NetStorage).
   182  
   183  #### --netstorage-host
   184  
   185  Domain+path of NetStorage host to connect to.
   186  
   187  Format should be `<domain>/<internal folders>`
   188  
   189  Properties:
   190  
   191  - Config:      host
   192  - Env Var:     RCLONE_NETSTORAGE_HOST
   193  - Type:        string
   194  - Required:    true
   195  
   196  #### --netstorage-account
   197  
   198  Set the NetStorage account name
   199  
   200  Properties:
   201  
   202  - Config:      account
   203  - Env Var:     RCLONE_NETSTORAGE_ACCOUNT
   204  - Type:        string
   205  - Required:    true
   206  
   207  #### --netstorage-secret
   208  
   209  Set the NetStorage account secret/G2O key for authentication.
   210  
   211  Please choose the 'y' option to set your own password then enter your secret.
   212  
   213  **NB** Input to this must be obscured - see [rclone obscure](/commands/rclone_obscure/).
   214  
   215  Properties:
   216  
   217  - Config:      secret
   218  - Env Var:     RCLONE_NETSTORAGE_SECRET
   219  - Type:        string
   220  - Required:    true
   221  
   222  ### Advanced options
   223  
   224  Here are the Advanced options specific to netstorage (Akamai NetStorage).
   225  
   226  #### --netstorage-protocol
   227  
   228  Select between HTTP or HTTPS protocol.
   229  
   230  Most users should choose HTTPS, which is the default.
   231  HTTP is provided primarily for debugging purposes.
   232  
   233  Properties:
   234  
   235  - Config:      protocol
   236  - Env Var:     RCLONE_NETSTORAGE_PROTOCOL
   237  - Type:        string
   238  - Default:     "https"
   239  - Examples:
   240      - "http"
   241          - HTTP protocol
   242      - "https"
   243          - HTTPS protocol
   244  
   245  #### --netstorage-description
   246  
   247  Description of the remote
   248  
   249  Properties:
   250  
   251  - Config:      description
   252  - Env Var:     RCLONE_NETSTORAGE_DESCRIPTION
   253  - Type:        string
   254  - Required:    false
   255  
   256  ## Backend commands
   257  
   258  Here are the commands specific to the netstorage backend.
   259  
   260  Run them with
   261  
   262      rclone backend COMMAND remote:
   263  
   264  The help below will explain what arguments each command takes.
   265  
   266  See the [backend](/commands/rclone_backend/) command for more
   267  info on how to pass options and arguments.
   268  
   269  These can be run on a running backend using the rc command
   270  [backend/command](/rc/#backend-command).
   271  
   272  ### du
   273  
   274  Return disk usage information for a specified directory
   275  
   276      rclone backend du remote: [options] [<arguments>+]
   277  
   278  The usage information returned, includes the targeted directory as well as all
   279  files stored in any sub-directories that may exist.
   280  
   281  ### symlink
   282  
   283  You can create a symbolic link in ObjectStore with the symlink action.
   284  
   285      rclone backend symlink remote: [options] [<arguments>+]
   286  
   287  The desired path location (including applicable sub-directories) ending in
   288  the object that will be the target of the symlink (for example, /links/mylink).
   289  Include the file extension for the object, if applicable.
   290  `rclone backend symlink <src> <path>`
   291  
   292  {{< rem autogenerated options stop >}}