github.com/rclone/rclone@v1.66.1-0.20240517100346-7b89735ae726/docs/content/hidrive.md (about)

     1  ---
     2  title: "HiDrive"
     3  description: "Rclone docs for HiDrive"
     4  versionIntroduced: "v1.59"
     5  ---
     6  
     7  # {{< icon "fa fa-cloud" >}} HiDrive
     8  
     9  Paths are specified as `remote:path`
    10  
    11  Paths may be as deep as required, e.g. `remote:directory/subdirectory`.
    12  
    13  The initial setup for hidrive involves getting a token from HiDrive
    14  which you need to do in your browser.
    15  `rclone config` walks you through it.
    16  
    17  ## Configuration
    18  
    19  Here is an example of how to make a remote called `remote`.  First run:
    20  
    21       rclone config
    22  
    23  This will guide you through an interactive setup process:
    24  
    25  ```
    26  No remotes found - make a new one
    27  n) New remote
    28  s) Set configuration password
    29  q) Quit config
    30  n/s/q> n
    31  name> remote
    32  Type of storage to configure.
    33  Choose a number from below, or type in your own value
    34  [snip]
    35  XX / HiDrive
    36     \ "hidrive"
    37  [snip]
    38  Storage> hidrive
    39  OAuth Client Id - Leave blank normally.
    40  client_id>
    41  OAuth Client Secret - Leave blank normally.
    42  client_secret>
    43  Access permissions that rclone should use when requesting access from HiDrive.
    44  Leave blank normally.
    45  scope_access>
    46  Edit advanced config?
    47  y/n> n
    48  Use web browser to automatically authenticate rclone with remote?
    49   * Say Y if the machine running rclone has a web browser you can use
    50   * Say N if running rclone on a (remote) machine without web browser access
    51  If not sure try Y. If Y failed, try N.
    52  y/n> y
    53  If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=xxxxxxxxxxxxxxxxxxxxxx
    54  Log in and authorize rclone for access
    55  Waiting for code...
    56  Got code
    57  --------------------
    58  [remote]
    59  type = hidrive
    60  token = {"access_token":"xxxxxxxxxxxxxxxxxxxx","token_type":"Bearer","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxx","expiry":"xxxxxxxxxxxxxxxxxxxxxxx"}
    61  --------------------
    62  y) Yes this is OK (default)
    63  e) Edit this remote
    64  d) Delete this remote
    65  y/e/d> y
    66  ```
    67  
    68  **You should be aware that OAuth-tokens can be used to access your account
    69  and hence should not be shared with other persons.**
    70  See the [below section](#keeping-your-tokens-safe) for more information.
    71  
    72  See the [remote setup docs](/remote_setup/) for how to set it up on a
    73  machine with no Internet browser available.
    74  
    75  Note that rclone runs a webserver on your local machine to collect the
    76  token as returned from HiDrive. This only runs from the moment it opens
    77  your browser to the moment you get back the verification code.
    78  The webserver runs on `http://127.0.0.1:53682/`.
    79  If local port `53682` is protected by a firewall you may need to temporarily
    80  unblock the firewall to complete authorization.
    81  
    82  Once configured you can then use `rclone` like this,
    83  
    84  List directories in top level of your HiDrive root folder
    85  
    86      rclone lsd remote:
    87  
    88  List all the files in your HiDrive filesystem
    89  
    90      rclone ls remote:
    91  
    92  To copy a local directory to a HiDrive directory called backup
    93  
    94      rclone copy /home/source remote:backup
    95  
    96  ### Keeping your tokens safe
    97  
    98  Any OAuth-tokens will be stored by rclone in the remote's configuration file as unencrypted text.
    99  Anyone can use a valid refresh-token to access your HiDrive filesystem without knowing your password.
   100  Therefore you should make sure no one else can access your configuration.
   101  
   102  It is possible to encrypt rclone's configuration file.
   103  You can find information on securing your configuration file by viewing the [configuration encryption docs](/docs/#configuration-encryption).
   104  
   105  ### Invalid refresh token
   106  
   107  As can be verified [here](https://developer.hidrive.com/basics-flows/),
   108  each `refresh_token` (for Native Applications) is valid for 60 days.
   109  If used to access HiDrivei, its validity will be automatically extended.
   110  
   111  This means that if you
   112  
   113    * Don't use the HiDrive remote for 60 days
   114  
   115  then rclone will return an error which includes a text
   116  that implies the refresh token is *invalid* or *expired*.
   117  
   118  To fix this you will need to authorize rclone to access your HiDrive account again.
   119  
   120  Using
   121  
   122      rclone config reconnect remote:
   123  
   124  the process is very similar to the process of initial setup exemplified before.
   125  
   126  ### Modification times and hashes
   127  
   128  HiDrive allows modification times to be set on objects accurate to 1 second.
   129  
   130  HiDrive supports [its own hash type](https://static.hidrive.com/dev/0001)
   131  which is used to verify the integrity of file contents after successful transfers.
   132  
   133  ### Restricted filename characters
   134  
   135  HiDrive cannot store files or folders that include
   136  `/` (0x2F) or null-bytes (0x00) in their name.
   137  Any other characters can be used in the names of files or folders.
   138  Additionally, files or folders cannot be named either of the following: `.` or `..`
   139  
   140  Therefore rclone will automatically replace these characters,
   141  if files or folders are stored or accessed with such names.
   142  
   143  You can read about how this filename encoding works in general
   144  [here](overview/#restricted-filenames).
   145  
   146  Keep in mind that HiDrive only supports file or folder names
   147  with a length of 255 characters or less.
   148  
   149  ### Transfers
   150  
   151  HiDrive limits file sizes per single request to a maximum of 2 GiB.
   152  To allow storage of larger files and allow for better upload performance,
   153  the hidrive backend will use a chunked transfer for files larger than 96 MiB.
   154  Rclone will upload multiple parts/chunks of the file at the same time.
   155  Chunks in the process of being uploaded are buffered in memory,
   156  so you may want to restrict this behaviour on systems with limited resources.
   157  
   158  You can customize this behaviour using the following options:
   159  
   160  * `chunk_size`: size of file parts
   161  * `upload_cutoff`: files larger or equal to this in size will use a chunked transfer
   162  * `upload_concurrency`: number of file-parts to upload at the same time
   163  
   164  See the below section about configuration options for more details.
   165  
   166  ### Root folder
   167  
   168  You can set the root folder for rclone.
   169  This is the directory that rclone considers to be the root of your HiDrive.
   170  
   171  Usually, you will leave this blank, and rclone will use the root of the account.
   172  
   173  However, you can set this to restrict rclone to a specific folder hierarchy.
   174  
   175  This works by prepending the contents of the `root_prefix` option
   176  to any paths accessed by rclone.
   177  For example, the following two ways to access the home directory are equivalent:
   178  
   179      rclone lsd --hidrive-root-prefix="/users/test/" remote:path
   180  
   181      rclone lsd remote:/users/test/path
   182  
   183  See the below section about configuration options for more details.
   184  
   185  ### Directory member count
   186  
   187  By default, rclone will know the number of directory members contained in a directory.
   188  For example, `rclone lsd` uses this information.
   189  
   190  The acquisition of this information will result in additional time costs for HiDrive's API.
   191  When dealing with large directory structures, it may be desirable to circumvent this time cost,
   192  especially when this information is not explicitly needed.
   193  For this, the `disable_fetching_member_count` option can be used.
   194  
   195  See the below section about configuration options for more details.
   196  
   197  {{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/hidrive/hidrive.go then run make backenddocs" >}}
   198  ### Standard options
   199  
   200  Here are the Standard options specific to hidrive (HiDrive).
   201  
   202  #### --hidrive-client-id
   203  
   204  OAuth Client Id.
   205  
   206  Leave blank normally.
   207  
   208  Properties:
   209  
   210  - Config:      client_id
   211  - Env Var:     RCLONE_HIDRIVE_CLIENT_ID
   212  - Type:        string
   213  - Required:    false
   214  
   215  #### --hidrive-client-secret
   216  
   217  OAuth Client Secret.
   218  
   219  Leave blank normally.
   220  
   221  Properties:
   222  
   223  - Config:      client_secret
   224  - Env Var:     RCLONE_HIDRIVE_CLIENT_SECRET
   225  - Type:        string
   226  - Required:    false
   227  
   228  #### --hidrive-scope-access
   229  
   230  Access permissions that rclone should use when requesting access from HiDrive.
   231  
   232  Properties:
   233  
   234  - Config:      scope_access
   235  - Env Var:     RCLONE_HIDRIVE_SCOPE_ACCESS
   236  - Type:        string
   237  - Default:     "rw"
   238  - Examples:
   239      - "rw"
   240          - Read and write access to resources.
   241      - "ro"
   242          - Read-only access to resources.
   243  
   244  ### Advanced options
   245  
   246  Here are the Advanced options specific to hidrive (HiDrive).
   247  
   248  #### --hidrive-token
   249  
   250  OAuth Access Token as a JSON blob.
   251  
   252  Properties:
   253  
   254  - Config:      token
   255  - Env Var:     RCLONE_HIDRIVE_TOKEN
   256  - Type:        string
   257  - Required:    false
   258  
   259  #### --hidrive-auth-url
   260  
   261  Auth server URL.
   262  
   263  Leave blank to use the provider defaults.
   264  
   265  Properties:
   266  
   267  - Config:      auth_url
   268  - Env Var:     RCLONE_HIDRIVE_AUTH_URL
   269  - Type:        string
   270  - Required:    false
   271  
   272  #### --hidrive-token-url
   273  
   274  Token server url.
   275  
   276  Leave blank to use the provider defaults.
   277  
   278  Properties:
   279  
   280  - Config:      token_url
   281  - Env Var:     RCLONE_HIDRIVE_TOKEN_URL
   282  - Type:        string
   283  - Required:    false
   284  
   285  #### --hidrive-scope-role
   286  
   287  User-level that rclone should use when requesting access from HiDrive.
   288  
   289  Properties:
   290  
   291  - Config:      scope_role
   292  - Env Var:     RCLONE_HIDRIVE_SCOPE_ROLE
   293  - Type:        string
   294  - Default:     "user"
   295  - Examples:
   296      - "user"
   297          - User-level access to management permissions.
   298          - This will be sufficient in most cases.
   299      - "admin"
   300          - Extensive access to management permissions.
   301      - "owner"
   302          - Full access to management permissions.
   303  
   304  #### --hidrive-root-prefix
   305  
   306  The root/parent folder for all paths.
   307  
   308  Fill in to use the specified folder as the parent for all paths given to the remote.
   309  This way rclone can use any folder as its starting point.
   310  
   311  Properties:
   312  
   313  - Config:      root_prefix
   314  - Env Var:     RCLONE_HIDRIVE_ROOT_PREFIX
   315  - Type:        string
   316  - Default:     "/"
   317  - Examples:
   318      - "/"
   319          - The topmost directory accessible by rclone.
   320          - This will be equivalent with "root" if rclone uses a regular HiDrive user account.
   321      - "root"
   322          - The topmost directory of the HiDrive user account
   323      - ""
   324          - This specifies that there is no root-prefix for your paths.
   325          - When using this you will always need to specify paths to this remote with a valid parent e.g. "remote:/path/to/dir" or "remote:root/path/to/dir".
   326  
   327  #### --hidrive-endpoint
   328  
   329  Endpoint for the service.
   330  
   331  This is the URL that API-calls will be made to.
   332  
   333  Properties:
   334  
   335  - Config:      endpoint
   336  - Env Var:     RCLONE_HIDRIVE_ENDPOINT
   337  - Type:        string
   338  - Default:     "https://api.hidrive.strato.com/2.1"
   339  
   340  #### --hidrive-disable-fetching-member-count
   341  
   342  Do not fetch number of objects in directories unless it is absolutely necessary.
   343  
   344  Requests may be faster if the number of objects in subdirectories is not fetched.
   345  
   346  Properties:
   347  
   348  - Config:      disable_fetching_member_count
   349  - Env Var:     RCLONE_HIDRIVE_DISABLE_FETCHING_MEMBER_COUNT
   350  - Type:        bool
   351  - Default:     false
   352  
   353  #### --hidrive-chunk-size
   354  
   355  Chunksize for chunked uploads.
   356  
   357  Any files larger than the configured cutoff (or files of unknown size) will be uploaded in chunks of this size.
   358  
   359  The upper limit for this is 2147483647 bytes (about 2.000Gi).
   360  That is the maximum amount of bytes a single upload-operation will support.
   361  Setting this above the upper limit or to a negative value will cause uploads to fail.
   362  
   363  Setting this to larger values may increase the upload speed at the cost of using more memory.
   364  It can be set to smaller values smaller to save on memory.
   365  
   366  Properties:
   367  
   368  - Config:      chunk_size
   369  - Env Var:     RCLONE_HIDRIVE_CHUNK_SIZE
   370  - Type:        SizeSuffix
   371  - Default:     48Mi
   372  
   373  #### --hidrive-upload-cutoff
   374  
   375  Cutoff/Threshold for chunked uploads.
   376  
   377  Any files larger than this will be uploaded in chunks of the configured chunksize.
   378  
   379  The upper limit for this is 2147483647 bytes (about 2.000Gi).
   380  That is the maximum amount of bytes a single upload-operation will support.
   381  Setting this above the upper limit will cause uploads to fail.
   382  
   383  Properties:
   384  
   385  - Config:      upload_cutoff
   386  - Env Var:     RCLONE_HIDRIVE_UPLOAD_CUTOFF
   387  - Type:        SizeSuffix
   388  - Default:     96Mi
   389  
   390  #### --hidrive-upload-concurrency
   391  
   392  Concurrency for chunked uploads.
   393  
   394  This is the upper limit for how many transfers for the same file are running concurrently.
   395  Setting this above to a value smaller than 1 will cause uploads to deadlock.
   396  
   397  If you are uploading small numbers of large files over high-speed links
   398  and these uploads do not fully utilize your bandwidth, then increasing
   399  this may help to speed up the transfers.
   400  
   401  Properties:
   402  
   403  - Config:      upload_concurrency
   404  - Env Var:     RCLONE_HIDRIVE_UPLOAD_CONCURRENCY
   405  - Type:        int
   406  - Default:     4
   407  
   408  #### --hidrive-encoding
   409  
   410  The encoding for the backend.
   411  
   412  See the [encoding section in the overview](/overview/#encoding) for more info.
   413  
   414  Properties:
   415  
   416  - Config:      encoding
   417  - Env Var:     RCLONE_HIDRIVE_ENCODING
   418  - Type:        Encoding
   419  - Default:     Slash,Dot
   420  
   421  #### --hidrive-description
   422  
   423  Description of the remote
   424  
   425  Properties:
   426  
   427  - Config:      description
   428  - Env Var:     RCLONE_HIDRIVE_DESCRIPTION
   429  - Type:        string
   430  - Required:    false
   431  
   432  {{< rem autogenerated options stop >}}
   433  
   434  ## Limitations
   435  
   436  ### Symbolic links
   437  
   438  HiDrive is able to store symbolic links (*symlinks*) by design,
   439  for example, when unpacked from a zip archive.
   440  
   441  There exists no direct mechanism to manage native symlinks in remotes.
   442  As such this implementation has chosen to ignore any native symlinks present in the remote.
   443  rclone will not be able to access or show any symlinks stored in the hidrive-remote.
   444  This means symlinks cannot be individually removed, copied, or moved,
   445  except when removing, copying, or moving the parent folder.
   446  
   447  *This does not affect the `.rclonelink`-files
   448  that rclone uses to encode and store symbolic links.*
   449  
   450  ### Sparse files
   451  
   452  It is possible to store sparse files in HiDrive.
   453  
   454  Note that copying a sparse file will expand the holes
   455  into null-byte (0x00) regions that will then consume disk space.
   456  Likewise, when downloading a sparse file,
   457  the resulting file will have null-byte regions in the place of file holes.