github.com/rclone/rclone@v1.66.1-0.20240517100346-7b89735ae726/docs/content/hidrive.md (about) 1 --- 2 title: "HiDrive" 3 description: "Rclone docs for HiDrive" 4 versionIntroduced: "v1.59" 5 --- 6 7 # {{< icon "fa fa-cloud" >}} HiDrive 8 9 Paths are specified as `remote:path` 10 11 Paths may be as deep as required, e.g. `remote:directory/subdirectory`. 12 13 The initial setup for hidrive involves getting a token from HiDrive 14 which you need to do in your browser. 15 `rclone config` walks you through it. 16 17 ## Configuration 18 19 Here is an example of how to make a remote called `remote`. First run: 20 21 rclone config 22 23 This will guide you through an interactive setup process: 24 25 ``` 26 No remotes found - make a new one 27 n) New remote 28 s) Set configuration password 29 q) Quit config 30 n/s/q> n 31 name> remote 32 Type of storage to configure. 33 Choose a number from below, or type in your own value 34 [snip] 35 XX / HiDrive 36 \ "hidrive" 37 [snip] 38 Storage> hidrive 39 OAuth Client Id - Leave blank normally. 40 client_id> 41 OAuth Client Secret - Leave blank normally. 42 client_secret> 43 Access permissions that rclone should use when requesting access from HiDrive. 44 Leave blank normally. 45 scope_access> 46 Edit advanced config? 47 y/n> n 48 Use web browser to automatically authenticate rclone with remote? 49 * Say Y if the machine running rclone has a web browser you can use 50 * Say N if running rclone on a (remote) machine without web browser access 51 If not sure try Y. If Y failed, try N. 52 y/n> y 53 If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth?state=xxxxxxxxxxxxxxxxxxxxxx 54 Log in and authorize rclone for access 55 Waiting for code... 56 Got code 57 -------------------- 58 [remote] 59 type = hidrive 60 token = {"access_token":"xxxxxxxxxxxxxxxxxxxx","token_type":"Bearer","refresh_token":"xxxxxxxxxxxxxxxxxxxxxxx","expiry":"xxxxxxxxxxxxxxxxxxxxxxx"} 61 -------------------- 62 y) Yes this is OK (default) 63 e) Edit this remote 64 d) Delete this remote 65 y/e/d> y 66 ``` 67 68 **You should be aware that OAuth-tokens can be used to access your account 69 and hence should not be shared with other persons.** 70 See the [below section](#keeping-your-tokens-safe) for more information. 71 72 See the [remote setup docs](/remote_setup/) for how to set it up on a 73 machine with no Internet browser available. 74 75 Note that rclone runs a webserver on your local machine to collect the 76 token as returned from HiDrive. This only runs from the moment it opens 77 your browser to the moment you get back the verification code. 78 The webserver runs on `http://127.0.0.1:53682/`. 79 If local port `53682` is protected by a firewall you may need to temporarily 80 unblock the firewall to complete authorization. 81 82 Once configured you can then use `rclone` like this, 83 84 List directories in top level of your HiDrive root folder 85 86 rclone lsd remote: 87 88 List all the files in your HiDrive filesystem 89 90 rclone ls remote: 91 92 To copy a local directory to a HiDrive directory called backup 93 94 rclone copy /home/source remote:backup 95 96 ### Keeping your tokens safe 97 98 Any OAuth-tokens will be stored by rclone in the remote's configuration file as unencrypted text. 99 Anyone can use a valid refresh-token to access your HiDrive filesystem without knowing your password. 100 Therefore you should make sure no one else can access your configuration. 101 102 It is possible to encrypt rclone's configuration file. 103 You can find information on securing your configuration file by viewing the [configuration encryption docs](/docs/#configuration-encryption). 104 105 ### Invalid refresh token 106 107 As can be verified [here](https://developer.hidrive.com/basics-flows/), 108 each `refresh_token` (for Native Applications) is valid for 60 days. 109 If used to access HiDrivei, its validity will be automatically extended. 110 111 This means that if you 112 113 * Don't use the HiDrive remote for 60 days 114 115 then rclone will return an error which includes a text 116 that implies the refresh token is *invalid* or *expired*. 117 118 To fix this you will need to authorize rclone to access your HiDrive account again. 119 120 Using 121 122 rclone config reconnect remote: 123 124 the process is very similar to the process of initial setup exemplified before. 125 126 ### Modification times and hashes 127 128 HiDrive allows modification times to be set on objects accurate to 1 second. 129 130 HiDrive supports [its own hash type](https://static.hidrive.com/dev/0001) 131 which is used to verify the integrity of file contents after successful transfers. 132 133 ### Restricted filename characters 134 135 HiDrive cannot store files or folders that include 136 `/` (0x2F) or null-bytes (0x00) in their name. 137 Any other characters can be used in the names of files or folders. 138 Additionally, files or folders cannot be named either of the following: `.` or `..` 139 140 Therefore rclone will automatically replace these characters, 141 if files or folders are stored or accessed with such names. 142 143 You can read about how this filename encoding works in general 144 [here](overview/#restricted-filenames). 145 146 Keep in mind that HiDrive only supports file or folder names 147 with a length of 255 characters or less. 148 149 ### Transfers 150 151 HiDrive limits file sizes per single request to a maximum of 2 GiB. 152 To allow storage of larger files and allow for better upload performance, 153 the hidrive backend will use a chunked transfer for files larger than 96 MiB. 154 Rclone will upload multiple parts/chunks of the file at the same time. 155 Chunks in the process of being uploaded are buffered in memory, 156 so you may want to restrict this behaviour on systems with limited resources. 157 158 You can customize this behaviour using the following options: 159 160 * `chunk_size`: size of file parts 161 * `upload_cutoff`: files larger or equal to this in size will use a chunked transfer 162 * `upload_concurrency`: number of file-parts to upload at the same time 163 164 See the below section about configuration options for more details. 165 166 ### Root folder 167 168 You can set the root folder for rclone. 169 This is the directory that rclone considers to be the root of your HiDrive. 170 171 Usually, you will leave this blank, and rclone will use the root of the account. 172 173 However, you can set this to restrict rclone to a specific folder hierarchy. 174 175 This works by prepending the contents of the `root_prefix` option 176 to any paths accessed by rclone. 177 For example, the following two ways to access the home directory are equivalent: 178 179 rclone lsd --hidrive-root-prefix="/users/test/" remote:path 180 181 rclone lsd remote:/users/test/path 182 183 See the below section about configuration options for more details. 184 185 ### Directory member count 186 187 By default, rclone will know the number of directory members contained in a directory. 188 For example, `rclone lsd` uses this information. 189 190 The acquisition of this information will result in additional time costs for HiDrive's API. 191 When dealing with large directory structures, it may be desirable to circumvent this time cost, 192 especially when this information is not explicitly needed. 193 For this, the `disable_fetching_member_count` option can be used. 194 195 See the below section about configuration options for more details. 196 197 {{< rem autogenerated options start" - DO NOT EDIT - instead edit fs.RegInfo in backend/hidrive/hidrive.go then run make backenddocs" >}} 198 ### Standard options 199 200 Here are the Standard options specific to hidrive (HiDrive). 201 202 #### --hidrive-client-id 203 204 OAuth Client Id. 205 206 Leave blank normally. 207 208 Properties: 209 210 - Config: client_id 211 - Env Var: RCLONE_HIDRIVE_CLIENT_ID 212 - Type: string 213 - Required: false 214 215 #### --hidrive-client-secret 216 217 OAuth Client Secret. 218 219 Leave blank normally. 220 221 Properties: 222 223 - Config: client_secret 224 - Env Var: RCLONE_HIDRIVE_CLIENT_SECRET 225 - Type: string 226 - Required: false 227 228 #### --hidrive-scope-access 229 230 Access permissions that rclone should use when requesting access from HiDrive. 231 232 Properties: 233 234 - Config: scope_access 235 - Env Var: RCLONE_HIDRIVE_SCOPE_ACCESS 236 - Type: string 237 - Default: "rw" 238 - Examples: 239 - "rw" 240 - Read and write access to resources. 241 - "ro" 242 - Read-only access to resources. 243 244 ### Advanced options 245 246 Here are the Advanced options specific to hidrive (HiDrive). 247 248 #### --hidrive-token 249 250 OAuth Access Token as a JSON blob. 251 252 Properties: 253 254 - Config: token 255 - Env Var: RCLONE_HIDRIVE_TOKEN 256 - Type: string 257 - Required: false 258 259 #### --hidrive-auth-url 260 261 Auth server URL. 262 263 Leave blank to use the provider defaults. 264 265 Properties: 266 267 - Config: auth_url 268 - Env Var: RCLONE_HIDRIVE_AUTH_URL 269 - Type: string 270 - Required: false 271 272 #### --hidrive-token-url 273 274 Token server url. 275 276 Leave blank to use the provider defaults. 277 278 Properties: 279 280 - Config: token_url 281 - Env Var: RCLONE_HIDRIVE_TOKEN_URL 282 - Type: string 283 - Required: false 284 285 #### --hidrive-scope-role 286 287 User-level that rclone should use when requesting access from HiDrive. 288 289 Properties: 290 291 - Config: scope_role 292 - Env Var: RCLONE_HIDRIVE_SCOPE_ROLE 293 - Type: string 294 - Default: "user" 295 - Examples: 296 - "user" 297 - User-level access to management permissions. 298 - This will be sufficient in most cases. 299 - "admin" 300 - Extensive access to management permissions. 301 - "owner" 302 - Full access to management permissions. 303 304 #### --hidrive-root-prefix 305 306 The root/parent folder for all paths. 307 308 Fill in to use the specified folder as the parent for all paths given to the remote. 309 This way rclone can use any folder as its starting point. 310 311 Properties: 312 313 - Config: root_prefix 314 - Env Var: RCLONE_HIDRIVE_ROOT_PREFIX 315 - Type: string 316 - Default: "/" 317 - Examples: 318 - "/" 319 - The topmost directory accessible by rclone. 320 - This will be equivalent with "root" if rclone uses a regular HiDrive user account. 321 - "root" 322 - The topmost directory of the HiDrive user account 323 - "" 324 - This specifies that there is no root-prefix for your paths. 325 - When using this you will always need to specify paths to this remote with a valid parent e.g. "remote:/path/to/dir" or "remote:root/path/to/dir". 326 327 #### --hidrive-endpoint 328 329 Endpoint for the service. 330 331 This is the URL that API-calls will be made to. 332 333 Properties: 334 335 - Config: endpoint 336 - Env Var: RCLONE_HIDRIVE_ENDPOINT 337 - Type: string 338 - Default: "https://api.hidrive.strato.com/2.1" 339 340 #### --hidrive-disable-fetching-member-count 341 342 Do not fetch number of objects in directories unless it is absolutely necessary. 343 344 Requests may be faster if the number of objects in subdirectories is not fetched. 345 346 Properties: 347 348 - Config: disable_fetching_member_count 349 - Env Var: RCLONE_HIDRIVE_DISABLE_FETCHING_MEMBER_COUNT 350 - Type: bool 351 - Default: false 352 353 #### --hidrive-chunk-size 354 355 Chunksize for chunked uploads. 356 357 Any files larger than the configured cutoff (or files of unknown size) will be uploaded in chunks of this size. 358 359 The upper limit for this is 2147483647 bytes (about 2.000Gi). 360 That is the maximum amount of bytes a single upload-operation will support. 361 Setting this above the upper limit or to a negative value will cause uploads to fail. 362 363 Setting this to larger values may increase the upload speed at the cost of using more memory. 364 It can be set to smaller values smaller to save on memory. 365 366 Properties: 367 368 - Config: chunk_size 369 - Env Var: RCLONE_HIDRIVE_CHUNK_SIZE 370 - Type: SizeSuffix 371 - Default: 48Mi 372 373 #### --hidrive-upload-cutoff 374 375 Cutoff/Threshold for chunked uploads. 376 377 Any files larger than this will be uploaded in chunks of the configured chunksize. 378 379 The upper limit for this is 2147483647 bytes (about 2.000Gi). 380 That is the maximum amount of bytes a single upload-operation will support. 381 Setting this above the upper limit will cause uploads to fail. 382 383 Properties: 384 385 - Config: upload_cutoff 386 - Env Var: RCLONE_HIDRIVE_UPLOAD_CUTOFF 387 - Type: SizeSuffix 388 - Default: 96Mi 389 390 #### --hidrive-upload-concurrency 391 392 Concurrency for chunked uploads. 393 394 This is the upper limit for how many transfers for the same file are running concurrently. 395 Setting this above to a value smaller than 1 will cause uploads to deadlock. 396 397 If you are uploading small numbers of large files over high-speed links 398 and these uploads do not fully utilize your bandwidth, then increasing 399 this may help to speed up the transfers. 400 401 Properties: 402 403 - Config: upload_concurrency 404 - Env Var: RCLONE_HIDRIVE_UPLOAD_CONCURRENCY 405 - Type: int 406 - Default: 4 407 408 #### --hidrive-encoding 409 410 The encoding for the backend. 411 412 See the [encoding section in the overview](/overview/#encoding) for more info. 413 414 Properties: 415 416 - Config: encoding 417 - Env Var: RCLONE_HIDRIVE_ENCODING 418 - Type: Encoding 419 - Default: Slash,Dot 420 421 #### --hidrive-description 422 423 Description of the remote 424 425 Properties: 426 427 - Config: description 428 - Env Var: RCLONE_HIDRIVE_DESCRIPTION 429 - Type: string 430 - Required: false 431 432 {{< rem autogenerated options stop >}} 433 434 ## Limitations 435 436 ### Symbolic links 437 438 HiDrive is able to store symbolic links (*symlinks*) by design, 439 for example, when unpacked from a zip archive. 440 441 There exists no direct mechanism to manage native symlinks in remotes. 442 As such this implementation has chosen to ignore any native symlinks present in the remote. 443 rclone will not be able to access or show any symlinks stored in the hidrive-remote. 444 This means symlinks cannot be individually removed, copied, or moved, 445 except when removing, copying, or moving the parent folder. 446 447 *This does not affect the `.rclonelink`-files 448 that rclone uses to encode and store symbolic links.* 449 450 ### Sparse files 451 452 It is possible to store sparse files in HiDrive. 453 454 Note that copying a sparse file will expand the holes 455 into null-byte (0x00) regions that will then consume disk space. 456 Likewise, when downloading a sparse file, 457 the resulting file will have null-byte regions in the place of file holes.