github.com/rclone/rclone@v1.66.1-0.20240517100346-7b89735ae726/docs/content/docs.md (about) 1 --- 2 title: "Documentation" 3 description: "Rclone Usage" 4 --- 5 6 # Usage 7 8 Rclone is a command line program to manage files on cloud storage. 9 After [download](/downloads/) and [install](/install), continue 10 here to learn how to use it: Initial [configuration](#configure), 11 what the [basic syntax](#basic-syntax) looks like, describes the 12 various [subcommands](#subcommands), the various [options](#options), 13 and more. 14 15 Configure 16 --------- 17 18 First, you'll need to configure rclone. As the object storage systems 19 have quite complicated authentication these are kept in a config file. 20 (See the [`--config`](#config-config-file) entry for how to find the config 21 file and choose its location.) 22 23 The easiest way to make the config is to run rclone with the config 24 option: 25 26 rclone config 27 28 See the following for detailed instructions for 29 30 * [1Fichier](/fichier/) 31 * [Akamai Netstorage](/netstorage/) 32 * [Alias](/alias/) 33 * [Amazon S3](/s3/) 34 * [Backblaze B2](/b2/) 35 * [Box](/box/) 36 * [Chunker](/chunker/) - transparently splits large files for other remotes 37 * [Citrix ShareFile](/sharefile/) 38 * [Compress](/compress/) 39 * [Combine](/combine/) 40 * [Crypt](/crypt/) - to encrypt other remotes 41 * [DigitalOcean Spaces](/s3/#digitalocean-spaces) 42 * [Digi Storage](/koofr/#digi-storage) 43 * [Dropbox](/dropbox/) 44 * [Enterprise File Fabric](/filefabric/) 45 * [FTP](/ftp/) 46 * [Google Cloud Storage](/googlecloudstorage/) 47 * [Google Drive](/drive/) 48 * [Google Photos](/googlephotos/) 49 * [Hasher](/hasher/) - to handle checksums for other remotes 50 * [HDFS](/hdfs/) 51 * [HiDrive](/hidrive/) 52 * [HTTP](/http/) 53 * [Internet Archive](/internetarchive/) 54 * [Jottacloud](/jottacloud/) 55 * [Koofr](/koofr/) 56 * [Linkbox](/linkbox/) 57 * [Mail.ru Cloud](/mailru/) 58 * [Mega](/mega/) 59 * [Memory](/memory/) 60 * [Microsoft Azure Blob Storage](/azureblob/) 61 * [Microsoft Azure Files Storage](/azurefiles/) 62 * [Microsoft OneDrive](/onedrive/) 63 * [OpenStack Swift / Rackspace Cloudfiles / Blomp Cloud Storage / Memset Memstore](/swift/) 64 * [OpenDrive](/opendrive/) 65 * [Oracle Object Storage](/oracleobjectstorage/) 66 * [Pcloud](/pcloud/) 67 * [PikPak](/pikpak/) 68 * [premiumize.me](/premiumizeme/) 69 * [put.io](/putio/) 70 * [Proton Drive](/protondrive/) 71 * [QingStor](/qingstor/) 72 * [Quatrix by Maytech](/quatrix/) 73 * [Seafile](/seafile/) 74 * [SFTP](/sftp/) 75 * [Sia](/sia/) 76 * [SMB](/smb/) 77 * [Storj](/storj/) 78 * [SugarSync](/sugarsync/) 79 * [Union](/union/) 80 * [Uloz.to](/ulozto/) 81 * [Uptobox](/uptobox/) 82 * [WebDAV](/webdav/) 83 * [Yandex Disk](/yandex/) 84 * [Zoho WorkDrive](/zoho/) 85 * [The local filesystem](/local/) 86 87 Basic syntax 88 ----- 89 90 Rclone syncs a directory tree from one storage system to another. 91 92 Its syntax is like this 93 94 rclone subcommand [options] <parameters> <parameters...> 95 96 A `subcommand` is a the rclone operation required, (e.g. `sync`, 97 `copy`, `ls`). 98 99 An `option` is a single letter flag (e.g. `-v`) or a group of single 100 letter flags (e.g. `-Pv`) or a long flag (e.g. `--progress`). No 101 options are required. Options can come after the `subcommand` or in 102 between parameters too or on the end, but only global options can be 103 used before the `subcommand`. Anything after a `--` option will not be 104 interpreted as an option so if you need to add a parameter which 105 starts with a `-` then put a `--` on its own first, eg 106 107 rclone lsf -- -directory-starting-with-dash 108 109 A `parameter` is usually a file path or [rclone remote](#syntax-of-remote-paths), eg 110 `/path/to/file` or `remote:path/to/file` but it can be other things - 111 the `subcommand` help will tell you what. 112 113 Source and destination paths are specified by the name you gave the 114 storage system in the config file then the sub path, e.g. 115 "drive:myfolder" to look at "myfolder" in Google drive. 116 117 You can define as many storage paths as you like in the config file. 118 119 Please use the [`--interactive`/`-i`](#interactive) flag while 120 learning rclone to avoid accidental data loss. 121 122 Subcommands 123 ----------- 124 125 rclone uses a system of subcommands. For example 126 127 rclone ls remote:path # lists a remote 128 rclone copy /local/path remote:path # copies /local/path to the remote 129 rclone sync --interactive /local/path remote:path # syncs /local/path to the remote 130 131 The main rclone commands with most used first 132 133 * [rclone config](/commands/rclone_config/) - Enter an interactive configuration session. 134 * [rclone copy](/commands/rclone_copy/) - Copy files from source to dest, skipping already copied. 135 * [rclone sync](/commands/rclone_sync/) - Make source and dest identical, modifying destination only. 136 * [rclone bisync](/commands/rclone_bisync/) - [Bidirectional synchronization](/bisync/) between two paths. 137 * [rclone move](/commands/rclone_move/) - Move files from source to dest. 138 * [rclone delete](/commands/rclone_delete/) - Remove the contents of path. 139 * [rclone purge](/commands/rclone_purge/) - Remove the path and all of its contents. 140 * [rclone mkdir](/commands/rclone_mkdir/) - Make the path if it doesn't already exist. 141 * [rclone rmdir](/commands/rclone_rmdir/) - Remove the path. 142 * [rclone rmdirs](/commands/rclone_rmdirs/) - Remove any empty directories under the path. 143 * [rclone check](/commands/rclone_check/) - Check if the files in the source and destination match. 144 * [rclone ls](/commands/rclone_ls/) - List all the objects in the path with size and path. 145 * [rclone lsd](/commands/rclone_lsd/) - List all directories/containers/buckets in the path. 146 * [rclone lsl](/commands/rclone_lsl/) - List all the objects in the path with size, modification time and path. 147 * [rclone md5sum](/commands/rclone_md5sum/) - Produce an md5sum file for all the objects in the path. 148 * [rclone sha1sum](/commands/rclone_sha1sum/) - Produce a sha1sum file for all the objects in the path. 149 * [rclone size](/commands/rclone_size/) - Return the total size and number of objects in remote:path. 150 * [rclone version](/commands/rclone_version/) - Show the version number. 151 * [rclone cleanup](/commands/rclone_cleanup/) - Clean up the remote if possible. 152 * [rclone dedupe](/commands/rclone_dedupe/) - Interactively find duplicate files and delete/rename them. 153 * [rclone authorize](/commands/rclone_authorize/) - Remote authorization. 154 * [rclone cat](/commands/rclone_cat/) - Concatenate any files and send them to stdout. 155 * [rclone copyto](/commands/rclone_copyto/) - Copy files from source to dest, skipping already copied. 156 * [rclone genautocomplete](/commands/rclone_genautocomplete/) - Output shell completion scripts for rclone. 157 * [rclone gendocs](/commands/rclone_gendocs/) - Output markdown docs for rclone to the directory supplied. 158 * [rclone listremotes](/commands/rclone_listremotes/) - List all the remotes in the config file. 159 * [rclone mount](/commands/rclone_mount/) - Mount the remote as a mountpoint. 160 * [rclone moveto](/commands/rclone_moveto/) - Move file or directory from source to dest. 161 * [rclone obscure](/commands/rclone_obscure/) - Obscure password for use in the rclone.conf 162 * [rclone cryptcheck](/commands/rclone_cryptcheck/) - Check the integrity of an encrypted remote. 163 * [rclone about](/commands/rclone_about/) - Get quota information from the remote. 164 165 See the [commands index](/commands/) for the full list. 166 167 Copying single files 168 -------------------- 169 170 rclone normally syncs or copies directories. However, if the source 171 remote points to a file, rclone will just copy that file. The 172 destination remote must point to a directory - rclone will give the 173 error `Failed to create file system for "remote:file": is a file not a 174 directory` if it isn't. 175 176 For example, suppose you have a remote with a file in called 177 `test.jpg`, then you could copy just that file like this 178 179 rclone copy remote:test.jpg /tmp/download 180 181 The file `test.jpg` will be placed inside `/tmp/download`. 182 183 This is equivalent to specifying 184 185 rclone copy --files-from /tmp/files remote: /tmp/download 186 187 Where `/tmp/files` contains the single line 188 189 test.jpg 190 191 It is recommended to use `copy` when copying individual files, not `sync`. 192 They have pretty much the same effect but `copy` will use a lot less 193 memory. 194 195 Syntax of remote paths 196 ---------------------- 197 198 The syntax of the paths passed to the rclone command are as follows. 199 200 ### /path/to/dir 201 202 This refers to the local file system. 203 204 On Windows `\` may be used instead of `/` in local paths **only**, 205 non local paths must use `/`. See [local filesystem](https://rclone.org/local/#paths-on-windows) 206 documentation for more about Windows-specific paths. 207 208 These paths needn't start with a leading `/` - if they don't then they 209 will be relative to the current directory. 210 211 ### remote:path/to/dir 212 213 This refers to a directory `path/to/dir` on `remote:` as defined in 214 the config file (configured with `rclone config`). 215 216 ### remote:/path/to/dir 217 218 On most backends this is refers to the same directory as 219 `remote:path/to/dir` and that format should be preferred. On a very 220 small number of remotes (FTP, SFTP, Dropbox for business) this will 221 refer to a different directory. On these, paths without a leading `/` 222 will refer to your "home" directory and paths with a leading `/` will 223 refer to the root. 224 225 ### :backend:path/to/dir 226 227 This is an advanced form for creating remotes on the fly. `backend` 228 should be the name or prefix of a backend (the `type` in the config 229 file) and all the configuration for the backend should be provided on 230 the command line (or in environment variables). 231 232 Here are some examples: 233 234 rclone lsd --http-url https://pub.rclone.org :http: 235 236 To list all the directories in the root of `https://pub.rclone.org/`. 237 238 rclone lsf --http-url https://example.com :http:path/to/dir 239 240 To list files and directories in `https://example.com/path/to/dir/` 241 242 rclone copy --http-url https://example.com :http:path/to/dir /tmp/dir 243 244 To copy files and directories in `https://example.com/path/to/dir` to `/tmp/dir`. 245 246 rclone copy --sftp-host example.com :sftp:path/to/dir /tmp/dir 247 248 To copy files and directories from `example.com` in the relative 249 directory `path/to/dir` to `/tmp/dir` using sftp. 250 251 ### Connection strings {#connection-strings} 252 253 The above examples can also be written using a connection string 254 syntax, so instead of providing the arguments as command line 255 parameters `--http-url https://pub.rclone.org` they are provided as 256 part of the remote specification as a kind of connection string. 257 258 rclone lsd ":http,url='https://pub.rclone.org':" 259 rclone lsf ":http,url='https://example.com':path/to/dir" 260 rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir 261 rclone copy :sftp,host=example.com:path/to/dir /tmp/dir 262 263 These can apply to modify existing remotes as well as create new 264 remotes with the on the fly syntax. This example is equivalent to 265 adding the `--drive-shared-with-me` parameter to the remote `gdrive:`. 266 267 rclone lsf "gdrive,shared_with_me:path/to/dir" 268 269 The major advantage to using the connection string style syntax is 270 that it only applies to the remote, not to all the remotes of that 271 type of the command line. A common confusion is this attempt to copy a 272 file shared on google drive to the normal drive which **does not 273 work** because the `--drive-shared-with-me` flag applies to both the 274 source and the destination. 275 276 rclone copy --drive-shared-with-me gdrive:shared-file.txt gdrive: 277 278 However using the connection string syntax, this does work. 279 280 rclone copy "gdrive,shared_with_me:shared-file.txt" gdrive: 281 282 Note that the connection string only affects the options of the immediate 283 backend. If for example gdriveCrypt is a crypt based on gdrive, then the 284 following command **will not work** as intended, because 285 `shared_with_me` is ignored by the crypt backend: 286 287 rclone copy "gdriveCrypt,shared_with_me:shared-file.txt" gdriveCrypt: 288 289 The connection strings have the following syntax 290 291 remote,parameter=value,parameter2=value2:path/to/dir 292 :backend,parameter=value,parameter2=value2:path/to/dir 293 294 If the `parameter` has a `:` or `,` then it must be placed in quotes `"` or 295 `'`, so 296 297 remote,parameter="colon:value",parameter2="comma,value":path/to/dir 298 :backend,parameter='colon:value',parameter2='comma,value':path/to/dir 299 300 If a quoted value needs to include that quote, then it should be 301 doubled, so 302 303 remote,parameter="with""quote",parameter2='with''quote':path/to/dir 304 305 This will make `parameter` be `with"quote` and `parameter2` be 306 `with'quote`. 307 308 If you leave off the `=parameter` then rclone will substitute `=true` 309 which works very well with flags. For example, to use s3 configured in 310 the environment you could use: 311 312 rclone lsd :s3,env_auth: 313 314 Which is equivalent to 315 316 rclone lsd :s3,env_auth=true: 317 318 Note that on the command line you might need to surround these 319 connection strings with `"` or `'` to stop the shell interpreting any 320 special characters within them. 321 322 If you are a shell master then you'll know which strings are OK and 323 which aren't, but if you aren't sure then enclose them in `"` and use 324 `'` as the inside quote. This syntax works on all OSes. 325 326 rclone copy ":http,url='https://example.com':path/to/dir" /tmp/dir 327 328 On Linux/macOS some characters are still interpreted inside `"` 329 strings in the shell (notably `\` and `$` and `"`) so if your strings 330 contain those you can swap the roles of `"` and `'` thus. (This syntax 331 does not work on Windows.) 332 333 rclone copy ':http,url="https://example.com":path/to/dir' /tmp/dir 334 335 #### Connection strings, config and logging 336 337 If you supply extra configuration to a backend by command line flag, 338 environment variable or connection string then rclone will add a 339 suffix based on the hash of the config to the name of the remote, eg 340 341 rclone -vv lsf --s3-chunk-size 20M s3: 342 343 Has the log message 344 345 DEBUG : s3: detected overridden config - adding "{Srj1p}" suffix to name 346 347 This is so rclone can tell the modified remote apart from the 348 unmodified remote when caching the backends. 349 350 This should only be noticeable in the logs. 351 352 This means that on the fly backends such as 353 354 rclone -vv lsf :s3,env_auth: 355 356 Will get their own names 357 358 DEBUG : :s3: detected overridden config - adding "{YTu53}" suffix to name 359 360 ### Valid remote names 361 362 Remote names are case sensitive, and must adhere to the following rules: 363 - May contain number, letter, `_`, `-`, `.`, `+`, `@` and space. 364 - May not start with `-` or space. 365 - May not end with space. 366 367 Starting with rclone version 1.61, any Unicode numbers and letters are allowed, 368 while in older versions it was limited to plain ASCII (0-9, A-Z, a-z). If you use 369 the same rclone configuration from different shells, which may be configured with 370 different character encoding, you must be cautious to use characters that are 371 possible to write in all of them. This is mostly a problem on Windows, where 372 the console traditionally uses a non-Unicode character set - defined 373 by the so-called "code page". 374 375 Do not use single character names on Windows as it creates ambiguity with Windows 376 drives' names, e.g.: remote called `C` is indistinguishable from `C` drive. Rclone 377 will always assume that single letter name refers to a drive. 378 379 Quoting and the shell 380 --------------------- 381 382 When you are typing commands to your computer you are using something 383 called the command line shell. This interprets various characters in 384 an OS specific way. 385 386 Here are some gotchas which may help users unfamiliar with the shell rules 387 388 ### Linux / OSX ### 389 390 If your names have spaces or shell metacharacters (e.g. `*`, `?`, `$`, 391 `'`, `"`, etc.) then you must quote them. Use single quotes `'` by default. 392 393 rclone copy 'Important files?' remote:backup 394 395 If you want to send a `'` you will need to use `"`, e.g. 396 397 rclone copy "O'Reilly Reviews" remote:backup 398 399 The rules for quoting metacharacters are complicated and if you want 400 the full details you'll have to consult the manual page for your 401 shell. 402 403 ### Windows ### 404 405 If your names have spaces in you need to put them in `"`, e.g. 406 407 rclone copy "E:\folder name\folder name\folder name" remote:backup 408 409 If you are using the root directory on its own then don't quote it 410 (see [#464](https://github.com/rclone/rclone/issues/464) for why), e.g. 411 412 rclone copy E:\ remote:backup 413 414 Copying files or directories with `:` in the names 415 -------------------------------------------------- 416 417 rclone uses `:` to mark a remote name. This is, however, a valid 418 filename component in non-Windows OSes. The remote name parser will 419 only search for a `:` up to the first `/` so if you need to act on a 420 file or directory like this then use the full path starting with a 421 `/`, or use `./` as a current directory prefix. 422 423 So to sync a directory called `sync:me` to a remote called `remote:` use 424 425 rclone sync --interactive ./sync:me remote:path 426 427 or 428 429 rclone sync --interactive /full/path/to/sync:me remote:path 430 431 Server Side Copy 432 ---------------- 433 434 Most remotes (but not all - see [the 435 overview](/overview/#optional-features)) support server-side copy. 436 437 This means if you want to copy one folder to another then rclone won't 438 download all the files and re-upload them; it will instruct the server 439 to copy them in place. 440 441 Eg 442 443 rclone copy s3:oldbucket s3:newbucket 444 445 Will copy the contents of `oldbucket` to `newbucket` without 446 downloading and re-uploading. 447 448 Remotes which don't support server-side copy **will** download and 449 re-upload in this case. 450 451 Server side copies are used with `sync` and `copy` and will be 452 identified in the log when using the `-v` flag. The `move` command 453 may also use them if remote doesn't support server-side move directly. 454 This is done by issuing a server-side copy then a delete which is much 455 quicker than a download and re-upload. 456 457 Server side copies will only be attempted if the remote names are the 458 same. 459 460 This can be used when scripting to make aged backups efficiently, e.g. 461 462 rclone sync --interactive remote:current-backup remote:previous-backup 463 rclone sync --interactive /path/to/files remote:current-backup 464 465 ## Metadata support {#metadata} 466 467 Metadata is data about a file (or directory) which isn't the contents 468 of the file (or directory). Normally rclone only preserves the 469 modification time and the content (MIME) type where possible. 470 471 Rclone supports preserving all the available metadata on files and 472 directories when using the `--metadata` or `-M` flag. 473 474 Exactly what metadata is supported and what that support means depends 475 on the backend. Backends that support metadata have a metadata section 476 in their docs and are listed in the [features table](/overview/#features) 477 (Eg [local](/local/#metadata), [s3](/s3/#metadata)) 478 479 Some backends don't support metadata, some only support metadata on 480 files and some support metadata on both files and directories. 481 482 Rclone only supports a one-time sync of metadata. This means that 483 metadata will be synced from the source object to the destination 484 object only when the source object has changed and needs to be 485 re-uploaded. If the metadata subsequently changes on the source object 486 without changing the object itself then it won't be synced to the 487 destination object. This is in line with the way rclone syncs 488 `Content-Type` without the `--metadata` flag. 489 490 Using `--metadata` when syncing from local to local will preserve file 491 attributes such as file mode, owner, extended attributes (not 492 Windows). 493 494 Note that arbitrary metadata may be added to objects using the 495 `--metadata-set key=value` flag when the object is first uploaded. 496 This flag can be repeated as many times as necessary. 497 498 The [--metadata-mapper](#metadata-mapper) flag can be used to pass the 499 name of a program in which can transform metadata when it is being 500 copied from source to destination. 501 502 Rclone supports `--metadata-set` and `--metadata-mapper` when doing 503 sever side `Move` and server side `Copy`, but not when doing server 504 side `DirMove` (renaming a directory) as this would involve recursing 505 into the directory. Note that you can disable `DirMove` with 506 `--disable DirMove` and rclone will revert back to using `Move` for 507 each individual object where `--metadata-set` and `--metadata-mapper` 508 are supported. 509 510 ### Types of metadata 511 512 Metadata is divided into two type. System metadata and User metadata. 513 514 Metadata which the backend uses itself is called system metadata. For 515 example on the local backend the system metadata `uid` will store the 516 user ID of the file when used on a unix based platform. 517 518 Arbitrary metadata is called user metadata and this can be set however 519 is desired. 520 521 When objects are copied from backend to backend, they will attempt to 522 interpret system metadata if it is supplied. Metadata may change from 523 being user metadata to system metadata as objects are copied between 524 different backends. For example copying an object from s3 sets the 525 `content-type` metadata. In a backend which understands this (like 526 `azureblob`) this will become the Content-Type of the object. In a 527 backend which doesn't understand this (like the `local` backend) this 528 will become user metadata. However should the local object be copied 529 back to s3, the Content-Type will be set correctly. 530 531 ### Metadata framework 532 533 Rclone implements a metadata framework which can read metadata from an 534 object and write it to the object when (and only when) it is being 535 uploaded. 536 537 This metadata is stored as a dictionary with string keys and string 538 values. 539 540 There are some limits on the names of the keys (these may be clarified 541 further in the future). 542 543 - must be lower case 544 - may be `a-z` `0-9` containing `.` `-` or `_` 545 - length is backend dependent 546 547 Each backend can provide system metadata that it understands. Some 548 backends can also store arbitrary user metadata. 549 550 Where possible the key names are standardized, so, for example, it is 551 possible to copy object metadata from s3 to azureblob for example and 552 metadata will be translated appropriately. 553 554 Some backends have limits on the size of the metadata and rclone will 555 give errors on upload if they are exceeded. 556 557 ### Metadata preservation 558 559 The goal of the implementation is to 560 561 1. Preserve metadata if at all possible 562 2. Interpret metadata if at all possible 563 564 The consequences of 1 is that you can copy an S3 object to a local 565 disk then back to S3 losslessly. Likewise you can copy a local file 566 with file attributes and xattrs from local disk to s3 and back again 567 losslessly. 568 569 The consequence of 2 is that you can copy an S3 object with metadata 570 to Azureblob (say) and have the metadata appear on the Azureblob 571 object also. 572 573 ### Standard system metadata 574 575 Here is a table of standard system metadata which, if appropriate, a 576 backend may implement. 577 578 | key | description | example | 579 |---------------------|-------------|---------| 580 | mode | File type and mode: octal, unix style | 0100664 | 581 | uid | User ID of owner: decimal number | 500 | 582 | gid | Group ID of owner: decimal number | 500 | 583 | rdev | Device ID (if special file) => hexadecimal | 0 | 584 | atime | Time of last access: RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | 585 | mtime | Time of last modification: RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | 586 | btime | Time of file creation (birth): RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | 587 | utime | Time of file upload: RFC 3339 | 2006-01-02T15:04:05.999999999Z07:00 | 588 | cache-control | Cache-Control header | no-cache | 589 | content-disposition | Content-Disposition header | inline | 590 | content-encoding | Content-Encoding header | gzip | 591 | content-language | Content-Language header | en-US | 592 | content-type | Content-Type header | text/plain | 593 594 The metadata keys `mtime` and `content-type` will take precedence if 595 supplied in the metadata over reading the `Content-Type` or 596 modification time of the source object. 597 598 Hashes are not included in system metadata as there is a well defined 599 way of reading those already. 600 601 Options 602 ------- 603 604 Rclone has a number of options to control its behaviour. 605 606 Options that take parameters can have the values passed in two ways, 607 `--option=value` or `--option value`. However boolean (true/false) 608 options behave slightly differently to the other options in that 609 `--boolean` sets the option to `true` and the absence of the flag sets 610 it to `false`. It is also possible to specify `--boolean=false` or 611 `--boolean=true`. Note that `--boolean false` is not valid - this is 612 parsed as `--boolean` and the `false` is parsed as an extra command 613 line argument for rclone. 614 615 ### Time or duration options {#time-option} 616 617 TIME or DURATION options can be specified as a duration string or a 618 time string. 619 620 A duration string is a possibly signed sequence of decimal numbers, 621 each with optional fraction and a unit suffix, such as "300ms", 622 "-1.5h" or "2h45m". Default units are seconds or the following 623 abbreviations are valid: 624 625 * `ms` - Milliseconds 626 * `s` - Seconds 627 * `m` - Minutes 628 * `h` - Hours 629 * `d` - Days 630 * `w` - Weeks 631 * `M` - Months 632 * `y` - Years 633 634 These can also be specified as an absolute time in the following 635 formats: 636 637 - RFC3339 - e.g. `2006-01-02T15:04:05Z` or `2006-01-02T15:04:05+07:00` 638 - ISO8601 Date and time, local timezone - `2006-01-02T15:04:05` 639 - ISO8601 Date and time, local timezone - `2006-01-02 15:04:05` 640 - ISO8601 Date - `2006-01-02` (YYYY-MM-DD) 641 642 ### Size options {#size-option} 643 644 Options which use SIZE use KiB (multiples of 1024 bytes) by default. 645 However, a suffix of `B` for Byte, `K` for KiB, `M` for MiB, 646 `G` for GiB, `T` for TiB and `P` for PiB may be used. These are 647 the binary units, e.g. 1, 2\*\*10, 2\*\*20, 2\*\*30 respectively. 648 649 ### --backup-dir=DIR ### 650 651 When using `sync`, `copy` or `move` any files which would have been 652 overwritten or deleted are moved in their original hierarchy into this 653 directory. 654 655 If `--suffix` is set, then the moved files will have the suffix added 656 to them. If there is a file with the same path (after the suffix has 657 been added) in DIR, then it will be overwritten. 658 659 The remote in use must support server-side move or copy and you must 660 use the same remote as the destination of the sync. The backup 661 directory must not overlap the destination directory without it being 662 excluded by a filter rule. 663 664 For example 665 666 rclone sync --interactive /path/to/local remote:current --backup-dir remote:old 667 668 will sync `/path/to/local` to `remote:current`, but for any files 669 which would have been updated or deleted will be stored in 670 `remote:old`. 671 672 If running rclone from a script you might want to use today's date as 673 the directory name passed to `--backup-dir` to store the old files, or 674 you might want to pass `--suffix` with today's date. 675 676 See `--compare-dest` and `--copy-dest`. 677 678 ### --bind string ### 679 680 Local address to bind to for outgoing connections. This can be an 681 IPv4 address (1.2.3.4), an IPv6 address (1234::789A) or host name. If 682 the host name doesn't resolve or resolves to more than one IP address 683 it will give an error. 684 685 You can use `--bind 0.0.0.0` to force rclone to use IPv4 addresses and 686 `--bind ::0` to force rclone to use IPv6 addresses. 687 688 ### --bwlimit=BANDWIDTH_SPEC ### 689 690 This option controls the bandwidth limit. For example 691 692 --bwlimit 10M 693 694 would mean limit the upload and download bandwidth to 10 MiB/s. 695 **NB** this is **bytes** per second not **bits** per second. To use a 696 single limit, specify the desired bandwidth in KiB/s, or use a 697 suffix B|K|M|G|T|P. The default is `0` which means to not limit bandwidth. 698 699 The upload and download bandwidth can be specified separately, as 700 `--bwlimit UP:DOWN`, so 701 702 --bwlimit 10M:100k 703 704 would mean limit the upload bandwidth to 10 MiB/s and the download 705 bandwidth to 100 KiB/s. Either limit can be "off" meaning no limit, so 706 to just limit the upload bandwidth you would use 707 708 --bwlimit 10M:off 709 710 this would limit the upload bandwidth to 10 MiB/s but the download 711 bandwidth would be unlimited. 712 713 When specified as above the bandwidth limits last for the duration of 714 run of the rclone binary. 715 716 It is also possible to specify a "timetable" of limits, which will 717 cause certain limits to be applied at certain times. To specify a 718 timetable, format your entries as `WEEKDAY-HH:MM,BANDWIDTH 719 WEEKDAY-HH:MM,BANDWIDTH...` where: `WEEKDAY` is optional element. 720 721 - `BANDWIDTH` can be a single number, e.g.`100k` or a pair of numbers 722 for upload:download, e.g.`10M:1M`. 723 - `WEEKDAY` can be written as the whole word or only using the first 3 724 characters. It is optional. 725 - `HH:MM` is an hour from 00:00 to 23:59. 726 727 An example of a typical timetable to avoid link saturation during daytime 728 working hours could be: 729 730 `--bwlimit "08:00,512k 12:00,10M 13:00,512k 18:00,30M 23:00,off"` 731 732 In this example, the transfer bandwidth will be set to 512 KiB/s 733 at 8am every day. At noon, it will rise to 10 MiB/s, and drop back 734 to 512 KiB/sec at 1pm. At 6pm, the bandwidth limit will be set to 735 30 MiB/s, and at 11pm it will be completely disabled (full speed). 736 Anything between 11pm and 8am will remain unlimited. 737 738 An example of timetable with `WEEKDAY` could be: 739 740 `--bwlimit "Mon-00:00,512 Fri-23:59,10M Sat-10:00,1M Sun-20:00,off"` 741 742 It means that, the transfer bandwidth will be set to 512 KiB/s on 743 Monday. It will rise to 10 MiB/s before the end of Friday. At 10:00 744 on Saturday it will be set to 1 MiB/s. From 20:00 on Sunday it will 745 be unlimited. 746 747 Timeslots without `WEEKDAY` are extended to the whole week. So this 748 example: 749 750 `--bwlimit "Mon-00:00,512 12:00,1M Sun-20:00,off"` 751 752 Is equivalent to this: 753 754 `--bwlimit "Mon-00:00,512Mon-12:00,1M Tue-12:00,1M Wed-12:00,1M Thu-12:00,1M Fri-12:00,1M Sat-12:00,1M Sun-12:00,1M Sun-20:00,off"` 755 756 Bandwidth limit apply to the data transfer for all backends. For most 757 backends the directory listing bandwidth is also included (exceptions 758 being the non HTTP backends, `ftp`, `sftp` and `storj`). 759 760 Note that the units are **Byte/s**, not **bit/s**. Typically 761 connections are measured in bit/s - to convert divide by 8. For 762 example, let's say you have a 10 Mbit/s connection and you wish rclone 763 to use half of it - 5 Mbit/s. This is 5/8 = 0.625 MiB/s so you would 764 use a `--bwlimit 0.625M` parameter for rclone. 765 766 On Unix systems (Linux, macOS, …) the bandwidth limiter can be toggled by 767 sending a `SIGUSR2` signal to rclone. This allows to remove the limitations 768 of a long running rclone transfer and to restore it back to the value specified 769 with `--bwlimit` quickly when needed. Assuming there is only one rclone instance 770 running, you can toggle the limiter like this: 771 772 kill -SIGUSR2 $(pidof rclone) 773 774 If you configure rclone with a [remote control](/rc) then you can use 775 change the bwlimit dynamically: 776 777 rclone rc core/bwlimit rate=1M 778 779 ### --bwlimit-file=BANDWIDTH_SPEC ### 780 781 This option controls per file bandwidth limit. For the options see the 782 `--bwlimit` flag. 783 784 For example use this to allow no transfers to be faster than 1 MiB/s 785 786 --bwlimit-file 1M 787 788 This can be used in conjunction with `--bwlimit`. 789 790 Note that if a schedule is provided the file will use the schedule in 791 effect at the start of the transfer. 792 793 ### --buffer-size=SIZE ### 794 795 Use this sized buffer to speed up file transfers. Each `--transfer` 796 will use this much memory for buffering. 797 798 When using `mount` or `cmount` each open file descriptor will use this much 799 memory for buffering. 800 See the [mount](/commands/rclone_mount/#file-buffering) documentation for more details. 801 802 Set to `0` to disable the buffering for the minimum memory usage. 803 804 Note that the memory allocation of the buffers is influenced by the 805 [--use-mmap](#use-mmap) flag. 806 807 ### --cache-dir=DIR ### 808 809 Specify the directory rclone will use for caching, to override 810 the default. 811 812 Default value is depending on operating system: 813 - Windows `%LocalAppData%\rclone`, if `LocalAppData` is defined. 814 - macOS `$HOME/Library/Caches/rclone` if `HOME` is defined. 815 - Unix `$XDG_CACHE_HOME/rclone` if `XDG_CACHE_HOME` is defined, else `$HOME/.cache/rclone` if `HOME` is defined. 816 - Fallback (on all OS) to `$TMPDIR/rclone`, where `TMPDIR` is the value from [--temp-dir](#temp-dir-dir). 817 818 You can use the [config paths](/commands/rclone_config_paths/) 819 command to see the current value. 820 821 Cache directory is heavily used by the [VFS File Caching](/commands/rclone_mount/#vfs-file-caching) 822 mount feature, but also by [serve](/commands/rclone_serve/), [GUI](/gui) and other parts of rclone. 823 824 ### --check-first ### 825 826 If this flag is set then in a `sync`, `copy` or `move`, rclone will do 827 all the checks to see whether files need to be transferred before 828 doing any of the transfers. Normally rclone would start running 829 transfers as soon as possible. 830 831 This flag can be useful on IO limited systems where transfers 832 interfere with checking. 833 834 It can also be useful to ensure perfect ordering when using 835 `--order-by`. 836 837 If both `--check-first` and `--order-by` are set when doing `rclone move` 838 then rclone will use the transfer thread to delete source files which 839 don't need transferring. This will enable perfect ordering of the 840 transfers and deletes but will cause the transfer stats to have more 841 items in than expected. 842 843 Using this flag can use more memory as it effectively sets 844 `--max-backlog` to infinite. This means that all the info on the 845 objects to transfer is held in memory before the transfers start. 846 847 ### --checkers=N ### 848 849 Originally controlling just the number of file checkers to run in parallel, 850 e.g. by `rclone copy`. Now a fairly universal parallelism control 851 used by `rclone` in several places. 852 853 Note: checkers do the equality checking of files during a sync. 854 For some storage systems (e.g. S3, Swift, Dropbox) this can take 855 a significant amount of time so they are run in parallel. 856 857 The default is to run 8 checkers in parallel. However, in case 858 of slow-reacting backends you may need to lower (rather than increase) 859 this default by setting `--checkers` to 4 or less threads. This is 860 especially advised if you are experiencing backend server crashes 861 during file checking phase (e.g. on subsequent or top-up backups 862 where little or no file copying is done and checking takes up 863 most of the time). Increase this setting only with utmost care, 864 while monitoring your server health and file checking throughput. 865 866 ### -c, --checksum ### 867 868 Normally rclone will look at modification time and size of files to 869 see if they are equal. If you set this flag then rclone will check 870 the file hash and size to determine if files are equal. 871 872 This is useful when the remote doesn't support setting modified time 873 and a more accurate sync is desired than just checking the file size. 874 875 This is very useful when transferring between remotes which store the 876 same hash type on the object, e.g. Drive and Swift. For details of which 877 remotes support which hash type see the table in the [overview 878 section](/overview/). 879 880 Eg `rclone --checksum sync s3:/bucket swift:/bucket` would run much 881 quicker than without the `--checksum` flag. 882 883 When using this flag, rclone won't update mtimes of remote files if 884 they are incorrect as it would normally. 885 886 ### --color WHEN ### 887 888 Specify when colors (and other ANSI codes) should be added to the output. 889 890 `AUTO` (default) only allows ANSI codes when the output is a terminal 891 892 `NEVER` never allow ANSI codes 893 894 `ALWAYS` always add ANSI codes, regardless of the output format (terminal or file) 895 896 ### --compare-dest=DIR ### 897 898 When using `sync`, `copy` or `move` DIR is checked in addition to the 899 destination for files. If a file identical to the source is found that 900 file is NOT copied from source. This is useful to copy just files that 901 have changed since the last backup. 902 903 You must use the same remote as the destination of the sync. The 904 compare directory must not overlap the destination directory. 905 906 See `--copy-dest` and `--backup-dir`. 907 908 ### --config=CONFIG_FILE ### 909 910 Specify the location of the rclone configuration file, to override 911 the default. E.g. `rclone config --config="rclone.conf"`. 912 913 The exact default is a bit complex to describe, due to changes 914 introduced through different versions of rclone while preserving 915 backwards compatibility, but in most cases it is as simple as: 916 917 - `%APPDATA%/rclone/rclone.conf` on Windows 918 - `~/.config/rclone/rclone.conf` on other 919 920 The complete logic is as follows: Rclone will look for an existing 921 configuration file in any of the following locations, in priority order: 922 923 1. `rclone.conf` (in program directory, where rclone executable is) 924 2. `%APPDATA%/rclone/rclone.conf` (only on Windows) 925 3. `$XDG_CONFIG_HOME/rclone/rclone.conf` (on all systems, including Windows) 926 4. `~/.config/rclone/rclone.conf` (see below for explanation of ~ symbol) 927 5. `~/.rclone.conf` 928 929 If no existing configuration file is found, then a new one will be created 930 in the following location: 931 932 - On Windows: Location 2 listed above, except in the unlikely event 933 that `APPDATA` is not defined, then location 4 is used instead. 934 - On Unix: Location 3 if `XDG_CONFIG_HOME` is defined, else location 4. 935 - Fallback to location 5 (on all OS), when the rclone directory cannot be 936 created, but if also a home directory was not found then path 937 `.rclone.conf` relative to current working directory will be used as 938 a final resort. 939 940 The `~` symbol in paths above represent the home directory of the current user 941 on any OS, and the value is defined as following: 942 943 - On Windows: `%HOME%` if defined, else `%USERPROFILE%`, or else `%HOMEDRIVE%\%HOMEPATH%`. 944 - On Unix: `$HOME` if defined, else by looking up current user in OS-specific user database 945 (e.g. passwd file), or else use the result from shell command `cd && pwd`. 946 947 If you run `rclone config file` you will see where the default 948 location is for you. 949 950 The fact that an existing file `rclone.conf` in the same directory 951 as the rclone executable is always preferred, means that it is easy 952 to run in "portable" mode by downloading rclone executable to a 953 writable directory and then create an empty file `rclone.conf` in the 954 same directory. 955 956 If the location is set to empty string `""` or path to a file 957 with name `notfound`, or the os null device represented by value `NUL` on 958 Windows and `/dev/null` on Unix systems, then rclone will keep the 959 config file in memory only. 960 961 The file format is basic [INI](https://en.wikipedia.org/wiki/INI_file#Format): 962 Sections of text, led by a `[section]` header and followed by 963 `key=value` entries on separate lines. In rclone each remote is 964 represented by its own section, where the section name defines the 965 name of the remote. Options are specified as the `key=value` entries, 966 where the key is the option name without the `--backend-` prefix, 967 in lowercase and with `_` instead of `-`. E.g. option `--mega-hard-delete` 968 corresponds to key `hard_delete`. Only backend options can be specified. 969 A special, and required, key `type` identifies the [storage system](/overview/), 970 where the value is the internal lowercase name as returned by command 971 `rclone help backends`. Comments are indicated by `;` or `#` at the 972 beginning of a line. 973 974 Example: 975 976 [megaremote] 977 type = mega 978 user = you@example.com 979 pass = PDPcQVVjVtzFY-GTdDFozqBhTdsPg3qH 980 981 Note that passwords are in [obscured](/commands/rclone_obscure/) 982 form. Also, many storage systems uses token-based authentication instead 983 of passwords, and this requires additional steps. It is easier, and safer, 984 to use the interactive command `rclone config` instead of manually 985 editing the configuration file. 986 987 The configuration file will typically contain login information, and 988 should therefore have restricted permissions so that only the current user 989 can read it. Rclone tries to ensure this when it writes the file. 990 You may also choose to [encrypt](#configuration-encryption) the file. 991 992 When token-based authentication are used, the configuration file 993 must be writable, because rclone needs to update the tokens inside it. 994 995 To reduce risk of corrupting an existing configuration file, rclone 996 will not write directly to it when saving changes. Instead it will 997 first write to a new, temporary, file. If a configuration file already 998 existed, it will (on Unix systems) try to mirror its permissions to 999 the new file. Then it will rename the existing file to a temporary 1000 name as backup. Next, rclone will rename the new file to the correct name, 1001 before finally cleaning up by deleting the backup file. 1002 1003 If the configuration file path used by rclone is a symbolic link, then 1004 this will be evaluated and rclone will write to the resolved path, instead 1005 of overwriting the symbolic link. Temporary files used in the process 1006 (described above) will be written to the same parent directory as that 1007 of the resolved configuration file, but if this directory is also a 1008 symbolic link it will not be resolved and the temporary files will be 1009 written to the location of the directory symbolic link. 1010 1011 ### --contimeout=TIME ### 1012 1013 Set the connection timeout. This should be in go time format which 1014 looks like `5s` for 5 seconds, `10m` for 10 minutes, or `3h30m`. 1015 1016 The connection timeout is the amount of time rclone will wait for a 1017 connection to go through to a remote object storage system. It is 1018 `1m` by default. 1019 1020 ### --copy-dest=DIR ### 1021 1022 When using `sync`, `copy` or `move` DIR is checked in addition to the 1023 destination for files. If a file identical to the source is found that 1024 file is server-side copied from DIR to the destination. This is useful 1025 for incremental backup. 1026 1027 The remote in use must support server-side copy and you must 1028 use the same remote as the destination of the sync. The compare 1029 directory must not overlap the destination directory. 1030 1031 See `--compare-dest` and `--backup-dir`. 1032 1033 ### --dedupe-mode MODE ### 1034 1035 Mode to run dedupe command in. One of `interactive`, `skip`, `first`, 1036 `newest`, `oldest`, `rename`. The default is `interactive`. 1037 See the dedupe command for more information as to what these options mean. 1038 1039 ### --default-time TIME ### 1040 1041 If a file or directory does have a modification time rclone can read 1042 then rclone will display this fixed time instead. 1043 1044 The default is `2000-01-01 00:00:00 UTC`. This can be configured in 1045 any of the ways shown in [the time or duration options](#time-option). 1046 1047 For example `--default-time 2020-06-01` to set the default time to the 1048 1st of June 2020 or `--default-time 0s` to set the default time to the 1049 time rclone started up. 1050 1051 ### --disable FEATURE,FEATURE,... ### 1052 1053 This disables a comma separated list of optional features. For example 1054 to disable server-side move and server-side copy use: 1055 1056 --disable move,copy 1057 1058 The features can be put in any case. 1059 1060 To see a list of which features can be disabled use: 1061 1062 --disable help 1063 1064 The features a remote has can be seen in JSON format with: 1065 1066 rclone backend features remote: 1067 1068 See the overview [features](/overview/#features) and 1069 [optional features](/overview/#optional-features) to get an idea of 1070 which feature does what. 1071 1072 Note that some features can be set to `true` if they are `true`/`false` 1073 feature flag features by prefixing them with `!`. For example the 1074 `CaseInsensitive` feature can be forced to `false` with `--disable CaseInsensitive` 1075 and forced to `true` with `--disable '!CaseInsensitive'`. In general 1076 it isn't a good idea doing this but it may be useful in extremis. 1077 1078 (Note that `!` is a shell command which you will 1079 need to escape with single quotes or a backslash on unix like 1080 platforms.) 1081 1082 This flag can be useful for debugging and in exceptional circumstances 1083 (e.g. Google Drive limiting the total volume of Server Side Copies to 1084 100 GiB/day). 1085 1086 ### --disable-http2 1087 1088 This stops rclone from trying to use HTTP/2 if available. This can 1089 sometimes speed up transfers due to a 1090 [problem in the Go standard library](https://github.com/golang/go/issues/37373). 1091 1092 ### --dscp VALUE ### 1093 1094 Specify a DSCP value or name to use in connections. This could help QoS 1095 system to identify traffic class. BE, EF, DF, LE, CSx and AFxx are allowed. 1096 1097 See the description of [differentiated services](https://en.wikipedia.org/wiki/Differentiated_services) to get an idea of 1098 this field. Setting this to 1 (LE) to identify the flow to SCAVENGER class 1099 can avoid occupying too much bandwidth in a network with DiffServ support ([RFC 8622](https://tools.ietf.org/html/rfc8622)). 1100 1101 For example, if you configured QoS on router to handle LE properly. Running: 1102 ``` 1103 rclone copy --dscp LE from:/from to:/to 1104 ``` 1105 would make the priority lower than usual internet flows. 1106 1107 This option has no effect on Windows (see [golang/go#42728](https://github.com/golang/go/issues/42728)). 1108 1109 ### -n, --dry-run ### 1110 1111 Do a trial run with no permanent changes. Use this to see what rclone 1112 would do without actually doing it. Useful when setting up the `sync` 1113 command which deletes files in the destination. 1114 1115 ### --expect-continue-timeout=TIME ### 1116 1117 This specifies the amount of time to wait for a server's first 1118 response headers after fully writing the request headers if the 1119 request has an "Expect: 100-continue" header. Not all backends support 1120 using this. 1121 1122 Zero means no timeout and causes the body to be sent immediately, 1123 without waiting for the server to approve. This time does not include 1124 the time to send the request header. 1125 1126 The default is `1s`. Set to `0` to disable. 1127 1128 ### --error-on-no-transfer ### 1129 1130 By default, rclone will exit with return code 0 if there were no errors. 1131 1132 This option allows rclone to return exit code 9 if no files were transferred 1133 between the source and destination. This allows using rclone in scripts, and 1134 triggering follow-on actions if data was copied, or skipping if not. 1135 1136 NB: Enabling this option turns a usually non-fatal error into a potentially 1137 fatal one - please check and adjust your scripts accordingly! 1138 1139 ### --fix-case ### 1140 1141 Normally, a sync to a case insensitive dest (such as macOS / Windows) will 1142 not result in a matching filename if the source and dest filenames have 1143 casing differences but are otherwise identical. For example, syncing `hello.txt` 1144 to `HELLO.txt` will normally result in the dest filename remaining `HELLO.txt`. 1145 If `--fix-case` is set, then `HELLO.txt` will be renamed to `hello.txt` 1146 to match the source. 1147 1148 NB: 1149 - directory names with incorrect casing will also be fixed 1150 - `--fix-case` will be ignored if `--immutable` is set 1151 - using `--local-case-sensitive` instead is not advisable; 1152 it will cause `HELLO.txt` to get deleted! 1153 - the old dest filename must not be excluded by filters. 1154 Be especially careful with [`--files-from`](/filtering/#files-from-read-list-of-source-file-names), 1155 which does not respect [`--ignore-case`](/filtering/#ignore-case-make-searches-case-insensitive)! 1156 - on remotes that do not support server-side move, `--fix-case` will require 1157 downloading the file and re-uploading it. To avoid this, do not use `--fix-case`. 1158 1159 ### --fs-cache-expire-duration=TIME 1160 1161 When using rclone via the API rclone caches created remotes for 5 1162 minutes by default in the "fs cache". This means that if you do 1163 repeated actions on the same remote then rclone won't have to build it 1164 again from scratch, which makes it more efficient. 1165 1166 This flag sets the time that the remotes are cached for. If you set it 1167 to `0` (or negative) then rclone won't cache the remotes at all. 1168 1169 Note that if you use some flags, eg `--backup-dir` and if this is set 1170 to `0` rclone may build two remotes (one for the source or destination 1171 and one for the `--backup-dir` where it may have only built one 1172 before. 1173 1174 ### --fs-cache-expire-interval=TIME 1175 1176 This controls how often rclone checks for cached remotes to expire. 1177 See the `--fs-cache-expire-duration` documentation above for more 1178 info. The default is 60s, set to 0 to disable expiry. 1179 1180 ### --header ### 1181 1182 Add an HTTP header for all transactions. The flag can be repeated to 1183 add multiple headers. 1184 1185 If you want to add headers only for uploads use `--header-upload` and 1186 if you want to add headers only for downloads use `--header-download`. 1187 1188 This flag is supported for all HTTP based backends even those not 1189 supported by `--header-upload` and `--header-download` so may be used 1190 as a workaround for those with care. 1191 1192 ``` 1193 rclone ls remote:test --header "X-Rclone: Foo" --header "X-LetMeIn: Yes" 1194 ``` 1195 1196 ### --header-download ### 1197 1198 Add an HTTP header for all download transactions. The flag can be repeated to 1199 add multiple headers. 1200 1201 ``` 1202 rclone sync --interactive s3:test/src ~/dst --header-download "X-Amz-Meta-Test: Foo" --header-download "X-Amz-Meta-Test2: Bar" 1203 ``` 1204 1205 See the GitHub issue [here](https://github.com/rclone/rclone/issues/59) for 1206 currently supported backends. 1207 1208 ### --header-upload ### 1209 1210 Add an HTTP header for all upload transactions. The flag can be repeated to add 1211 multiple headers. 1212 1213 ``` 1214 rclone sync --interactive ~/src s3:test/dst --header-upload "Content-Disposition: attachment; filename='cool.html'" --header-upload "X-Amz-Meta-Test: FooBar" 1215 ``` 1216 1217 See the GitHub issue [here](https://github.com/rclone/rclone/issues/59) for 1218 currently supported backends. 1219 1220 ### --human-readable ### 1221 1222 Rclone commands output values for sizes (e.g. number of bytes) and 1223 counts (e.g. number of files) either as *raw* numbers, or 1224 in *human-readable* format. 1225 1226 In human-readable format the values are scaled to larger units, indicated with 1227 a suffix shown after the value, and rounded to three decimals. Rclone consistently 1228 uses binary units (powers of 2) for sizes and decimal units (powers of 10) for counts. 1229 The unit prefix for size is according to IEC standard notation, e.g. `Ki` for kibi. 1230 Used with byte unit, `1 KiB` means 1024 Byte. In list type of output, only the 1231 unit prefix appended to the value (e.g. `9.762Ki`), while in more textual output 1232 the full unit is shown (e.g. `9.762 KiB`). For counts the SI standard notation is 1233 used, e.g. prefix `k` for kilo. Used with file counts, `1k` means 1000 files. 1234 1235 The various [list](/commands/rclone_ls/) commands output raw numbers by default. 1236 Option `--human-readable` will make them output values in human-readable format 1237 instead (with the short unit prefix). 1238 1239 The [about](/commands/rclone_about/) command outputs human-readable by default, 1240 with a command-specific option `--full` to output the raw numbers instead. 1241 1242 Command [size](/commands/rclone_size/) outputs both human-readable and raw numbers 1243 in the same output. 1244 1245 The [tree](/commands/rclone_tree/) command also considers `--human-readable`, but 1246 it will not use the exact same notation as the other commands: It rounds to one 1247 decimal, and uses single letter suffix, e.g. `K` instead of `Ki`. The reason for 1248 this is that it relies on an external library. 1249 1250 The interactive command [ncdu](/commands/rclone_ncdu/) shows human-readable by 1251 default, and responds to key `u` for toggling human-readable format. 1252 1253 ### --ignore-case-sync ### 1254 1255 Using this option will cause rclone to ignore the case of the files 1256 when synchronizing so files will not be copied/synced when the 1257 existing filenames are the same, even if the casing is different. 1258 1259 ### --ignore-checksum ### 1260 1261 Normally rclone will check that the checksums of transferred files 1262 match, and give an error "corrupted on transfer" if they don't. 1263 1264 You can use this option to skip that check. You should only use it if 1265 you have had the "corrupted on transfer" error message and you are 1266 sure you might want to transfer potentially corrupted data. 1267 1268 ### --ignore-existing ### 1269 1270 Using this option will make rclone unconditionally skip all files 1271 that exist on the destination, no matter the content of these files. 1272 1273 While this isn't a generally recommended option, it can be useful 1274 in cases where your files change due to encryption. However, it cannot 1275 correct partial transfers in case a transfer was interrupted. 1276 1277 When performing a `move`/`moveto` command, this flag will leave skipped 1278 files in the source location unchanged when a file with the same name 1279 exists on the destination. 1280 1281 ### --ignore-size ### 1282 1283 Normally rclone will look at modification time and size of files to 1284 see if they are equal. If you set this flag then rclone will check 1285 only the modification time. If `--checksum` is set then it only 1286 checks the checksum. 1287 1288 It will also cause rclone to skip verifying the sizes are the same 1289 after transfer. 1290 1291 This can be useful for transferring files to and from OneDrive which 1292 occasionally misreports the size of image files (see 1293 [#399](https://github.com/rclone/rclone/issues/399) for more info). 1294 1295 ### -I, --ignore-times ### 1296 1297 Using this option will cause rclone to unconditionally upload all 1298 files regardless of the state of files on the destination. 1299 1300 Normally rclone would skip any files that have the same 1301 modification time and are the same size (or have the same checksum if 1302 using `--checksum`). 1303 1304 ### --immutable ### 1305 1306 Treat source and destination files as immutable and disallow 1307 modification. 1308 1309 With this option set, files will be created and deleted as requested, 1310 but existing files will never be updated. If an existing file does 1311 not match between the source and destination, rclone will give the error 1312 `Source and destination exist but do not match: immutable file modified`. 1313 1314 Note that only commands which transfer files (e.g. `sync`, `copy`, 1315 `move`) are affected by this behavior, and only modification is 1316 disallowed. Files may still be deleted explicitly (e.g. `delete`, 1317 `purge`) or implicitly (e.g. `sync`, `move`). Use `copy --immutable` 1318 if it is desired to avoid deletion as well as modification. 1319 1320 This can be useful as an additional layer of protection for immutable 1321 or append-only data sets (notably backup archives), where modification 1322 implies corruption and should not be propagated. 1323 1324 ### --inplace {#inplace} 1325 1326 The `--inplace` flag changes the behaviour of rclone when uploading 1327 files to some backends (backends with the `PartialUploads` feature 1328 flag set) such as: 1329 1330 - local 1331 - ftp 1332 - sftp 1333 1334 Without `--inplace` (the default) rclone will first upload to a 1335 temporary file with an extension like this, where `XXXXXX` represents a 1336 random string and `.partial` is [--partial-suffix](#partial-suffix) value 1337 (`.partial` by default). 1338 1339 original-file-name.XXXXXX.partial 1340 1341 (rclone will make sure the final name is no longer than 100 characters 1342 by truncating the `original-file-name` part if necessary). 1343 1344 When the upload is complete, rclone will rename the `.partial` file to 1345 the correct name, overwriting any existing file at that point. If the 1346 upload fails then the `.partial` file will be deleted. 1347 1348 This prevents other users of the backend from seeing partially 1349 uploaded files in their new names and prevents overwriting the old 1350 file until the new one is completely uploaded. 1351 1352 If the `--inplace` flag is supplied, rclone will upload directly to 1353 the final name without creating a `.partial` file. 1354 1355 This means that an incomplete file will be visible in the directory 1356 listings while the upload is in progress and any existing files will 1357 be overwritten as soon as the upload starts. If the transfer fails 1358 then the file will be deleted. This can cause data loss of the 1359 existing file if the transfer fails. 1360 1361 Note that on the local file system if you don't use `--inplace` hard 1362 links (Unix only) will be broken. And if you do use `--inplace` you 1363 won't be able to update in use executables. 1364 1365 Note also that versions of rclone prior to v1.63.0 behave as if the 1366 `--inplace` flag is always supplied. 1367 1368 ### -i, --interactive {#interactive} 1369 1370 This flag can be used to tell rclone that you wish a manual 1371 confirmation before destructive operations. 1372 1373 It is **recommended** that you use this flag while learning rclone 1374 especially with `rclone sync`. 1375 1376 For example 1377 1378 ``` 1379 $ rclone delete --interactive /tmp/dir 1380 rclone: delete "important-file.txt"? 1381 y) Yes, this is OK (default) 1382 n) No, skip this 1383 s) Skip all delete operations with no more questions 1384 !) Do all delete operations with no more questions 1385 q) Exit rclone now. 1386 y/n/s/!/q> n 1387 ``` 1388 1389 The options mean 1390 1391 - `y`: **Yes**, this operation should go ahead. You can also press Return 1392 for this to happen. You'll be asked every time unless you choose `s` 1393 or `!`. 1394 - `n`: **No**, do not do this operation. You'll be asked every time unless 1395 you choose `s` or `!`. 1396 - `s`: **Skip** all the following operations of this type with no more 1397 questions. This takes effect until rclone exits. If there are any 1398 different kind of operations you'll be prompted for them. 1399 - `!`: **Do all** the following operations with no more 1400 questions. Useful if you've decided that you don't mind rclone doing 1401 that kind of operation. This takes effect until rclone exits . If 1402 there are any different kind of operations you'll be prompted for 1403 them. 1404 - `q`: **Quit** rclone now, just in case! 1405 1406 ### --leave-root #### 1407 1408 During rmdirs it will not remove root directory, even if it's empty. 1409 1410 ### --log-file=FILE ### 1411 1412 Log all of rclone's output to FILE. This is not active by default. 1413 This can be useful for tracking down problems with syncs in 1414 combination with the `-v` flag. See the [Logging section](#logging) 1415 for more info. 1416 1417 If FILE exists then rclone will append to it. 1418 1419 Note that if you are using the `logrotate` program to manage rclone's 1420 logs, then you should use the `copytruncate` option as rclone doesn't 1421 have a signal to rotate logs. 1422 1423 ### --log-format LIST ### 1424 1425 Comma separated list of log format options. Accepted options are `date`, 1426 `time`, `microseconds`, `pid`, `longfile`, `shortfile`, `UTC`. Any other 1427 keywords will be silently ignored. `pid` will tag log messages with process 1428 identifier which useful with `rclone mount --daemon`. Other accepted 1429 options are explained in the [go documentation](https://pkg.go.dev/log#pkg-constants). 1430 The default log format is "`date`,`time`". 1431 1432 ### --log-level LEVEL ### 1433 1434 This sets the log level for rclone. The default log level is `NOTICE`. 1435 1436 `DEBUG` is equivalent to `-vv`. It outputs lots of debug info - useful 1437 for bug reports and really finding out what rclone is doing. 1438 1439 `INFO` is equivalent to `-v`. It outputs information about each transfer 1440 and prints stats once a minute by default. 1441 1442 `NOTICE` is the default log level if no logging flags are supplied. It 1443 outputs very little when things are working normally. It outputs 1444 warnings and significant events. 1445 1446 `ERROR` is equivalent to `-q`. It only outputs error messages. 1447 1448 ### --use-json-log ### 1449 1450 This switches the log format to JSON for rclone. The fields of json log 1451 are level, msg, source, time. 1452 1453 ### --low-level-retries NUMBER ### 1454 1455 This controls the number of low level retries rclone does. 1456 1457 A low level retry is used to retry a failing operation - typically one 1458 HTTP request. This might be uploading a chunk of a big file for 1459 example. You will see low level retries in the log with the `-v` 1460 flag. 1461 1462 This shouldn't need to be changed from the default in normal operations. 1463 However, if you get a lot of low level retries you may wish 1464 to reduce the value so rclone moves on to a high level retry (see the 1465 `--retries` flag) quicker. 1466 1467 Disable low level retries with `--low-level-retries 1`. 1468 1469 ### --max-backlog=N ### 1470 1471 This is the maximum allowable backlog of files in a sync/copy/move 1472 queued for being checked or transferred. 1473 1474 This can be set arbitrarily large. It will only use memory when the 1475 queue is in use. Note that it will use in the order of N KiB of memory 1476 when the backlog is in use. 1477 1478 Setting this large allows rclone to calculate how many files are 1479 pending more accurately, give a more accurate estimated finish 1480 time and make `--order-by` work more accurately. 1481 1482 Setting this small will make rclone more synchronous to the listings 1483 of the remote which may be desirable. 1484 1485 Setting this to a negative number will make the backlog as large as 1486 possible. 1487 1488 ### --max-delete=N ### 1489 1490 This tells rclone not to delete more than N files. If that limit is 1491 exceeded then a fatal error will be generated and rclone will stop the 1492 operation in progress. 1493 1494 ### --max-delete-size=SIZE ### 1495 1496 Rclone will stop deleting files when the total size of deletions has 1497 reached the size specified. It defaults to off. 1498 1499 If that limit is exceeded then a fatal error will be generated and 1500 rclone will stop the operation in progress. 1501 1502 ### --max-depth=N ### 1503 1504 This modifies the recursion depth for all the commands except purge. 1505 1506 So if you do `rclone --max-depth 1 ls remote:path` you will see only 1507 the files in the top level directory. Using `--max-depth 2` means you 1508 will see all the files in first two directory levels and so on. 1509 1510 For historical reasons the `lsd` command defaults to using a 1511 `--max-depth` of 1 - you can override this with the command line flag. 1512 1513 You can use this command to disable recursion (with `--max-depth 1`). 1514 1515 Note that if you use this with `sync` and `--delete-excluded` the 1516 files not recursed through are considered excluded and will be deleted 1517 on the destination. Test first with `--dry-run` if you are not sure 1518 what will happen. 1519 1520 ### --max-duration=TIME ### 1521 1522 Rclone will stop transferring when it has run for the 1523 duration specified. 1524 Defaults to off. 1525 1526 When the limit is reached all transfers will stop immediately. 1527 Use `--cutoff-mode` to modify this behaviour. 1528 1529 Rclone will exit with exit code 10 if the duration limit is reached. 1530 1531 ### --max-transfer=SIZE ### 1532 1533 Rclone will stop transferring when it has reached the size specified. 1534 Defaults to off. 1535 1536 When the limit is reached all transfers will stop immediately. 1537 Use `--cutoff-mode` to modify this behaviour. 1538 1539 Rclone will exit with exit code 8 if the transfer limit is reached. 1540 1541 ### --cutoff-mode=hard|soft|cautious ### 1542 1543 This modifies the behavior of `--max-transfer` and `--max-duration` 1544 Defaults to `--cutoff-mode=hard`. 1545 1546 Specifying `--cutoff-mode=hard` will stop transferring immediately 1547 when Rclone reaches the limit. 1548 1549 Specifying `--cutoff-mode=soft` will stop starting new transfers 1550 when Rclone reaches the limit. 1551 1552 Specifying `--cutoff-mode=cautious` will try to prevent Rclone 1553 from reaching the limit. Only applicable for `--max-transfer` 1554 1555 ## -M, --metadata 1556 1557 Setting this flag enables rclone to copy the metadata from the source 1558 to the destination. For local backends this is ownership, permissions, 1559 xattr etc. See the [metadata section](#metadata) for more info. 1560 1561 ### --metadata-mapper SpaceSepList {#metadata-mapper} 1562 1563 If you supply the parameter `--metadata-mapper /path/to/program` then 1564 rclone will use that program to map metadata from source object to 1565 destination object. 1566 1567 The argument to this flag should be a command with an optional space separated 1568 list of arguments. If one of the arguments has a space in then enclose 1569 it in `"`, if you want a literal `"` in an argument then enclose the 1570 argument in `"` and double the `"`. See [CSV encoding](https://godoc.org/encoding/csv) 1571 for more info. 1572 1573 --metadata-mapper "python bin/test_metadata_mapper.py" 1574 --metadata-mapper 'python bin/test_metadata_mapper.py "argument with a space"' 1575 --metadata-mapper 'python bin/test_metadata_mapper.py "argument with ""two"" quotes"' 1576 1577 This uses a simple JSON based protocol with input on STDIN and output 1578 on STDOUT. This will be called for every file and directory copied and 1579 may be called concurrently. 1580 1581 The program's job is to take a metadata blob on the input and turn it 1582 into a metadata blob on the output suitable for the destination 1583 backend. 1584 1585 Input to the program (via STDIN) might look like this. This provides 1586 some context for the `Metadata` which may be important. 1587 1588 - `SrcFs` is the config string for the remote that the object is currently on. 1589 - `SrcFsType` is the name of the source backend. 1590 - `DstFs` is the config string for the remote that the object is being copied to 1591 - `DstFsType` is the name of the destination backend. 1592 - `Remote` is the path of the object relative to the root. 1593 - `Size`, `MimeType`, `ModTime` are attributes of the object. 1594 - `IsDir` is `true` if this is a directory (not yet implemented). 1595 - `ID` is the source `ID` of the object if known. 1596 - `Metadata` is the backend specific metadata as described in the backend docs. 1597 1598 ```json 1599 { 1600 "SrcFs": "gdrive:", 1601 "SrcFsType": "drive", 1602 "DstFs": "newdrive:user", 1603 "DstFsType": "onedrive", 1604 "Remote": "test.txt", 1605 "Size": 6, 1606 "MimeType": "text/plain; charset=utf-8", 1607 "ModTime": "2022-10-11T17:53:10.286745272+01:00", 1608 "IsDir": false, 1609 "ID": "xyz", 1610 "Metadata": { 1611 "btime": "2022-10-11T16:53:11Z", 1612 "content-type": "text/plain; charset=utf-8", 1613 "mtime": "2022-10-11T17:53:10.286745272+01:00", 1614 "owner": "user1@domain1.com", 1615 "permissions": "...", 1616 "description": "my nice file", 1617 "starred": "false" 1618 } 1619 } 1620 ``` 1621 1622 The program should then modify the input as desired and send it to 1623 STDOUT. The returned `Metadata` field will be used in its entirety for 1624 the destination object. Any other fields will be ignored. Note in this 1625 example we translate user names and permissions and add something to 1626 the description: 1627 1628 ```json 1629 { 1630 "Metadata": { 1631 "btime": "2022-10-11T16:53:11Z", 1632 "content-type": "text/plain; charset=utf-8", 1633 "mtime": "2022-10-11T17:53:10.286745272+01:00", 1634 "owner": "user1@domain2.com", 1635 "permissions": "...", 1636 "description": "my nice file [migrated from domain1]", 1637 "starred": "false" 1638 } 1639 } 1640 ``` 1641 1642 Metadata can be removed here too. 1643 1644 An example python program might look something like this to implement 1645 the above transformations. 1646 1647 ```python 1648 import sys, json 1649 1650 i = json.load(sys.stdin) 1651 metadata = i["Metadata"] 1652 # Add tag to description 1653 if "description" in metadata: 1654 metadata["description"] += " [migrated from domain1]" 1655 else: 1656 metadata["description"] = "[migrated from domain1]" 1657 # Modify owner 1658 if "owner" in metadata: 1659 metadata["owner"] = metadata["owner"].replace("domain1.com", "domain2.com") 1660 o = { "Metadata": metadata } 1661 json.dump(o, sys.stdout, indent="\t") 1662 ``` 1663 1664 You can find this example (slightly expanded) in the rclone source code at 1665 [bin/test_metadata_mapper.py](https://github.com/rclone/rclone/blob/master/bin/test_metadata_mapper.py). 1666 1667 If you want to see the input to the metadata mapper and the output 1668 returned from it in the log you can use `-vv --dump mapper`. 1669 1670 See the [metadata section](#metadata) for more info. 1671 1672 ### --metadata-set key=value 1673 1674 Add metadata `key` = `value` when uploading. This can be repeated as 1675 many times as required. See the [metadata section](#metadata) for more 1676 info. 1677 1678 ### --modify-window=TIME ### 1679 1680 When checking whether a file has been modified, this is the maximum 1681 allowed time difference that a file can have and still be considered 1682 equivalent. 1683 1684 The default is `1ns` unless this is overridden by a remote. For 1685 example OS X only stores modification times to the nearest second so 1686 if you are reading and writing to an OS X filing system this will be 1687 `1s` by default. 1688 1689 This command line flag allows you to override that computed default. 1690 1691 ### --multi-thread-write-buffer-size=SIZE ### 1692 1693 When transferring with multiple threads, rclone will buffer SIZE bytes 1694 in memory before writing to disk for each thread. 1695 1696 This can improve performance if the underlying filesystem does not deal 1697 well with a lot of small writes in different positions of the file, so 1698 if you see transfers being limited by disk write speed, you might want 1699 to experiment with different values. Specially for magnetic drives and 1700 remote file systems a higher value can be useful. 1701 1702 Nevertheless, the default of `128k` should be fine for almost all use 1703 cases, so before changing it ensure that network is not really your 1704 bottleneck. 1705 1706 As a final hint, size is not the only factor: block size (or similar 1707 concept) can have an impact. In one case, we observed that exact 1708 multiples of 16k performed much better than other values. 1709 1710 ### --multi-thread-chunk-size=SizeSuffix ### 1711 1712 Normally the chunk size for multi thread transfers is set by the backend. 1713 However some backends such as `local` and `smb` (which implement `OpenWriterAt` 1714 but not `OpenChunkWriter`) don't have a natural chunk size. 1715 1716 In this case the value of this option is used (default 64Mi). 1717 1718 ### --multi-thread-cutoff=SIZE {#multi-thread-cutoff} 1719 1720 When transferring files above SIZE to capable backends, rclone will 1721 use multiple threads to transfer the file (default 256M). 1722 1723 Capable backends are marked in the 1724 [overview](/overview/#optional-features) as `MultithreadUpload`. (They 1725 need to implement either the `OpenWriterAt` or `OpenChunkWriter` 1726 internal interfaces). These include include, `local`, `s3`, 1727 `azureblob`, `b2`, `oracleobjectstorage` and `smb` at the time of 1728 writing. 1729 1730 On the local disk, rclone preallocates the file (using 1731 `fallocate(FALLOC_FL_KEEP_SIZE)` on unix or `NTSetInformationFile` on 1732 Windows both of which takes no time) then each thread writes directly 1733 into the file at the correct place. This means that rclone won't 1734 create fragmented or sparse files and there won't be any assembly time 1735 at the end of the transfer. 1736 1737 The number of threads used to transfer is controlled by 1738 `--multi-thread-streams`. 1739 1740 Use `-vv` if you wish to see info about the threads. 1741 1742 This will work with the `sync`/`copy`/`move` commands and friends 1743 `copyto`/`moveto`. Multi thread transfers will be used with `rclone 1744 mount` and `rclone serve` if `--vfs-cache-mode` is set to `writes` or 1745 above. 1746 1747 **NB** that this **only** works with supported backends as the 1748 destination but will work with any backend as the source. 1749 1750 **NB** that multi-thread copies are disabled for local to local copies 1751 as they are faster without unless `--multi-thread-streams` is set 1752 explicitly. 1753 1754 **NB** on Windows using multi-thread transfers to the local disk will 1755 cause the resulting files to be [sparse](https://en.wikipedia.org/wiki/Sparse_file). 1756 Use `--local-no-sparse` to disable sparse files (which may cause long 1757 delays at the start of transfers) or disable multi-thread transfers 1758 with `--multi-thread-streams 0` 1759 1760 ### --multi-thread-streams=N ### 1761 1762 When using multi thread transfers (see above `--multi-thread-cutoff`) 1763 this sets the number of streams to use. Set to `0` to disable multi 1764 thread transfers (Default 4). 1765 1766 If the backend has a `--backend-upload-concurrency` setting (eg 1767 `--s3-upload-concurrency`) then this setting will be used as the 1768 number of transfers instead if it is larger than the value of 1769 `--multi-thread-streams` or `--multi-thread-streams` isn't set. 1770 1771 ### --no-check-dest ### 1772 1773 The `--no-check-dest` can be used with `move` or `copy` and it causes 1774 rclone not to check the destination at all when copying files. 1775 1776 This means that: 1777 1778 - the destination is not listed minimising the API calls 1779 - files are always transferred 1780 - this can cause duplicates on remotes which allow it (e.g. Google Drive) 1781 - `--retries 1` is recommended otherwise you'll transfer everything again on a retry 1782 1783 This flag is useful to minimise the transactions if you know that none 1784 of the files are on the destination. 1785 1786 This is a specialized flag which should be ignored by most users! 1787 1788 ### --no-gzip-encoding ### 1789 1790 Don't set `Accept-Encoding: gzip`. This means that rclone won't ask 1791 the server for compressed files automatically. Useful if you've set 1792 the server to return files with `Content-Encoding: gzip` but you 1793 uploaded compressed files. 1794 1795 There is no need to set this in normal operation, and doing so will 1796 decrease the network transfer efficiency of rclone. 1797 1798 ### --no-traverse ### 1799 1800 The `--no-traverse` flag controls whether the destination file system 1801 is traversed when using the `copy` or `move` commands. 1802 `--no-traverse` is not compatible with `sync` and will be ignored if 1803 you supply it with `sync`. 1804 1805 If you are only copying a small number of files (or are filtering most 1806 of the files) and/or have a large number of files on the destination 1807 then `--no-traverse` will stop rclone listing the destination and save 1808 time. 1809 1810 However, if you are copying a large number of files, especially if you 1811 are doing a copy where lots of the files under consideration haven't 1812 changed and won't need copying then you shouldn't use `--no-traverse`. 1813 1814 See [rclone copy](/commands/rclone_copy/) for an example of how to use it. 1815 1816 ### --no-unicode-normalization ### 1817 1818 Don't normalize unicode characters in filenames during the sync routine. 1819 1820 Sometimes, an operating system will store filenames containing unicode 1821 parts in their decomposed form (particularly macOS). Some cloud storage 1822 systems will then recompose the unicode, resulting in duplicate files if 1823 the data is ever copied back to a local filesystem. 1824 1825 Using this flag will disable that functionality, treating each unicode 1826 character as unique. For example, by default é and é will be normalized 1827 into the same character. With `--no-unicode-normalization` they will be 1828 treated as unique characters. 1829 1830 ### --no-update-modtime ### 1831 1832 When using this flag, rclone won't update modification times of remote 1833 files if they are incorrect as it would normally. 1834 1835 This can be used if the remote is being synced with another tool also 1836 (e.g. the Google Drive client). 1837 1838 ### --no-update-dir-modtime ### 1839 1840 When using this flag, rclone won't update modification times of remote 1841 directories if they are incorrect as it would normally. 1842 1843 ### --order-by string ### 1844 1845 The `--order-by` flag controls the order in which files in the backlog 1846 are processed in `rclone sync`, `rclone copy` and `rclone move`. 1847 1848 The order by string is constructed like this. The first part 1849 describes what aspect is being measured: 1850 1851 - `size` - order by the size of the files 1852 - `name` - order by the full path of the files 1853 - `modtime` - order by the modification date of the files 1854 1855 This can have a modifier appended with a comma: 1856 1857 - `ascending` or `asc` - order so that the smallest (or oldest) is processed first 1858 - `descending` or `desc` - order so that the largest (or newest) is processed first 1859 - `mixed` - order so that the smallest is processed first for some threads and the largest for others 1860 1861 If the modifier is `mixed` then it can have an optional percentage 1862 (which defaults to `50`), e.g. `size,mixed,25` which means that 25% of 1863 the threads should be taking the smallest items and 75% the 1864 largest. The threads which take the smallest first will always take 1865 the smallest first and likewise the largest first threads. The `mixed` 1866 mode can be useful to minimise the transfer time when you are 1867 transferring a mixture of large and small files - the large files are 1868 guaranteed upload threads and bandwidth and the small files will be 1869 processed continuously. 1870 1871 If no modifier is supplied then the order is `ascending`. 1872 1873 For example 1874 1875 - `--order-by size,desc` - send the largest files first 1876 - `--order-by modtime,ascending` - send the oldest files first 1877 - `--order-by name` - send the files with alphabetically by path first 1878 1879 If the `--order-by` flag is not supplied or it is supplied with an 1880 empty string then the default ordering will be used which is as 1881 scanned. With `--checkers 1` this is mostly alphabetical, however 1882 with the default `--checkers 8` it is somewhat random. 1883 1884 #### Limitations 1885 1886 The `--order-by` flag does not do a separate pass over the data. This 1887 means that it may transfer some files out of the order specified if 1888 1889 - there are no files in the backlog or the source has not been fully scanned yet 1890 - there are more than [--max-backlog](#max-backlog-n) files in the backlog 1891 1892 Rclone will do its best to transfer the best file it has so in 1893 practice this should not cause a problem. Think of `--order-by` as 1894 being more of a best efforts flag rather than a perfect ordering. 1895 1896 If you want perfect ordering then you will need to specify 1897 [--check-first](#check-first) which will find all the files which need 1898 transferring first before transferring any. 1899 1900 ### --partial-suffix {#partial-suffix} 1901 1902 When [--inplace](#inplace) is not used, it causes rclone to use 1903 the `--partial-suffix` as suffix for temporary files. 1904 1905 Suffix length limit is 16 characters. 1906 1907 The default is `.partial`. 1908 1909 ### --password-command SpaceSepList ### 1910 1911 This flag supplies a program which should supply the config password 1912 when run. This is an alternative to rclone prompting for the password 1913 or setting the `RCLONE_CONFIG_PASS` variable. 1914 1915 The argument to this should be a command with a space separated list 1916 of arguments. If one of the arguments has a space in then enclose it 1917 in `"`, if you want a literal `"` in an argument then enclose the 1918 argument in `"` and double the `"`. See [CSV encoding](https://godoc.org/encoding/csv) 1919 for more info. 1920 1921 Eg 1922 1923 --password-command "echo hello" 1924 --password-command 'echo "hello with space"' 1925 --password-command 'echo "hello with ""quotes"" and space"' 1926 1927 See the [Configuration Encryption](#configuration-encryption) for more info. 1928 1929 See a [Windows PowerShell example on the Wiki](https://github.com/rclone/rclone/wiki/Windows-Powershell-use-rclone-password-command-for-Config-file-password). 1930 1931 ### -P, --progress ### 1932 1933 This flag makes rclone update the stats in a static block in the 1934 terminal providing a realtime overview of the transfer. 1935 1936 Any log messages will scroll above the static block. Log messages 1937 will push the static block down to the bottom of the terminal where it 1938 will stay. 1939 1940 Normally this is updated every 500mS but this period can be overridden 1941 with the `--stats` flag. 1942 1943 This can be used with the `--stats-one-line` flag for a simpler 1944 display. 1945 1946 Note: On Windows until [this bug](https://github.com/Azure/go-ansiterm/issues/26) 1947 is fixed all non-ASCII characters will be replaced with `.` when 1948 `--progress` is in use. 1949 1950 ### --progress-terminal-title ### 1951 1952 This flag, when used with `-P/--progress`, will print the string `ETA: %s` 1953 to the terminal title. 1954 1955 ### -q, --quiet ### 1956 1957 This flag will limit rclone's output to error messages only. 1958 1959 ### --refresh-times ### 1960 1961 The `--refresh-times` flag can be used to update modification times of 1962 existing files when they are out of sync on backends which don't 1963 support hashes. 1964 1965 This is useful if you uploaded files with the incorrect timestamps and 1966 you now wish to correct them. 1967 1968 This flag is **only** useful for destinations which don't support 1969 hashes (e.g. `crypt`). 1970 1971 This can be used any of the sync commands `sync`, `copy` or `move`. 1972 1973 To use this flag you will need to be doing a modification time sync 1974 (so not using `--size-only` or `--checksum`). The flag will have no 1975 effect when using `--size-only` or `--checksum`. 1976 1977 If this flag is used when rclone comes to upload a file it will check 1978 to see if there is an existing file on the destination. If this file 1979 matches the source with size (and checksum if available) but has a 1980 differing timestamp then instead of re-uploading it, rclone will 1981 update the timestamp on the destination file. If the checksum does not 1982 match rclone will upload the new file. If the checksum is absent (e.g. 1983 on a `crypt` backend) then rclone will update the timestamp. 1984 1985 Note that some remotes can't set the modification time without 1986 re-uploading the file so this flag is less useful on them. 1987 1988 Normally if you are doing a modification time sync rclone will update 1989 modification times without `--refresh-times` provided that the remote 1990 supports checksums **and** the checksums match on the file. However if the 1991 checksums are absent then rclone will upload the file rather than 1992 setting the timestamp as this is the safe behaviour. 1993 1994 ### --retries int ### 1995 1996 Retry the entire sync if it fails this many times it fails (default 3). 1997 1998 Some remotes can be unreliable and a few retries help pick up the 1999 files which didn't get transferred because of errors. 2000 2001 Disable retries with `--retries 1`. 2002 2003 ### --retries-sleep=TIME ### 2004 2005 This sets the interval between each retry specified by `--retries` 2006 2007 The default is `0`. Use `0` to disable. 2008 2009 ### --server-side-across-configs ### 2010 2011 Allow server-side operations (e.g. copy or move) to work across 2012 different configurations. 2013 2014 This can be useful if you wish to do a server-side copy or move 2015 between two remotes which use the same backend but are configured 2016 differently. 2017 2018 Note that this isn't enabled by default because it isn't easy for 2019 rclone to tell if it will work between any two configurations. 2020 2021 ### --size-only ### 2022 2023 Normally rclone will look at modification time and size of files to 2024 see if they are equal. If you set this flag then rclone will check 2025 only the size. 2026 2027 This can be useful transferring files from Dropbox which have been 2028 modified by the desktop sync client which doesn't set checksums of 2029 modification times in the same way as rclone. 2030 2031 ### --stats=TIME ### 2032 2033 Commands which transfer data (`sync`, `copy`, `copyto`, `move`, 2034 `moveto`) will print data transfer stats at regular intervals to show 2035 their progress. 2036 2037 This sets the interval. 2038 2039 The default is `1m`. Use `0` to disable. 2040 2041 If you set the stats interval then all commands can show stats. This 2042 can be useful when running other commands, `check` or `mount` for 2043 example. 2044 2045 Stats are logged at `INFO` level by default which means they won't 2046 show at default log level `NOTICE`. Use `--stats-log-level NOTICE` or 2047 `-v` to make them show. See the [Logging section](#logging) for more 2048 info on log levels. 2049 2050 Note that on macOS you can send a SIGINFO (which is normally ctrl-T in 2051 the terminal) to make the stats print immediately. 2052 2053 ### --stats-file-name-length integer ### 2054 By default, the `--stats` output will truncate file names and paths longer 2055 than 40 characters. This is equivalent to providing 2056 `--stats-file-name-length 40`. Use `--stats-file-name-length 0` to disable 2057 any truncation of file names printed by stats. 2058 2059 ### --stats-log-level string ### 2060 2061 Log level to show `--stats` output at. This can be `DEBUG`, `INFO`, 2062 `NOTICE`, or `ERROR`. The default is `INFO`. This means at the 2063 default level of logging which is `NOTICE` the stats won't show - if 2064 you want them to then use `--stats-log-level NOTICE`. See the [Logging 2065 section](#logging) for more info on log levels. 2066 2067 ### --stats-one-line ### 2068 2069 When this is specified, rclone condenses the stats into a single line 2070 showing the most important stats only. 2071 2072 ### --stats-one-line-date ### 2073 2074 When this is specified, rclone enables the single-line stats and prepends 2075 the display with a date string. The default is `2006/01/02 15:04:05 - ` 2076 2077 ### --stats-one-line-date-format ### 2078 2079 When this is specified, rclone enables the single-line stats and prepends 2080 the display with a user-supplied date string. The date string MUST be 2081 enclosed in quotes. Follow [golang specs](https://golang.org/pkg/time/#Time.Format) for 2082 date formatting syntax. 2083 2084 ### --stats-unit=bits|bytes ### 2085 2086 By default, data transfer rates will be printed in bytes per second. 2087 2088 This option allows the data rate to be printed in bits per second. 2089 2090 Data transfer volume will still be reported in bytes. 2091 2092 The rate is reported as a binary unit, not SI unit. So 1 Mbit/s 2093 equals 1,048,576 bit/s and not 1,000,000 bit/s. 2094 2095 The default is `bytes`. 2096 2097 ### --suffix=SUFFIX ### 2098 2099 When using `sync`, `copy` or `move` any files which would have been 2100 overwritten or deleted will have the suffix added to them. If there 2101 is a file with the same path (after the suffix has been added), then 2102 it will be overwritten. 2103 2104 The remote in use must support server-side move or copy and you must 2105 use the same remote as the destination of the sync. 2106 2107 This is for use with files to add the suffix in the current directory 2108 or with `--backup-dir`. See `--backup-dir` for more info. 2109 2110 For example 2111 2112 rclone copy --interactive /path/to/local/file remote:current --suffix .bak 2113 2114 will copy `/path/to/local` to `remote:current`, but for any files 2115 which would have been updated or deleted have .bak added. 2116 2117 If using `rclone sync` with `--suffix` and without `--backup-dir` then 2118 it is recommended to put a filter rule in excluding the suffix 2119 otherwise the `sync` will delete the backup files. 2120 2121 rclone sync --interactive /path/to/local/file remote:current --suffix .bak --exclude "*.bak" 2122 2123 ### --suffix-keep-extension ### 2124 2125 When using `--suffix`, setting this causes rclone put the SUFFIX 2126 before the extension of the files that it backs up rather than after. 2127 2128 So let's say we had `--suffix -2019-01-01`, without the flag `file.txt` 2129 would be backed up to `file.txt-2019-01-01` and with the flag it would 2130 be backed up to `file-2019-01-01.txt`. This can be helpful to make 2131 sure the suffixed files can still be opened. 2132 2133 If a file has two (or more) extensions and the second (or subsequent) 2134 extension is recognised as a valid mime type, then the suffix will go 2135 before that extension. So `file.tar.gz` would be backed up to 2136 `file-2019-01-01.tar.gz` whereas `file.badextension.gz` would be 2137 backed up to `file.badextension-2019-01-01.gz`. 2138 2139 ### --syslog ### 2140 2141 On capable OSes (not Windows or Plan9) send all log output to syslog. 2142 2143 This can be useful for running rclone in a script or `rclone mount`. 2144 2145 ### --syslog-facility string ### 2146 2147 If using `--syslog` this sets the syslog facility (e.g. `KERN`, `USER`). 2148 See `man syslog` for a list of possible facilities. The default 2149 facility is `DAEMON`. 2150 2151 ### --temp-dir=DIR ### 2152 2153 Specify the directory rclone will use for temporary files, to override 2154 the default. Make sure the directory exists and have accessible permissions. 2155 2156 By default the operating system's temp directory will be used: 2157 - On Unix systems, `$TMPDIR` if non-empty, else `/tmp`. 2158 - On Windows, the first non-empty value from `%TMP%`, `%TEMP%`, `%USERPROFILE%`, or the Windows directory. 2159 2160 When overriding the default with this option, the specified path will be 2161 set as value of environment variable `TMPDIR` on Unix systems 2162 and `TMP` and `TEMP` on Windows. 2163 2164 You can use the [config paths](/commands/rclone_config_paths/) 2165 command to see the current value. 2166 2167 ### --tpslimit float ### 2168 2169 Limit transactions per second to this number. Default is 0 which is 2170 used to mean unlimited transactions per second. 2171 2172 A transaction is roughly defined as an API call; its exact meaning 2173 will depend on the backend. For HTTP based backends it is an HTTP 2174 PUT/GET/POST/etc and its response. For FTP/SFTP it is a round trip 2175 transaction over TCP. 2176 2177 For example, to limit rclone to 10 transactions per second use 2178 `--tpslimit 10`, or to 1 transaction every 2 seconds use `--tpslimit 2179 0.5`. 2180 2181 Use this when the number of transactions per second from rclone is 2182 causing a problem with the cloud storage provider (e.g. getting you 2183 banned or rate limited). 2184 2185 This can be very useful for `rclone mount` to control the behaviour of 2186 applications using it. 2187 2188 This limit applies to all HTTP based backends and to the FTP and SFTP 2189 backends. It does not apply to the local backend or the Storj backend. 2190 2191 See also `--tpslimit-burst`. 2192 2193 ### --tpslimit-burst int ### 2194 2195 Max burst of transactions for `--tpslimit` (default `1`). 2196 2197 Normally `--tpslimit` will do exactly the number of transaction per 2198 second specified. However if you supply `--tps-burst` then rclone can 2199 save up some transactions from when it was idle giving a burst of up 2200 to the parameter supplied. 2201 2202 For example if you provide `--tpslimit-burst 10` then if rclone has 2203 been idle for more than 10*`--tpslimit` then it can do 10 transactions 2204 very quickly before they are limited again. 2205 2206 This may be used to increase performance of `--tpslimit` without 2207 changing the long term average number of transactions per second. 2208 2209 ### --track-renames ### 2210 2211 By default, rclone doesn't keep track of renamed files, so if you 2212 rename a file locally then sync it to a remote, rclone will delete the 2213 old file on the remote and upload a new copy. 2214 2215 An rclone sync with `--track-renames` runs like a normal sync, but keeps 2216 track of objects which exist in the destination but not in the source 2217 (which would normally be deleted), and which objects exist in the 2218 source but not the destination (which would normally be transferred). 2219 These objects are then candidates for renaming. 2220 2221 After the sync, rclone matches up the source only and destination only 2222 objects using the `--track-renames-strategy` specified and either 2223 renames the destination object or transfers the source and deletes the 2224 destination object. `--track-renames` is stateless like all of 2225 rclone's syncs. 2226 2227 To use this flag the destination must support server-side copy or 2228 server-side move, and to use a hash based `--track-renames-strategy` 2229 (the default) the source and the destination must have a compatible 2230 hash. 2231 2232 If the destination does not support server-side copy or move, rclone 2233 will fall back to the default behaviour and log an error level message 2234 to the console. 2235 2236 Encrypted destinations are not currently supported by `--track-renames` 2237 if `--track-renames-strategy` includes `hash`. 2238 2239 Note that `--track-renames` is incompatible with `--no-traverse` and 2240 that it uses extra memory to keep track of all the rename candidates. 2241 2242 Note also that `--track-renames` is incompatible with 2243 `--delete-before` and will select `--delete-after` instead of 2244 `--delete-during`. 2245 2246 ### --track-renames-strategy (hash,modtime,leaf,size) ### 2247 2248 This option changes the file matching criteria for `--track-renames`. 2249 2250 The matching is controlled by a comma separated selection of these tokens: 2251 2252 - `modtime` - the modification time of the file - not supported on all backends 2253 - `hash` - the hash of the file contents - not supported on all backends 2254 - `leaf` - the name of the file not including its directory name 2255 - `size` - the size of the file (this is always enabled) 2256 2257 The default option is `hash`. 2258 2259 Using `--track-renames-strategy modtime,leaf` would match files 2260 based on modification time, the leaf of the file name and the size 2261 only. 2262 2263 Using `--track-renames-strategy modtime` or `leaf` can enable 2264 `--track-renames` support for encrypted destinations. 2265 2266 Note that the `hash` strategy is not supported with encrypted destinations. 2267 2268 ### --delete-(before,during,after) ### 2269 2270 This option allows you to specify when files on your destination are 2271 deleted when you sync folders. 2272 2273 Specifying the value `--delete-before` will delete all files present 2274 on the destination, but not on the source *before* starting the 2275 transfer of any new or updated files. This uses two passes through the 2276 file systems, one for the deletions and one for the copies. 2277 2278 Specifying `--delete-during` will delete files while checking and 2279 uploading files. This is the fastest option and uses the least memory. 2280 2281 Specifying `--delete-after` (the default value) will delay deletion of 2282 files until all new/updated files have been successfully transferred. 2283 The files to be deleted are collected in the copy pass then deleted 2284 after the copy pass has completed successfully. The files to be 2285 deleted are held in memory so this mode may use more memory. This is 2286 the safest mode as it will only delete files if there have been no 2287 errors subsequent to that. If there have been errors before the 2288 deletions start then you will get the message `not deleting files as 2289 there were IO errors`. 2290 2291 ### --fast-list ### 2292 2293 When doing anything which involves a directory listing (e.g. `sync`, 2294 `copy`, `ls` - in fact nearly every command), rclone has different 2295 strategies to choose from. 2296 2297 The basic strategy is to list one directory and processes it before using 2298 more directory lists to process any subdirectories. This is a mandatory 2299 backend feature, called `List`, which means it is supported by all backends. 2300 This strategy uses small amount of memory, and because it can be parallelised 2301 it is fast for operations involving processing of the list results. 2302 2303 Some backends provide the support for an alternative strategy, where all 2304 files beneath a directory can be listed in one (or a small number) of 2305 transactions. Rclone supports this alternative strategy through an optional 2306 backend feature called [`ListR`](/overview/#listr). You can see in the storage 2307 system overview documentation's [optional features](/overview/#optional-features) 2308 section which backends it is enabled for (these tend to be the bucket-based 2309 ones, e.g. S3, B2, GCS, Swift). This strategy requires fewer transactions 2310 for highly recursive operations, which is important on backends where this 2311 is charged or heavily rate limited. It may be faster (due to fewer transactions) 2312 or slower (because it can't be parallelized) depending on different parameters, 2313 and may require more memory if rclone has to keep the whole listing in memory. 2314 2315 Which listing strategy rclone picks for a given operation is complicated, but 2316 in general it tries to choose the best possible. It will prefer `ListR` in 2317 situations where it doesn't need to store the listed files in memory, e.g. 2318 for unlimited recursive `ls` command variants. In other situations it will 2319 prefer `List`, e.g. for `sync` and `copy`, where it needs to keep the listed 2320 files in memory, and is performing operations on them where parallelization 2321 may be a huge advantage. 2322 2323 Rclone is not able to take all relevant parameters into account for deciding 2324 the best strategy, and therefore allows you to influence the choice in two ways: 2325 You can stop rclone from using `ListR` by disabling the feature, using the 2326 [--disable](#disable-feature-feature) option (`--disable ListR`), or you can 2327 allow rclone to use `ListR` where it would normally choose not to do so due to 2328 higher memory usage, using the `--fast-list` option. Rclone should always 2329 produce identical results either way. Using `--disable ListR` or `--fast-list` 2330 on a remote which doesn't support `ListR` does nothing, rclone will just ignore 2331 it. 2332 2333 A rule of thumb is that if you pay for transactions and can fit your entire 2334 sync listing into memory, then `--fast-list` is recommended. If you have a 2335 very big sync to do, then don't use `--fast-list`, otherwise you will run out 2336 of memory. Run some tests and compare before you decide, and if in doubt then 2337 just leave the default, let rclone decide, i.e. not use `--fast-list`. 2338 2339 ### --timeout=TIME ### 2340 2341 This sets the IO idle timeout. If a transfer has started but then 2342 becomes idle for this long it is considered broken and disconnected. 2343 2344 The default is `5m`. Set to `0` to disable. 2345 2346 ### --transfers=N ### 2347 2348 The number of file transfers to run in parallel. It can sometimes be 2349 useful to set this to a smaller number if the remote is giving a lot 2350 of timeouts or bigger if you have lots of bandwidth and a fast remote. 2351 2352 The default is to run 4 file transfers in parallel. 2353 2354 Look at --multi-thread-streams if you would like to control single file transfers. 2355 2356 ### -u, --update ### 2357 2358 This forces rclone to skip any files which exist on the destination 2359 and have a modified time that is newer than the source file. 2360 2361 This can be useful in avoiding needless transfers when transferring to 2362 a remote which doesn't support modification times directly (or when 2363 using `--use-server-modtime` to avoid extra API calls) as it is more 2364 accurate than a `--size-only` check and faster than using 2365 `--checksum`. On such remotes (or when using `--use-server-modtime`) 2366 the time checked will be the uploaded time. 2367 2368 If an existing destination file has a modification time older than the 2369 source file's, it will be updated if the sizes are different. If the 2370 sizes are the same, it will be updated if the checksum is different or 2371 not available. 2372 2373 If an existing destination file has a modification time equal (within 2374 the computed modify window) to the source file's, it will be updated 2375 if the sizes are different. The checksum will not be checked in this 2376 case unless the `--checksum` flag is provided. 2377 2378 In all other cases the file will not be updated. 2379 2380 Consider using the `--modify-window` flag to compensate for time skews 2381 between the source and the backend, for backends that do not support 2382 mod times, and instead use uploaded times. However, if the backend 2383 does not support checksums, note that syncing or copying within the 2384 time skew window may still result in additional transfers for safety. 2385 2386 ### --use-mmap ### 2387 2388 If this flag is set then rclone will use anonymous memory allocated by 2389 mmap on Unix based platforms and VirtualAlloc on Windows for its 2390 transfer buffers (size controlled by `--buffer-size`). Memory 2391 allocated like this does not go on the Go heap and can be returned to 2392 the OS immediately when it is finished with. 2393 2394 If this flag is not set then rclone will allocate and free the buffers 2395 using the Go memory allocator which may use more memory as memory 2396 pages are returned less aggressively to the OS. 2397 2398 It is possible this does not work well on all platforms so it is 2399 disabled by default; in the future it may be enabled by default. 2400 2401 ### --use-server-modtime ### 2402 2403 Some object-store backends (e.g, Swift, S3) do not preserve file modification 2404 times (modtime). On these backends, rclone stores the original modtime as 2405 additional metadata on the object. By default it will make an API call to 2406 retrieve the metadata when the modtime is needed by an operation. 2407 2408 Use this flag to disable the extra API call and rely instead on the server's 2409 modified time. In cases such as a local to remote sync using `--update`, 2410 knowing the local file is newer than the time it was last uploaded to the 2411 remote is sufficient. In those cases, this flag can speed up the process and 2412 reduce the number of API calls necessary. 2413 2414 Using this flag on a sync operation without also using `--update` would cause 2415 all files modified at any time other than the last upload time to be uploaded 2416 again, which is probably not what you want. 2417 2418 ### -v, -vv, --verbose ### 2419 2420 With `-v` rclone will tell you about each file that is transferred and 2421 a small number of significant events. 2422 2423 With `-vv` rclone will become very verbose telling you about every 2424 file it considers and transfers. Please send bug reports with a log 2425 with this setting. 2426 2427 When setting verbosity as an environment variable, use 2428 `RCLONE_VERBOSE=1` or `RCLONE_VERBOSE=2` for `-v` and `-vv` respectively. 2429 2430 ### -V, --version ### 2431 2432 Prints the version number 2433 2434 SSL/TLS options 2435 --------------- 2436 2437 The outgoing SSL/TLS connections rclone makes can be controlled with 2438 these options. For example this can be very useful with the HTTP or 2439 WebDAV backends. Rclone HTTP servers have their own set of 2440 configuration for SSL/TLS which you can find in their documentation. 2441 2442 ### --ca-cert stringArray 2443 2444 This loads the PEM encoded certificate authority certificates and uses 2445 it to verify the certificates of the servers rclone connects to. 2446 2447 If you have generated certificates signed with a local CA then you 2448 will need this flag to connect to servers using those certificates. 2449 2450 ### --client-cert string 2451 2452 This loads the PEM encoded client side certificate. 2453 2454 This is used for [mutual TLS authentication](https://en.wikipedia.org/wiki/Mutual_authentication). 2455 2456 The `--client-key` flag is required too when using this. 2457 2458 ### --client-key string 2459 2460 This loads the PEM encoded client side private key used for mutual TLS 2461 authentication. Used in conjunction with `--client-cert`. 2462 2463 ### --no-check-certificate=true/false ### 2464 2465 `--no-check-certificate` controls whether a client verifies the 2466 server's certificate chain and host name. 2467 If `--no-check-certificate` is true, TLS accepts any certificate 2468 presented by the server and any host name in that certificate. 2469 In this mode, TLS is susceptible to man-in-the-middle attacks. 2470 2471 This option defaults to `false`. 2472 2473 **This should be used only for testing.** 2474 2475 Configuration Encryption 2476 ------------------------ 2477 Your configuration file contains information for logging in to 2478 your cloud services. This means that you should keep your 2479 `rclone.conf` file in a secure location. 2480 2481 If you are in an environment where that isn't possible, you can 2482 add a password to your configuration. This means that you will 2483 have to supply the password every time you start rclone. 2484 2485 To add a password to your rclone configuration, execute `rclone config`. 2486 2487 ``` 2488 >rclone config 2489 Current remotes: 2490 2491 e) Edit existing remote 2492 n) New remote 2493 d) Delete remote 2494 s) Set configuration password 2495 q) Quit config 2496 e/n/d/s/q> 2497 ``` 2498 2499 Go into `s`, Set configuration password: 2500 ``` 2501 e/n/d/s/q> s 2502 Your configuration is not encrypted. 2503 If you add a password, you will protect your login information to cloud services. 2504 a) Add Password 2505 q) Quit to main menu 2506 a/q> a 2507 Enter NEW configuration password: 2508 password: 2509 Confirm NEW password: 2510 password: 2511 Password set 2512 Your configuration is encrypted. 2513 c) Change Password 2514 u) Unencrypt configuration 2515 q) Quit to main menu 2516 c/u/q> 2517 ``` 2518 2519 Your configuration is now encrypted, and every time you start rclone 2520 you will have to supply the password. See below for details. 2521 In the same menu, you can change the password or completely remove 2522 encryption from your configuration. 2523 2524 There is no way to recover the configuration if you lose your password. 2525 2526 rclone uses [nacl secretbox](https://godoc.org/golang.org/x/crypto/nacl/secretbox) 2527 which in turn uses XSalsa20 and Poly1305 to encrypt and authenticate 2528 your configuration with secret-key cryptography. 2529 The password is SHA-256 hashed, which produces the key for secretbox. 2530 The hashed password is not stored. 2531 2532 While this provides very good security, we do not recommend storing 2533 your encrypted rclone configuration in public if it contains sensitive 2534 information, maybe except if you use a very strong password. 2535 2536 If it is safe in your environment, you can set the `RCLONE_CONFIG_PASS` 2537 environment variable to contain your password, in which case it will be 2538 used for decrypting the configuration. 2539 2540 You can set this for a session from a script. For unix like systems 2541 save this to a file called `set-rclone-password`: 2542 2543 ``` 2544 #!/bin/echo Source this file don't run it 2545 2546 read -s RCLONE_CONFIG_PASS 2547 export RCLONE_CONFIG_PASS 2548 ``` 2549 2550 Then source the file when you want to use it. From the shell you 2551 would do `source set-rclone-password`. It will then ask you for the 2552 password and set it in the environment variable. 2553 2554 An alternate means of supplying the password is to provide a script 2555 which will retrieve the password and print on standard output. This 2556 script should have a fully specified path name and not rely on any 2557 environment variables. The script is supplied either via 2558 `--password-command="..."` command line argument or via the 2559 `RCLONE_PASSWORD_COMMAND` environment variable. 2560 2561 One useful example of this is using the `passwordstore` application 2562 to retrieve the password: 2563 2564 ``` 2565 export RCLONE_PASSWORD_COMMAND="pass rclone/config" 2566 ``` 2567 2568 If the `passwordstore` password manager holds the password for the 2569 rclone configuration, using the script method means the password 2570 is primarily protected by the `passwordstore` system, and is never 2571 embedded in the clear in scripts, nor available for examination 2572 using the standard commands available. It is quite possible with 2573 long running rclone sessions for copies of passwords to be innocently 2574 captured in log files or terminal scroll buffers, etc. Using the 2575 script method of supplying the password enhances the security of 2576 the config password considerably. 2577 2578 If you are running rclone inside a script, unless you are using the 2579 `--password-command` method, you might want to disable 2580 password prompts. To do that, pass the parameter 2581 `--ask-password=false` to rclone. This will make rclone fail instead 2582 of asking for a password if `RCLONE_CONFIG_PASS` doesn't contain 2583 a valid password, and `--password-command` has not been supplied. 2584 2585 Whenever running commands that may be affected by options in a 2586 configuration file, rclone will look for an existing file according 2587 to the rules described [above](#config-config-file), and load any it 2588 finds. If an encrypted file is found, this includes decrypting it, 2589 with the possible consequence of a password prompt. When executing 2590 a command line that you know are not actually using anything from such 2591 a configuration file, you can avoid it being loaded by overriding the 2592 location, e.g. with one of the documented special values for 2593 memory-only configuration. Since only backend options can be stored 2594 in configuration files, this is normally unnecessary for commands 2595 that do not operate on backends, e.g. `genautocomplete`. However, 2596 it will be relevant for commands that do operate on backends in 2597 general, but are used without referencing a stored remote, e.g. 2598 listing local filesystem paths, or 2599 [connection strings](#connection-strings): `rclone --config="" ls .` 2600 2601 Developer options 2602 ----------------- 2603 2604 These options are useful when developing or debugging rclone. There 2605 are also some more remote specific options which aren't documented 2606 here which are used for testing. These start with remote name e.g. 2607 `--drive-test-option` - see the docs for the remote in question. 2608 2609 ### --cpuprofile=FILE ### 2610 2611 Write CPU profile to file. This can be analysed with `go tool pprof`. 2612 2613 #### --dump flag,flag,flag #### 2614 2615 The `--dump` flag takes a comma separated list of flags to dump info 2616 about. 2617 2618 Note that some headers including `Accept-Encoding` as shown may not 2619 be correct in the request and the response may not show `Content-Encoding` 2620 if the go standard libraries auto gzip encoding was in effect. In this case 2621 the body of the request will be gunzipped before showing it. 2622 2623 The available flags are: 2624 2625 #### --dump headers #### 2626 2627 Dump HTTP headers with `Authorization:` lines removed. May still 2628 contain sensitive info. Can be very verbose. Useful for debugging 2629 only. 2630 2631 Use `--dump auth` if you do want the `Authorization:` headers. 2632 2633 #### --dump bodies #### 2634 2635 Dump HTTP headers and bodies - may contain sensitive info. Can be 2636 very verbose. Useful for debugging only. 2637 2638 Note that the bodies are buffered in memory so don't use this for 2639 enormous files. 2640 2641 #### --dump requests #### 2642 2643 Like `--dump bodies` but dumps the request bodies and the response 2644 headers. Useful for debugging download problems. 2645 2646 #### --dump responses #### 2647 2648 Like `--dump bodies` but dumps the response bodies and the request 2649 headers. Useful for debugging upload problems. 2650 2651 #### --dump auth #### 2652 2653 Dump HTTP headers - will contain sensitive info such as 2654 `Authorization:` headers - use `--dump headers` to dump without 2655 `Authorization:` headers. Can be very verbose. Useful for debugging 2656 only. 2657 2658 #### --dump filters #### 2659 2660 Dump the filters to the output. Useful to see exactly what include 2661 and exclude options are filtering on. 2662 2663 #### --dump goroutines #### 2664 2665 This dumps a list of the running go-routines at the end of the command 2666 to standard output. 2667 2668 #### --dump openfiles #### 2669 2670 This dumps a list of the open files at the end of the command. It 2671 uses the `lsof` command to do that so you'll need that installed to 2672 use it. 2673 2674 #### --dump mapper #### 2675 2676 This shows the JSON blobs being sent to the program supplied with 2677 `--metadata-mapper` and received from it. It can be useful for 2678 debugging the metadata mapper interface. 2679 2680 ### --memprofile=FILE ### 2681 2682 Write memory profile to file. This can be analysed with `go tool pprof`. 2683 2684 Filtering 2685 --------- 2686 2687 For the filtering options 2688 2689 * `--delete-excluded` 2690 * `--filter` 2691 * `--filter-from` 2692 * `--exclude` 2693 * `--exclude-from` 2694 * `--exclude-if-present` 2695 * `--include` 2696 * `--include-from` 2697 * `--files-from` 2698 * `--files-from-raw` 2699 * `--min-size` 2700 * `--max-size` 2701 * `--min-age` 2702 * `--max-age` 2703 * `--dump filters` 2704 * `--metadata-include` 2705 * `--metadata-include-from` 2706 * `--metadata-exclude` 2707 * `--metadata-exclude-from` 2708 * `--metadata-filter` 2709 * `--metadata-filter-from` 2710 2711 See the [filtering section](/filtering/). 2712 2713 Remote control 2714 -------------- 2715 2716 For the remote control options and for instructions on how to remote control rclone 2717 2718 * `--rc` 2719 * and anything starting with `--rc-` 2720 2721 See [the remote control section](/rc/). 2722 2723 Logging 2724 ------- 2725 2726 rclone has 4 levels of logging, `ERROR`, `NOTICE`, `INFO` and `DEBUG`. 2727 2728 By default, rclone logs to standard error. This means you can redirect 2729 standard error and still see the normal output of rclone commands (e.g. 2730 `rclone ls`). 2731 2732 By default, rclone will produce `Error` and `Notice` level messages. 2733 2734 If you use the `-q` flag, rclone will only produce `Error` messages. 2735 2736 If you use the `-v` flag, rclone will produce `Error`, `Notice` and 2737 `Info` messages. 2738 2739 If you use the `-vv` flag, rclone will produce `Error`, `Notice`, 2740 `Info` and `Debug` messages. 2741 2742 You can also control the log levels with the `--log-level` flag. 2743 2744 If you use the `--log-file=FILE` option, rclone will redirect `Error`, 2745 `Info` and `Debug` messages along with standard error to FILE. 2746 2747 If you use the `--syslog` flag then rclone will log to syslog and the 2748 `--syslog-facility` control which facility it uses. 2749 2750 Rclone prefixes all log messages with their level in capitals, e.g. INFO 2751 which makes it easy to grep the log file for different kinds of 2752 information. 2753 2754 Exit Code 2755 --------- 2756 2757 If any errors occur during the command execution, rclone will exit with a 2758 non-zero exit code. This allows scripts to detect when rclone 2759 operations have failed. 2760 2761 During the startup phase, rclone will exit immediately if an error is 2762 detected in the configuration. There will always be a log message 2763 immediately before exiting. 2764 2765 When rclone is running it will accumulate errors as it goes along, and 2766 only exit with a non-zero exit code if (after retries) there were 2767 still failed transfers. For every error counted there will be a high 2768 priority log message (visible with `-q`) showing the message and 2769 which file caused the problem. A high priority message is also shown 2770 when starting a retry so the user can see that any previous error 2771 messages may not be valid after the retry. If rclone has done a retry 2772 it will log a high priority message if the retry was successful. 2773 2774 ### List of exit codes ### 2775 * `0` - success 2776 * `1` - Syntax or usage error 2777 * `2` - Error not otherwise categorised 2778 * `3` - Directory not found 2779 * `4` - File not found 2780 * `5` - Temporary error (one that more retries might fix) (Retry errors) 2781 * `6` - Less serious errors (like 461 errors from dropbox) (NoRetry errors) 2782 * `7` - Fatal error (one that more retries won't fix, like account suspended) (Fatal errors) 2783 * `8` - Transfer exceeded - limit set by --max-transfer reached 2784 * `9` - Operation successful, but no files transferred (Requires [`--error-on-no-transfer`](#error-on-no-transfer)) 2785 * `10` - Duration exceeded - limit set by --max-duration reached 2786 2787 Environment Variables 2788 --------------------- 2789 2790 Rclone can be configured entirely using environment variables. These 2791 can be used to set defaults for options or config file entries. 2792 2793 ### Options ### 2794 2795 Every option in rclone can have its default set by environment 2796 variable. 2797 2798 To find the name of the environment variable, first, take the long 2799 option name, strip the leading `--`, change `-` to `_`, make 2800 upper case and prepend `RCLONE_`. 2801 2802 For example, to always set `--stats 5s`, set the environment variable 2803 `RCLONE_STATS=5s`. If you set stats on the command line this will 2804 override the environment variable setting. 2805 2806 Or to always use the trash in drive `--drive-use-trash`, set 2807 `RCLONE_DRIVE_USE_TRASH=true`. 2808 2809 Verbosity is slightly different, the environment variable 2810 equivalent of `--verbose` or `-v` is `RCLONE_VERBOSE=1`, 2811 or for `-vv`, `RCLONE_VERBOSE=2`. 2812 2813 The same parser is used for the options and the environment variables 2814 so they take exactly the same form. 2815 2816 The options set by environment variables can be seen with the `-vv` flag, e.g. `rclone version -vv`. 2817 2818 ### Config file ### 2819 2820 You can set defaults for values in the config file on an individual 2821 remote basis. The names of the config items are documented in the page 2822 for each backend. 2823 2824 To find the name of the environment variable, you need to set, take 2825 `RCLONE_CONFIG_` + name of remote + `_` + name of config file option 2826 and make it all uppercase. 2827 Note one implication here is the remote's name must be 2828 convertible into a valid environment variable name, 2829 so it can only contain letters, digits, or the `_` (underscore) character. 2830 2831 For example, to configure an S3 remote named `mys3:` without a config 2832 file (using unix ways of setting environment variables): 2833 2834 ``` 2835 $ export RCLONE_CONFIG_MYS3_TYPE=s3 2836 $ export RCLONE_CONFIG_MYS3_ACCESS_KEY_ID=XXX 2837 $ export RCLONE_CONFIG_MYS3_SECRET_ACCESS_KEY=XXX 2838 $ rclone lsd mys3: 2839 -1 2016-09-21 12:54:21 -1 my-bucket 2840 $ rclone listremotes | grep mys3 2841 mys3: 2842 ``` 2843 2844 Note that if you want to create a remote using environment variables 2845 you must create the `..._TYPE` variable as above. 2846 2847 Note that the name of a remote created using environment variable is 2848 case insensitive, in contrast to regular remotes stored in config 2849 file as documented [above](#valid-remote-names). 2850 You must write the name in uppercase in the environment variable, but 2851 as seen from example above it will be listed and can be accessed in 2852 lowercase, while you can also refer to the same remote in uppercase: 2853 ``` 2854 $ rclone lsd mys3: 2855 -1 2016-09-21 12:54:21 -1 my-bucket 2856 $ rclone lsd MYS3: 2857 -1 2016-09-21 12:54:21 -1 my-bucket 2858 ``` 2859 2860 2861 Note that you can only set the options of the immediate backend, 2862 so RCLONE_CONFIG_MYS3CRYPT_ACCESS_KEY_ID has no effect, if myS3Crypt is 2863 a crypt remote based on an S3 remote. However RCLONE_S3_ACCESS_KEY_ID will 2864 set the access key of all remotes using S3, including myS3Crypt. 2865 2866 Note also that now rclone has [connection strings](#connection-strings), 2867 it is probably easier to use those instead which makes the above example 2868 2869 rclone lsd :s3,access_key_id=XXX,secret_access_key=XXX: 2870 2871 ### Precedence 2872 2873 The various different methods of backend configuration are read in 2874 this order and the first one with a value is used. 2875 2876 - Parameters in connection strings, e.g. `myRemote,skip_links:` 2877 - Flag values as supplied on the command line, e.g. `--skip-links` 2878 - Remote specific environment vars, e.g. `RCLONE_CONFIG_MYREMOTE_SKIP_LINKS` (see above). 2879 - Backend-specific environment vars, e.g. `RCLONE_LOCAL_SKIP_LINKS`. 2880 - Backend generic environment vars, e.g. `RCLONE_SKIP_LINKS`. 2881 - Config file, e.g. `skip_links = true`. 2882 - Default values, e.g. `false` - these can't be changed. 2883 2884 So if both `--skip-links` is supplied on the command line and an 2885 environment variable `RCLONE_LOCAL_SKIP_LINKS` is set, the command line 2886 flag will take preference. 2887 2888 The backend configurations set by environment variables can be seen with the `-vv` flag, e.g. `rclone about myRemote: -vv`. 2889 2890 For non backend configuration the order is as follows: 2891 2892 - Flag values as supplied on the command line, e.g. `--stats 5s`. 2893 - Environment vars, e.g. `RCLONE_STATS=5s`. 2894 - Default values, e.g. `1m` - these can't be changed. 2895 2896 ### Other environment variables ### 2897 2898 - `RCLONE_CONFIG_PASS` set to contain your config file password (see [Configuration Encryption](#configuration-encryption) section) 2899 - `HTTP_PROXY`, `HTTPS_PROXY` and `NO_PROXY` (or the lowercase versions thereof). 2900 - `HTTPS_PROXY` takes precedence over `HTTP_PROXY` for https requests. 2901 - The environment values may be either a complete URL or a "host[:port]" for, in which case the "http" scheme is assumed. 2902 - `USER` and `LOGNAME` values are used as fallbacks for current username. The primary method for looking up username is OS-specific: Windows API on Windows, real user ID in /etc/passwd on Unix systems. In the documentation the current username is simply referred to as `$USER`. 2903 - `RCLONE_CONFIG_DIR` - rclone **sets** this variable for use in config files and sub processes to point to the directory holding the config file. 2904 2905 The options set by environment variables can be seen with the `-vv` and `--log-level=DEBUG` flags, e.g. `rclone version -vv`.